When adding a new source in Streamlabs OBS, select Alert Box under Widgets. Then position the box wherever you would like the alerts to appear on your stream. Voila! Your alerts are all set up.

In XSplit and normal OBS, adding sources and changing their settings is fast and easy, but in Streamlabs OBS this area of the app often hangs for me.