I've heard Quassel Core is multi user capable? How do I make use of that?

Yes, but you can't use it yet.

The Quassel Core in fact is already multi user capable. In fact we use our development core already in that way.
But since there aren't configuration options at the time being, there is no way to setup additional accounts or manage existing ones.
Once this is done you can add accounts.