I am hosting both XMPP and Matrix now, and my main concern with Matrix is storage. I am afraid it would eat up the very limited disk space I have on my VPS. Conduit offers no built-in way to clean files up, and media is stored in a weird way that makes it a PITA to see which ones can and cannot be deleted. I now know that neither the database files nor the media can be just deleted.
I sorta like the idea of a chatroom existing on more than one server, but that MUST have been opt-in or at the very least opt-out.