I have a Mikrotik router, 2x VLAN-enabled switches, and 3x VLAN-enabled APs. My Internet access broke every day for a month while I figured out what I was doing.
It's going to haunt you for years. Wife: "why doesn't this iot thing work?" Oh its on the wrong network. "I don't understand it just doesn't work". Then I go add more exceptions in pfsense and the cycle continues.
The correct way of doing this is to never interact with an iot device directly. Put all of them on the same network with Home Assistant and then control all of them only via Home Assistant. Then you make one exception for home assistant to be accessible to the other networks.
This also allows you to disable Internet access for every single iot device Except home assistant.
I have so much trash on my network. For instance my security cameras are decentralized and we use tinycam on android to view them so I had to allow an exception for rtsp to get between the dirty iot WiFi and our normal WiFi. Our WiFi connected Bose speakers and Spotify is also another set of annoyance, they will only work if you try to connect to them from the same subnet via Spotify and app. I've tried to NAT the traffic and it didn't work.
It's not easy but they only way to make it all work without creating massive security holes is to only buy things that allow connection with open standards (which means home Assistant can connect to it.