AMBA/AXI-bus in the case of the Pi. GPUs existed long before PCIe did lol.
One some x86 systems the CPU and GPU aren't connected with PCIe either. AMD has infinity fabric that they use for things like the Instinct MI300 and some of their other APUs
Edit: Oh yeah also ARM isn't just low power anymore. It's used in data centers and super computers these days. Even if it was there is lots of stuff you can do with a low power node, including running file servers, DNS or Pi hole, web servers, torrent/usenet downloaders, image and music servers, etc. I have also seen them used to maintain cluster quorum after loss of one more powerful node. A two node cluster won't have quorum if one fails, so adding a pi as a third node makes sense.