PCIe is absolutely plug and play. Cards have been PnP since the ISA era. You probably meant hot-plug, but it's hot-pluggable too: https://lwn.net/Articles/767885/
Any buffered data will sit in the buffer, and eventually be dropped. Any data sent to the buffer while the buffer is full will be dropped. I'm not intimately familiar with communicating with GPUs, but I imagine the only buffers are in the GPU driver (which would either handle the removal or crash) or in the application (which would probably not handle the removal and just crash). Buffering is not really where I would expect to see a problem.
That said, a GPU disappearing unexpectedly will probably crash your program, if not your whole OS. Physical damage is unlikely, though I definitely wouldn't recommend connecting two PSUs to one system due to the potential for unexpected... well, potential. Inrush current wouldn't really be my concern, since it would be pulling from the external PSU which should have plenty of capacity (and over-current protection too, I would hope). And it's mostly a concern for AC systems, rarely for DC.