I thought that worse case scenario this would be an overnight transfer, but only 40 GB have been transferred so far.
The server's bandwidth use is well below the limit, so I think that the issue might be on the receiving end (Contabo's S3-compatible object storage).
I had to decrease the number of parallel connections because otherwise I would get too many errors and the transfer would stop, so it might not be the transfer speed of a single file, but rather the amount of time the server has to wait between initiating transfers. I am not sure, but at ~50GB per 12 hours this will end up taking almost 4 days... I will ask in the Matrix chat to see if this was also the experience of others who migrated.