@pluralistic
I think you miss two key points in here.
No mention of "commercial impact" prong of fair use - LLMs directly diminish the market for the original works (esp. stuff like promoting midjourney with "in the style of X").
The memorization problem isn't easy - it's impossible. LLM providers today are accounting for it on a vanishingly small number of potential outputs, and it's not by altering the models (they basically can't). It's by slapping output filters on at the end.