johnvoorhees , 15 days ago Six days later, what started as a ‘WTF is going on?’ moment for @robb, @viticci, and me has become something else entirely. The latest from Business Insider, which reports that OpenAI and Anthropic are ignoring publishers’ robots.txt files too. https://www.businessinsider.com/openai-anthropic-ai-ignore-rule-scraping-web-contect-robotstxt See also: https://www.reuters.com/technology/artificial-intelligence/multiple-ai-companies-bypassing-web-standard-scrape-publisher-sites-licensing-2024-06-21/
Six days later, what started as a ‘WTF is going on?’ moment for @robb, @viticci, and me has become something else entirely.
The latest from Business Insider, which reports that OpenAI and Anthropic are ignoring publishers’ robots.txt files too.
https://www.businessinsider.com/openai-anthropic-ai-ignore-rule-scraping-web-contect-robotstxt
See also: https://www.reuters.com/technology/artificial-intelligence/multiple-ai-companies-bypassing-web-standard-scrape-publisher-sites-licensing-2024-06-21/