Category: data

OpenAI launches web crawling GPTBot, sparking blocking effort by website owners and creators

When OpenAI added the GPTBot support page, it also introduced a way to block the service from scraping your website through robots.txt files.