The biggest names in AI are backing a new way to opt out of scraping
中文摘要
OpenAI、Google、微软和 Anthropic 将采用新标准,让出版商能通过“策略令牌”一次性拒绝多家 AI 公司的抓取。
English Summary
OpenAI, Google, Microsoft, and Anthropic are adopting a new standard, letting publishers opt out of AI scraping across multiple companies simultaneously via "policy tokens."
OpenAI, Google, Microsoft, and Anthropic say they will adopt a new standard that publishers can use to opt out of being scraped by AI, as reported earlier by Search Engine Roundtable. With this control, publishers can apply their AI training preferences across multiple AI companies, instead of setting them individually. The shared "policy token" signal is proposed by Kevin Kelley, a technical lead for standards at Anthropic, as part of the Internet Engineering Task Force (IETF). It will live within a website's robots.txt file, which has long signaled which web crawlers can and can't scrape a site - whether for search indexing, archiving, or … Read the full story at The Verge.