Online media brands hope that a new protocol will stop unwanted AI crawlers

Online media brands, including Yahoo, Quora and Medium, take a new step to prevent AI companies from copying and using their content to train models without their permission.

The publishers, including Cnet’s parent company Ziff Davis, see this new tool called RSLIf another way to ensure that large AI developers do not use their work without payment or compensation – a problem that has already led to a large number of lawsuits.

RSL, which stands for really simple licenses, is inspired by Really simple syndicationAn old web stand that offers up-to-date and automatic content updates in a computer-readable format. Just like RSS, RSL is open, decentralized and can work with almost every piece of content online, including web pages, videos and data sets.

View this: The new iPhone air changes the game for pre -orders

5:34 AM

At the moment, when the wandering internet robot of an AI company, known as a crawler, wants to suck the information on a site, it must go through Robots.txt, which acts as a basic input or non-instort door. AI companies have found ways around robots.txt or ignored it and have Then were charged. The goal for RSL is to be a more robust layer of technology to deal with AI crawlers, which are now good More than half of all internet traffic. (Disclosure: Ziff Davis, CNET’s parent company, brought a lawsuit against OpenAi in April, which claimed that it infringed Ziff Davis authentic rights in training and operating his AI systems.)

“RSL builds directly on the inheritance of RSS and offers the missing license layer for the ai-first internet,” said Tim O’Reilly, CEO of O’Reilly Media, in a press release. “It ensures that the makers and publishers feeding AI innovation are not only part of the conversation, but are reasonably compensated for the value they create.”

Brands that have registered with RSL are Reddit, People, Internet Brands, Fastly, Wikihow, O’Reilly, Daily Beast, The Mit Press, Miso, Adweek, Ranker, Evolve Media and Raptive.

“If AI is trained at the work of our writers, it has to pay for work,” said Medium CEO Tony Stubblebine in a press release. “AI is currently on stolen content. The adoption of this RSL standard is how we force those AI companies to pay for what they use, stop using or are closed.”

The arrival of RSL is delivered because online web traffic is clatered with changes to Google and the superiority of AI. Google’s integrated AI-generated answers at the top of the Google search have been Criticized by publishers As a way of potential clicks, they would have received differently. Google argues that AI overviews send “Clicks of higher quality“For sites, people who are more involved and stay in locations for longer. AI chatbots such as chatgpt also help with research and synthesis, which means that people do not have to jump around different sites to bring pieces of information together in the same way as before. Crime.

“Widespread acceptance of the RSL standard will protect the integrity of the original work and speed up a mutual affordable framework for publishers and AI providers,” said Ziff Davis CEO Vivek Shah.

In response, publishers AI companies or ink license -deals continue. In other cases, sites turn to services such as Tollbit, who want AI crawlers to ask every time they ask to investigate the content of a site. Content delivery networks such as Cloudflare, which ensure that people have quick access to sites online, are Block ai crawlers.

RSL-MEDE founder Eckart Walther said that the RSL standard and such efforts are complementary by Cloudflare, with many of the same media companies participating in both. Walther compared the tools such as Cloudflare with bouncers that protect a website against unwanted crawlers, while RSL De Cawler simply enables the rules and the admission price to understand. “These compensation methods can also work together. For example, a publisher may want to charge for crawling their content and then also requires a royalty payment every time the content is used by an AI model to answer a question,” Walther said in an e -mail to CNET.