Cloudflare announced a new system for continuously evaluating the behavior of verified bots, in a move aimed at linking a bot’s identity to a practical record showing how well it respects site preferences. The company believes that knowing which entity operates a bot is not enough on its own to determine its trustworthiness, as a known bot may ignore site instructions or continue sending requests despite receiving clear signals to reduce its activity.
Cloudflare uses the term Verified Bot for bots that meet three basic conditions: accurately and securely identifying themselves, doing what they say they will do, and not engaging in abusive behavior. Identification is based on Web Bot Auth for signing requests, while BotBase catalogs these identities in an open format. The new system focuses on the second and third conditions by monitoring repeated behavior rather than issuing a permanent judgment based on a single incident.
From Identity to Accumulated Trust
Cloudflare distinguishes between risk and trust. Risk describes the likelihood that a particular request will be harmful at a specific moment, while trust is built through repeated and consistent behavior. The company will turn behavioral observations into an explainable “trust range” showing how consistently a bot respects site preferences and the factors that led to the evaluation.
One example cited by the company is the removal of the Perplexity bot from the verified bots list in 2025, after it attempted to access resources on sites that had chosen not to allow scraping and then tried to bypass controls when those requests were denied.
Signals the System Monitors
The initial evaluation focuses on two questions: Does the bot respect published instructions? And does it respond to direct feedback from the site? Cloudflare considers robots.txt the primary mechanism sites use to publish their crawling preferences. The system therefore monitors whether bot traffic aligns with these preferences over time, focusing on recurring patterns rather than isolated violations.
The company explains that robots.txt defines the minimum rules a bot should follow and does not mean that everything not mentioned in it is automatically permitted. A site may send other signals indicating that continued access is unwanted.
The system also monitors how a bot responds to HTTP 429, which means “Too Many Requests” and asks the visitor to reduce its request rate; it may also include a waiting period before retrying. A bot that slows down or stops after receiving this signal strengthens its trust range, while repeatedly ignoring it lowers the evaluation.
What Changes in Practice?
Cloudflare says the system is currently operating internally to evaluate how well verified bots comply with site preferences. The company plans to make it available to all its customers, giving them the ability to specify the level of trust required when deciding how to handle Verified Bot traffic.
The system is part of an accountability loop that combines it with Bot Preference Sync, which helps site owners publish their preferences, and Bot Management, which allows them to determine how to handle automated traffic. In practice, this adds a behavioral layer to the allow decision: a site will not rely solely on a bot’s identity but can also draw on its record of respecting instructions and responding to pressure signals.
However, Cloudflare has not yet specified the details of how the trust range will be calculated or when it will become generally available. The system also remains in the stage of building a historical record and refining how observations are converted into guidance. The company intends to publish these scores on Radar in the future and expand the signals included in the evaluation.