Cloudflare has launched BotBase for Operators to give bot operators a clearer way to list their bots in the BotBase directory and track the status of requests after submitting them. The new feature shows whether a request is under review, accepted, or rejected, along with the reason for rejection or any change Cloudflare made to the bot’s classification.
The move follows Cloudflare’s launch of a searchable directory of known bots within the dashboard, along with tools that help site owners understand how crawlers interact with their content. The company believes that managing automated traffic does not depend solely on site owners’ decisions, but also requires bot operators to identify themselves, explain how they operate, and keep their data accurate.
A Unified Space for Submission Requests
The bot submission form was previously available under Manage Account → Configurations, making the process more closely tied to the account than to the bot ecosystem. Now, BotBase for Operators can be accessed through Protect & Connect → Application Security → BotBase in the Cloudflare dashboard, and the company says all customers can access it.
The interface is divided into three main sections:
- Bots directory: Browse, search, and filter the bots tracked by Cloudflare.
- Submission form: Submit a request to list a new bot.
- Submission history: Track requests submitted by the account.
The submission history page displays three primary statuses: Waiting for review when the request is received and placed in the review queue, Accepted after the bot is approved and added to the directory, and Rejected when the request requires changes. In the latter case, the reason for rejection and the steps required for resubmission are shown.
Editing Bot Data and Describing Its Behavior
Operators can now edit a previously submitted request instead of creating a new listing when details change, such as the endpoint for an IP address list or moving from an IP allowlist to using a Web Bot Auth signature. They can also cancel the request as long as it is awaiting review. Cloudflare recommends keeping this data up to date because its accuracy contributes to a bot obtaining and retaining Verified status, while the final decision on whether to allow traffic remains with each site owner.
The new listing form is based on the behavior and content-use model Cloudflare introduced on July 1, rather than requiring the operator to select a single classification. The form asks for three aspects:
- What the bot does, such as indexing pages, acting on behalf of a user, collecting data, training models, or supporting search engine optimization tools.
- How it uses the content it reads, specifying the level of use according to the Content Signals model.
- The entity operating it: a direct operator managing its infrastructure, or an intermediary executing requests originating from another product or platform.
Cloudflare explains that a bot reading a page to create a search snippet differs from a bot storing the page to train a model. It cites an example of a signal that could include Content-Signal: search=yes, ai-train=no, use=reference, allowing indexing for search and retention as a reference, but not permitting AI training.
What Changes in Practice?
Cloudflare has also rebuilt the review process to include automated checks after the number of new annual requests increased roughly sevenfold since 2023. The systems check for potential bot duplication, the uniqueness of the user-agent pattern, and the validity of the declared verification method. They can retrieve an IP address list and verify reverse DNS or check a Web Bot Auth signature. If a request requires human review, it is referred to the team with the reason for the check specified instead of being placed in an undescribed queue.
The update represents BotBase’s transition from a directory primarily aimed at site owners to a space that also includes bot operators. However, the current launch focuses only on visibility; claiming ownership of and managing a listing, as well as understanding how sites handle it, are planned for later, while the idea of creating a dialogue channel between operators and site owners remains a more distant goal. Therefore, BotBase does not eliminate individual site decisions, but provides more consistent information on which those decisions can be based.