GeoMonoBot

CRAWLER NOTICE

If you found this name, or our former name GoPersonaBot, in your server logs, you are in the right place. Every line below is read from the code; none of it is guessed.

Can AI bots get into your site?

Type your domain. The free readiness check reads your robots.txt rules for each AI bot separately and measures whether your server treats those bots differently from how it treats our own crawler.

What you will see

The robots.txt verdict for each search and user bot: OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Googlebot, bingbot
A separate note for training bots (GPTBot, ClaudeBot, Google-Extended, CCBot) that never lowers the score
Whether the same page, requested under AI bot names, gets a different response from the one our own crawler gets
Which CDN sits in front of your site
The Sitemap line in robots.txt, and whether the sitemap really is XML
Directives that restrict quoting of the page in AI Overviews

Crawler notice

About our crawler

Who we are

Our requests identify themselves as: Mozilla/5.0 (compatible; GeoMonoBot/1.0; +https://geomono.com/bot)

Until 29 September 2026 our name was GoPersonaBot. robots.txt rules you wrote for the old name still apply: for every decision about our own crawler we read the rules for both names and apply the stricter one.

GeoMono measures how a brand appears in AI answer engines. The requests we send you are the technical half of that measurement: whether your page can be reached and read by machines.

Exactly what one scan requests

When a person types an address, we fetch the ONE page at it. Then helper files from the same origin: /robots.txt, /sitemap.xml, /llms.txt and the .well-known paths for agent descriptors (mcp/server-card.json, mcp.json, agent-card.json, api-catalog, agent-skills/index.json, http-message-signatures-directory, oauth-authorization-server, oauth-protected-resource).

We also request that same page once as Markdown (Accept: text/markdown) and seven more times, about a second apart, each identifying as a different AI crawler. The reason is below.

This happens once per scan: a limited number of requests for your page, robots.txt, llms.txt and sitemap files. The page request times out at 12 seconds and the helpers at 6; we do not read beyond 5 MB.

Why seven of our requests carry an AI crawler's name

Seven requests carry the names OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot, GPTBot and ClaudeBot, each followed by our own identity: "GeoMonoBot-probe; +https://geomono.com/bot". What is measured is whether your edge treats those crawler names differently from how it treats our own crawler, and since the rule keys on the name, the request has to carry it.

These requests carry no contact address of OpenAI, Anthropic or Perplexity, because we are the ones sending them. If you see those lines in your logs, this is where they came from. A rule that matches the whole user-agent string exactly will not fire on them, which is why a "treated the same" result is not a guarantee.

Our relationship with robots.txt

We read your robots.txt, but not as permission for our own request; we read it to report on it. A person typed an address and that single page was fetched; this is an audit, not a crawl.

We write it that way because "we respect robots.txt" would read better and would not be true of this request.

How to block us

Add to your robots.txt: User-agent: GeoMonoBot followed by Disallow: /. The daily readiness check obeys it fully: if your homepage is disallowed, it reads only robots.txt and fetches nothing else. The weekly re-measurement still reads your homepage and skips every other disallowed page. The single page read when someone types your address into the free scan is an audit and is also fetched. To stop those too, block our identity at your edge.

We visit your site in two cases: when someone types your address into the free scan, or, if your brand is tracked in GeoMono, through our scheduled jobs (a daily readiness check and a weekly re-measurement). We do not visit an untracked site on our own. Blocking our identity at your edge works too.

What we do not do

We do not send requests to private or internal addresses; a guard refuses them. We do not fill in forms, sign in, or click anything on your page. We read nothing beyond the bytes your server already serves publicly.

If you saw something in your logs you did not expect, write to us:

info@geomono.com

How to use it

Type your domain
Type your site's address in the box; the free readiness check opens and the scan starts straight away.
Open the AI access heading
The result shows a separate row for each bot. The rows to fix are where that bot is stopped, in robots.txt or at the edge.
Decide on purpose
You can keep search bots open and make a separate decision about training bots. The example below shows what a robots.txt that separates the jobs looks like.

Guide

Three kinds of AI bot

The requests an AI company sends your site serve one of three jobs, and each job has its own bot name. The table below shows the bots our scan reads one by one, by the job their companies' own documentation gives them.

AI bots the scan reads in robots.txt by name
Bot	Company	Job	Weight in the scan
OAI-SearchBot	OpenAI	Search index	Critical
ChatGPT-User	OpenAI	User request	Medium
GPTBot	OpenAI	Model training	For information
Claude-SearchBot	Anthropic	Search index	Critical
Claude-User	Anthropic	User request	Medium
ClaudeBot	Anthropic	Model training	For information
PerplexityBot	Perplexity	Search index	Critical
Perplexity-User	Perplexity	User request	Medium
Googlebot	Google	Search index	Critical
Google-Extended	Google	Model training	For information
bingbot	Microsoft	Search index	Medium
CCBot	Common Crawl	Model training	For information

The weight column is the weight in the scan. The search bots of OpenAI, Anthropic, Perplexity and Google are critical, because they are the way to being a source in that engine's answers. User bots are medium: they open your page the moment a user asks about it, and OpenAI and Perplexity document that robots.txt rules may not apply to those requests. Training bots are for information only; blocking them is a data policy decision.

Google-Extended is a separate control token. According to Google's documentation it governs whether your content may be used to train Gemini models and for grounding in some products; it does not affect inclusion or ranking in Google Search.

Telling AI bots apart in robots.txt

In robots.txt each bot can be addressed by its own name, and one group can carry several bot names. A bot with no group of its own follows the star group (User-agent: *). A common mistake is closing search bots with the same general rule while trying to stop training bots.

# Search and user bots: allowed
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: PerplexityBot
User-agent: Perplexity-User
Allow: /

# Training bots: closed in this example
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
User-agent: CCBot
Disallow: /

Sitemap: https://yourdomain.com/sitemap.xml

An example robots.txt. Not a recommendation, an example of a decision that separates the jobs.

Leaving training bots open is a legitimate choice too; what matters is that each group is written knowing what it is for. Googlebot and bingbot are not listed in this example: when there is neither a group for a crawler's name nor a star group, the protocol places no restriction on it. After writing the rule, run the scan: each bot's robots.txt verdict appears on its own row.

A note: robots.txt is a protocol of requests, not access control. The protocol's own document (RFC 9309) says plainly that these rules are not a form of authorisation. If you need a hard block, write the rule on the server or in the CDN.

robots.txt allows it, the CDN stops it

Even when robots.txt allows it, a request can be stopped before it reaches your server. Cloudflare announced in July 2025 that it was changing its default to block AI crawlers for new domains. A firewall rule or a bot management setting can do the same, and none of it shows in robots.txt.

To measure this, our scan requests the same page once under our own identity and three times under identities carrying the names GPTBot, ClaudeBot and PerplexityBot, and compares the responses. Those requests carry our identity too; the notice above says why.

Its limit, stated: this is evidence from one address, at the level of the user agent. Some CDNs verify bots by IP address, and the real bot may be treated differently. The proof is in your server logs.

What controls quoting?

How much of a page may be quoted in AI Overviews is decided by page-level directives, not by robots.txt. Google documents that the controls governing search result previews (nosnippet, data-nosnippet, max-snippet, noindex) govern its AI features too.

Added by mistake, these directives do damage nobody sees: the page stays indexed and ranks, but the text available for quoting in an AI Overview is restricted. Our scan reads them as a critical item. Some meta tags proposed for AI (such as noai) are not in that Google document; use the documented controls.

Frequently asked questions

Should I block AI bots?

It is your decision, but keep two decisions apart. Blocking search and user bots closes the way to being a source in that engine's answers. Blocking training bots is a preference about your content being used to train models. If you want one and not the other, write separate groups in robots.txt.

I allowed it in robots.txt and it still shows as blocked. Why?

The most common reason is the CDN or firewall in front of the server. A bot rule that does not show in robots.txt can stop the request before it reaches your page. Our scan shows this by requesting the same page under bot names and comparing the responses; check the bot settings in your CDN's dashboard.

If I block Google-Extended, do I leave AI Overviews?

According to Google's documentation, no. Google-Extended does not affect inclusion in Google Search; it governs training of Gemini models and use in some other products. The documented way to limit what AI Overviews show from your pages is the page-level preview controls.

How do I recognise your measurement requests in my logs?

The user-agent string carries the name GeoMonoBot (GoPersonaBot until 29 September 2026). Our three measurement requests that carry AI bot names also end with the GeoMonoBot-probe identity and this page's address.

Does publishing llms.txt make bots read me?

We know of no public evidence that it does. Google writes that no such file is needed for its AI features. Our scan reads llms.txt for information only; its absence does not lower the score.

Other free tools

All free tools

Ready is not the same as named. Do the engines name you?

The free tools tell you whether your site is ready for AI engines. Whether the engines actually name you, who they recommend instead and whether what they say about you is true, we measure by putting real questions to real engines. That is the paid measurement, and we scope it with you.

Get a quote See Visibility Measurement

Sources

Public documents behind the outside claims on this page. Sentences about how the scan works are read from its code and from the reasons given in the checklist.