CCBot

About this crawler

CCBot is Common Crawl’s shared-dataset crawler. Common Crawl's crawler. Common Crawl is the public corpus most large language models train on. Being in Common Crawl means being seen by every major LLM lab indirectly.

How this page works

1. Every request to findloc.ai passes through a Next.js middleware.
2. If the User-Agent matches a known AI bot pattern, one row is inserted into the ai_crawler_visits Postgres table. Fire-and-forget so the bot’s response isn’t slowed.
3. This page queries that table via three SECURITY DEFINER RPCs (ai_bot_hit_counts, ai_bot_daily_hits, ai_bot_top_paths) on every render.
4. Page response is cached for 1 hour via Next.js ISR — so DB load is bounded even under load, and the freshest number is always at most 60 minutes old.