AI Crawler Access Check
io.github.hamzawamiqhq/ai-crawler-check
Can GPTBot, ClaudeBot and PerplexityBot reach a site? Real fetch per crawler, firewall/CDN blocks.
This endpoint completed initialize and tools/list. No tool was called.
- Rank
- #3034
- Score
- 81
- Tool tokens
- 201
- Tools
- 1
Tools
Token cost is the o200k count of the tools list JSON, including schemas, before any tool is called.
check_ai_crawler_access
Checks whether AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended and others) can actually reach a website. Goes beyond robots.txt: fetches the site as each cr
Nearby in GitHub
Up-to-date code docs for any prompt
MCP server for Financial Modeling Prep API with 250+ financial data tools
Official SerpApi MCP server for Google, Bing, and other search engines.
Comprehensive PostgreSQL documentation and best practices, including ecosystem tools