Connections
AI crawler access and llms.txt
AI engines can only cite pages they are allowed to read. Here is what Nalza checks and fixes.
Which crawlers Nalza checks
Nalza reads your robots.txt and reports any rule that blocks the crawlers AI engines use to read and cite the web:
- GPTBot, OAI-SearchBot and ChatGPT-User (OpenAI)
- ClaudeBot, Claude-Web and anthropic-ai (Anthropic)
- PerplexityBot and Perplexity-User
- Google-Extended
- Applebot-Extended
- CCBot (Common Crawl)
- Bytespider
If one is blocked, a Move drafts the robots.txt change that lets it in. If you block a crawler on purpose, leave that Move unapproved.
llms.txt
llms.txt is a short file at the root of your site that tells AI engines which pages matter most. Nalza drafts one from the pages it crawled, and the connector publishes it once you approve it.
Renderability
AI crawlers read the HTML your server returns. If the words on a page only appear after JavaScript runs, they see an empty page. Nalza measures how much of each page's text is in that HTML and flags the pages that depend on scripts.
Related
- Install the WordPress connectorThe connector applies the changes you approve and reports back what actually landed.
- Connect Search Console, Analytics 4 and Google Business ProfileWhat each Google connection adds to Nalza, and what Nalza does with the access.
Did not find what you needed? Ask us and we will reply the same business day.