Can crawlers see content rendered by JavaScript?

Why rendering matters

Some can and some may not. Google renders JavaScript, but many other crawlers, including several used by AI products, appear to read only the HTML your server sends. If your main content, links or metadata exist only after JavaScript runs in the browser, those crawlers may see an empty or partial page.

The safe approach is to make sure everything that matters is in the initial HTML response, and to treat JavaScript as an enhancement rather than the only way content appears.

How Google handles JavaScript

Google processes JavaScript pages in three phases: crawling, rendering and indexing. After fetching the HTML, Googlebot queues the page for rendering in an up-to-date version of Chromium. Google says a page may wait in that queue for a few seconds or longer, so content that depends on rendering can be processed later than content already in the HTML.

• Googlebot does not click buttons or fill in forms, so content that only loads after a user action may not be seen.
• Scripts and stylesheets blocked in robots.txt cannot be used for rendering.
• In practice, heavy or failing scripts can leave parts of a page unrendered.

AI crawlers and raw HTML

OpenAI, Anthropic and Perplexity document their crawlers' names and how to control them with robots.txt, but they do not publicly document whether those crawlers execute JavaScript. In practice they appear to fetch the HTML without running scripts. Until that is documented otherwise, assume that content which only appears after client-side rendering may be invisible to them.

Google says pages need to be indexed in Google Search to appear as links in its AI features, such as AI Overviews, and that index is built from rendered pages. A page can therefore look complete to Google and still be thin or empty to other AI systems.

Put important content in the HTML

• Use server-side rendering, which builds the HTML for each request, or static generation, which builds it ahead of time. Both send complete HTML that scripts can then make interactive.
• Include the main text, headings, prices, product details, structured data, canonical tag and meta robots tag in the initial HTML.
• Make links real anchor elements with an href attribute that holds a URL. Google says it can only reliably follow links built this way, not click handlers on other elements.
• Give each distinct view its own URL rather than relying on the fragment after a # symbol.
• Avoid dynamic rendering, which serves crawlers a separate prerendered version. Google describes it as a workaround, not a long-term solution.

<!-- Crawlable -->
<a href="/pricing">Pricing</a>

<!-- Not reliably crawlable -->
<span onclick="goTo('/pricing')">Pricing</span>

How to check what crawlers receive

1. Open view-source for the page, not the Elements panel in developer tools, which shows the page after scripts have run. Search it for your main heading and a sentence of body text.
2. Load the URL with JavaScript disabled, or request it with a command-line tool, and confirm the content and links are still there.
3. Run the URL Inspection live test in Search Console to see the HTML Google rendered, and compare it with the raw source.

If the main content is missing from the raw HTML but present after rendering, Google can probably still index it, but crawlers that do not run JavaScript likely cannot.