How to Appear in ChatGPT Search Results

ChatGPT can only show what it can fetch

A page can rank in Google and still be missing from ChatGPT search answers. One common cause is simple: the right crawler could not reach the page, or the useful text was missing from the raw HTML.

For ChatGPT search, the crawler to allow is OAI-SearchBot. If you block it in robots.txt, you are telling OpenAI not to use that content for live search answers. That is separate from GPTBot, which is a training crawler. Blocking GPTBot does not remove you from ChatGPT search results.

Start with the boring checks. The page should return HTTP 200, be open without a login, have a clear canonical URL, and expose the main answer in server-rendered HTML. Put the direct answer near the top. Use plain headings. Link to the page from related pages. If the page needs JavaScript before the text appears, fix that first. Google documents JavaScript rendering for Googlebot, but most AI-search crawler behavior around rendering is less clear. Treat client-side-only content as a risk.

• Allow OAI-SearchBot in robots.txt for pages you want ChatGPT search to find.
• Make the main text readable in view-source HTML before JavaScript runs.
• Keep pages indexable, public, fast, and internally linked.
• Use a clear title, one main topic, and answers that can stand alone when cited.
• Check crawler access with the AI visibility checker before you wait for traffic.

Use the right robots.txt rules

AI crawler names are easy to mix up. That mistake can hide your site from the answer engines you want while doing nothing for the ones you meant to block.

The crawlers that matter for live AI answers are OAI-SearchBot for ChatGPT search, Claude-SearchBot for Claude, PerplexityBot for Perplexity, Googlebot for Google AI Overviews, and Applebot for Apple Intelligence. Google AI Overviews use the normal Google Search index. There is no separate Google AI Overviews crawler to block without also affecting Search visibility.

Training and opt-out controls are different. GPTBot, ClaudeBot, CCBot, Google-Extended, and Applebot-Extended are training or data-use controls. Blocking them does not block live AI-search visibility by itself. Google-Extended and Applebot-Extended are robots-only control tokens, with no separate crawl user-agent you should expect to see in server logs.

Robots.txt is a stated policy. It is not proof of what happened on your server. Well-behaved crawlers use it, but public reports have said Perplexity-User and Bytespider may ignore it. Use logs to see requests, and use vendor docs to decide which user agents to allow or block.

• For ChatGPT search visibility, do not disallow OAI-SearchBot.
• For Claude answer visibility, do not disallow Claude-SearchBot.
• For Perplexity answer visibility, do not disallow PerplexityBot.
• For Google AI Overviews, keep Googlebot access if you want normal Search and AI Overview visibility.
• For Apple Intelligence visibility, keep Applebot access if you want Apple to crawl the page.

Write pages that answer engines can trust

Crawler access gets you considered. Clear, useful content gets you cited.

A good answer page says the answer early, then supports it with details a reader can verify. Do not bury the answer below a long intro. Do not split the real content across tabs that are empty in the raw HTML. Use normal links, not button-only navigation. If you quote a standard or vendor policy, name the source and keep the claim narrow.

For technical pages, use the published standard or vendor doc. SPF is defined by RFC 7208, DKIM by RFC 6376, and DMARC by RFC 7489. Check the current Google sender guidelines and Microsoft sender guidance for mailbox rules. For crawler names, use the docs from OpenAI, Anthropic, Perplexity, Google, Apple, and Common Crawl.

Answer engines also need a reason to pick your page over a thin rewrite. Show steps, failure modes, examples, and what to check first. For a page about email deliverability, explain the DNS records, the inbox-provider behavior, and the tradeoffs. A short page with one clear fix is better than a long page that never makes a decision.

• Put the direct answer in the first screen of content.
• Use descriptive headings that match the questions people ask.
• Keep important text in HTML and avoid image-only explanations.
• Link to related context, such as your main guide hub at InboxRadar guides.
• Refresh pages when vendor crawler names, sender rules, or product behavior changes.

Do not ignore domain trust

Search visibility and email trust meet at the same place: your domain. If your domain looks broken, throwaway, or easy to spoof, people and filters trust it less.

SPF is a TXT record that lists the servers allowed to send mail for your domain. Publish one SPF record, keep it under the RFC 7208 limit of 10 DNS lookups, and avoid +all. Use ~all while you are still testing or cleaning up shared senders. Use -all only after every real sender is covered and DMARC reports look clean.

DKIM signs each message with a private key at your mail provider and a public key in DNS. The DNS name uses a selector, such as selector._domainkey.example.com. Each sender that sends mail for you should sign with DKIM. Selectors let you rotate keys without breaking every sender at once.

DMARC tells receivers what to do when a message fails DMARC and checks whether the visible From domain aligns with SPF or DKIM. Start with p=none and a rua aggregate report address so you can see every sender. Then move to p=quarantine or p=reject after your real mail passes. A strict policy too early can block your own mail.

MX records tell the internet where replies should go. They are separate from SPF, DKIM, and DMARC. Broken MX can make replies bounce. Blocklists are worth checking after authentication, especially if a domain was compromised, warmed too fast, or used with a bad list.

Gmail, Outlook, and other mailbox providers do not route mail by one DNS record alone. They weigh authentication, domain and IP reputation, complaint rate, engagement, sending volume, content, links, and past behavior. A clean DNS setup does not guarantee inbox placement, but a broken one is one of the fastest ways to lose it. A quick domain scorecard can show SPF, DKIM, DMARC, MX, and blocklist problems before you send from the domain.

• Publish one SPF record and keep the lookup count below 10.
• Enable DKIM signing for every real sender and check the selector records.
• Use DMARC reports before moving from p=none to enforcement.
• Keep MX records working so replies do not bounce.
• Check blocklists after you confirm SPF, DKIM, and DMARC are healthy.

A simple order of work

Do the checks in the order that removes the biggest blockers first.

First, make sure the page is public and readable in raw HTML. Second, allow the live search crawlers you want. Third, write the page so it answers one query cleanly. Fourth, protect the sending domain so your brand does not look spoofable or abandoned. Then watch the page and the domain over time, because both robots.txt and DNS can drift when tools, hosts, and email providers change.

• Fetch the page without cookies and confirm the answer is visible.
• Review robots.txt for OAI-SearchBot, Claude-SearchBot, PerplexityBot, Googlebot, and Applebot.
• Separate training opt-outs from live search crawler access.
• Fix SPF, DKIM, DMARC, MX, and obvious blocklist issues.
• Recheck after site deploys, DNS changes, or sender changes.