Overview
PerplexityBot is the web crawler for Perplexity AI, an AI-powered search engine that provides direct answers to questions with cited sources. Unlike GPTBot or ClaudeBot, PerplexityBot is primarily a search crawler — it fetches pages in near-real-time to answer user queries.
PerplexityBot crawls content to surface and link websites in Perplexity's search results. Perplexity also uses a separate agent (Perplexity-User) for live browsing during user queries. Together, they make Perplexity function more like a search engine than a training data collector.
Because Perplexity links back to source pages, allowing PerplexityBot can drive referral traffic. This differentiates it from pure training crawlers like GPTBot, where the value exchange is less direct.
User-Agent String
PerplexityBot identifies itself with the following user-agent string:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)How PerplexityBot Handles OG Images
PerplexityBot fetches page content for AI search results. OG images may appear alongside citations in Perplexity's search results.
Meta Tags Read
og:imageog:titleog:description| Preferred Size | 1200 × 630 |
| Cache Duration | PerplexityBot crawls periodically to build the search index. The separate Perplexity-User agent fetches pages on demand during user queries. Content freshness depends on crawl frequency. |
Caching Behavior
PerplexityBot crawls periodically to build the search index. The separate Perplexity-User agent fetches pages on demand during user queries. Content freshness depends on crawl frequency.
Fallback Behavior
PerplexityBot reads the full page content. OG tags help with result presentation but the full text content is used for generating answers.
Things to Know
- PerplexityBot crawls content to build Perplexity's search index, while the separate Perplexity-User agent handles live browsing during user queries.
- Perplexity cites sources with links, potentially driving referral traffic to your site.
- PerplexityBot may fetch your page periodically to keep its search index fresh.
- The crawler focuses on text content — images are secondary to the text-based answer generation.
robots.txt
Respects robots.txt. Blocking PerplexityBot prevents your content from appearing in Perplexity AI search results.
# To allow PerplexityBot:
User-agent: PerplexityBot
Allow: /
# To block PerplexityBot:
User-agent: PerplexityBot
Disallow: /How to Test
Search for topics related to your content on Perplexity.ai. If your site appears in the cited sources, PerplexityBot has successfully crawled your content.
- Search for your brand or specific topics on Perplexity.ai to see if your content is being cited.
- Check server logs for the PerplexityBot user-agent to track crawl activity.
- Consider allowing PerplexityBot if referral traffic from AI search is valuable to you.
You can also use the MyOG OG Preview tool to check how your OG tags are configured before testing with PerplexityBot.
FAQ
What is PerplexityBot?
PerplexityBot is the web crawler for Perplexity AI, an AI-powered search engine. Unlike training-focused crawlers (GPTBot, ClaudeBot), PerplexityBot primarily fetches pages in real-time to answer user search queries, with citations linking back to the source pages.
Does Perplexity drive traffic to my site?
Yes. Unlike pure AI training crawlers, Perplexity cites sources with clickable links in its search results. If PerplexityBot crawls your content and it's used to answer a query, users can click through to your original page.
What is the difference between PerplexityBot and GPTBot?
GPTBot collects content for training GPT models. PerplexityBot crawls content to build Perplexity's search index and surface results with source citations. Perplexity also uses a separate Perplexity-User agent for live browsing during queries. PerplexityBot is more like a search engine crawler, while GPTBot is a training data collector.
Related Bots
Test Your OG Images
Check how your Open Graph images appear to bots and crawlers. Preview your link cards before sharing.
Already have an account?