I also made a code snippet for blocking all the AI bots (if you use The SEO Framework)

[ ‘by’ => ‘AddSearch’, ‘link’ => ‘https://www.addsearch.com’ ],
‘Agentic’ => [ ‘by’ => ‘Agentic’, ‘link’ => ‘https://agentic.ai’ ],
‘AgentQL’ => [ ‘by’ => ‘AgentQL’, ‘link’ => ‘https://agentql.com’ ],
‘Alexa’ => [ ‘by’ => ‘Amazon’, ‘link’ => ‘https://developer.amazon.com/amazonbot’ ],
‘Amazonbot’ => [ ‘by’ => ‘Amazon’, ‘link’ => ‘https://developer.amazon.com/amazonbot’ ],
‘Anthropic’ => [ ‘by’ => ‘Anthropic’, ‘link’ => ‘https://support.anthropic.com/en/articles/8896518’ ],
‘anthropic-ai’ => [ ‘by’ => ‘Anthropic’, ‘link’ => ‘https://support.anthropic.com/en/articles/8896518’ ],
‘Anyword’ => [ ‘by’ => ‘Anyword’, ‘link’ => ‘https://www.anyword.com’ ],
‘Applebot’ => [ ‘by’ => ‘Apple’, ‘link’ => ‘https://support.apple.com/en-us/119829’ ],
‘Applebot-Extended’ => [ ‘by’ => ‘Apple’, ‘link’ => ‘https://support.apple.com/en-us/119829’ ],
‘AnyPicker’ => [ ‘by’ => ‘AnyPicker’, ‘link’ => ‘https://www.anypicker.com’ ],
‘Articoolo’ => [ ‘by’ => ‘Articoolo’, ‘link’ => ‘https://articoolo.com’ ],
‘Botsonic’ => [ ‘by’ => ‘Botsonic’, ‘link’ => ‘https://www.botsonic.com’ ],
‘Bytespider’ => [ ‘by’ => ‘ByteDance’, ‘link’ => ‘https://www.bytedance.com’ ],
‘CCBot’ => [ ‘by’ => ‘Common Crawl’, ‘link’ => ‘https://commoncrawl.org/ccbot’ ],
‘Chatsonic’ => [ ‘by’ => ‘Chatsonic’, ‘link’ => ‘https://www.chatsonic.com’ ],
‘Claude’ => [ ‘by’ => ‘Anthropic’, ‘link’ => ‘https://support.anthropic.com/en/articles/8896518’ ],
‘ClaudeBot’ => [ ‘by’ => ‘Anthropic’, ‘link’ => ‘https://support.anthropic.com/en/articles/8896518’ ],
‘Cohere’ => [ ‘by’ => ‘Cohere’, ‘link’ => ‘https://cohere.ai’ ],
‘cohere-ai’ => [ ‘by’ => ‘Cohere’, ‘link’ => ‘https://cohere.ai’ ],
‘ContentAtScale’ => [ ‘by’ => ‘ContentAtScale’, ‘link’ => ‘https://www.contentatscale.ai’ ],
‘Copyscape’ => [ ‘by’ => ‘Copyscape’, ‘link’ => ‘https://www.copyscape.com’ ],
‘CrewAI’ => [ ‘by’ => ‘CrewAI’, ‘link’ => ‘https://www.crewai.com’ ],
‘Crawl4AI’ => [ ‘by’ => ‘Crawl4AI’, ‘link’ => ‘https://crawl4ai.com’ ],
‘DALL-E’ => [ ‘by’ => ‘OpenAI’, ‘link’ => ‘https://openai.com/dall-e’ ],
‘DeepAI’ => [ ‘by’ => ‘DeepAI’, ‘link’ => ‘https://deepai.org’ ],
‘DeepL’ => [ ‘by’ => ‘DeepL’, ‘link’ => ‘https://www.deepl.com’ ],
‘DeepMind’ => [ ‘by’ => ‘Google’, ‘link’ => ‘https://deepmind.com’ ],
‘DeepSeek’ => [ ‘by’ => ‘DeepSeek’, ‘link’ => ‘https://www.deepseek.com’ ],
‘Diffbot’ => [ ‘by’ => ‘Diffbot’, ‘link’ => ‘https://www.diffbot.com’ ],
‘FacebookBot’ => [ ‘by’ => ‘Meta’, ‘link’ => ‘https://developers.facebook.com/docs/sharing/bot’ ],
‘FacebookExternalHit’ => [ ‘by’ => ‘Meta’, ‘link’ => ‘https://developers.facebook.com/docs/sharing/bot’ ],
‘Firecrawl’ => [ ‘by’ => ‘Firecrawl’, ‘link’ => ‘https://firecrawl.dev’ ],
‘Genspark’ => [ ‘by’ => ‘Genspark’, ‘link’ => ‘https://www.genspark.ai’ ],
‘Gemini’ => [ ‘by’ => ‘Google’, ‘link’ => ‘https://gemini.google.com’ ],
‘GigaChat’ => [ ‘by’ => ‘GigaChat’, ‘link’ => ‘https://gigachat.ai’ ],
‘Google-Extended’ => [ ‘by’ => ‘Google’, ‘link’ => ‘https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers’ ],
‘Google-CloudVertexBot’ => [ ‘by’ => ‘Google’, ‘link’ => ‘https://cloud.google.com/vertex-ai’ ],
‘GPTBot’ => [ ‘by’ => ‘OpenAI’, ‘link’ => ‘https://platform.openai.com/docs/bots’ ],
‘Grok’ => [ ‘by’ => ‘xAI’, ‘link’ => ‘https://grok.x.com’ ],
‘Grammarly’ => [ ‘by’ => ‘Grammarly’, ‘link’ => ‘https://www.grammarly.com’ ],
‘Jasper’ => [ ‘by’ => ‘Jasper’, ‘link’ => ‘https://www.jasper.ai’ ],
‘Jinja’ => [ ‘by’ => ‘Jinja’, ‘link’ => ‘https://jinja.ai’ ],
‘Kaggle’ => [ ‘by’ => ‘Kaggle’, ‘link’ => ‘https://www.kaggle.com’ ],
‘Kimi’ => [ ‘by’ => ‘Kimi’, ‘link’ => ‘https://kimi.moonshot.cn’ ],
‘LangChain’ => [ ‘by’ => ‘LangChain’, ‘link’ => ‘https://www.langchain.com’ ],
‘Lightpanda’ => [ ‘by’ => ‘Lightpanda’, ‘link’ => ‘https://www.lightpanda.io’ ],
‘LLaMA’ => [ ‘by’ => ‘Meta’, ‘link’ => ‘https://www.llama.com’ ],
‘LLM’ => [ ‘by’ => ‘LLM’, ‘link’ => ‘https://llm.ai’ ],
‘magpie-crawler’ => [ ‘by’ => ‘Magpie’, ‘link’ => ‘https://magpie.com’ ],
‘Maroofy’ => [ ‘by’ => ‘Maroofy’, ‘link’ => ‘https://maroofy.com’ ],
‘Meta-External’ => [ ‘by’ => ‘Meta’, ‘link’ => ‘https://developers.facebook.com/docs/sharing/webmasters/web-crawlers’ ],
‘Meta-ExternalAgent’ => [ ‘by’ => ‘Meta’, ‘link’ => ‘https://developers.facebook.com/docs/sharing/webmasters/web-crawlers’ ],
‘Meta-Webindexer’ => [ ‘by’ => ‘Meta’, ‘link’ => ‘https://developers.facebook.com/docs/sharing/webmasters/web-crawlers’ ],
‘MetaAI’ => [ ‘by’ => ‘Meta’, ‘link’ => ‘https://www.meta.ai’ ],
‘Midjourney’ => [ ‘by’ => ‘Midjourney’, ‘link’ => ‘https://www.midjourney.com’ ],
‘Mistral’ => [ ‘by’ => ‘Mistral AI’, ‘link’ => ‘https://mistral.ai’ ],
‘Mixtral’ => [ ‘by’ => ‘Mistral AI’, ‘link’ => ‘https://mistral.ai’ ],
‘Monica’ => [ ‘by’ => ‘Monica’, ‘link’ => ‘https://www.monica.im’ ],
‘Midjourney’ => [ ‘by’ => ‘Midjourney’, ‘link’ => ‘https://www.midjourney.com’ ],
‘Mistral’ => [ ‘by’ => ‘Mistral’, ‘link’ => ‘https://mistral.ai’ ],
‘OAI-SearchBot’ => [ ‘by’ => ‘OpenAI’, ‘link’ => ‘https://platform.openai.com/docs/bots’ ],
‘OpenAI’ => [ ‘by’ => ‘OpenAI’, ‘link’ => ‘https://openai.com’ ],
‘OpenRouter’ => [ ‘by’ => ‘OpenRouter’, ‘link’ => ‘https://openrouter.ai’ ],
‘Perplexity’ => [ ‘by’ => ‘Perplexity’, ‘link’ => ‘https://perplexity.ai’ ],
‘PerplexityBot’ => [ ‘by’ => ‘Perplexity’, ‘link’ => ‘https://perplexity.ai’ ],
‘Perplexity-User’ => [ ‘by’ => ‘Perplexity’, ‘link’ => ‘https://perplexity.ai’ ],
‘Phind’ => [ ‘by’ => ‘Phind’, ‘link’ => ‘https://www.phind.com’ ],
‘Puppeteer’ => [ ‘by’ => ‘Puppeteer’, ‘link’ => ‘https://pptr.dev’ ],
‘Qwen’ => [ ‘by’ => ‘Alibaba’, ‘link’ => ‘https://www.aliyun.com’ ],
‘QuillBot’ => [ ‘by’ => ‘QuillBot’, ‘link’ => ‘https://quillbot.com’ ],
‘Rytr’ => [ ‘by’ => ‘Rytr’, ‘link’ => ‘https://rytr.me’ ],
‘SemrushBot’ => [ ‘by’ => ‘SEMrush’, ‘link’ => ‘https://www.semrush.com/bot’ ],
‘SemrushBot-BA’ => [ ‘by’ => ‘SEMrush’, ‘link’ => ‘https://www.semrush.com/bot’ ],
‘SiteAuditBot’ => [ ‘by’ => ‘SEMrush’, ‘link’ => ‘https://www.semrush.com/bot’ ],
‘Serper’ => [ ‘by’ => ‘Serper’, ‘link’ => ‘https://serper.dev’ ],
‘Sora’ => [ ‘by’ => ‘OpenAI’, ‘link’ => ‘https://openai.com/sora’ ],
‘StableDiffusionBot’ => [ ‘by’ => ‘Stability AI’, ‘link’ => ‘https://stability.ai’ ],
‘Sudowrite’ => [ ‘by’ => ‘Sudowrite’, ‘link’ => ‘https://www.sudowrite.com’ ],
‘Surfer’ => [ ‘by’ => ‘Surfer’, ‘link’ => ‘https://surferseo.com’ ],
‘TextCortex’ => [ ‘by’ => ‘TextCortex’, ‘link’ => ‘https://textcortex.com’ ],
‘Together’ => [ ‘by’ => ‘Together AI’, ‘link’ => ‘https://www.together.ai’ ],
‘TurnitinBot’ => [ ‘by’ => ‘Turnitin’, ‘link’ => ‘https://www.turnitin.com’ ],
‘Writesonic’ => [ ‘by’ => ‘Writesonic’, ‘link’ => ‘https://writesonic.com’ ],
‘Writescope’ => [ ‘by’ => ‘Writescope’, ‘link’ => ‘https://www.writescope.ai’ ],
‘Wordtune’ => [ ‘by’ => ‘Wordtune’, ‘link’ => ‘https://www.wordtune.com’ ],
‘xAI’ => [ ‘by’ => ‘xAI’, ‘link’ => ‘https://x.ai’ ],
‘YandexAdditional’ => [ ‘by’ => ‘Yandex’, ‘link’ => ‘https://yandex.com’ ],
‘Zhipu’ => [ ‘by’ => ‘Zhipu’, ‘link’ => ‘https://www.zhipuai.com’ ],
];

// Merge with existing agents
$agents = array_merge( $agents, $ai_bots );
break;

case ‘seo’:
// Add SEO bots to the blocklist if needed
// allowing them to crawl your site again, while still blocking others.
unset( $agents[‘AhrefsBot’], $agents[‘AhrefsSiteAudit’] );

break;
}
return $agents;
}, 10, 2 );
}, 1 );

Also just to clarify — nothing I mention in this post will block you from appearing in AI Overviews. The only way to do that is to block Google-Bot entirely (at least until Google offers us more granular control mechanisms). We’re only talking about other AI Search bots like Perplexity, or the training bots like Google-Extended. But that doesn’t affect Search, AI Overviews, or AI Mode.

Hi Paul!

It works if enough publishers do it that Google feels a need to negotiate. We know they are doing private deals already. Just last week it came out via testimony in the ad tech antitrust case that Google is paying an AI licensing deal to WikiHow. They have told the UK government they are only willing to pay for content where they cannot access data.

If you give away your content for free why would Google or any other AI company pay you?

Personally, I don’t feel that trading 1% traffic now for all my bargaining chips is a good deal. AI content licensing deals are likely to be the primary monetization method of the future web, and the best deals are going to go to the publishers with the most leverage. Again, all just my opinion.

Thank you for reading, thinking about this, and commenting!

Have I got this right or am I missing something?