Standard · Discover
Content Signals
Three yes or no lines in robots.txt: search, ai-input, ai-train.
What it is
Content Signals adds a line to robots.txt that states how your content may be used: for search (search), as input to an answer an agent gives (ai-input) and to train models (ai-train). Each is yes or no.
An analogy
At a glance
- Owner
- Cloudflare (released CC0)
- Maturity
- Emerging Published September 2025. On by default in Cloudflare’s managed robots.txt, which gives it wide reach quickly.
- Where it lives
- contentsignals.org
- Framework domain
- Discover: Be found
- What it serves
- State how your content may be used
- Allow answers while declining training
Who should care
The roles this standard touches most. See all the roles on the personas page.
- Marketing and brand. Own the story agents tell: accurate facts, positioning, reputation and presence where agents look.
- Legal, risk and security. Set the policies: which agents may do what, identity, data, disclosure and liability.
- Engineering and product. Build the access, data, endpoints and actions that agents use, on open standards.
How to adopt it
- 1
Decide your position on search, ai-input and ai-train with legal and marketing.
- 2
Add the signal line to robots.txt.
- 3
Test that the file still parses and that the lines say what you meant.
The Institute’s view
It is cheap and clear. It is a statement of preference, so it does not stop a crawler that ignores it. It may be superseded by the IETF work on the same problem, so keep your policy decision separate from the syntax.
Standards change fast. Last reviewed November 2026.