Search optimisation
robots.txt reachable
robots.txt returns 200 and is parseable
What we check
robots.txt returns 200 and is parseable
Why it matters
Crawlers request robots.txt first; an error there can slow crawling.
How to fix it
Serve a valid robots.txt at the domain root.
How it affects your score
This check sits in Search optimisation, which is worth 25% of the overall score. A critical result costs the full weight of the check; a warning costs half. If we can't measure it on your page, we leave it out of the score entirely.