Relevant visits
Help readers exploring your topic find the original page. Visits and conversions still need separate measurement.
Clear evidence. Practical next steps.
Is robots.txt blocking search? A practical example
Learn how to spot an accidental crawl restriction, preserve private paths and verify what search engines can actually access.
◎ For marketers⌘ For developers✧ For researchers
Audit in progress
Checking public access, discovery files and page content. This may take a moment.
Latest 100 reports · Stored in your account for one year.
Your focused check
The overall score below covers all eight audit checks, not just this tool.
Citable 2.0https://citable-2.onrender.comPAGE AUDIT · CHECKLIST POINTS
What we measured
Green: present · Amber: partial · Coral: improvement · Grey: unknown
Your next moves
Start with the largest measured gaps. Unknown checks need investigation, not assumed fixes.
DETAILED REPORT · ONE DOMAIN
Turn findings into a reviewable action plan.
Unlock the evidence behind all eight checks, training crawler permissions, priorities, and draft llms.txt and JSON-LD files. Review drafts before applying them.
One payment for this domain snapshot. No subscription. Saved to your account for one year from the audit. Download a copy to keep it longer.
Payments are not active.
CITABLE CERTIFICATE · SEPARATE PURCHASE
A verifiable record of a complete 100/100 page audit.
Includes the page address, scan and issue dates, checklist explanation, certificate number, and public verification link. Saved in your account and emailed as a link. Open the certificate to print or save a PDF using your browser.
Issued by Citable for this page and scan date. No external accreditation or guarantee of AI citations. Purchase within seven days of a complete audit. Refunded or disputed certificates show a revoked status.
Technical evidence
Observed page evidence is shown in its original language. Recommendations are in the separate action plan.
Training permissions (not scored)
Draft files
robots-review.txt is guidance, not a replacement robots.txt. Keep existing private-path rules. JSON-LD uses observed page facts and does not invent an Organization.
MY CERTIFICATES
Return to your certificates
Inspect the rule before changing the file
PRACTICAL EXAMPLE · CHECKED 12 OCTOBER 2026
Before buying an AI visibility service, check whether automated search services can read the page you want discovered. Start with the exact published URL, because rules can differ between paths and user agents.
A teaching example: blocking the entire site
User-agent: *
Disallow: /This illustrative group blocks crawling across the site for crawlers that respect it. It may be intentional on a staging site. It is not a finding about a customer website, and you should not replace a production file before reviewing the full rules and your access policy.
What we observed on our own website
On 12 October 2026, Citable's public robots.txt returned these rules:
User-agent: *
Allow: /
Disallow: /api/
Disallow: /certificates/
Sitemap: https://citableaudit.com/sitemap.xmlThe public pages are allowed while those two paths are disallowed. Private routes still require server-side authorization: robots.txt is not a security barrier. A different website may have more specific user-agent groups, so inspect the complete file rather than one line.
Turn the result into a safe next step
- Use the audit form to check a public page you own and read the focused access evidence.
- Check whether the restriction is intentional. Preserve staging and private content restrictions; decide search and training permissions separately.
- Review the page's status, indexing directives and Google Search Console URL Inspection before claiming it is indexed.
- If a real mistake is corrected, rescan the same URL and keep dated reports. Record the technical change separately from any later traffic change.
A blocked URL can sometimes appear in search without its content being crawled. An allowed rule is permission, not proof of a visit, a ranking or an AI citation. Network protection and timeouts may leave the audit unknown.
Open the focused robots.txt checker · Review the WordPress workflow · Tell us which recommendation was unclear
Read the official Google documentation ↗
This tool runs Citable's full eight-check audit and highlights the relevant check. No signup required during public beta.
FROM DISCOVERY TO TRUST
Why AI search visibility matters
When an AI answer cites your work, readers can follow the source to your website. Clear, accessible content makes your expertise easier to understand and verify.
Verifiable expertise
Give researchers clear facts, authorship and sources they can check. Publish evidence worth referencing.
A shared action plan
Connect marketing priorities with concrete technical tasks your developer can review.
Educational links, not integrations or endorsements. Citable checks your page; it does not query these services, measure actual citations or guarantee inclusion.
LET’S TALK
Contact us
Have a question, a confusing result or a suggestion? Tell us which page you checked and what you noticed.
Please do not send passwords, access keys or private customer information.
WHAT THE SCORE MEASURES
Eight checks, with their limits visible.
AI search tool permissions 30 · Initial HTML text 15 · Metadata 15 · JSON-LD 15 · Sitemap 10 · Heading outline 5 · Canonical/social tags 5 · Optional llms.txt 5.
Unavailable resources remain unknown. An incomplete audit displays a possible score range. Short text does not prove JavaScript dependence. robots.txt is a permission signal; this scanner does not impersonate provider crawlers.
Your page summary
Checklist points, check states, scan date and measurement limits.
Evidence and next steps
Review the observed evidence, choose your next action and download a dated report. All report details and downloads are free during the public beta.