Open Technical SEO Evidence Lab

A working evidence system, not a checklist score

Generic audit templates are useful for remembering categories, but they often collapse observation, ownership, repair, and verification into one checkbox. The Open Evidence Lab keeps those states separate. Each route register begins with the canonical task, a source-backed detection sequence, remediation steps, validation steps, false-positive checks, and a visible claim boundary.

Operating fields are intentionally blank. A downloadable register does not claim that a site was audited, that a defect exists, or that a repair passed. It gives an operator a reproducible place to record the owner, status, evidence URL, verification time, and notes without replacing unknown provider data with a favorable value.

The result is a bridge between an explanatory article and an accountable technical change. Readers can inspect the method in HTML, download a route-specific CSV, follow the primary sources, and retain the completed evidence beside their own deployment receipt.

Start with one of four operating lanes

Crawl and canonical work begins with URL ownership, discovery, response state, robots access, rendering, directives, and representative-page selection. AI crawler policy begins with the exact product token, the effective group and path rule, current vendor documentation, and request attribution. Answer readiness begins with visible claims, nearby sources, entity identity, extractable passages, and explicit uncertainty. JavaScript delivery begins by comparing raw HTML, rendered DOM, interactive state, and dependency failure.

Those lanes overlap, but they are not interchangeable. A robots rule cannot prove identity. Schema validity cannot prove selection. A rendered page cannot prove indexing. A Search Console sample cannot describe traffic that privacy thresholds withhold. Each curated route below owns one bounded decision and links back to its wider technical or AI-search collection.

• Crawl, indexation, canonicalization, and internal-link evidence.
• OpenAI crawler roles, robots policy, request attribution, and provider boundaries.
• Visible claims, source support, entity identity, and answer-ready passages.
• Raw-versus-rendered delivery, JavaScript failure states, and deployed validation.

Downloadable artifacts retain provenance and unknown states

The master index enumerates 54 route-specific registers across issue playbooks, structured-data references, audit checklists, crawler controls, and answer-readiness guides. Every row names its canonical HTML owner, source count, workflow size, review date, and claim boundary. The individual CSVs then expand that contract into detection, repair, verification, false-positive, and source rows.

The crawler fixture matrix and execution receipt form a second artifact chain. The matrix declares ten public inputs and expected RFC-bound policy results. The receipt names the evaluator, records actual results, and reports the comparison. Request identity stays unverified, provider application stays not tested or explicitly bounded, and search, training, citation, ranking, traffic, and conversion outcomes stay unknown.

What the robots-policy execution proves

The bounded evaluator parses user-agent groups, combines repeated exact groups, falls back to the wildcard group only when a specific group is absent, selects the most specific matching path, prefers Allow when equally specific rules conflict, and preserves the RFC 9309 distinction between a resolved rule and an unavailable 4xx robots resource.

That is a deterministic protocol result, not a universal crawler simulator. The fixtures deliberately include GPTBot, OAI-SearchBot, and ChatGPT-User so vendor identity does not erase product-role differences. OpenAI documents search discovery, training-crawl preference, and certain user-triggered requests as separate surfaces; the receipt preserves those boundaries beside the parser output.

Use the lab as an authority map, then return to the route owner

This page curates the highest-leverage operating routes; it does not duplicate their full guidance. Technical SEO and AI Search remain the canonical collection owners, while individual playbooks own exact detection, remediation, validation, and source language. Atlas remains the canonical project dossier for the larger observed-versus-derived evidence architecture.

A useful research path therefore moves from the lab to one task owner, through its related technical routes, and back to a dated artifact or production receipt. That structure compounds: new verified methods can join the lab without creating keyword variants, and every improvement strengthens a finite set of canonical pages instead of scattering authority across disconnected notes.

Open evidence artifacts

Download the inputs, operating registers, and execution receipts behind this collection. Unknown external outcomes remain unknown.

• Download the 54-route evidence-register index (CSV)

  Canonical route owners, source counts, workflow sizes, review dates, and claim boundaries.

• Download the OpenAI crawler robots fixture matrix (CSV)

  Ten public inputs for group matching, path precedence, crawler separation, and unavailable states.

• Download the executed robots-policy receipt (JSON)

  Actual-versus-expected results with identity, provider application, and downstream outcomes kept separate.

• Download the sanitized Atlas demonstration run (ZIP)

  A reviewed non-client bundle showing the broader Atlas observation-to-claim artifact contract.