# A Reproducible Public-Data Model Is More Than a Notebook That Runs

A notebook can execute twice and still produce an indefensible chart. Reproducibility has two layers: byte reproducibility and interpretive reproducibility.

Byte reproducibility asks whether the same frozen inputs and configuration generate the same tables and figures. Interpretive reproducibility asks whether a new researcher can tell why those inputs, rows, release states, transformations, and exceptions were chosen. Public-data work needs both.

## Freeze the native evidence

The F01 build starts with six files: two Census CSVs, a BEA workbook, an EIA bulk archive, a source-located BLS excerpt, and a controlled missingness fixture. Each has an expected SHA-256 hash. The analysis refuses to run if any observed hash differs.

That turns “download the latest file” into an explicit update operation rather than a silent mutation. A new vintage can be added, but it becomes a new input and a new observation. It does not overwrite the file that supported an earlier claim.

The raw manifest also preserves URLs, byte sizes, and retrieval evidence. Large narrative pages are not copied; the source ledger retains first-party links and claim roles.

## Give every observation a versioned identity

The canonical table contains 710 observations from four publisher bundles. Every row stores raw and numeric values, source URL and hash, source-row locator, geography and period bounds, frequency, unit, seasonal adjustment, price basis, reference year, vintage ID, release date, retrieval time, and status.

An observation ID is a stable SHA-256 prefix over the full source-specific grain. A Census 2024 estimate from Vintage 2024 and the same reference date from Vintage 2025 are two observations. Their matched pair has its own ID. Derived scenario rows have model IDs and parent edges.

This is intentionally more verbose than a spreadsheet keyed by state and year. A state-year key alone would make one vintage overwrite another—the exact evidence needed to measure revision risk.

## Treat transformations as code and data

Every derived operation in F01 has a transform ID, formula, input/output units, version, and guardrail. Revision to a published growth rate is a percentage-point difference. A level-relative revision is a percent with an undefined result when the earlier level is zero. Rankings use a frozen frame and stable tie-break. Scenario compounding requires a named starting vintage and assumption ID.

Lineage is stored as 742 child-parent edges. That lets a reviewer traverse from a charted conclusion to a revision pair and then to both raw observations. It also exposes where a claim depends on an analyst-created assumption rather than a publisher value.

## Fail on discrepancies; do not repair them invisibly

Three source behaviors show why deterministic code needs semantic receipts.

First, the Census Vintage 2024 file has 75 columns and Vintage 2025 has 97. The pipeline selects named population fields and asserts geography identity. It never trusts column position.

Second, three BLS wage rows differ by one million dollars between the subtraction of separately rounded displayed levels and the separately displayed delta. The pipeline retains both and uses a documented ±$1 million tolerance only for that measure. A universal fuzzy-equality setting would be unsafe.

Third, all three selected EIA records declare `start=2024` but contain data keys only for 2025–2050. A naive period generator would create an absent year. F01 validates actual keys, records the metadata mismatch, and leaves 2024 out.

These decisions cannot be recovered from an output hash alone. They belong in data-quality issues, method notes, and the worklog.

## Make charts deterministic outputs

The six F01 figures are standalone SVGs generated by a standard-library Python pipeline. Each figure has a preregistered chart contract, fixed dimensions, an accessible title and description, a backing processed table, a source/method note, and a SHA-256 entry in the figure manifest.

The build does not fetch data from the network. `reproduce.py` extracts the frozen BEA workbook through the audited bundled spreadsheet runtime, then executes the hash-verifying analysis. `determinism_check.py` performs the full build twice and compares every processed CSV/JSON and SVG. `verify.py` checks schema, IDs, counts, lineage, source/claim ledgers, figures, hypotheses, and article packets.

## A reproducibility receipt should answer five questions

1. **What exact bytes entered the model?** Source URL, retrieval time, byte size, and hash.
2. **What does each row mean?** Concept signature, grain, units, adjustments, and release state.
3. **How was each output derived?** Versioned formula, parent IDs, assumptions, and missingness treatment.
4. **What failed or required judgment?** Access limits, schema drift, rounding, missing metadata, and downgrade decisions.
5. **Can a stranger rerun and challenge it?** One command, validation checks, deterministic manifests, counterevidence, and a continuation report.

That is the difference between a model that merely reruns and an evidence product that survives handoff.

## Evidence

- [Raw input manifest](../../ANALYSIS/processed/RAW_MANIFEST.csv)
- [Canonical observation panel](../../ANALYSIS/processed/canonical_observations.csv)
- [Lineage edges](../../ANALYSIS/processed/lineage_edges.csv)
- [Transform registry](../../ANALYSIS/processed/transformation_registry.csv)
- [Quality issues](../../ANALYSIS/processed/data_quality_issues.csv)
- [Reproduction entry point](../../ANALYSIS/reproduce.py)
- [Determinism checker](../../ANALYSIS/determinism_check.py)
- [Acceptance validator](../../ANALYSIS/verify.py)