Part VIII. Applications and Strategy · Chapter 71
Application Evidence Levels
A simulator result and a paying customer are both "evidence," and confusing them is how money gets lost. This chapter gives you a ladder of evidence levels and the habit of asking, for every claim, which rung it actually stands on.
In this chapter 9 sections
Classify the strongest reproducible link in the claim chain—from theorem through controlled hardware comparison to operational outcome—and let the weakest required link cap the decision; press releases and customer names do not raise the level.
Evidence levels belong to individual claims. The sentence 'the algorithm exists' can be strongly supported while 'the workflow saves money' remains unsupported on the same page. Assigning one maturity label to a company or application launders evidence upward from the strongest claim to the weakest.
Evidence attaches to a claim, not a company
Seven rungs are useful when their requirements are explicit: mathematical result; executable reference; controlled simulator comparison; controlled hardware comparison; representative-scale reproduction; buyer pilot with a measured workflow outcome; repeated operational outcome. Each rung inherits every relevant lower-level requirement and adds a new one.
- Mechanism and executable artifact.
- Controlled task result against a named comparator.
- Independent reproduction on a representative workload.
- Operational outcome retained by the workflow owner.
Seven rungs from theorem to operating outcome
Dated ledger for three apparently similar claims
FIELDS = ("artifact", "comparator", "reproduction", "representative", "outcome")
LABELS = ("mathematical", "artifact", "controlled", "reproduced", "representative", "operational")
def classify(record, maximum_age_days):
errors = []
for name in ("scenario", "claim", "evidence_origin"):
if type(record.get(name)) is not str or not record.get(name):
errors.append(name)
if type(record.get("source_ids")) is not list or not record.get("source_ids"):
errors.append("source_ids")
age = record.get("age", {})
if type(age.get("value")) is not int or age.get("unit") != "days":
errors.append("age_schema")
for name in FIELDS:
if type(record.get(name)) is not bool:
errors.append(name)
if errors:
return {"decision": "invalid", "errors": sorted(set(errors)), "ceiling": None}
level = 0
for name in FIELDS:
if not record[name]:
break
level += 1
if record["evidence_origin"] == "marketing" or age["value"] > maximum_age_days:
level = min(level, 1)
return {"decision": "classified", "errors": [], "ceiling": LABELS[level], "level": level}
common = {"scenario": "three-claim calibration packet", "source_ids": ["acm-artifact-review"],
"age": {"value": 30, "unit": "days"}, "evidence_origin": "primary record"}
hardware = {**common, "claim": "circuit improves metric", "artifact": True, "comparator": True,
"reproduction": True, "representative": False, "outcome": False}
pilot = {**common, "claim": "pilot creates value", "artifact": True, "comparator": True,
"reproduction": False, "representative": True, "outcome": False}
hardware_result = classify(hardware, 90)
pilot_result = classify(pilot, 90)
bad = {key: value for key, value in hardware.items() if key != "comparator"}
bad_result = classify(bad, 90)
promoted_result = classify({**hardware, "representative": True}, 90)
assert hardware_result["ceiling"] == "reproduced" and pilot_result["ceiling"] == "controlled"
assert bad_result["decision"] == "invalid" and "comparator" in bad_result["errors"]
assert promoted_result["ceiling"] == "representative" and promoted_result["level"] > hardware_result["level"]
print(f"PASS: 71 evidence classifier hardware={hardware_result['ceiling']} pilot={pilot_result['ceiling']} invalid={bad_result['errors']} promoted={promoted_result['ceiling']}")Three claims share the word demonstrated. A theorem with a proof receives the mathematical rung. A small hardware run with code, calibration, and comparator receives controlled hardware evidence. A named customer pilot without an outcome metric remains a reported pilot, not an operational outcome. The classifier makes a missing comparator or reproduction field lower the ceiling automatically.
Artifact contract. A dated claim-by-claim ledger and executable monotone classifier. Missing reproducibility, comparator, representative scale, or operating metric can only lower a level; marketing evidence cannot raise a technical claim.
| Claim | Strongest record | Missing promotion field | Ceiling |
|---|---|---|---|
| Method exists | proof and original paper | executable reference | mathematical |
| Circuit improves metric | controlled hardware comparison | representative reproduction | controlled hardware |
| Pilot creates value | customer report | measured counterfactual outcome | reported pilot |
Exact validation command: python3 tools/validate_briefs.py --briefs data/editorial_briefs_64_87.json --from 64 --through 87 --check-rewritten-sources --execute-artifacts
Reproduction and comparator quality can lower a rung
Use the level to choose the next evidence purchase. A mathematical result needs an executable instance; a controlled result needs representative scale; a pilot needs a predeclared buyer metric and counterfactual. Promotion occurs when the new record exists, not when a press release changes the verb.
Calibrate a mixed evidence packet
Prompt. Assign evidence levels to a theorem, a small hardware benchmark, and a reported customer pilot.
Deliverable. Three claim ledgers with primary sources, comparator and scale fields, reproducibility status, level, and permitted decision.
Pass condition. Each level is capped by its weakest required field, the pilot is not called operational without an outcome metric, and promotion criteria name an observable event.
Model answer: three claims, three different ceilings
Format. Three-level adjudication separating algorithm existence, controlled experiment, and buyer outcome.
The reference adjudication assigns three different ceilings and lists the field needed for each promotion. The hardware claim cannot rise above controlled comparison without representative scale and independent reproduction. The pilot cannot rise above buyer-pilot evidence without an operating outcome and counterfactual. The conclusion preserves strong algorithmic evidence without lending it to commercial claims.
Verification. The classifier returns the documented ceilings and refuses a higher level when any required field is false.
Companion work
Artifacts for this chapter
These entries resolve to checked-in local source. Commands are reproduced exactly from the chapter manifest, and source-embedded fixtures are exported as direct downloads.
evidence dossier
Application evidence-level classifier
Reproduce or test
python3 tools/validate_briefs.py --briefs data/editorial_briefs_64_87.json --from 64 --through 87 --check-rewritten-sources --execute-artifacts
Provenance
Sources and review
- U.S. Government Accountability Office. Quantum Computing and Communications: Status and Prospects. GAO. 2021government technology assessment
- National Academies of Sciences, Engineering, and Medicine. Quantum Computing: Progress and Prospects. National Academies Press. 2019consensus study report
- Timothy Proctor et al.. Benchmarking quantum computers. Nature Reviews Physics. 2025peer-reviewed perspective
- John Preskill. Quantum Computing in the NISQ era and beyond. Quantum. 2018peer-reviewed perspective
- Association for Computing Machinery. Artifact Review and Badging. ACM Publications. 2026official reproducibility policy
- Easwar Magesan, J. M. Gambetta, and Joseph Emerson. Scalable and robust randomized benchmarking of quantum processes. Physical Review Letters. 2011primary paper
- Edward Farhi, Jeffrey Goldstone, and Sam Gutmann. A Quantum Approximate Optimization Algorithm. arXiv. 2014primary preprint
- Alberto Peruzzo et al.. A variational eigenvalue solver on a photonic quantum processor. Nature Communications. 2014primary paper
The load-bearing claims in the chapter are mapped inline to this registered source set. A citation supports only the bounded claim beside it.