Reproducible Quantum Labs
A lab nobody else can rerun is a diary entry. This chapter turns private quantum experiments into public artifacts: exact commands, seeded sampling, expected outputs, stated failure modes, and the limitations that keep a toy simulator from impersonating hardware.
Reproducibility is the cheapest credibility a quantum project can buy — and the most commonly skipped, because it makes small results look small.
By the end of this chapter you will be able to package any experiment in this book so that a stranger can predict the outcome, run it, compare the output, and say exactly what the result does not show.
Core concepts: benchmarking and reproducibility, assessment evidence, public explanation.
What makes a lab reproducible
A quantum lab is publishable only when another learner can rerun it, predict what should happen, and state what limitations remain. That requires both deterministic checks — exact simulator probabilities — and stochastic interpretation: what finite shots are allowed to look like before anyone declares a bug.
The package has ten parts:
- objective and prerequisites
- files and setup steps
- the exact run command
- expected exact output
- expected sampled behavior
- failure modes
- a mastery check
- the source boundary: what this result is and is not evidence for
This is how private study becomes a learning object other people can build on.
Deterministic checks, stochastic honesty
Sampled output obeys : with shots, proportions wobble around the true probability by roughly that much. A reproducible lab says so up front, fixes a seed, and tells the reader how much variation is ordinary.
The teaching standard has four parts: . A lab that only runs code is incomplete. A lab that only explains concepts is incomplete. The learner needs action, evidence, transfer, and limits.
Worked example: from lab note to lesson
A private note says: "I ran the Bell simulator and got counts near 50/50." As a public lab, the same work reads:
- Objective: prepare and sample a Bell state.
- Run command: the exact command from the workspace root.
- Expected exact probabilities: 00 and 11 only, each 1/2.
- Expected sampled behavior: finite counts vary around 50/50; the seed controls reproducibility.
- Check: compare exact probabilities before sampling.
- Limitation: statevector simulation exposes amplitudes that hardware never will.
- Transfer: change the seed or the measurement basis and explain what happens.
The note recorded an event. The lab teaches a skill.
Where labs fail
The first trap is works-on-my-machine: missing files, unstated versions, commands that assume the author's directory layout. The second is hiding uncertainty because clean output feels better — quantum output is sampled, and a lesson that conceals the wobble teaches the reader to misread real data.
The third trap is the missing limitation. Without one, learners overgeneralize a toy simulator into hardware evidence, and the lab has taught a falsehood by omission.
The engineering view
A reproducible lab is a small software package: tests, docs, expected outputs. The same metadata that lets a learner rerun it later powers route pages, progress tracking, quizzes, and source-refresh warnings on a site like steven-geller.com/playground/quantum — an explanation pane, a code pane, the run command, an expected-output panel, a histogram or table, a mastery gate, a decision implication. A lab without structured metadata may read well, but it will resist becoming an interactive lesson.
Before promoting a lab, run a peer-review pass. Can a reader predict the result before running it? Rerun the command? Compare output to expectation? Name the failure modes? State one decision implication? Any "no" means the lab is not ready for public placement.
What this buys you in diligence
A team that cannot package an experiment with rerun instructions, expected output, and failure modes is not ready for outside scrutiny. A team that can has given you evidence even when the result is modest — the artifact itself demonstrates engineering discipline, which is often the real diligence question.
Exercise
SDK reproducibility check. Choose one existing lab and write the exact command a learner should run from the workspace root.
- Record: objective, files, Python version, package assumptions, command, expected output, shot count, seed, failure modes, and limitation.
- Check: wherever sampling appears, use the shot standard error to explain why the observed counts are ordinary.
- Submit: the lab record plus a readiness verdict — private study, team review, partner diligence, or public platform — with one sentence defending it.
- Repair: if the record lacks rerun instructions, expected output, or a limitation, rework it after Chapter 41 (Noise Models and Noisy Simulation), which covers simulator-output reproducibility.
Check your understanding
Without notes: write a lab record with objective, code, expected output, uncertainty, failure modes, and rerun instructions.
Oral defense: explain why two runs of the same circuit with different seeds can disagree — and why that is not automatically a bug.
If you get stuck
If you cannot reproduce exact simulator output, or cannot tell sampled counts apart from the true state, work through Chapter 41 (Noise Models and Noisy Simulation) and Chapter 42 (Benchmarking Quantum Programs) before publishing anything.