Simulation-Based Calibration Checking
A guide to checking Bayesian inference algorithms by simulating parameters and datasets from a model and examining posterior ranks or coverage.
External resource. No execution or independent verification is claimed here.
Can a deliberately biased posterior sampler be distinguished from an exact conjugate sampler by a prespecified rank-uniformity diagnostic?
The evaluator owns the prior-predictive generator, exact posterior reference, rank calculation and test threshold; a passing finite calibration test is not a proof of universal correctness.
What you could produce
- A versioned minimal protocol, evaluator-owned test cases, and a comparison report with numerical/statistical uncertainty and failures.
Before you use it
- A separately qualified Stan runtime and compatible compiled toolchain; no Stan environment is available in the current pilot
Limits to keep in view
- No upstream example, source package, build hook or submitted code was executed.
- The current qualified pilot accepts only self-contained Python 3.13 with a 90-second deadline. This reference's package/runtime is not qualified for that path.
- The proposed protocol requires bounded resource estimates and an independently controlled evaluator before any scientific execution claim.
Source and permission context
Stan User's Guide / Stan Development Team. Simulation-Based Calibration Checking. https://raw.githubusercontent.com/stan-dev/docs/1bc688a6f9a6c6931b7527c333345d2dcf6ee0bb/src/stan-users-guide/simulation-based-calibration.qmd; observed version 1bc688a6f9a6c6931b7527c333345d2dcf6ee0bb.
Catalog listing reviewed. This review covers the description and source links displayed here.
Pinned Stan commit reviewed: CC-BY-ND-4.0 text/images and BSD-3-Clause code.
Reviewed 2026-09-14. Copying or adapting source files remains subject to their own terms.
documentation · CC-BY-ND-4.0
The LICENSE file in the exact fetched documentation commit assigns CC BY-ND 4.0 to text and images, separately from its BSD-licensed code. No text adaptation or redistribution permission beyond those terms is inferred.
Inspect the license evidence ↗code · BSD-3-Clause
The same pinned LICENSE file explicitly assigns BSD 3-clause terms to code. This separate code scope does not remove the text/images' no-derivatives condition.
Inspect the license evidence ↗Before copying source material
- Third-party figures, linked papers, datasets and dependency licenses have not been assessed; no external content is copied into this catalog.
- CC BY-ND has a no-derivatives condition for shared adapted text/images; references and original descriptions do not license redistribution of adapted documentation.