QPUBench

Quantum benchmarking · one record format

Benchmark any quantum computer, compare the results years later.

QPUBench runs quantum benchmarks and records every result in one common format, so runs from different SDKs, machines and even different quantum computing paradigms can be compared side by side. You describe what to run as plain typed data, execute it through a pluggable backend adapter, and every run is stored as the same self-describing BenchmarkRecord — readable without any quantum SDK installed.

Read the guide Install Supported packages

  • Python ≥ 3.11
  • LGPL-3.0-or-later
  • Schema v6.1.0
  • pydantic is the only required dependency
bell.py
from qpubench import BenchmarkRunner, CircuitSpec

circuit = CircuitSpec(
    num_qubits=2,
    serialized=bell_qasm,
    observables=[zz],
)

runner = BenchmarkRunner(store="results/bell.ndjson")
runner.register(AerAdapter(), name="aer")
record = runner.run(circuit, "aer", shots=4096)

ev = record.result.expectation_values[0]
# <ZZ> = 1.0000 ± 0.0000

Swap "aer" for "ibm", "iqm", "braket" or a stub — the rest of the script, and the record it produces, do not change.

Three layers, and that is the whole framework

1

Schemas — what you benchmark

CircuitSpec, BackendSpec, ExecutionOptions and QuantumResult are plain Pydantic v2 models. They serialize to JSON, validate on construction, and never import a quantum SDK.

Schema reference →
2

Adapters — how it runs

Any object with spec, validate() and run() is a backend. Libraries that build their own circuits implement the sibling AlgorithmAdapter protocol instead. No base class to inherit.

Backends & adapters →
3

Stores — where results go

NDJSONStore (append-only, zero-dependency), ParquetStore (columnar, for pandas) and S3Store (one object per record, safe for distributed sweeps) share one interface.

Persistence →

Supported packages, backends and integrations

Every project below is reachable from QPUBench — either as a runnable adapter that executes circuits, or as a typed schema bridge that captures its run configuration and results in the shared record format. Schema bridges never import the external library, so installing QPUBench pulls in none of them. Follow a tile for the QPUBench documentation, or “upstream” for the project itself.

Execution backends

Simulators and QPUs with a runnable adapter in qpubench.backends. Register one with a BenchmarkRunner and every run lands in the same record format.

Algorithms & quantum chemistry

Packages whose run configurations, wavefunctions and results are mirrored as typed schemas, so their output is directly comparable with everything else in the store.

Modalities & hardware vendors

Beyond gate-based superconducting qubits. ComputingModel and QubitModality are independent axes on every record, so these runs share one store with everything above.

Mitigation, compilation, decomposition & orchestration

The layers around execution: making a noisy run usable, making a large circuit fit, and running the whole campaign as scheduled compute.

Marks shown here are generated placeholder wordmarks, not vendor artwork — see docs/site/README for how to drop in a real logo.

What QPUBench does and does not do

Yes

  • Absolute performance of an algorithm. Expectation values with error bars, energy error against a stored classical reference, timings and QPU-cost estimates.
  • Comparing implementations of the same algorithm. One ADAPT-VQE configuration runs unchanged against three different engines.
  • Comparing different algorithms. Records carry a package-agnostic algorithm family, so different algorithms on the same problem stay comparable.
  • Comparing hardware. Register several backends and sweep the same circuits across all of them.
  • Comparing modalities. The record format spans gate-based, MBQC, boson sampling and neutral-atom analog runs.

No

  • Modelling noise. Backends bring their own noise models; QPUBench records what ran, it does not define noise.
  • Measuring noise. It can store device-characterization results from external tools, but performs no characterization itself.
  • Benchmarking classical algorithms as first-class runs. Classical references (FCI, exact diagonalization) are computed and stored for comparison only.
  • Shipping a fixed benchmark suite. This is a framework for running your own campaigns, not a leaderboard.

A note on “chemical accuracy”. QPUBench reports an energy error — the gap between a run and a classically computed reference — and flags whether it is below 1.6 mHartree. That is a numerical-convergence check against a computed value, not agreement with an experimentally measured quantity. Treat the stored reference as a computed baseline, not as ground truth.

Documentation

Start here

Reference

Integrations

Development

Plugging in your own backend

An adapter is a plain class. Two rules keep the ecosystem healthy: SDK imports live inside methods, so importing your adapter never requires the SDK; and recoverable failures return status=FAILED rather than raising, so one bad point does not kill a sweep. 

class MySimulatorAdapter:
    @property
    def spec(self) -> BackendSpec:
        return BackendSpec(name="my_sim", provider="me", simulator=True)

    def validate(self, circuit: CircuitSpec) -> list[str]:
        return [] if circuit.num_qubits <= 30 else ["max 30 qubits"]

    def run(self, circuit: CircuitSpec, options: ExecutionOptions) -> QuantumResult:
        import my_sdk                    # SDK imports stay inside methods
        counts = my_sdk.sample(circuit.serialized, shots=options.shots)
        return QuantumResult(
            computing_model=circuit.computing_model,
            shots=ShotResult(num_qubits=circuit.num_qubits,
                             num_shots=options.shots, counts=counts),
            status=JobStatus.SUCCEEDED,
        )

Templates for both protocols live in integrations/template/, and Backends & adapters walks through testing an adapter with the SDK mocked out.