bencher.run

Convenience entry point for running benchmarks.

Attributes

_active_runners

_sigterm_installed

_prev_sigterm_handler

Functions

_shutdown_all_servers(→ None)

Stop all active panel servers during interpreter exit.

_sigterm_handler(→ None)

Handle SIGTERM so servers are stopped even when the process is killed.

_install_sigterm_handler(→ None)

Install SIGTERM handler lazily, only when servers are actually running.

run(→ list[bencher.bench_cfg.BenchCfg])

Run a benchmark target with sensible defaults.

Module Contents

bencher.run._active_runners: list = []
bencher.run._sigterm_installed: bool = False
bencher.run._shutdown_all_servers() → None

Stop all active panel servers during interpreter exit.

bencher.run._prev_sigterm_handler = None
bencher.run._sigterm_handler(signum, frame) → None

Handle SIGTERM so servers are stopped even when the process is killed.

bencher.run._install_sigterm_handler() → None

Install SIGTERM handler lazily, only when servers are actually running.

bencher.run.run(target: Callable | type | bencher.variables.parametrised_sweep.ParametrizedSweep, *, subsampling_divisions=UNSET, repeats: int = 1, max_subsampling_divisions: int | None = None, max_repeats: int | None = None, run_cfg: bencher.bench_cfg.BenchRunCfg | None = None, show: bool | str | bencher.bench_cfg.ShowMode = True, save: bool = False, publish: bool = False, publisher: bencher.bench_report.Publisher | bencher.bench_report.GithubPagesCfg | Callable | None = None, grouped: bool = False, cache_samples: bool | None = None, over_time: bool | None = None, backend: str | None = None, optimise: int | bool = 0, sampling_context: contextlib.AbstractContextManager[Any] | None = None, **kwargs) → list[bencher.bench_cfg.BenchCfg]

Run a benchmark target with sensible defaults.

Handles three cases: 1. Callable (e.g. bn.run(example_fn)) — wraps in BenchRunner. 2. ParametrizedSweep subclass (e.g. bn.run(SimpleFloat)) — instantiates, calls

to_bench() + plot_sweep().

  1. ParametrizedSweep instance (e.g. bn.run(SimpleFloat())) — same as above without instantiation.

Parameters:

  • target – A benchmark function, ParametrizedSweep class, or ParametrizedSweep instance.

  • subsampling_divisions – Benchmark sampling resolution subsampling_divisions. Defaults to 2.

  • repeats – Number of repeats. Defaults to 1.

  • max_subsampling_divisions – Maximum subsampling_divisions for progressive runs. Defaults to None (single subsampling_divisions).

  • max_repeats – Maximum repeats for progressive runs. Defaults to None (single repeat count).

  • run_cfg – Optional explicit BenchRunCfg. Defaults to None.

  • show – Where to view the report. Accepts True/ShowMode.LIVE (default — start a Panel server and block on input() until the user presses Enter), ShowMode.HTML (save an embedded HTML file and open it in the browser, then return), ShowMode.PUBLISHED (open the URL returned by publish — requires publish=True), or False/ShowMode.NONE (display nothing).

  • save – Save results to disk. Defaults to False.

  • publish – Publish results. Defaults to False.

  • publisher – An object conforming to the Publisher protocol (i.e. has a publish(report) method). Passed to BenchRunner and called after each progressive iteration when publish is True.

  • grouped – Produce a single HTML page with all benchmarks. Defaults to False.

  • cache_samples – Use sample cache for previous results. None (default) auto-enables for progressive runs. Pass False to disable even for progressive runs.

  • over_time – Enable time-series benchmarking. None preserves run_cfg value.

  • backend – Visualization backend (‘panel’ or ‘rerun’). None preserves run_cfg value.

  • optimise – When > 0, appends optuna analysis plots (parameter importance, with/without repeats comparison, best parameters) from the sweep results to the report. Defaults to 0 (no optimisation analysis).

  • sampling_context –

    An optional context manager that wraps only the sampling phase (br.run(...)). Its __exit__ is guaranteed to run before the Panel/Bokeh server starts, so resources held by the context (DB pools, GPU handles, simulators, etc.) are released while nothing blocks. save and publish still execute inside the context (they happen during br.run(show=False, ...)). Defaults to None (no wrapper, fully backward-compatible).

    Anti-pattern — wrapping the whole call keeps resources held during the interactive viewing session:

    with gpu_context():          # held for the entire viewing session!
    bn.run(target, show=True)

    Recommended — use sampling_context so the context exits before the server starts:

    bn.run(target, show=True, sampling_context=gpu_context())

Returns:

A list of benchmark configuration objects with results.

Return type:

list[BenchCfg]