bencher ======= .. py:module:: bencher Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/bencher/bench_cfg/index /autoapi/bencher/bench_plot_server/index /autoapi/bencher/bench_report/index /autoapi/bencher/bench_runner/index /autoapi/bencher/bencher/index /autoapi/bencher/cache_management/index /autoapi/bencher/class_enum/index /autoapi/bencher/factories/index /autoapi/bencher/file_server/index /autoapi/bencher/git_info/index /autoapi/bencher/job/index /autoapi/bencher/optuna_conversions/index /autoapi/bencher/perf_tracker/index /autoapi/bencher/plotting/index /autoapi/bencher/regression/index /autoapi/bencher/render/index /autoapi/bencher/result_collector/index /autoapi/bencher/results/index /autoapi/bencher/run/index /autoapi/bencher/sample_order/index /autoapi/bencher/sweep_executor/index /autoapi/bencher/sweep_timings/index /autoapi/bencher/test_timing_report/index /autoapi/bencher/utils/index /autoapi/bencher/utils_rerun/index /autoapi/bencher/utils_rrd/index /autoapi/bencher/variables/index /autoapi/bencher/video_writer/index /autoapi/bencher/worker_job/index /autoapi/bencher/worker_manager/index Attributes ---------- .. autoapisummary:: bencher.SUBSAMPLING_DIVISIONS_SAMPLES bencher.SCALAR_RESULT_TYPES bencher.DEFAULT_CACHE_SIZE_BYTES bencher.VideoResult bencher._DEPRECATED_ALIASES Exceptions ---------- .. autoapisummary:: bencher.RegressionError Classes ------- .. autoapisummary:: bencher.Bench bencher.BenchCfg bencher.BenchRunCfg bencher.ShowMode bencher.BenchRunner bencher.BenchPlotServer bencher.IntSweep bencher.FloatSweep bencher.StringSweep bencher.EnumSweep bencher.BoolSweep bencher.SweepBase bencher.YamlSweep bencher.TimeSnapshot bencher.ResultFloat bencher.ResultVar bencher.ResultBool bencher.ResultVec bencher.ResultHmap bencher.ResultPath bencher.ResultVideo bencher.ResultImage bencher.ResultString bencher.ResultContainer bencher.ResultRerun bencher.ResultReference bencher.ResultVolume bencher.OptDir bencher.ResultDataSet bencher.ComposeType bencher.ComposableContainerBase bencher.PaneLayout bencher.ComposableContainerVideo bencher.RenderCfg bencher.ComposableContainerPanel bencher.ComposableContainerDataset bencher.BoxWhiskerResult bencher.ViolinResult bencher.ScatterResult bencher.ScatterJitterResult bencher.BarResult bencher.LineResult bencher.CurveResult bencher.HeatmapResult bencher.BandResult bencher.SurfaceResult bencher.TabulatorResult bencher.TableResult bencher.VolumeResult bencher.HistogramResult bencher.ExplorerResult bencher.DataSetResult bencher.RegressionResult bencher.RegressionReport bencher.MethodCells bencher.PerfTracker bencher.PerfReport bencher.VarRange bencher.PlotFilter bencher.ParametrizedSweep bencher.ParametrizedSweepSingleton bencher.SampleOrder bencher.CacheDirStats bencher.CacheStats bencher.BenchResult bencher.OptimizeResult bencher.PaneResult bencher.ReduceType bencher.HoloviewResult bencher.BenchReport bencher.GithubPagesCfg bencher.Publisher bencher.Executors bencher.SweepTimings bencher.VideoWriter bencher.ClassEnum bencher.ExampleEnum Functions --------- .. autoapisummary:: bencher.render_report bencher.save_result bencher.load_result bencher.hash_sha1 bencher.box bencher.p bencher.sweep bencher.with_subsampling_divisions bencher.curve bencher.hmap_canonical_input bencher.get_nearest_coords bencher.make_namedtuple bencher.gen_path bencher.gen_image_path bencher.gen_video_path bencher.gen_rerun_data_path bencher.lerp bencher.tabs_in_markdown bencher.publish_file bencher.github_content bencher.publish_and_view_rrd bencher.rrd_to_pane bencher.rrd_file_to_pane bencher.run_file_server bencher.method_cells bencher.git_time_event bencher.cache_stats bencher.print_cache_stats bencher.clear_all bencher.clear_media bencher.clean_orphaned_media bencher.cleanup_job_media bencher.ensure_cache_version bencher.add_image bencher.create_bench bencher.create_bench_runner bencher.run bencher.__getattr__ Package Contents ---------------- .. py:class:: Bench(bench_name: str | None = None, worker: Callable | bencher.variables.parametrised_sweep.ParametrizedSweep | None = None, worker_input_cfg: bencher.variables.parametrised_sweep.ParametrizedSweep | None = None, run_cfg: bencher.bench_cfg.BenchRunCfg | None = None, report: bencher.bench_report.BenchReport | None = None) Bases: :py:obj:`bencher.bench_plot_server.BenchPlotServer` A server for display plots of benchmark results .. py:attribute:: bench_name :value: None .. py:attribute:: cache_size :value: 0 .. py:attribute:: _worker_mgr .. py:attribute:: _executor .. py:attribute:: _collector .. py:attribute:: run_cfg :value: None .. py:attribute:: results :value: [] .. py:attribute:: bench_cfg_hashes :value: [] .. py:attribute:: last_run_cfg :value: None .. py:attribute:: input_vars :value: None .. py:attribute:: result_vars :value: None .. py:attribute:: const_vars :value: None .. py:attribute:: plot_callbacks :value: [] .. py:attribute:: plot :value: True .. py:method:: close() -> None Close sample and collector caches so on-disk resources are released. .. py:property:: sample_cache Access the sample cache from the executor (for backward compatibility). .. py:property:: ds_dynamic Access the dynamic dataset from the collector (for backward compatibility). .. py:method:: add_plot_callback(callback: Callable[[bencher.results.bench_result.BenchResult], panel.panel], **kwargs) -> None Add a plotting callback to be called on benchmark results. This method registers a plotting function that will be automatically called on any BenchResult produced when running a sweep. You can pass additional arguments to the plotting function using keyword arguments. :param callback: A function that takes a BenchResult and returns a panel object. For example: BenchResult.to_video_grid :type callback: Callable[[BenchResult], pn.panel] :param \*\*kwargs: Additional keyword arguments to pass to the callback function .. rubric:: Examples >>> bench.add_plot_callback(BenchResult.to_video_grid, width=800) .. py:method:: set_worker(worker: Callable | bencher.variables.parametrised_sweep.ParametrizedSweep, worker_input_cfg: bencher.variables.parametrised_sweep.ParametrizedSweep | None = None) -> None Set the benchmark worker function and its input configuration. This method sets up the worker function to be benchmarked. The worker can be either a callable function that takes a ParametrizedSweep instance or a ParametrizedSweep instance with a __call__ method. In the latter case, worker_input_cfg is not needed. :param worker: Either a function that will be benchmarked or a ParametrizedSweep instance with a __call__ method. When a ParametrizedSweep is provided, its __call__ method becomes the worker function. :type worker: Callable | ParametrizedSweep :param worker_input_cfg: The class defining the input parameters for the worker function. Only needed if worker is a function rather than a ParametrizedSweep instance. Defaults to None. :type worker_input_cfg: ParametrizedSweep, optional :raises RuntimeError: If worker is a class type instead of an instance. .. py:method:: sweep_sequential(title: str = '', input_vars: list[bencher.variables.parametrised_sweep.ParametrizedSweep] | None = None, result_vars: list[bencher.variables.parametrised_sweep.ParametrizedSweep] | None = None, const_vars: list[bencher.variables.parametrised_sweep.ParametrizedSweep] | None = None, optimise_var: bencher.variables.parametrised_sweep.ParametrizedSweep | None = None, run_cfg: bencher.bench_cfg.BenchRunCfg | None = None, group_size: int = 1, iterations: int = 1, relationship_cb: Callable | None = None, plot_callbacks: list[Callable] | bool | None = None, aggregate: bool | int | list[str] | None = None, agg_fn: str = 'mean') -> list[bencher.results.bench_result.BenchResult] Run a sequence of benchmarks by sweeping through groups of input variables. This method performs sweeps on combinations of input variables, potentially using the optimal value from each sweep as constants for the next iteration. :param title: Base title for all the benchmark sweeps. Defaults to "". :type title: str, optional :param input_vars: Input variables to sweep through. If None, defaults to all input variables from the worker class instance. :type input_vars: list[ParametrizedSweep], optional :param result_vars: Result variables to collect. Defaults to None. :type result_vars: list[ParametrizedSweep], optional :param const_vars: Variables to keep constant. Defaults to None. :type const_vars: list[ParametrizedSweep], optional :param optimise_var: Variable to optimize on each sweep iteration. The optimal value will be used as constant input for subsequent sweeps. Defaults to None. :type optimise_var: ParametrizedSweep, optional :param run_cfg: Run configuration. Defaults to None. :type run_cfg: BenchRunCfg, optional :param group_size: Number of input variables to sweep together in each run. Defaults to 1. :type group_size: int, optional :param iterations: Number of optimization iterations to perform. Defaults to 1. :type iterations: int, optional :param relationship_cb: Function to determine how to group variables for sweeping. Defaults to itertools.combinations if None. :type relationship_cb: Callable, optional :param plot_callbacks: Callbacks for plotting or bool to enable/disable. Defaults to None. :type plot_callbacks: list[Callable] | bool, optional :returns: A list of results from all the sweep runs :rtype: list[BenchResult] .. py:method:: plot_sweep(title: str | None = None, input_vars: list[bencher.variables.parametrised_sweep.ParametrizedSweep] | None = None, result_vars: list[bencher.variables.parametrised_sweep.ParametrizedSweep] | None = None, const_vars: list[bencher.variables.parametrised_sweep.ParametrizedSweep] | None = None, time_src: datetime.datetime | None = None, description: str | None = None, post_description: str | None = None, pass_repeat: bool = False, tag: str = '', run_cfg: bencher.bench_cfg.BenchRunCfg | None = None, plot_callbacks: list[Callable] | bool | None = None, sample_order: bencher.sample_order.SampleOrder = SampleOrder.INORDER, aggregate: bool | int | list[str] | None = None, agg_fn: str = 'mean', auto_plot: bool | None = None) -> bencher.results.bench_result.BenchResult The all-in-one function for benchmarking and results plotting. This is the main function for performing benchmark sweeps. It handles all the setup, execution, and visualization of benchmarks based on the input parameters. When ``input_vars``, ``result_vars``, and ``const_vars`` are all ``None`` (the default), bencher **auto-discovers** all sweep inputs and result variables from the ``ParametrizedSweep`` class definition. This means a bare ``bench.plot_sweep()`` call with no arguments will sweep every input and collect every result. :param title: The title of the benchmark. If None, a title will be generated based on the input variables. Defaults to None. :type title: str, optional :param input_vars: Variables to sweep through in the benchmark. If None and worker_class_instance exists, auto-discovers all input sweep variables from the class. Defaults to None. :type input_vars: list[ParametrizedSweep], optional :param result_vars: Variables to collect results for. If None and worker_class_instance exists, auto-discovers all result variables from the class. Defaults to None. :type result_vars: list[ParametrizedSweep], optional :param const_vars: Variables to keep constant with specified values. If None and worker_class_instance exists, uses default input values. Defaults to None. :type const_vars: list[ParametrizedSweep], optional :param time_src: The timestamp for the benchmark. Used for time-series benchmarks. Defaults to None, which will use the current time. :type time_src: datetime, optional :param description: A description displayed before the benchmark plots. Defaults to None. :type description: str, optional :param post_description: A description displayed after the benchmark plots. Defaults to a generic message recommending to set a custom description. :type post_description: str, optional :param pass_repeat: If True, passes the 'repeat' parameter to the worker function. Defaults to False. :type pass_repeat: bool, optional :param tag: Tag to group different benchmarks together. Defaults to "". :type tag: str, optional :param run_cfg: Configuration for how the benchmarks are run. If None, uses the instance's run_cfg or creates a default one. Defaults to None. :type run_cfg: BenchRunCfg, optional :param plot_callbacks: Callbacks for plotting results. If True, uses default plotting. If False, disables plotting. If a list, uses the provided callbacks. Defaults to None. :type plot_callbacks: list[Callable] | bool, optional :param sample_order: Controls the traversal order of sampling only. Defaults to SampleOrder.INORDER. Plotting and dataset dimension order are unchanged. :type sample_order: SampleOrder, optional :param auto_plot: Whether to build the holoviews/panel report immediately after the sweep. ``None`` (default) respects ``run_cfg.auto_plot`` (itself ``True`` by default), so behaviour is unchanged unless a caller opts out. ``False`` collects samples and computes regression detection WITHOUT constructing any plotting objects — the returned BenchResult is fully populated (dataset + regression_report) and can be rendered later, in a separate process, via :func:`bencher.render_report`. Useful when the collecting process holds foreign C-extension state (e.g. ROS/rclpy) that makes in-process holoviews/bokeh garbage collection unsafe. See also :meth:`Bench.collect`. Because ``None`` defers to ``run_cfg``, setting ``run_cfg.auto_plot = False`` once disables plotting for every ``plot_sweep`` call that uses that config — including calls nested inside benchmark functions you don't control. :type auto_plot: bool, optional :returns: An object containing all the benchmark data and results :rtype: BenchResult :raises RuntimeError: If an unsupported input variable type is provided :raises TypeError: If variable parameters are not of the correct type :raises FileNotFoundError: If only_plot=True and no cached results exist .. py:method:: collect(*args, **kwargs) -> bencher.results.bench_result.BenchResult Run a sweep and collect results WITHOUT building any plots. Equivalent to :meth:`plot_sweep` with ``auto_plot=False``: it executes the sweep, merges over-time history, and computes regression detection, but constructs **no** holoviews/panel/bokeh objects. The returned :class:`BenchResult` is fully populated (dataset + ``regression_report``) and is the safe artifact to persist (:func:`bencher.save_result`) and render later — in a separate, clean process — via :func:`bencher.render_report`. This is the collection half of a collect/render split, intended for callers whose process holds foreign C-extension state (e.g. ROS/rclpy/DDS) where in-process holoviews/bokeh allocation and the resulting garbage collection can segfault. Accepts the same arguments as :meth:`plot_sweep` (``auto_plot`` is forced to ``False``). :returns: Fully-populated result with no plots built. :rtype: BenchResult .. py:method:: filter_overridable_params(bench_cfg: bencher.bench_cfg.BenchCfg, run_cfg: bencher.bench_cfg.BenchRunCfg) -> tuple[dict, list[str], list[str]] :staticmethod: Filter run_cfg parameters to only those that can override bench_cfg. Param 2.3 enforces constant Parameters (e.g. the implicit `name`), which cannot be overridden. This helper identifies which parameters from run_cfg can be applied to bench_cfg and reports any that must be skipped. :param bench_cfg: The benchmark configuration to be updated :param run_cfg: The run configuration providing override values :returns: - valid_params: dict of parameters that can be applied - missing_keys: list of run_cfg keys not found on bench_cfg - constant_keys: list of run_cfg keys that are constant on bench_cfg :rtype: A tuple of (valid_params, missing_keys, constant_keys) where .. py:method:: run_sweep(bench_cfg: bencher.bench_cfg.BenchCfg, run_cfg: bencher.bench_cfg.BenchRunCfg, time_src: datetime.datetime | None = None, sample_order: bencher.sample_order.SampleOrder = SampleOrder.INORDER) -> bencher.results.bench_result.BenchResult Execute a benchmark sweep based on the provided configuration. This method handles the caching, execution, and post-processing of a benchmark sweep according to the provided configurations. It's typically called by `plot_sweep` rather than directly by users. :param bench_cfg: Configuration defining inputs, results, and other benchmark parameters :type bench_cfg: BenchCfg :param run_cfg: Configuration for how the benchmark should be executed :type run_cfg: BenchRunCfg :param time_src: The timestamp for the benchmark. Used for time-series benchmarks. Defaults to None, which will use the current time. :type time_src: datetime, optional :param sample_order: Controls the traversal order of sampling only. Defaults to SampleOrder.INORDER. :type sample_order: SampleOrder, optional :returns: An object containing all benchmark data, results, and visualization :rtype: BenchResult :raises FileNotFoundError: If only_plot=True and no cached results exist .. py:method:: _append_result_via_split(bench_res: bencher.results.bench_result.BenchResult) -> None Append a result to the report through the collect/render split. Used only when the ``BENCHER_FORCE_SPLIT_RENDER`` env var is set. Instead of rendering ``bench_res`` in-process, it round-trips the result through pickle (:func:`bencher.save_result` / :func:`bencher.load_result`) and rebuilds the report tab from the *deserialized* copy — the same serialize then render-from-loaded steps that :func:`bencher.render_report` performs out of process. This lets a dedicated CI job re-run the entire existing test/example suite with the split pipeline forced on, so any divergence between in-process and split rendering (unpicklable result types, render paths that relied on live state) surfaces in the existing assertions. The round-trip stays in-process here so the full suite remains fast; a separate test covers the subprocess boundary. .. py:method:: convert_vars_to_params(variable: param.Parameter | str | dict | tuple, var_type: str, run_cfg: bencher.bench_cfg.BenchRunCfg | None) -> param.Parameter Convert various input formats (str, dict, tuple) to param.Parameter objects. .. py:method:: cache_results(bench_res: bencher.results.bench_result.BenchResult, bench_cfg_hash: str) -> None Cache benchmark results to disk using the config hash as key. .. py:method:: load_history_cache(dataset: xarray.Dataset, bench_cfg_hash: str, clear_history: bool, max_time_events: int | None = None, result_vars: list | None = None) -> xarray.Dataset Load and concatenate historical benchmark data from cache. .. py:method:: setup_dataset(bench_cfg: bencher.bench_cfg.BenchCfg, time_src: datetime.datetime | str) -> tuple[bencher.results.bench_result.BenchResult, zip, list[str], int] Initialize n-dimensional xarray dataset for storing benchmark results. .. py:method:: define_const_inputs(const_vars: list[tuple[param.Parameter, Any]]) -> dict | None Convert constant variable tuples into a name-value dictionary. .. py:method:: define_extra_vars(bench_cfg: bencher.bench_cfg.BenchCfg, repeats: int, time_src: datetime.datetime | str) -> list[bencher.variables.inputs.IntSweep] Define meta variables (repeat count, timestamps) for benchmark tracking. .. py:method:: calculate_benchmark_results(bench_cfg: bencher.bench_cfg.BenchCfg, time_src: datetime.datetime | str, bench_cfg_sample_hash: str, bench_run_cfg: bencher.bench_cfg.BenchRunCfg, sample_order: bencher.sample_order.SampleOrder = SampleOrder.INORDER, timings: bencher.sweep_timings.SweepTimings | None = None) -> bencher.results.bench_result.BenchResult Execute the benchmark runs and collect results into an n-dimensional array. This method handles the core benchmark execution process. It sets up the dataset, initializes worker jobs, submits them to the sample cache for execution or retrieval, and collects and stores the results. :param bench_cfg: Configuration defining the benchmark parameters :type bench_cfg: BenchCfg :param time_src: Timestamp or event name for the benchmark run :type time_src: datetime | str :param bench_cfg_sample_hash: Hash of the benchmark configuration without repeats :type bench_cfg_sample_hash: str :param bench_run_cfg: Configuration for how the benchmark should be executed :type bench_run_cfg: BenchRunCfg :param timings: Timing collector to populate. Defaults to None. :type timings: SweepTimings, optional :returns: An object containing all the benchmark data and results :rtype: BenchResult .. py:method:: store_results(job_result: bencher.job.JobFuture, bench_res: bencher.results.bench_result.BenchResult, worker_job: bencher.worker_job.WorkerJob, bench_run_cfg: bencher.bench_cfg.BenchRunCfg, rv_arrays: dict[str, numpy.ndarray] | None = None) -> None Store worker job results into the n-dimensional result dataset. .. py:method:: init_sample_cache(run_cfg: bencher.bench_cfg.BenchRunCfg) -> bencher.job.FutureCache Initialize the FutureCache for storing benchmark function results. .. py:method:: clear_tag_from_sample_cache(tag: str, run_cfg: bencher.bench_cfg.BenchRunCfg) -> None Clear all cached samples matching a specific tag. .. py:method:: add_metadata_to_dataset(bench_res: bencher.results.bench_result.BenchResult, input_var: bencher.variables.parametrised_sweep.ParametrizedSweep) -> None Add units, long names, and descriptions to xarray dataset attributes. .. py:method:: report_results(bench_res: bencher.results.bench_result.BenchResult, print_xarray: bool, print_pandas: bool) -> None Log benchmark results as xarray or pandas DataFrame. .. py:method:: clear_call_counts() -> None Clear the worker and cache call counts, to help debug and assert caching is happening properly .. py:method:: get_result(index: int = -1) -> bencher.results.bench_result.BenchResult Get a specific benchmark result from the results list. :param index: Index of the result to retrieve. Negative indices are supported, with -1 (default) returning the most recent result. :type index: int, optional :returns: The benchmark result at the specified index :rtype: BenchResult .. py:method:: get_ds(index: int = -1) -> xarray.Dataset Get the xarray Dataset from a specific benchmark result. This is a convenience method that retrieves a result and returns its dataset. :param index: Index of the result to retrieve the dataset from. Negative indices are supported, with -1 (default) returning the most recent result. :type index: int, optional :returns: The xarray Dataset from the benchmark result :rtype: xr.Dataset .. py:method:: publish(remote_callback: Callable[[str], str]) -> str Publish the benchmark report to a remote location. Uses the provided callback to publish the benchmark report to a remote location such as a GitHub Pages site. :param remote_callback: A function that takes a branch name and publishes the report, returning the URL where it's published :type remote_callback: Callable[[str], str] :returns: The URL where the report has been published :rtype: str .. py:method:: get_result_vars(as_str: bool = True) -> list[str | bencher.variables.parametrised_sweep.ParametrizedSweep] Retrieve the result variables from the worker class instance. :param as_str: If True, the result variables are returned as strings. If False, they are returned in their original form. Default is True. :type as_str: bool :returns: A list of result variables, either as strings or in their original form. :rtype: list[str | ParametrizedSweep] :raises RuntimeError: If the worker class instance is not set. .. py:method:: optimize(title: str | None = None, input_vars=None, result_vars=None, const_vars=None, n_trials: int = 100, sampler: optuna.samplers.BaseSampler | None = None, warm_start: bool = True, aggregate: bool | int | list[str] | None = None, agg_fn: str = 'mean', repeats: int = 1, tag: str = '', run_cfg: bencher.bench_cfg.BenchRunCfg | None = None, plot: bool = True) -> bencher.results.optimize_result.OptimizeResult | None Run optuna optimization directly — no full grid sweep required. :param title: Study name. Auto-generated when *None*. :param input_vars: Input variables to optimize over. Detected from ``worker_class_instance`` when *None*. :param result_vars: Result variables (objectives). Detected when *None*. :param const_vars: Constant variables. Detected when *None*. :param n_trials: Number of new optuna trials to run. :param sampler: Optuna sampler. Defaults to ``TPESampler``. :param warm_start: Seed the study with previously cached evaluations. :param aggregate: Dimensions to aggregate inside the objective function. Same semantics as ``plot_sweep``: *True* aggregates all but the first input dim, an *int N* aggregates the last N dims, or a *list[str]* names specific dims. Aggregated dims are looped over internally so Optuna sees the aggregated value. :param agg_fn: Aggregation function name (``"mean"``, ``"sum"``, ``"max"``, ``"min"``, ``"median"``). Applied when *aggregate* is set or *repeats* > 1. :param repeats: Number of times to evaluate each parameter combination. Each repeat uses a different ``repeat`` index and gets its own cache key. Results are aggregated with *agg_fn*. :param tag: Cache tag (same semantics as ``plot_sweep``). :param run_cfg: Run configuration. Defaults to ``BenchRunCfg()``. :param plot: If *True*, append visualisation to ``self.report``. :returns: OptimizeResult wrapping the completed ``optuna.Study``. .. py:method:: _resolve_optimize_vars(input_vars, result_vars, const_vars, run_cfg) Deep-copy and convert variable lists to param.Parameter objects. .. py:method:: _build_cache_key(inputs: dict, tag: str) -> str :staticmethod: Build a deterministic cache key from an input dict and tag. .. py:method:: _warm_from_results(study: optuna.Study) -> int Seed *study* from in-memory BenchResult objects. Returns count added. .. py:method:: _warm_from_sample_cache(study: optuna.Study, bench_cfg: bencher.bench_cfg.BenchCfg, input_vars: list, constant_inputs: dict, target_names: list[str]) -> int Seed *study* from the on-disk sample cache. Returns count added. .. py:method:: _warm_start_from_cache(study: optuna.Study, bench_cfg: bencher.bench_cfg.BenchCfg, input_vars: list, constant_inputs: dict, target_names: list[str]) -> int Seed *study* with cached evaluations. Returns count of added trials. .. py:method:: _split_optuna_and_agg_vars(input_vars, aggregate) :staticmethod: Partition *input_vars* into Optuna-tuned and aggregated lists. .. py:method:: _run_optuna_job(trial, job_args, repeat, constant_inputs, tag) Submit a single worker evaluation and return the result dict. .. py:method:: _make_optuna_objective(input_vars, constant_inputs, target_names, tag, agg_vars=None, agg_callable=None, repeats=1) Return an objective function compatible with ``study.optimize()``. .. py:method:: to(result_type: bencher.results.bench_result.BenchResult, result_var: param.Parameter | None = None, override: bool = True, **kwargs: Any) -> bencher.results.bench_result.BenchResult Return the current instance of BenchResult. :returns: The current instance of the benchmark result :rtype: BenchResult .. py:method:: add(result_type: bencher.results.bench_result.BenchResult, result_var: param.Parameter | None = None, override: bool = True, **kwargs: Any) .. py:class:: BenchCfg(**params: Any) Bases: :py:obj:`BenchRunCfg` Complete configuration for a benchmark protocol. This class extends BenchRunCfg and provides a comprehensive set of parameters for configuring benchmark runs. It maintains a unique hash value based on its configuration to ensure that benchmark results can be consistently referenced and that plots are uniquely identified across runs. The class handles input variables, result variables, constant values, meta variables, and various presentation options. It also provides methods for generating descriptive summaries and visualizations of the benchmark configuration. .. attribute:: input_vars A list of ParameterizedSweep variables to perform a parameter sweep over :type: list .. attribute:: result_vars A list of ParameterizedSweep results to collect and plot :type: list .. attribute:: const_vars Variables to keep constant but are different from the default value :type: list .. attribute:: result_hmaps A list of holomap results :type: list .. attribute:: meta_vars Meta variables such as recording time and repeat id :type: list .. attribute:: all_vars Stores a list of both the input_vars and meta_vars :type: list .. attribute:: iv_time Parameter for sampling the same inputs over time :type: list[TimeSnapshot | TimeEvent] .. attribute:: over_time Controls whether the function is sampled over time :type: bool .. attribute:: name The name of the benchmarkCfg :type: str .. attribute:: title The title of the benchmark :type: str .. attribute:: raise_duplicate_exception Used for debugging filename generation uniqueness :type: bool .. attribute:: bench_name The name of the benchmark and save folder :type: str .. attribute:: description A longer description of the benchmark function :type: str .. attribute:: post_description Comments on the output of the graphs :type: str .. attribute:: has_results Whether this config has results :type: bool .. attribute:: pass_repeat Whether to pass the 'repeat' kwarg to the benchmark function :type: bool .. attribute:: tag Tags for grouping different benchmarks :type: str .. attribute:: hash_value Stored hash value of the config :type: str .. attribute:: plot_callbacks Callables that take a BenchResult and return panel representation :type: list .. py:attribute:: input_vars .. py:attribute:: result_vars .. py:attribute:: const_vars .. py:attribute:: result_hmaps .. py:attribute:: meta_vars .. py:attribute:: all_vars .. py:attribute:: iv_time .. py:attribute:: name :type: str | None .. py:attribute:: title :type: str | None .. py:attribute:: raise_duplicate_exception :type: bool .. py:attribute:: bench_name :type: str | None .. py:attribute:: description :type: str | None .. py:attribute:: post_description :type: str | None .. py:attribute:: has_results :type: bool .. py:attribute:: pass_repeat :type: bool .. py:attribute:: tag :type: str .. py:attribute:: hash_value :type: str .. py:attribute:: plot_callbacks .. py:attribute:: agg_over_dims .. py:attribute:: agg_fn .. py:attribute:: plot_lib :value: None .. py:attribute:: hmap_kdims :value: None .. py:attribute:: iv_repeat :value: None .. py:method:: hash_persistent(include_repeats: bool) -> str Generate a persistent hash for the benchmark configuration. Overrides the default hash function because the default hash function does not return the same value for the same inputs. This method references only stable variables that are consistent across instances of BenchCfg with the same configuration. :param include_repeats: Whether to include repeats as part of the hash (True by default except when using the sample cache) :type include_repeats: bool :returns: A persistent hash value for the benchmark configuration :rtype: str .. py:method:: inputs_as_str() -> list[str] Get a list of input variable names. :returns: List of the names of input variables :rtype: list[str] .. py:method:: to_latex() -> panel.pane.LaTeX | None Convert benchmark configuration to LaTeX representation. :returns: LaTeX representation of the benchmark configuration :rtype: pn.pane.LaTeX | None .. py:method:: to_cartesian_animation() -> str | None Render an animation of the Cartesian product data collection. Delegates to :func:`bencher.results.manim_cartesian.render_animation`, which currently uses a PIL-based renderer. Returns the filesystem path to the generated animated PNG (or other format, depending on the renderer), or ``None`` on failure so callers can degrade gracefully. :returns: Path to the rendered animation file, or None on failure. :rtype: str | None .. py:method:: describe_sweep(width: int = 800, accordion: bool = True) -> panel.pane.Markdown | panel.Column Produce a markdown summary of the sweep settings. :param width: Width of the markdown panel in pixels. Defaults to 800. :type width: int :param accordion: Whether to wrap the description in an accordion. Defaults to True. :type accordion: bool :returns: Panel containing the sweep description :rtype: pn.pane.Markdown | pn.Column .. py:method:: sweep_sentence() -> panel.pane.Markdown Generate a concise summary sentence of the sweep configuration. :returns: A panel containing a markdown summary sentence :rtype: pn.pane.Markdown .. py:method:: describe_benchmark() -> str Generate a detailed string summary of the inputs and results from a BenchCfg. :returns: Comprehensive summary of BenchCfg :rtype: str .. py:method:: to_title(panel_name: str | None = None) -> panel.pane.Markdown Create a markdown panel with the benchmark title. :param panel_name: The name for the panel. Defaults to the benchmark title. :type panel_name: str | None :returns: A panel with the benchmark title as a heading :rtype: pn.pane.Markdown .. py:method:: to_description(width: int = 800) -> panel.pane.Markdown Create a markdown panel with the benchmark description. :param width: Width of the markdown panel in pixels. Defaults to 800. :type width: int :returns: A panel with the benchmark description :rtype: pn.pane.Markdown .. py:method:: to_post_description(width: int = 800) -> panel.pane.Markdown Create a markdown panel with the benchmark post-description. :param width: Width of the markdown panel in pixels. Defaults to 800. :type width: int :returns: A panel with the benchmark post-description :rtype: pn.pane.Markdown .. py:method:: to_sweep_summary(name: str | None = None, description: bool = True, describe_sweep: bool = True, results_suffix: bool = True, title: bool = True) -> panel.Column Produce panel output summarising the title, description and sweep setting. :param name: Name for the panel. Defaults to benchmark title or "Data Collection Parameters" if title is False. :type name: str | None :param description: Whether to include the benchmark description. Defaults to True. :type description: bool :param describe_sweep: Whether to include the sweep description. Defaults to True. :type describe_sweep: bool :param results_suffix: Whether to add a "Results:" heading. Defaults to True. :type results_suffix: bool :param title: Whether to include the benchmark title. Defaults to True. :type title: bool :returns: A panel with the benchmark summary :rtype: pn.Column .. py:method:: partition_input_vars(vars_) -> tuple[list, list] :staticmethod: Split variables into (optimized, non-optimized) based on the optimize flag. .. py:property:: optimized_input_vars :type: list Return input variables where optimize=True (suggested by Optuna). .. py:property:: non_optimized_input_vars :type: list Return input variables where optimize=False (swept/aggregated, not suggested). .. py:method:: optuna_targets(as_var: bool = False) -> list[Any] Get the list of result variables that are optimization targets. :param as_var: If True, return the variable objects rather than their names. Defaults to False. :type as_var: bool :returns: List of result variable names or objects that are optimization targets :rtype: list[Any] .. py:class:: BenchRunCfg(**params: Any) Bases: :py:obj:`BenchPlotSrvCfg` Configuration class for benchmark execution parameters. This class extends BenchPlotSrvCfg to provide comprehensive control over benchmark execution, including caching behavior, reporting options, visualization settings, and execution strategy. It defines numerous parameters that control how benchmark runs are performed, cached, and displayed to the user. Quick-start examples:: # Use defaults — each variable uses its own ``samples`` setting: run_cfg = BenchRunCfg() # Set a sampling subsampling_divisions (geometrically increasing sample counts): run_cfg = BenchRunCfg(subsampling_divisions=5) # 9 samples per variable run_cfg = BenchRunCfg(subsampling_divisions=8) # 65 samples per variable # Or set an exact sample count directly: run_cfg = BenchRunCfg(samples_per_var=20) Subsampling Divisions-to-samples mapping ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ========= ======= ========= ======= ========= ======= Subsampling Divisions Samples Subsampling Divisions Samples Subsampling Divisions Samples ========= ======= ========= ======= ========= ======= 1 1 5 9 9 129 2 2 6 17 10 257 3 3 7 33 11 513 4 5 8 65 12 1025 ========= ======= ========= ======= ========= ======= .. attribute:: repeats The number of times to sample the inputs :type: int .. attribute:: over_time If true each time the function is called it will plot a timeseries of historical and the latest result :type: bool .. attribute:: use_optuna Show optuna plots :type: bool .. attribute:: summarise_constant_inputs Print the inputs that are kept constant when describing the sweep parameters :type: bool .. attribute:: print_bench_inputs Print the inputs to the benchmark function every time it is called :type: bool .. attribute:: print_bench_results Print the results of the benchmark function every time it is called :type: bool .. attribute:: clear_history Clear historical results :type: bool .. attribute:: max_time_events Maximum number of over_time events to retain. None means unlimited. :type: int .. attribute:: max_slider_points Maximum time points in the over_time slider. Defaults to 10, None means all. :type: int .. attribute:: show_aggregated_time_tab Show the aggregated tab for over_time plots. Defaults to False. :type: bool .. attribute:: show_aggregate_plots Show aggregated BandResult plots when aggregate is set. :type: bool .. attribute:: print_pandas Print a pandas summary of the results to the console :type: bool .. attribute:: print_xarray Print an xarray summary of the results to the console :type: bool .. attribute:: serve_pandas Serve a pandas summary on the results webpage :type: bool .. attribute:: serve_pandas_flat Serve a flattened pandas summary on the results webpage :type: bool .. attribute:: serve_xarray Serve an xarray summary on the results webpage :type: bool .. attribute:: auto_plot Automatically deduce the best type of plot for the results :type: bool .. attribute:: raise_duplicate_exception Used to debug unique plot names :type: bool .. attribute:: cache_results Benchmark level cache for completed benchmark results :type: bool .. attribute:: clear_cache Clear the cache of saved input->output mappings :type: bool .. attribute:: cache_samples Enable per-sample caching :type: bool .. attribute:: only_hash_tag Use only the tag hash for cache identification :type: bool .. attribute:: clear_sample_cache Clear the per-sample cache :type: bool .. attribute:: overwrite_sample_cache Recalculate and overwrite cached sample values :type: bool .. attribute:: only_plot Do not calculate benchmarks if no results are found in cache :type: bool .. attribute:: use_holoview Use holoview for plotting :type: bool .. attribute:: nightly Run a more extensive set of tests for a nightly benchmark :type: bool .. attribute:: time_event String representation of a sequence over time :type: str .. attribute:: headless Run the benchmarks headlessly :type: bool .. attribute:: dry_run Preview sweep grid without executing the benchmark function :type: bool .. attribute:: subsampling_divisions Method of defining the number of samples to sweep over :type: int .. attribute:: samples_per_var Explicit sample count per variable (overrides subsampling_divisions) :type: int | None .. attribute:: run_tag Tag for isolating cached results :type: str .. attribute:: run_date Date the benchmark run was performed :type: datetime .. attribute:: executor Executor for running the benchmark :type: Executors .. attribute:: plot_size Sets both width and height of the plot :type: int .. attribute:: plot_width Sets width of the plots :type: int .. attribute:: plot_height Sets height of the plot :type: int .. py:attribute:: repeats :type: int .. py:attribute:: subsampling_divisions :type: int .. py:attribute:: samples_per_var :type: int | None .. py:attribute:: executor .. py:attribute:: nightly :type: bool .. py:attribute:: headless :type: bool .. py:attribute:: dry_run :type: bool .. py:attribute:: cache_results :type: bool .. py:attribute:: cache_samples :type: bool .. py:attribute:: clear_cache :type: bool .. py:attribute:: clear_sample_cache :type: bool .. py:attribute:: overwrite_sample_cache :type: bool .. py:attribute:: only_hash_tag :type: bool .. py:attribute:: only_plot :type: bool .. py:attribute:: cache_size :type: int .. py:attribute:: print_bench_inputs :type: bool .. py:attribute:: print_bench_results :type: bool .. py:attribute:: summarise_constant_inputs :type: bool .. py:attribute:: print_pandas :type: bool .. py:attribute:: print_xarray :type: bool .. py:attribute:: serve_pandas :type: bool .. py:attribute:: serve_pandas_flat :type: bool .. py:attribute:: serve_xarray :type: bool .. py:attribute:: auto_plot :type: bool .. py:attribute:: use_holoview :type: bool .. py:attribute:: use_optuna :type: bool .. py:attribute:: plot_size :type: int | None .. py:attribute:: plot_width :type: int | None .. py:attribute:: plot_height :type: int | None .. py:attribute:: raise_duplicate_exception :type: bool .. py:attribute:: pane_layout .. py:attribute:: backend .. py:attribute:: over_time :type: bool .. py:attribute:: clear_history :type: bool .. py:attribute:: max_time_events :type: int | None .. py:attribute:: max_slider_points :type: int | None .. py:attribute:: show_aggregated_time_tab :type: bool .. py:attribute:: show_aggregate_plots :type: bool .. py:attribute:: time_event :type: str | None .. py:attribute:: run_tag :type: str .. py:attribute:: run_date :type: datetime.datetime .. py:attribute:: regression_detection :type: bool .. py:attribute:: regression_method :type: str .. py:attribute:: regression_mad :type: float .. py:attribute:: regression_percentage :type: float .. py:attribute:: regression_delta :type: float .. py:attribute:: regression_absolute :type: float .. py:attribute:: regression_fail :type: bool .. py:method:: from_cmd_line() -> BenchRunCfg :staticmethod: Create a BenchRunCfg by parsing command line arguments. Parses command line arguments to create a configuration for benchmark runs. :returns: Configuration object with settings from command line arguments :rtype: BenchRunCfg .. py:method:: subsampling_divisions_to_samples(subsampling_divisions: int, max_subsampling_divisions: int = 12) -> int :staticmethod: Return the number of samples-per-variable for a given *subsampling_divisions*. :param subsampling_divisions: Sampling subsampling_divisions (1-12). :param max_subsampling_divisions: Cap applied before lookup. Defaults to 12. :returns: The sample count for this subsampling_divisions. :raises ValueError: If *subsampling_divisions* is out of range. Example:: >>> BenchRunCfg.subsampling_divisions_to_samples(5) 9 .. py:method:: level_to_samples(level: int, max_level: int = 12) -> int :staticmethod: Deprecated: use :meth:`subsampling_divisions_to_samples` instead. .. py:method:: deep() .. py:method:: with_defaults(run_cfg=None, **defaults) :classmethod: Merge *defaults* into *run_cfg*, creating a new instance when needed. When *run_cfg* is ``None`` a fresh ``BenchRunCfg`` is created with *defaults*. When *run_cfg* is provided, a shallow copy is made and each default is applied only if the corresponding field is still at its param-level default value (i.e. the caller did not explicitly set it). The original *run_cfg* is never mutated. This lets benchmark functions declare sensible defaults while still allowing callers to override:: run_cfg = bn.BenchRunCfg.with_defaults(run_cfg, repeats=5, subsampling_divisions=4) :raises ValueError: If any key in *defaults* is not a recognised parameter. .. py:class:: ShowMode Bases: :py:obj:`strenum.LowercaseStrEnum` Display mode for benchmark reports. .. py:attribute:: LIVE .. py:attribute:: HTML .. py:attribute:: PUBLISHED .. py:attribute:: NONE .. py:class:: BenchRunner(name: str | Benchable | None = None, bench_class: bencher.variables.parametrised_sweep.ParametrizedSweep | None = None, run_cfg: bencher.bench_cfg.BenchRunCfg | None = None, publisher: Callable | None = None, run_tag: str | None = None) A class to manage running multiple benchmarks in groups, or running the same benchmark but at multiple resolutions. BenchRunner provides a framework for organizing, configuring, and executing multiple benchmark runs with different parameters. It supports progressive refinement of benchmark resolution, caching of results, and publication of results to various formats. .. py:attribute:: bench_fns :value: [] .. py:attribute:: run_cfg .. py:attribute:: publisher :value: None .. py:attribute:: results :value: [] .. py:attribute:: servers :value: [] .. py:method:: _generate_name() -> str Generate a unique name for the BenchRunner instance. :returns: A unique name based on timestamp, object id, and random value :rtype: str .. py:method:: setup_run_cfg(run_cfg: bencher.bench_cfg.BenchRunCfg | None = None, subsampling_divisions=UNSET, cache_samples: bool = False, over_time: bool | None = None, level: int | None = None) -> bencher.bench_cfg.BenchRunCfg :staticmethod: Configure benchmark run settings with reasonable defaults. Creates a copy of the provided configuration with the specified subsampling_divisions and caching behavior settings applied. :param run_cfg: Base configuration to modify. Defaults to None. :type run_cfg: BenchRunCfg, optional :param subsampling_divisions: Benchmark sampling resolution subsampling_divisions. Defaults to 2. :type subsampling_divisions: int, optional :param cache_samples: Whether to enable sample caching. Defaults to False. :type cache_samples: bool, optional :param over_time: Enable time-series benchmarking. None preserves run_cfg value. :type over_time: bool, optional :param level: Deprecated. Use ``subsampling_divisions`` instead. :type level: int, optional :returns: A new configuration object with the specified settings :rtype: BenchRunCfg .. py:method:: from_parametrized_sweep(class_instance: bencher.variables.parametrised_sweep.ParametrizedSweep, run_cfg: bencher.bench_cfg.BenchRunCfg | None = None, report: bencher.bench_report.BenchReport | None = None) -> bencher.bencher.Bench :staticmethod: Create a Bench instance from a ParametrizedSweep class. :param class_instance: The parametrized sweep class instance to benchmark :type class_instance: ParametrizedSweep :param run_cfg: Configuration for benchmark execution. Defaults to None. :type run_cfg: BenchRunCfg, optional :param report: Report to store benchmark results. Defaults to None. :type report: BenchReport, optional :returns: A configured Bench instance ready to run the benchmark :rtype: Bench .. py:method:: add(bench_fn: Benchable) -> BenchRunner Add a benchmark function to be executed by this runner. :param bench_fn: A callable that implements the Benchable protocol :type bench_fn: Benchable :returns: Self for method chaining :rtype: BenchRunner .. py:method:: add_run(bench_fn: Benchable) -> None Add a benchmark function to be executed by this runner. .. deprecated:: Use :meth:`add` instead. :param bench_fn: A callable that implements the Benchable protocol :type bench_fn: Benchable .. py:method:: add_bench(class_instance: bencher.variables.parametrised_sweep.ParametrizedSweep) -> None Add a parametrized sweep class instance as a benchmark. Creates and adds a function that will create a Bench instance from the provided parametrized sweep class when executed. :param class_instance: The parametrized sweep to benchmark :type class_instance: ParametrizedSweep .. py:method:: _merge_reports(target: bencher.bench_report.BenchReport, source: bencher.bench_report.BenchReport | None) -> None Append all tabs from source report into the target report. .. py:method:: _execute_bench_fn(bench_fn: Benchable, run_cfg: bencher.bench_cfg.BenchRunCfg, report: bencher.bench_report.BenchReport | None) -> tuple[bencher.bench_cfg.BenchCfg, bencher.bench_report.BenchReport | None] Execute a bench function handling legacy and new signatures. .. py:method:: run(subsampling_divisions=UNSET, repeats: int = 1, max_subsampling_divisions: int | None = None, max_repeats: int | None = None, min_level: int | None = None, start_repeats: int | None = None, run_cfg: bencher.bench_cfg.BenchRunCfg | None = None, publish: bool = False, debug: bool = False, show: bool | str | bencher.bench_cfg.ShowMode = False, save: bool = False, grouped: bool = False, cache_samples: bool | None = None, over_time: bool | None = None, backend: str | None = None, **kwargs) -> list[bencher.bench_cfg.BenchCfg] Unified interface for running benchmarks. This function provides a single entry point for benchmark runs: - Single runs: Use subsampling_divisions and repeats parameters only - Progressive runs: Set max_subsampling_divisions and/or max_repeats for automatic progression :param # Primary parameters: :type # Primary parameters: starting values :param subsampling_divisions: Starting benchmark subsampling_divisions. Defaults to 2. :type subsampling_divisions: int :param repeats: Starting number of repeats. Defaults to 1. :type repeats: int :param max_subsampling_divisions: Maximum subsampling_divisions for progression. If None, uses single subsampling_divisions. :type max_subsampling_divisions: int, optional :param max_repeats: Maximum repeats for progression. If None, uses single repeat count. :type max_repeats: int, optional :param # Legacy parameters: :type # Legacy parameters: deprecated - use subsampling_divisions/max_subsampling_divisions instead :param min_level: DEPRECATED - use 'subsampling_divisions' parameter instead. :type min_level: int, optional :param start_repeats: DEPRECATED - use 'repeats' parameter instead. :type start_repeats: int, optional :param run_cfg: benchmark run configuration. Defaults to None. :type run_cfg: BenchRunCfg, optional :param publish: Publish the results to git, requires a publish url to be set up. Defaults to False. :type publish: bool, optional :param debug: Enable debug output during publishing. Defaults to False. :type debug: bool, optional :param show: How to display results. ``True``/``ShowMode.LIVE`` starts a Panel server (blocks); ``ShowMode.HTML`` saves HTML and opens it in the browser (returns); ``ShowMode.PUBLISHED`` opens the published URL (requires ``publish=True``); ``False``/``ShowMode.NONE`` displays nothing. Defaults to False. :type show: bool | str | ShowMode, optional :param save: save the results to disk in index.html. Defaults to False. :type save: bool, optional :param grouped: Produce a single html page with all the benchmarks included. Defaults to False. :type grouped: bool, optional :param cache_samples: Use the sample cache to reuse previous results. None (default) auto-enables for progressive runs. Pass False to disable even for progressive runs. :type cache_samples: bool | None, optional :param over_time: Enable time-series benchmarking. None preserves run_cfg value. :type over_time: bool, optional :param backend: Visualization backend ('panel' or 'rerun'). None preserves run_cfg value. :type backend: str, optional :returns: A list of benchmark configuration objects with results :rtype: list[BenchCfg] .. py:method:: show_publish(report: bencher.bench_report.BenchReport, show: bool | str | bencher.bench_cfg.ShowMode, publish: bool, save: bool, debug: bool) -> None Handle publishing, saving, and displaying of a benchmark report. :param report: The benchmark report to process :type report: BenchReport :param show: How to display the report. See :func:`bencher.bench_cfg.normalize_show` for accepted values. :type show: bool | str | ShowMode :param publish: Whether to publish the report :type publish: bool :param save: Whether to save the report to disk :type save: bool :param debug: Whether to enable debug mode for publishing :type debug: bool .. py:method:: show(report: bencher.bench_report.BenchReport | None = None, show: bool | str | bencher.bench_cfg.ShowMode = True, publish: bool = False, save: bool = False, debug: bool = False) -> None Display or publish a specific benchmark report. This is a convenience method to show, publish, or save a specific report. If no report is provided, it will use the most recent result. :param report: The report to process. Defaults to None (most recent). :type report: BenchReport, optional :param show: How to display. See :meth:`run` for accepted values. Defaults to True. :type show: bool | str | ShowMode, optional :param publish: Whether to publish the report. Defaults to False. :type publish: bool, optional :param save: Whether to save to disk. Defaults to False. :type save: bool, optional :param debug: Enable debug mode for publishing. Defaults to False. :type debug: bool, optional :raises RuntimeError: If no report is specified and no results are available .. py:method:: shutdown() -> None Stop all running panel servers launched by this benchmark runner. This method ensures that any web servers started to display benchmark results are properly shut down. .. py:method:: __del__() -> None Destructor that ensures proper cleanup of resources. Automatically calls shutdown() to stop any running servers when the BenchRunner instance is garbage collected. .. py:function:: render_report(result_or_path: bencher.results.bench_result.BenchResult | str | pathlib.Path, output_dir: str | pathlib.Path, *, report: bencher.bench_report.BenchReport | None = None, filename: str | None = None, in_html_folder: bool = False, portable: bool = False) -> pathlib.Path Render a collected result to an HTML report. Reconstructs the holoviews/panel report from a result produced by :meth:`Bench.collect` (or a path to one saved with :func:`save_result`) and writes it under *output_dir*. This is the only step that constructs plotting objects, and it is designed to run in a process free of foreign C-extension state. The result already carries its ``regression_report`` (computed during collection), so no sweep re-execution happens here. :param result_or_path: A :class:`BenchResult`, or a path to a saved one. :param output_dir: Directory to write the report into. :param report: An existing :class:`BenchReport` to append to. A new one is created (named after the benchmark) when omitted. :param filename: Output HTML filename. Defaults to ``.html``. :param in_html_folder: Forwarded to :meth:`BenchReport.save`. :param portable: Forwarded to :meth:`BenchReport.save` (base64-inline assets). :returns: The path to the saved report. .. py:function:: save_result(bench_res: bencher.results.bench_result.BenchResult, path: str | pathlib.Path) -> pathlib.Path Persist a collected :class:`BenchResult` to *path* via pickle. Mirrors how bencher already caches results internally: the (potentially non-pickleable) ``object_index`` is stripped before writing and restored afterwards, so the live object is unchanged. :param bench_res: A result from :meth:`Bench.collect` / ``plot_sweep``. :param path: Destination file path. :returns: The path written. .. py:function:: load_result(path: str | pathlib.Path) -> bencher.results.bench_result.BenchResult Load a :class:`BenchResult` previously written by :func:`save_result`. .. py:class:: BenchPlotServer A server for display plots of benchmark results .. py:method:: plot_server(bench_name: str, plot_cfg: bencher.bench_cfg.BenchPlotSrvCfg | None = None, plots_instance=None) -> threading.Thread Load previously calculated benchmark data from the database and start a plot server to display it :param bench_name: The name of the benchmark and output folder for the figures :type bench_name: str :param plot_cfg: Options for the plot server. Defaults to None. :type plot_cfg: BenchPlotSrvCfg, optional :raises FileNotFoundError: No data found was found in the database to plot .. py:method:: load_data_from_cache(bench_name: str) -> tuple[bencher.bench_cfg.BenchCfg, list[panel.panel]] | None Load previously calculated benchmark data from the database and start a plot server to display it :param bench_name: The name of the benchmark and output folder for the figures :type bench_name: str :returns: benchmark result data and any additional panels :rtype: tuple[BenchCfg, list[pn.panel]] | None :raises FileNotFoundError: No data found was found in the database to plot .. py:method:: _find_free_port() -> int :staticmethod: Find a free port by testing random ports in the dynamic/private range. Using ``port=0`` with Tornado/Bokeh can fail on some Linux kernels (notably 6.x) because the kernel deterministically assigns the same ephemeral port, causing ``EADDRINUSE`` when a previous server is still running. Picking a random port from the IANA dynamic range avoids this. Note: there is an inherent TOCTOU race between probing the port here and the actual ``bind()`` inside Panel/Bokeh. In practice the window is very small and the random selection makes collisions unlikely, but callers should be prepared for a rare ``OSError`` on server start. .. py:method:: serve(bench_name: str, plots_instance: list[panel.panel], port: int | None = None, show: bool = True) -> threading.Thread Launch a panel server to view results :param bench_cfg: benchmark results :type bench_cfg: BenchCfg :param plots_instance: list of panel objects to display :type plots_instance: list[pn.panel] :param port: use a fixed port to launch the server :type port: int .. py:method:: _rrd_extra_patterns() -> list :staticmethod: Return Tornado route patterns for serving .rrd files with CORS headers. Mounts ``cachedir/rrd/`` at ``/rrd_static/`` so that the local rerun viewer can fetch ``.rrd`` files from the Panel server. See the module docstring in ``utils_rerun.py`` for the full architecture explanation. .. py:function:: hash_sha1(var: Any) -> str A hash function that avoids the PYTHONHASHSEED 'feature' which returns a different hash value each time the program is run. Converts input to a consistent SHA1 hash string. :param var: The variable to hash :type var: Any :returns: A hexadecimal SHA1 hash of the string representation of the variable :rtype: str .. py:data:: SUBSAMPLING_DIVISIONS_SAMPLES :value: [0, 1, 2, 3, 5, 9, 17, 33, 65, 129, 257, 513, 1025, 2049] .. py:class:: IntSweep(units: str = 'ul', samples: int | None = None, sample_values: list[int] | None = None, optimize: bool = True, **params) Bases: :py:obj:`param.Integer`, :py:obj:`bencher.variables.sweep_base.SweepBase` A class representing a parameter sweep for integer values. This class extends both Integer and SweepBase to provide parameter sweeping capabilities specifically for integer values within specified bounds or with custom sample values. .. attribute:: units The units of measurement for the parameter :type: str .. attribute:: samples The number of samples to take from the range :type: int .. attribute:: sample_values Specific integer values to use as samples instead of generating them from bounds. If provided, overrides the samples parameter. :type: list[int], optional .. py:attribute:: __slots__ :value: ['units', 'samples', 'optimize', 'sample_values'] .. py:attribute:: units :value: 'ul' .. py:attribute:: optimize :value: True .. py:method:: _coerce_bound(value) Override in subclasses to coerce bound values to the correct type. .. py:method:: _sweep_identity() -> tuple Include bounds and sample_values so a reshaped sweep busts the cache. .. py:method:: values() -> list[int] Return all the values for the parameter sweep. If sample_values is provided, returns those values. Otherwise generates values within the specified bounds. :returns: A list of integer values to sweep through :rtype: list[int] .. py:method:: _validate_value(value, allow_None) Validate the parameter value against constraints. :param value: The value to be validated. :type value: Any :param allow_None: Whether `None` is allowed as a valid value. :type allow_None: bool :raises ValueError: If the value does not meet the parameter's constraints. .. py:method:: _validate_step(val, step) .. py:class:: FloatSweep(units: str = 'ul', samples: int = 10, sample_values: list[float] | None = None, step: float | None = None, optimize: bool = True, **params) Bases: :py:obj:`param.Number`, :py:obj:`bencher.variables.sweep_base.SweepBase` A class representing a parameter sweep for floating point values. This class extends both Number and SweepBase to provide parameter sweeping capabilities specifically for floating point values within specified bounds or with custom sample values. .. attribute:: units The units of measurement for the parameter :type: str .. attribute:: samples The number of samples to take from the range :type: int .. attribute:: sample_values Specific float values to use as samples instead of generating them from bounds. If provided, overrides the samples parameter. :type: list[float], optional .. attribute:: step Step size between samples when generating values from bounds :type: float, optional .. py:attribute:: __slots__ :value: ['units', 'samples', 'optimize', 'sample_values'] .. py:attribute:: units :value: 'ul' .. py:attribute:: optimize :value: True .. py:attribute:: sample_values .. py:method:: _coerce_bound(value) Override in subclasses to coerce bound values to the correct type. .. py:method:: _sweep_identity() -> tuple Include bounds, sample_values, and step so a reshaped sweep busts the cache. .. py:method:: values() -> list[float] Return all the values for the parameter sweep. If sample_values is provided, returns those values. Otherwise generates values within the specified bounds, either using linspace (when step is None) or arange. :returns: A list of float values to sweep through :rtype: list[float] .. py:class:: StringSweep(string_list: list[str], units: str = 'ul', samples: int | None = None, optimize: bool = True, **params) Bases: :py:obj:`SweepSelector` A class representing a parameter sweep for string values. This class extends SweepSelector to provide parameter sweeping capabilities specifically for a list of string values. .. attribute:: units The units of measurement for the parameter :type: str .. attribute:: samples The number of samples to take from the available strings :type: int .. py:method:: dynamic(*, placeholder: str | None = None, units: str = 'ul', doc: str | None = None, **params) -> StringSweep :classmethod: Create a StringSweep intended for later population. :param placeholder: Optional text to show before real values are loaded. Defaults to the sentinel's displayed text. :param units: Units label (optional, passed through). :param doc: Documentation string for the parameter. :param params: Additional param overrides. :returns: A sweep with a single sentinel placeholder value. :rtype: StringSweep .. py:class:: EnumSweep(enum_type: enum.Enum | list[enum.Enum], units: str = 'ul', samples: int | None = None, optimize: bool = True, **params) Bases: :py:obj:`SweepSelector` A class representing a parameter sweep for enum values. This class extends SweepSelector to provide parameter sweeping capabilities specifically for enumeration types, supporting both enum types and lists of enum values. .. attribute:: units The units of measurement for the parameter :type: str .. attribute:: samples The number of samples to take from the available enum values :type: int .. py:attribute:: __slots__ :value: ['units', 'samples', 'optimize'] .. py:class:: BoolSweep(units: str = 'ul', samples: int | None = None, default: bool = True, optimize: bool = True, **params) Bases: :py:obj:`SweepSelector` A class representing a parameter sweep for boolean values. This class extends SweepSelector to provide parameter sweeping capabilities specifically for boolean values (True and False). .. attribute:: units The units of measurement for the parameter :type: str .. attribute:: samples The number of samples to take (typically 2 for booleans) :type: int .. py:class:: SweepBase(default: Any = ..., *, doc: str | None = None, label: str | None = None, precedence: float | None = None, instantiate: bool = False, constant: bool = False, readonly: bool = False, pickle_default_value: bool = True, allow_None: Literal[False] = False, per_instance: bool = True, allow_refs: bool = False, nested_refs: bool = False, default_factory: collections.abc.Callable[[], Any] | None = None, metadata: dict[str, Any] | None = None) Bases: :py:obj:`param.Parameter` Base :class:`Parameter` type to hold any type of Python object. Parameters are a special kind of class attribute implemented as descriptor. Setting a Parameterized class attribute to a `Parameter` instance enables enhanced functionality, including type and range validation at assignment, support for constant and read-only parameters, documentation strings and dynamic parameter values. Parameters can only be used as class attributes of :class:`Parameterized` classes. Using them in standalone contexts or with non-:class:`Parameterized` classes will not provide the described behavior. .. rubric:: Notes Parameters provide lots of features. **Dynamic Behavior**: - Parameters provide support for dynamic values, type validation, and range checking. - Parameters can be declared as constant or read-only. **Automatic Initialization**: - Parameters can be set during object construction using keyword arguments. For example: ``myfoo = Foo(alpha=0.5); print(myfoo.alpha)``. - If custom constructors are implemented, they can still pass keyword arguments to the superclass to allow Parameter initialization. **Inheritance**: - :class:`Parameterized` classes automatically inherit parameters from their superclasses. Attributes can be selectively overridden. **Subclassing**: - The :class:`Parameter` class can be subclassed to create custom behavior, such as validating specific ranges or generating values dynamically. **GUI Integration**: - Parameters provide sufficient metadata for auto-generating property sheets in graphical user interfaces, enabling user-friendly parameter editing. .. rubric:: Examples Define a :class:`Parameterized` class with parameters: >>> import param >>> class Foo(param.Parameterized): ... alpha = param.Parameter(default=0.1, doc="The starting value.") ... beta = param.Parameter(default=0.5, doc="The standard deviation.", constant=True) When no initial value is provided the default is used: >>> Foo().alpha 0.1 When an initial value is provided it is used: >>> foo = Foo(alpha=0.5) >>> foo.alpha 0.5 Constant parameters cannot be modified: >>> foo.beta = 0.1 # Cannot be changed since it's constant ... TypeError: Constant parameter 'beta' cannot be modified .. py:property:: sweep_bounds :type: tuple | None Return the sweep range (low, high). FloatSweep/IntSweep store user-supplied bounds as param softbounds (not hard bounds) so that values outside the range are not rejected. This property provides a single access point. .. py:method:: values() -> list[Any] :abstractmethod: All sweep classes must implement this method. This generates sample values from based on the parameters bounds and sample number. :returns: A list of samples from the variable :rtype: list[Any] .. py:attribute:: _sweep_hash_exclude :type: tuple[str, Ellipsis] :value: ('optimize',) .. py:method:: _sweep_identity() -> tuple Return the tuple of values that uniquely identifies this sweep for the benchmark-level and over_time history caches. Subclasses MUST override and call ``super()._sweep_identity() + (...)`` to append any shape-affecting fields: bounds, sample_values, step, ``objects``, etc. Any field that changes the set of sampled values or the coordinate labels of the resulting xarray must contribute here, otherwise the benchmark-level cache and over_time history will silently serve stale data for a reshaped sweep. The class name is included so that different Sweep subclasses with the same identity tuple do not collide. Note: the sample cache is keyed solely by concrete input values (see :class:`bencher.worker_job.WorkerJob`) and is unaffected by this hash, so widening a sweep range still reuses per-sample cache entries for overlapping inputs. .. py:method:: hash_persistent() -> str Deterministic hash based on :meth:`_sweep_identity`. Avoids Python's per-process hash randomisation so two Bench runs (or two processes) compute identical cache keys for equivalent sweeps. .. py:method:: sampling_str() -> str Generate a string representation of the of the sampling procedure .. py:method:: as_slider() -> panel.widgets.slider.DiscreteSlider given a sweep variable (self), return the range of values as a panel slider :param debug: pass to the sweepvar to produce a full set of variables, or when debug=True, a reduces number of sweep vars. Defaults to False. :type debug: bool, optional :returns: A panel slider with the values() of the sweep variable :rtype: pn.widgets.slider.DiscreteSlider .. py:method:: as_dim(compute_values=False) -> holoviews.Dimension Takes a sweep variable and turns it into a holoview dimension :rtype: hv.Dimension .. py:method:: indices_to_samples(desires_num_samples, sample_values) .. py:method:: with_samples(samples: int) -> SweepBase .. py:method:: _coerce_bound(value) Override in subclasses to coerce bound values to the correct type. .. py:method:: with_bounds(low: float, high: float, samples: int | None = None) -> SweepBase Create a copy with overridden sweep bounds (and optionally sample count). :param low: Lower bound of the sweep range. :param high: Upper bound of the sweep range. :param samples: Number of samples. When *None* the existing sample count is kept. :returns: A new sweep with the specified bounds. :rtype: SweepBase :raises ValueError: If *low* >= *high* or the sweep has no bounds attributes. .. py:method:: with_sample_values(sample_values: list) -> SweepBase .. py:method:: __call__(values: list | None = None, *, samples: int | None = None, bounds: tuple[float, float] | None = None) -> SweepBase Shorthand for creating a sweep with specific values, sample count, or bounds. Usage:: Cfg.param.theta([0, 0.5, 1.0]) # explicit values Cfg.param.theta(samples=5) # override sample count Cfg.param.theta(bounds=(0, 1)) # override range Cfg.param.theta(bounds=(0, 1), samples=10) # override range and count :param values: Explicit list of values to sweep through. :param samples: Number of samples to take from the sweep range. :param bounds: ``(low, high)`` tuple to override the sweep range. :returns: A copy of this sweep with the specified configuration. :rtype: SweepBase .. py:method:: with_const(const_value: Any) -> tuple[SweepBase, Any] Create a new instance of SweepBase with a constant value. :param const_value: The constant value to be associated with the new instance. :type const_value: Any :returns: A tuple containing the new instance of SweepBase and the constant value. :rtype: tuple[SweepBase, Any] .. py:method:: with_subsampling_divisions(subsampling_divisions: int = 1, max_subsampling_divisions: int = 12) -> SweepBase .. py:method:: with_level(level: int = 1, max_level: int = 12) -> SweepBase Deprecated: use :meth:`with_subsampling_divisions` instead. .. py:class:: YamlSweep(yaml_path: str | pathlib.Path, units: str = 'ul', samples: int | None = None, default_key: str | None = None, optimize: bool = True, **params) Bases: :py:obj:`SweepSelector` Sweep over configurations stored in a YAML file. Loads the YAML mapping once during initialisation and exposes each top-level key as a sweep choice. Each sampled value is a :class:`YamlSelection` instance that exposes the underlying YAML content via the ``value`` attribute (and dict-like helpers). .. py:attribute:: __slots__ :value: ['units', 'samples', 'optimize', 'yaml_path', '_entries', 'default_key'] .. py:attribute:: _sweep_hash_exclude :value: ('yaml_path', '_entries', 'default_key') .. py:attribute:: yaml_path :value: '' .. py:attribute:: _entries .. py:attribute:: default_key :value: None .. py:method:: _load_yaml(path: pathlib.Path) -> Any :staticmethod: .. py:method:: keys() -> list[str] .. py:method:: items() -> list[tuple[str, Any]] .. py:method:: values() -> list[Any] Return all the values for the parameter sweep. :returns: A list of parameter values to sweep through :rtype: list[Any] .. py:method:: key_for_value(value: Any) -> str | None .. py:class:: TimeSnapshot(datetime_src: datetime.datetime | str, units: str = 'time', samples: int | None = None, **params) Bases: :py:obj:`TimeBase` A class to capture a time snapshot of benchmark values. Time is represent as a continuous value i.e a datetime which is converted into a np.datetime64. To represent time as a discrete value use the TimeEvent class. The distinction is because holoview and plotly code makes different assumptions about discrete vs continuous variables .. py:attribute:: __slots__ :value: ['units', 'samples', 'optimize'] .. py:attribute:: units :value: 'time' .. py:attribute:: optimize :value: False .. py:function:: box(name: str, center: float, width: float) -> FloatSweep Create a FloatSweep parameter centered around a value with a given width. This is a convenience function to create a bounded FloatSweep parameter with bounds centered on a specific value, extending by the width in both directions. :param name: The name of the parameter :type name: str :param center: The center value of the parameter :type center: float :param width: The distance from the center to the bounds in both directions :type width: float :returns: A FloatSweep parameter with the specified name, default, and bounds :rtype: FloatSweep .. py:function:: p(name: str | bencher.variables.sweep_base.SweepBase, values: list[Any] | None = None, *, samples: int | None = None, bounds: tuple[float, float] | None = None, max_subsampling_divisions: int | None = None) -> dict[str, Any] | bencher.variables.sweep_base.SweepBase Deprecated: use ``bn.sweep()`` instead. .. py:function:: sweep(name: str | bencher.variables.sweep_base.SweepBase, values: list[Any] | None = None, *, samples: int | None = None, bounds: tuple[float, float] | None = None, max_subsampling_divisions: int | None = None, max_level: int | None = None) -> dict[str, Any] | bencher.variables.sweep_base.SweepBase Create a parameter specification for use in plot_sweep input_vars. Accepts either a string parameter name (returns a dict for deferred lookup) or a SweepBase parameter object (returns a configured sweep directly). Examples:: bn.sweep("theta", [0, 0.5, 1.0]) # explicit values bn.sweep("theta", samples=5) # override sample count bn.sweep("theta", bounds=(0, 1)) # override range bn.sweep("theta", bounds=(0, 1), samples=10) # override range + count bn.sweep(Cfg.param.theta, bounds=(0, 1), samples=5) # SweepBase object :param name: The parameter name (str) or a param object (e.g. ``Cfg.param.theta``). :param values: A list of values for the parameter. :param samples: The number of samples. Must be > 0 if provided. :param bounds: ``(low, high)`` tuple to override the sweep range. :param max_subsampling_divisions: The maximum subsampling_divisions. Must be > 0 if provided. :returns: A parameter dict (for string names) or configured sweep object. :rtype: dict[str, Any] | SweepBase .. py:function:: with_subsampling_divisions(arr: list, subsampling_divisions: int) -> list Apply subsampling_divisions-based sampling to a list of values. Uses the same subsampling_divisions→sample-count table as SweepBase.with_subsampling_divisions and picks evenly spaced items from *arr* by index. :param arr: list of values to sample from :type arr: list :param subsampling_divisions: The sampling subsampling_divisions to apply (higher subsampling_divisions provides more samples) :type subsampling_divisions: int :returns: The subsampling_divisions-sampled values :rtype: list .. py:class:: ResultFloat(units='ul', direction: OptDir = OptDir.minimize, share_axis=True, max_time_events=None, default=0, **params) Bases: :py:obj:`param.Number` A class to represent continuous float result variables and the desired optimisation direction. For boolean (success/failure) outcomes, use ``ResultBool`` instead — it locks bounds to [0, 1] and produces correct boolean-style plots. .. py:attribute:: __slots__ :value: ['units', 'direction', 'share_axis', 'max_time_events'] .. py:attribute:: _hash_exclude :value: ('direction', 'share_axis', 'max_time_events') .. py:attribute:: units :value: 'ul' .. py:attribute:: default :value: 0 .. py:attribute:: direction .. py:attribute:: share_axis :value: True .. py:attribute:: max_time_events :value: None .. py:method:: as_dim() -> holoviews.Dimension .. py:method:: hash_persistent() -> str A hash function that avoids the PYTHONHASHSEED 'feature' which returns a different hash value each time the program is run .. py:class:: ResultVar(*args, **kwargs) Bases: :py:obj:`ResultFloat` Deprecated: use ResultFloat instead. .. py:class:: ResultBool(units='ratio', direction: OptDir = OptDir.minimize, default=0, **params) Bases: :py:obj:`ResultFloat` A result type for binary outcomes (success/failure, pass/fail, reachable/unreachable). Bounds are locked to [0, 1] and plots use boolean-style rendering. For continuous scalar metrics (time, distance, score), use ``ResultFloat`` instead. .. py:attribute:: default :value: 0 .. py:attribute:: bounds :value: (0, 1) .. py:data:: SCALAR_RESULT_TYPES .. py:class:: ResultVec(size, units='ul', direction: OptDir = OptDir.minimize, max_time_events=None, default=0, **params) Bases: :py:obj:`param.List` A class to represent fixed size vector result variable .. py:attribute:: __slots__ :value: ['units', 'direction', 'size', 'max_time_events'] .. py:attribute:: _hash_exclude :value: ('max_time_events',) .. py:attribute:: units :value: 'ul' .. py:attribute:: default :value: 0 .. py:attribute:: direction .. py:attribute:: size .. py:attribute:: max_time_events :value: None .. py:method:: hash_persistent() -> str A hash function that avoids the PYTHONHASHSEED 'feature' which returns a different hash value each time the program is run .. py:method:: index_name(idx: int) -> str given the index of the vector, return the column name that :param idx: index of the result vector :type idx: int :returns: column name of the vector for the xarray dataset :rtype: str .. py:method:: index_names() -> list[str] Returns a list of all the xarray column names for the result vector :returns: column names :rtype: list[str] .. py:class:: ResultHmap(default: Any = ..., *, doc: str | None = None, label: str | None = None, precedence: float | None = None, instantiate: bool = False, constant: bool = False, readonly: bool = False, pickle_default_value: bool = True, allow_None: Literal[False] = False, per_instance: bool = True, allow_refs: bool = False, nested_refs: bool = False, default_factory: collections.abc.Callable[[], Any] | None = None, metadata: dict[str, Any] | None = None) Bases: :py:obj:`param.Parameter` A class to represent a holomap return type. Note: this class has no __slots__, so _hash_slots hashes only the class name. Every ResultHmap instance produces the same hash. This is intentional — there are no configuration attributes that would differentiate instances. If a slot is added in the future, _hash_slots will automatically include it. .. py:method:: hash_persistent() -> str A hash function that avoids the PYTHONHASHSEED 'feature' which returns a different hash value each time the program is run .. py:class:: ResultPath(default=None, units='path', max_time_events=None, **params) Bases: :py:obj:`param.Filename` Parameter that can be set to a string specifying the path of a file. The string should be specified in UNIX style, but it will be returned in the format of the user's operating system. The specified path can be absolute, or relative to either: * any of the paths specified in the ``search_paths`` attribute (if ``search_paths`` is not ``None``); * any of the paths searched by :func:`resolve_path` (if ``search_paths`` is ``None``). .. py:attribute:: __slots__ :value: ['units', 'max_time_events'] .. py:attribute:: _hash_exclude :value: ('max_time_events',) .. py:attribute:: units :value: 'path' .. py:attribute:: max_time_events :value: None .. py:method:: hash_persistent() -> str A hash function that avoids the PYTHONHASHSEED 'feature' which returns a different hash value each time the program is run .. py:method:: to_container() Returns a partial function for creating a FileDownload widget with embedding enabled. This function is used to create a panel container to represent the ResultPath object .. py:class:: ResultVideo(default=None, units='path', max_time_events=None, **params) Bases: :py:obj:`param.Filename` Parameter that can be set to a string specifying the path of a file. The string should be specified in UNIX style, but it will be returned in the format of the user's operating system. The specified path can be absolute, or relative to either: * any of the paths specified in the ``search_paths`` attribute (if ``search_paths`` is not ``None``); * any of the paths searched by :func:`resolve_path` (if ``search_paths`` is ``None``). .. py:attribute:: __slots__ :value: ['units', 'max_time_events'] .. py:attribute:: _hash_exclude :value: ('max_time_events',) .. py:attribute:: units :value: 'path' .. py:attribute:: max_time_events :value: None .. py:method:: hash_persistent() -> str A hash function that avoids the PYTHONHASHSEED 'feature' which returns a different hash value each time the program is run .. py:class:: ResultImage(default=None, units='path', max_time_events=None, **params) Bases: :py:obj:`param.Filename` Parameter that can be set to a string specifying the path of a file. The string should be specified in UNIX style, but it will be returned in the format of the user's operating system. The specified path can be absolute, or relative to either: * any of the paths specified in the ``search_paths`` attribute (if ``search_paths`` is not ``None``); * any of the paths searched by :func:`resolve_path` (if ``search_paths`` is ``None``). .. py:attribute:: __slots__ :value: ['units', 'max_time_events'] .. py:attribute:: _hash_exclude :value: ('max_time_events',) .. py:attribute:: units :value: 'path' .. py:attribute:: max_time_events :value: None .. py:method:: hash_persistent() -> str A hash function that avoids the PYTHONHASHSEED 'feature' which returns a different hash value each time the program is run .. py:class:: ResultString(default=None, units='str', max_time_events=None, **params) Bases: :py:obj:`param.String` A String Parameter with optional regular expression (regex) validation. The ``String`` class extends the :class:`Parameter` class to specifically handle string values and provides additional support for validating values against a regular expression. :param default: The default value of the parameter. Default is an empty string (``""``). :type default: str, optional :param regex: A regular expression used to validate the string value. If ``None``, no regex validation is applied. Default is ``None``. :type regex: str or None, optional .. rubric:: Examples Define a ``String`` parameter with regex validation: >>> import param >>> class MyClass(param.Parameterized): ... user_name = param.String(default="John Doe", regex=r"^[A-Za-z ]+$", doc="Name of a person.") >>> instance = MyClass() Access the default value: >>> instance.user_name 'John Doe' Set a valid value: >>> instance.user_name = "Jane Smith" >>> instance.user_name 'Jane Smith' Attempt to set an invalid value (non-alphabetic characters): >>> instance.user_name = "Jane123" ... ValueError: String parameter 'MyClass.user_name' value 'Jane123' does not match regex '^[A-Za-z ]+$'. .. py:attribute:: __slots__ :value: ['units', 'max_time_events'] .. py:attribute:: _hash_exclude :value: ('max_time_events',) .. py:attribute:: units :value: 'str' .. py:attribute:: max_time_events :value: None .. py:method:: hash_persistent() -> str A hash function that avoids the PYTHONHASHSEED 'feature' which returns a different hash value each time the program is run .. py:class:: ResultContainer(default=None, units='container', max_time_events=None, **params) Bases: :py:obj:`param.Parameter` Base :class:`Parameter` type to hold any type of Python object. Parameters are a special kind of class attribute implemented as descriptor. Setting a Parameterized class attribute to a `Parameter` instance enables enhanced functionality, including type and range validation at assignment, support for constant and read-only parameters, documentation strings and dynamic parameter values. Parameters can only be used as class attributes of :class:`Parameterized` classes. Using them in standalone contexts or with non-:class:`Parameterized` classes will not provide the described behavior. .. rubric:: Notes Parameters provide lots of features. **Dynamic Behavior**: - Parameters provide support for dynamic values, type validation, and range checking. - Parameters can be declared as constant or read-only. **Automatic Initialization**: - Parameters can be set during object construction using keyword arguments. For example: ``myfoo = Foo(alpha=0.5); print(myfoo.alpha)``. - If custom constructors are implemented, they can still pass keyword arguments to the superclass to allow Parameter initialization. **Inheritance**: - :class:`Parameterized` classes automatically inherit parameters from their superclasses. Attributes can be selectively overridden. **Subclassing**: - The :class:`Parameter` class can be subclassed to create custom behavior, such as validating specific ranges or generating values dynamically. **GUI Integration**: - Parameters provide sufficient metadata for auto-generating property sheets in graphical user interfaces, enabling user-friendly parameter editing. .. rubric:: Examples Define a :class:`Parameterized` class with parameters: >>> import param >>> class Foo(param.Parameterized): ... alpha = param.Parameter(default=0.1, doc="The starting value.") ... beta = param.Parameter(default=0.5, doc="The standard deviation.", constant=True) When no initial value is provided the default is used: >>> Foo().alpha 0.1 When an initial value is provided it is used: >>> foo = Foo(alpha=0.5) >>> foo.alpha 0.5 Constant parameters cannot be modified: >>> foo.beta = 0.1 # Cannot be changed since it's constant ... TypeError: Constant parameter 'beta' cannot be modified .. py:attribute:: __slots__ :value: ['units', 'max_time_events'] .. py:attribute:: _hash_exclude :value: ('max_time_events',) .. py:attribute:: units :value: 'container' .. py:attribute:: max_time_events :value: None .. py:method:: hash_persistent() -> str A hash function that avoids the PYTHONHASHSEED 'feature' which returns a different hash value each time the program is run .. py:class:: ResultRerun(default=None, units='rerun', width=600, height=600, max_time_events=None, **params) Bases: :py:obj:`ResultContainer` Result type for rerun .rrd spatial visualizations. Stores a path to an .rrd file (like ResultContainer) but carries viewer sizing metadata and provides a dedicated ``to_container()`` that renders the file with the rerun web viewer. Usage in a ParametrizedSweep:: out_rerun = ResultRerun(width=600, height=600) def benchmark(self): rr.log("boxes", rr.Boxes2D(half_sizes=[self.theta, 1])) self.out_rerun = bn.capture_rerun_window(width=600, height=600) .. py:attribute:: __slots__ :value: ['width', 'height'] .. py:attribute:: _hash_exclude :value: ('width', 'height') .. py:attribute:: width :value: 600 .. py:attribute:: height :value: 600 .. py:method:: to_container() Return a callable that renders an .rrd file path as a rerun viewer pane. .. py:class:: ResultReference(obj: Any | None = None, container: Callable[Any, panel.pane.panel] | None = None, default: Any | None = None, units: str = 'container', max_time_events=None, **params) Bases: :py:obj:`param.Parameter` Use this class to save arbitrary objects that are not picklable or native to panel. You can pass a container callback that takes the object and returns a panel pane to be displayed .. py:attribute:: __slots__ :value: ['units', 'obj', 'container', 'max_time_events'] .. py:attribute:: _hash_exclude :value: ('obj', 'container', 'max_time_events') .. py:attribute:: units :value: 'container' .. py:attribute:: obj :value: None .. py:attribute:: container :value: None .. py:attribute:: max_time_events :value: None .. py:method:: hash_persistent() -> str A hash function that avoids the PYTHONHASHSEED 'feature' which returns a different hash value each time the program is run .. py:class:: ResultVolume(obj=None, default=None, units='container', max_time_events=None, **params) Bases: :py:obj:`param.Parameter` Base :class:`Parameter` type to hold any type of Python object. Parameters are a special kind of class attribute implemented as descriptor. Setting a Parameterized class attribute to a `Parameter` instance enables enhanced functionality, including type and range validation at assignment, support for constant and read-only parameters, documentation strings and dynamic parameter values. Parameters can only be used as class attributes of :class:`Parameterized` classes. Using them in standalone contexts or with non-:class:`Parameterized` classes will not provide the described behavior. .. rubric:: Notes Parameters provide lots of features. **Dynamic Behavior**: - Parameters provide support for dynamic values, type validation, and range checking. - Parameters can be declared as constant or read-only. **Automatic Initialization**: - Parameters can be set during object construction using keyword arguments. For example: ``myfoo = Foo(alpha=0.5); print(myfoo.alpha)``. - If custom constructors are implemented, they can still pass keyword arguments to the superclass to allow Parameter initialization. **Inheritance**: - :class:`Parameterized` classes automatically inherit parameters from their superclasses. Attributes can be selectively overridden. **Subclassing**: - The :class:`Parameter` class can be subclassed to create custom behavior, such as validating specific ranges or generating values dynamically. **GUI Integration**: - Parameters provide sufficient metadata for auto-generating property sheets in graphical user interfaces, enabling user-friendly parameter editing. .. rubric:: Examples Define a :class:`Parameterized` class with parameters: >>> import param >>> class Foo(param.Parameterized): ... alpha = param.Parameter(default=0.1, doc="The starting value.") ... beta = param.Parameter(default=0.5, doc="The standard deviation.", constant=True) When no initial value is provided the default is used: >>> Foo().alpha 0.1 When an initial value is provided it is used: >>> foo = Foo(alpha=0.5) >>> foo.alpha 0.5 Constant parameters cannot be modified: >>> foo.beta = 0.1 # Cannot be changed since it's constant ... TypeError: Constant parameter 'beta' cannot be modified .. py:attribute:: __slots__ :value: ['units', 'obj', 'max_time_events'] .. py:attribute:: _hash_exclude :value: ('obj', 'max_time_events') .. py:attribute:: units :value: 'container' .. py:attribute:: obj :value: None .. py:attribute:: max_time_events :value: None .. py:method:: hash_persistent() -> str A hash function that avoids the PYTHONHASHSEED 'feature' which returns a different hash value each time the program is run .. py:class:: OptDir Bases: :py:obj:`strenum.StrEnum` StrEnum is a Python ``enum.Enum`` that inherits from ``str``. The default ``auto()`` behavior uses the member name as its value. Example usage:: class Example(StrEnum): UPPER_CASE = auto() lower_case = auto() MixedCase = auto() assert Example.UPPER_CASE == "UPPER_CASE" assert Example.lower_case == "lower_case" assert Example.MixedCase == "MixedCase" .. py:attribute:: minimize .. py:attribute:: maximize .. py:attribute:: none .. py:class:: ResultDataSet(obj: Any | None = None, default: Any | None = None, units: str = 'dataset', max_time_events=None, **params) Bases: :py:obj:`param.Parameter` Base :class:`Parameter` type to hold any type of Python object. Parameters are a special kind of class attribute implemented as descriptor. Setting a Parameterized class attribute to a `Parameter` instance enables enhanced functionality, including type and range validation at assignment, support for constant and read-only parameters, documentation strings and dynamic parameter values. Parameters can only be used as class attributes of :class:`Parameterized` classes. Using them in standalone contexts or with non-:class:`Parameterized` classes will not provide the described behavior. .. rubric:: Notes Parameters provide lots of features. **Dynamic Behavior**: - Parameters provide support for dynamic values, type validation, and range checking. - Parameters can be declared as constant or read-only. **Automatic Initialization**: - Parameters can be set during object construction using keyword arguments. For example: ``myfoo = Foo(alpha=0.5); print(myfoo.alpha)``. - If custom constructors are implemented, they can still pass keyword arguments to the superclass to allow Parameter initialization. **Inheritance**: - :class:`Parameterized` classes automatically inherit parameters from their superclasses. Attributes can be selectively overridden. **Subclassing**: - The :class:`Parameter` class can be subclassed to create custom behavior, such as validating specific ranges or generating values dynamically. **GUI Integration**: - Parameters provide sufficient metadata for auto-generating property sheets in graphical user interfaces, enabling user-friendly parameter editing. .. rubric:: Examples Define a :class:`Parameterized` class with parameters: >>> import param >>> class Foo(param.Parameterized): ... alpha = param.Parameter(default=0.1, doc="The starting value.") ... beta = param.Parameter(default=0.5, doc="The standard deviation.", constant=True) When no initial value is provided the default is used: >>> Foo().alpha 0.1 When an initial value is provided it is used: >>> foo = Foo(alpha=0.5) >>> foo.alpha 0.5 Constant parameters cannot be modified: >>> foo.beta = 0.1 # Cannot be changed since it's constant ... TypeError: Constant parameter 'beta' cannot be modified .. py:attribute:: __slots__ :value: ['units', 'obj', 'max_time_events'] .. py:attribute:: _hash_exclude :value: ('obj', 'max_time_events') .. py:attribute:: units :value: 'dataset' .. py:attribute:: obj :value: None .. py:attribute:: max_time_events :value: None .. py:method:: hash_persistent() -> str A hash function that avoids the PYTHONHASHSEED 'feature' which returns a different hash value each time the program is run .. py:function:: curve(x_vals: list[float], y_vals: list[float], x_name: str, y_name: str, label: str | None = None, **kwargs) -> holoviews.Curve .. py:class:: ComposeType Bases: :py:obj:`strenum.StrEnum` StrEnum is a Python ``enum.Enum`` that inherits from ``str``. The default ``auto()`` behavior uses the member name as its value. Example usage:: class Example(StrEnum): UPPER_CASE = auto() lower_case = auto() MixedCase = auto() assert Example.UPPER_CASE == "UPPER_CASE" assert Example.lower_case == "lower_case" assert Example.MixedCase == "MixedCase" .. py:attribute:: right .. py:attribute:: down .. py:attribute:: sequence .. py:attribute:: overlay .. py:method:: flip() .. py:method:: from_horizontal(horizontal: bool) :staticmethod: .. py:class:: ComposableContainerBase A base class for renderer backends. A composable renderer .. py:attribute:: compose_method :type: ComposeType .. py:attribute:: container :type: list[Any] :value: [] .. py:attribute:: label_len :type: int :value: 0 .. py:method:: label_formatter(var_name: str, var_value: int | float | str) -> str :staticmethod: Take a variable name and values and return a pretty version with approximate fixed width :param var_name: The name of the variable, usually a dimension :type var_name: str :param var_value: The value of the dimension :type var_value: int | float | str :returns: Pretty string representation with fixed width :rtype: str .. py:method:: append(obj: Any) -> None Add an object to the container. The relationship between the objects is defined by the ComposeType :param obj: Object to add to the container :type obj: Any .. py:method:: render() Return a representation of the container that can be composed with other render() results. This function can also be used to defer layout and rending options until all the information about the container content is known. You may need to override this method depending on the container. See composable_container_video as an example. :returns: Visual representation of the container that can be combined with other containers :rtype: Any .. py:class:: PaneLayout Bases: :py:obj:`strenum.StrEnum` Controls how multi-dimensional data is laid out in panel displays. grid: Use rows/columns for all dimensions (default, existing behavior) tabs: Use tabs for all outer dimensions, only the innermost uses grid tabs_and_grid: Use tabs for the outermost dimension, grid for inner dimensions .. py:attribute:: grid .. py:attribute:: tabs .. py:attribute:: tabs_and_grid .. py:method:: all() -> list[PaneLayout] :classmethod: Return all layout values. Use this instead of hard-coded name lists. .. py:class:: ComposableContainerVideo Bases: :py:obj:`bencher.results.composable_container.composable_container_base.ComposableContainerBase` A base class for renderer backends. A composable renderer .. py:method:: append(obj: moviepy.VideoClip | moviepy.ImageClip | str | numpy.ndarray) -> None Appends an image or video to the container :param obj: Any representation of an image or video :type obj: VideoClip | ImageClip | str | np.ndarray :raises RuntimeWarning: if file format is not recognised .. py:method:: calculate_duration(frames, render_cfg: RenderCfg) .. py:method:: render(render_cfg: RenderCfg | None = None, **kwargs) -> moviepy.CompositeVideoClip Composes the images/videos into a single image/video based on the type of compose method :param compose_method: optionally override the default compose type. Defaults to None. :type compose_method: ComposeType, optional :returns: A composite video clip containing the images/videos added via append() :rtype: CompositeVideoClip .. py:method:: to_video(render_args: RenderCfg | None = None) -> str Returns the composite video clip as a webm file path :returns: webm filepath :rtype: str .. py:method:: deep() .. py:method:: extend_clip(clip: moviepy.VideoClip, desired_duration: float) .. py:class:: RenderCfg Configuration class for video rendering options. This class controls how videos and images are composed and rendered together. It provides options for timing, layout, appearance, and labeling of the output. .. attribute:: compose_method Method to compose multiple clips (sequence, right, down, overlay). Defaults to ComposeType.sequence. :type: ComposeType .. attribute:: var_name Variable name for labeling. Defaults to None. :type: str, optional .. attribute:: var_value Variable value for labeling. Defaults to None. :type: str, optional .. attribute:: background_col RGB color for background. Defaults to white (255, 255, 255). :type: tuple[int, int, int] .. attribute:: duration Target duration for the composed video in seconds. Defaults to 10.0. :type: float .. attribute:: default_duration Fallback duration when duration is None. Defaults to 10.0. :type: float .. attribute:: duration_target If True, tries to match target duration while respecting frame duration constraints. If False, uses exact duration. Defaults to True. :type: bool .. attribute:: min_frame_duration Minimum duration for each frame in seconds. Defaults to 1/30. :type: float .. attribute:: max_frame_duration Maximum duration for each frame in seconds. Defaults to 2.0. :type: float .. attribute:: margin Margin size in pixels to add around clips. Defaults to 0. :type: int .. py:attribute:: compose_method :type: bencher.results.composable_container.composable_container_base.ComposeType .. py:attribute:: var_name :type: str | None :value: None .. py:attribute:: var_value :type: str | None :value: None .. py:attribute:: background_col :type: tuple[int, int, int] :value: (255, 255, 255) .. py:attribute:: duration :type: float :value: 10.0 .. py:attribute:: default_duration :type: float :value: 10.0 .. py:attribute:: duration_target :type: bool :value: True .. py:attribute:: min_frame_duration :type: float :value: 0.03333333333333333 .. py:attribute:: max_frame_duration :type: float :value: 2.0 .. py:attribute:: margin :type: int :value: 0 .. py:class:: ComposableContainerPanel Bases: :py:obj:`bencher.results.composable_container.composable_container_base.ComposableContainerBase` A base class for renderer backends. A composable renderer .. py:attribute:: name :type: str | None :value: None .. py:attribute:: var_name :type: str | None :value: None .. py:attribute:: var_value :type: str | None :value: None .. py:attribute:: width :type: int | None :value: None .. py:attribute:: background_col :type: str | None :value: None .. py:attribute:: horizontal :type: bool | None :value: None .. py:method:: __post_init__() -> None .. py:method:: append(obj) Add an object to the container. The relationship between the objects is defined by the ComposeType :param obj: Object to add to the container :type obj: Any .. py:method:: render() Return a representation of the container that can be composed with other render() results. This function can also be used to defer layout and rending options until all the information about the container content is known. You may need to override this method depending on the container. See composable_container_video as an example. :returns: Visual representation of the container that can be combined with other containers :rtype: Any .. py:class:: ComposableContainerDataset Bases: :py:obj:`bencher.results.composable_container.composable_container_base.ComposableContainerBase` A base class for renderer backends. A composable renderer .. py:attribute:: var_name :type: str | None :value: None .. py:attribute:: var_value :type: str | None :value: None .. py:method:: render(**kwargs) Return a representation of the container that can be composed with other render() results. This function can also be used to defer layout and rending options until all the information about the container content is known. You may need to override this method depending on the container. See composable_container_video as an example. :returns: Visual representation of the container that can be combined with other containers :rtype: Any .. py:class:: BoxWhiskerResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.holoview_results.distribution_result.distribution_result.DistributionResult` A class for creating box and whisker plots from benchmark results. Box and whisker plots are useful for visualizing the distribution of data, including the median, quartiles, and potential outliers. This class provides methods to generate these plots from benchmark data, particularly useful for comparing distributions across different categorical variables or between different repetitions of the same benchmark. Box plots show: - The median (middle line in the box) - The interquartile range (IQR) as a box (25th to 75th percentile) - Whiskers extending to the furthest data points within 1.5*IQR - Outliers as individual points beyond the whiskers .. py:method:: to_plot(result_var: param.Parameter | None = None, override: bool = True, **kwargs: Any) -> panel.panel | None Generates a box and whisker plot from benchmark data. This method applies filters to ensure the data is appropriate for a box plot and then passes the filtered data to to_boxplot_ds for rendering. :param result_var: The result variable to plot. If None, uses the default. :type result_var: Parameter | None :param override: Whether to override filter restrictions. Defaults to True. :type override: bool :param \*\*kwargs: Additional keyword arguments passed to the plot rendering. :type \*\*kwargs: Any :returns: A panel containing the box plot if data is appropriate, otherwise returns filter match results. :rtype: pn.panel | None .. py:method:: to_boxplot_ds(dataset: xarray.Dataset, result_var: param.Parameter, **kwargs: Any) -> holoviews.BoxWhisker Creates a box and whisker plot from the provided dataset. Given a filtered dataset, this method generates a box and whisker visualization showing the distribution of values for a result variable, potentially grouped by a categorical variable. :param dataset: The dataset containing benchmark results. :type dataset: xr.Dataset :param result_var: The result variable to plot. :type result_var: Parameter :param \*\*kwargs: Additional keyword arguments for plot customization such as: - box_fill_color: Color for the box - whisker_color: Color for the whiskers - outlier_color: Color for outlier points - line_width: Width of lines in the plot :type \*\*kwargs: Any :returns: A HoloViews BoxWhisker plot of the benchmark data. :rtype: hv.BoxWhisker .. py:class:: ViolinResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.holoview_results.distribution_result.distribution_result.DistributionResult` A class for creating violin plots from benchmark results. Violin plots combine aspects of box plots with kernel density plots, showing the distribution shape of the data. This class provides methods to generate these plots from benchmark data, which is particularly useful for visualizing the distribution of metrics across different configurations or repetitions. Violin plots display: - The full probability density of the data (the width of the "violin" at each point) - Summary statistics like median and interquartile ranges - The overall distribution shape, revealing features like multi-modality that box plots might miss .. py:method:: to_plot(result_var: param.Parameter | None = None, override: bool = True, **kwargs: Any) -> panel.panel | None Generates a violin plot from benchmark data. This method applies filters to ensure the data is appropriate for a violin plot and then passes the filtered data to to_violin_ds for rendering. :param result_var: The result variable to plot. If None, uses the default. :type result_var: Parameter | None :param override: Whether to override filter restrictions. Defaults to True. :type override: bool :param \*\*kwargs: Additional keyword arguments passed to the plot rendering. :type \*\*kwargs: Any :returns: A panel containing the violin plot if data is appropriate, otherwise returns filter match results. :rtype: pn.panel | None .. py:method:: to_violin_ds(dataset: xarray.Dataset, result_var: param.Parameter, **kwargs: Any) -> holoviews.Violin Creates a violin plot from the provided dataset. Given a filtered dataset, this method generates a violin plot visualization showing the distribution of values for a result variable, potentially grouped by a categorical variable. :param dataset: The dataset containing benchmark results. :type dataset: xr.Dataset :param result_var: The result variable to plot. :type result_var: Parameter :param \*\*kwargs: Additional keyword arguments for plot customization, such as: - violin_color: Color for the violin body - inner_color: Color for inner statistics markers - line_width: Width of outline lines - bandwidth: Controls the smoothness of the density estimate :type \*\*kwargs: Any :returns: A HoloViews Violin plot of the benchmark data. :rtype: hv.Violin .. py:class:: ScatterResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.holoview_results.holoview_result.HoloviewResult` A class for creating scatter plots from benchmark results. Scatter plots are useful for visualizing the distribution of individual data points and identifying patterns, clusters, or outliers. This class generates scatter plots that can be grouped by categorical variables. .. py:method:: to_plot(**kwargs) -> panel.panel | None Creates a scatter plot. See ``to_scatter`` for parameters. .. py:method:: to_scatter(result_var: param.Parameter | None = None, override: bool = True, **kwargs) -> panel.panel | None Creates a standard scatter plot from benchmark data. :param result_var: The result variable to plot. :type result_var: Parameter, optional :param override: Whether to override filter restrictions. Defaults to True. :type override: bool, optional :param \*\*kwargs: Additional keyword arguments passed to the scatter plot options. :returns: A panel containing the scatter plot, or filter match results. :rtype: pn.panel | None .. py:method:: _to_scatter_ds(dataset: xarray.Dataset, result_var: param.Parameter, **kwargs) -> panel.panel | None Creates a scatter plot from the provided dataset. :param dataset: The dataset containing benchmark results. :type dataset: xr.Dataset :param result_var: The result variable to plot. :type result_var: Parameter :param \*\*kwargs: Additional keyword arguments passed to the scatter plot. :returns: A scatter plot visualization. :rtype: pn.panel | None .. py:class:: ScatterJitterResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.holoview_results.distribution_result.distribution_result.DistributionResult` A class for creating scatter jitter plots from benchmark results. Scatter jitter plots display individual data points with slight random offsets to avoid overlapping, making it easier to visualize the distribution of data. This is particularly useful for smaller datasets where showing individual points provides more insight than aggregate statistics, or alongside box plots to show the actual data distribution. Key features: - Displays individual data points rather than statistical summaries - Applies controlled random offsets to avoid point overlap - Useful for revealing the actual sample size and distribution - Complements statistical plots like box plots or violin plots .. py:method:: to_plot(result_var: param.Parameter | None = None, override: bool = True, jitter: float = 0.1, target_dimension: int | None = None, **kwargs: Any) -> panel.panel | None Generates a scatter jitter plot from benchmark data. This method applies filters to ensure the data is appropriate for a scatter plot and then passes the filtered data to to_scatter_jitter_ds for rendering. :param result_var: The result variable to plot. If None, uses the default. :param override: Whether to override filter restrictions. Defaults to True. :param jitter: Amount of jitter to apply to points. Defaults to 0.1. :param \*\*kwargs: Additional keyword arguments passed to the plot rendering. :returns: A panel containing the scatter jitter plot if data is appropriate, otherwise returns filter match results. .. py:method:: to_scatter_jitter_ds(dataset: xarray.Dataset, result_var: param.Parameter, jitter: float = 0.1, **kwargs: Any) -> holoviews.Scatter Creates a scatter jitter plot from the provided dataset. Given a filtered dataset, this method generates a scatter visualization showing individual data points with random jitter to avoid overlapping, making the distribution of values more visible. :param dataset: The dataset containing benchmark results. :param result_var: The result variable to plot. :param jitter: Amount of jitter to apply to points. Defaults to 0.1. :param \*\*kwargs: Additional keyword arguments for plot customization, such as: - color: Color for data points - size: Size of data points - alpha: Transparency of data points - marker: Shape of data points ('o', 's', 'd', etc.) :returns: A HoloViews Scatter plot of the benchmark data with jittered points. .. py:class:: BarResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.holoview_results.holoview_result.HoloviewResult` A class for creating bar chart visualizations from benchmark results. Bar charts are effective for comparing values across categorical variables or discrete data points. This class provides methods to generate bar charts that display benchmark results, particularly useful for comparing performance metrics between different configurations or categories. .. py:method:: to_plot(result_var: param.Parameter | None = None, override: bool = True, **kwargs) -> panel.panel | None .. py:method:: to_bar(result_var: param.Parameter | None = None, override: bool = True, target_dimension: int = 2, **kwargs) -> panel.panel | None Generates a bar chart from benchmark data. This method applies filters to ensure the data is appropriate for a bar chart and then passes the filtered data to to_bar_ds for rendering. :param result_var: The result variable to plot. If None, uses the default. :type result_var: Parameter, optional :param override: Whether to override filter restrictions. Defaults to True. :type override: bool, optional :param target_dimension: The target dimensionality for data filtering. Defaults to 2. :type target_dimension: int, optional :param \*\*kwargs: Additional keyword arguments passed to the plot rendering. :returns: A panel containing the bar chart if data is appropriate, otherwise returns filter match results. :rtype: pn.panel | None .. py:method:: to_bar_ds(dataset: xarray.Dataset, result_var: param.Parameter | None = None, **kwargs) Creates a bar chart from the provided dataset. Given a filtered dataset, this method generates a bar chart visualization showing values of the result variable, potentially grouped by categorical variables. :param dataset: The dataset containing benchmark results. :type dataset: xr.Dataset :param result_var: The result variable to plot. If None, uses the default. :type result_var: Parameter, optional :param \*\*kwargs: Additional keyword arguments passed to the bar chart options. :returns: A bar chart visualization of the benchmark data. :rtype: hvplot.element.Bars | hv.HoloMap .. py:class:: LineResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.holoview_results.holoview_result.HoloviewResult` A class for creating line plot visualizations from benchmark results. Line plots are effective for visualizing trends in data over a continuous variable. This class provides methods to generate interactive line plots from benchmark data, with options for adding interactive tap functionality to display detailed information about specific data points. .. py:method:: to_plot(**kwargs) -> panel.panel | None Generates a line plot. See ``to_line`` for parameters. .. py:method:: to_line(result_var: param.Parameter | None = None, tap_var=None, tap_container: panel.pane.panel = None, target_dimension=2, override: bool = True, use_tap: bool = _USE_TAP, **kwargs) -> panel.panel | None Generates a line plot from benchmark data. :param result_var: The result variable to plot. :type result_var: Parameter, optional :param tap_var: Variables to display when tapping on line plot points. :param tap_container: Container to hold tapped information. :type tap_container: pn.pane.panel, optional :param target_dimension: Target dimensionality for the plot. Defaults to 2. :type target_dimension: int, optional :param override: Whether to override filter restrictions. Defaults to True. :type override: bool, optional :param use_tap: Whether to enable tap functionality. :type use_tap: bool, optional :param \*\*kwargs: Additional keyword arguments passed to the plot rendering. :returns: A panel containing the line plot, or filter match results. :rtype: pn.panel | None .. py:method:: to_line_ds(dataset: xarray.Dataset, result_var: param.Parameter, **kwargs) Creates a basic line plot from the provided dataset. When over_time is active with multiple time points, creates an hv.HoloMap with a slider. :param dataset: The dataset containing benchmark results. :type dataset: xr.Dataset :param result_var: The result variable to plot. :type result_var: Parameter :param \*\*kwargs: Additional keyword arguments passed to the line plot options. :returns: A line plot visualization. :rtype: hvplot.element.Curve | pn.Column .. py:method:: _to_line_tap_ds(dataset: xarray.Dataset, result_var: param.Parameter, result_var_plots: list[param.Parameter] | None = None, container: panel.pane.panel = pn.pane.panel, **kwargs) -> panel.Row Creates an interactive line plot with tap functionality. :param dataset: The dataset containing benchmark results. :type dataset: xr.Dataset :param result_var: The primary result variable to plot. :type result_var: Parameter :param result_var_plots: Additional result variables to display when a point is tapped. :type result_var_plots: list[Parameter], optional :param container: Container to display tapped information. :type container: pn.pane.panel, optional :param \*\*kwargs: Additional keyword arguments passed to the line plot options. :returns: A panel row containing the interactive line plot and tap info. :rtype: pn.Row .. py:class:: CurveResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.holoview_results.holoview_result.HoloviewResult` A class for creating curve plots with optional standard-deviation spread. Curve plots show the relationship between a continuous input variable and a result variable. When multiple benchmark repetitions are available, standard deviation bounds are displayed using an ``hv.Spread`` overlay. .. py:method:: to_plot(**kwargs) -> holoviews.Curve | None Generates a curve plot. See ``to_curve`` for parameters. .. py:method:: to_curve(result_var: param.Parameter | None = None, override: bool = True, **kwargs) -> holoviews.Curve | None Generates a curve plot from benchmark data. :param result_var: The result variable to plot. :type result_var: Parameter, optional :param override: Whether to override filter restrictions. Defaults to True. :type override: bool, optional :param \*\*kwargs: Additional keyword arguments passed to the plot rendering. :returns: A curve plot, or filter match results. :rtype: hv.Curve | None .. py:method:: to_curve_ds(dataset: xarray.Dataset, result_var: param.Parameter, **kwargs) -> holoviews.Curve | None Creates a curve plot from the provided dataset. Generates a curve with optional standard deviation spread overlay. When over_time is active with multiple time points, builds per-time-point curves inside an hv.HoloMap so the slider controls the time dimension. :param dataset: The dataset containing benchmark results. :type dataset: xr.Dataset :param result_var: The result variable to plot. :type result_var: Parameter :param \*\*kwargs: Additional keyword arguments passed to the curve plot options. :returns: A curve plot with optional standard deviation spread. :rtype: hv.Curve | None .. py:class:: HeatmapResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.holoview_results.holoview_result.HoloviewResult` A class for creating heatmap visualizations from benchmark results. Heatmaps are effective for visualizing the relationship between two input variables and a result variable by using color intensity to represent the result values. This class provides methods for generating interactive heatmaps that can display additional information when hovering over or selecting points on the heatmap. .. py:method:: to_plot(**kwargs) -> panel.panel | None Generates a heatmap visualization. See ``to_heatmap`` for parameters. .. py:method:: to_heatmap(result_var: param.Parameter | None = None, tap_var=None, tap_container: panel.pane.panel = None, tap_container_direction: panel.Column | panel.Row | None = None, target_dimension=2, override: bool = True, use_tap: bool = _USE_TAP, **kwargs) -> panel.panel | None Generates a heatmap visualization from benchmark data. :param result_var: The result variable to plot. :type result_var: Parameter, optional :param tap_var: Variables to display when tapping on heatmap points. :param tap_container: Container to hold tapped information. :type tap_container: pn.pane.panel, optional :param tap_container_direction: Layout direction for tap containers. :type tap_container_direction: pn.Column | pn.Row, optional :param target_dimension: Target dimensionality. Defaults to 2. :type target_dimension: int, optional :param override: Whether to override filter restrictions. Defaults to True. :type override: bool, optional :param use_tap: Whether to enable tap functionality. :type use_tap: bool, optional :param \*\*kwargs: Additional keyword arguments passed to the plot rendering. :returns: A panel containing the heatmap, or filter match results. :rtype: pn.panel | None .. py:method:: _pick_xy_axes() -> tuple[str, str] Pick x/y axis names, preferring float vars then falling back to cat vars. .. py:method:: to_heatmap_ds(dataset: xarray.Dataset, result_var: param.Parameter, **kwargs) -> holoviews.HeatMap | holoviews.HoloMap | None Creates a basic heatmap from the provided dataset. When over_time is active with multiple time points, creates an hv.HoloMap with a slider. :param dataset: The dataset containing benchmark results. :type dataset: xr.Dataset :param result_var: The result variable to plot. :type result_var: Parameter :param \*\*kwargs: Additional keyword arguments passed to the heatmap options. :returns: A heatmap visualization, or None if the dataset has fewer than 2 dimensions. :rtype: hv.HeatMap | hv.HoloMap | None .. py:method:: _to_heatmap_tap_ds(dataset: xarray.Dataset, result_var: param.Parameter, result_var_plots: list[param.Parameter] | None = None, container: panel.pane.panel = None, tap_container_direction: panel.Column | panel.Row | None = None, **kwargs) -> panel.Row Creates an interactive heatmap with tap functionality. :param dataset: The dataset containing benchmark results. :type dataset: xr.Dataset :param result_var: The primary result variable to plot. :type result_var: Parameter :param result_var_plots: Additional result variables to display when a point is tapped. :type result_var_plots: list[Parameter], optional :param container: Container to display tapped information. :type container: pn.pane.panel, optional :param tap_container_direction: Layout direction for tap containers. :type tap_container_direction: pn.Column | pn.Row, optional :param \*\*kwargs: Additional keyword arguments passed to the heatmap options. :returns: A panel row containing the interactive heatmap and tap info. :rtype: pn.Row .. py:method:: to_heatmap_tap(result_var: param.Parameter, reduce: bencher.results.bench_result_base.ReduceType = ReduceType.AUTO, width=800, height=800, **kwargs) Creates a tappable heatmap that shows details when tapped. Uses ``hv.streams.Tap`` for static click coordinates rather than PointerXY hover tracking. :param result_var: The result variable to plot. :type result_var: Parameter :param reduce: How to reduce the data. Defaults to ReduceType.AUTO. :type reduce: ReduceType, optional :param width: Width of the plot in pixels. Defaults to 800. :type width: int, optional :param height: Height of the plot in pixels. Defaults to 800. :type height: int, optional :param \*\*kwargs: Additional keyword arguments. :returns: A layout containing the heatmap and a dynamically updated detail view. :rtype: hv.Layout .. py:class:: BandResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.holoview_results.holoview_result.HoloviewResult` Percentile band plot showing distribution spread over a continuous axis. Displays nested shaded bands (e.g. 10th-90th and 25th-75th percentiles) with a median line and individual scatter points, giving a richer view of the distribution than mean +/- std. Particularly useful with ``agg_over_dims`` to show how a high-dimensional sweep's distribution evolves over time. .. py:method:: to_plot(result_var: param.Parameter | None = None, override: bool = True, **kwargs) -> holoviews.Overlay | None .. py:method:: to_band(result_var: param.Parameter | None = None, override: bool = True, **kwargs) .. py:method:: to_band_ds(dataset: xarray.Dataset, result_var: param.Parameter, **kwargs) -> holoviews.Overlay | None Create a percentile band plot from the provided dataset. Flattens all dimensions except the continuous axis (typically over_time) into a sample pool and computes percentiles across those samples. .. py:method:: _band_over_time(dataset: xarray.Dataset, var: str, title: str | None, units: str = '', **kwargs) -> holoviews.Overlay | None Build percentile bands with time on x-axis. .. py:method:: _band_static(dataset: xarray.Dataset, var: str, title: str | None, agg_over_dims: list[str] | None, units: str = '', **kwargs) -> holoviews.Overlay | None Build percentile bands over a non-time continuous axis. .. py:method:: _build_scatter_data(x_coords, values, **kwargs) -> tuple[numpy.ndarray | None, numpy.ndarray | None] :staticmethod: Build scatter arrays from the 2-D values grid, with optional downsampling. :param x_coords: 1-D array of x-axis coordinates. :param values: 2-D array of shape (n_x, n_samples). :param \*\*kwargs: Optional ``max_scatter_points`` (int, default 50_000) to cap the number of scatter points, and ``enable_scatter`` (bool, default True) to disable the scatter layer entirely. :returns: (scatter_x, scatter_y) arrays, or (None, None) when scatter is disabled. .. py:method:: _build_band_overlay(x_coords, p10, p25, p50, p75, p90, scatter_x, scatter_y, var: str, title: str, x_dim: str = 'x', units: str = '', **_kwargs) -> holoviews.Overlay :staticmethod: Construct the overlay of Area bands + median Curve + scatter points. :param x_dim: Name of the x-axis dimension, used as the kdim label so that axis labels reflect the original coordinate (e.g. ``over_time``). .. py:class:: SurfaceResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.holoview_results.holoview_result.HoloviewResult` A class for creating 3D surface plots from benchmark results. This class provides methods to visualize benchmark data as 3D surface plots, which are useful for showing relationships between two input variables and a result variable. Surface plots can also display standard deviation bounds when benchmark runs include multiple repetitions. .. py:method:: to_plot(result_var: param.Parameter | None = None, override: bool = True, **kwargs) -> panel.pane.Pane | None Generates a 3D surface plot from benchmark data. This is a convenience method that calls to_surface() with the same parameters. :param result_var: The result variable to plot. If None, uses the default. :type result_var: Parameter, optional :param override: Whether to override filter restrictions. Defaults to True. :type override: bool, optional :param \*\*kwargs: Additional keyword arguments passed to the plot rendering. :returns: A panel containing the surface plot if data is appropriate, otherwise returns filter match results. :rtype: pn.pane.Pane | None .. py:method:: to_surface(result_var: param.Parameter | None = None, override: bool = True, target_dimension: int = 2, **kwargs) -> panel.pane.Pane | None Generates a 3D surface plot from benchmark data. This method applies filters to ensure the data is appropriate for a surface plot and then passes the filtered data to to_surface_ds for rendering. :param result_var: The result variable to plot. If None, uses the default. :type result_var: Parameter, optional :param override: Whether to override filter restrictions. Defaults to True. :type override: bool, optional :param target_dimension: The target dimensionality for data filtering. Defaults to 2. :type target_dimension: int, optional :param \*\*kwargs: Additional keyword arguments passed to the plot rendering. :returns: A panel containing the surface plot if data is appropriate, otherwise returns filter match results. :rtype: pn.pane.Pane | None .. py:method:: to_surface_ds(dataset: xarray.Dataset, result_var: param.Parameter, override: bool = True, alpha: float = 0.3, width: int = 600, height: int = 600) -> panel.panel | None Creates a 3D surface plot from the provided dataset. Uses plotly directly (like VolumeResult) to avoid HoloViews backend contamination issues while ensuring reliable 3D rendering. Coordinates are sorted to guarantee monotonic x/y grids for plotly. :param dataset: The dataset containing benchmark results. :type dataset: xr.Dataset :param result_var: The result variable to plot. :type result_var: Parameter :param override: Whether to override filter restrictions. Defaults to True. :type override: bool, optional :param alpha: The transparency for std-dev surfaces. Defaults to 0.3. :type alpha: float, optional :param width: Plot width in pixels. Defaults to 600. :type width: int, optional :param height: Plot height in pixels. Defaults to 600. :type height: int, optional :returns: A panel containing the surface plot if data matches criteria, otherwise returns filter match results. :rtype: pn.panel | None .. py:class:: TabulatorResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.holoview_results.holoview_result.HoloviewResult` .. py:method:: to_plot(**kwargs) -> panel.widgets.Tabulator Create an interactive table visualization of the data. Passes the data to the panel Tabulator type to display an interactive table. See https://panel.holoviz.org/reference/widgets/Tabulator.html for extra options. :param \*\*kwargs: Additional parameters to pass to the Tabulator constructor. :returns: An interactive table widget. :rtype: pn.widgets.Tabulator .. py:method:: to_tabulator(result_var: param.Parameter | None = None, **kwargs) -> panel.widgets.Tabulator Generates a Tabulator widget from benchmark data. This is a convenience method that calls to_tabulator_ds() with the same parameters. :param result_var: The result variable to include in the table. If None, uses the default. :type result_var: Parameter, optional :param \*\*kwargs: Additional keyword arguments passed to the Tabulator constructor. :returns: An interactive table widget. :rtype: pn.widgets.Tabulator .. py:method:: to_tabulator_ds(dataset: xarray.Dataset, result_var: param.Parameter, **kwargs) -> panel.widgets.Tabulator Creates a Tabulator widget from the provided dataset. Given a filtered dataset, this method generates an interactive table visualization. :param dataset: The filtered dataset to visualize. :type dataset: xr.Dataset :param result_var: The result variable to include in the table. :type result_var: Parameter :param \*\*kwargs: Additional keyword arguments passed to the Tabulator constructor. :returns: An interactive table widget. :rtype: pn.widgets.Tabulator .. py:class:: TableResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.holoview_results.holoview_result.HoloviewResult` .. py:method:: to_plot(**kwargs) -> holoviews.Table Convert the dataset to a Table visualization. :returns: A HoloViews Table object. :rtype: hv.Table .. py:class:: VolumeResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.bench_result_base.BenchResultBase` .. py:method:: to_plot(result_var: param.Parameter | None = None, override: bool = True, **kwargs: Any) -> panel.panel | None Generates a 3d volume plot from benchmark data. :param result_var: The result variable to plot. If None, uses the default. :type result_var: Parameter | None :param override: Whether to override filter restrictions. Defaults to True. :type override: bool :param \*\*kwargs: Additional keyword arguments passed to the plot rendering. :type \*\*kwargs: Any :returns: A panel containing the volume plot if data is appropriate, otherwise returns filter match results. :rtype: pn.panel | None .. py:method:: to_volume(result_var: param.Parameter | None = None, override: bool = True, target_dimension: int = 3, **kwargs) Generates a 3D volume plot from benchmark data. This method applies filters to ensure the data is appropriate for a volume plot and then passes the filtered data to to_volume_ds for rendering. :param result_var: The result variable to plot. If None, uses the default. :type result_var: Parameter, optional :param override: Whether to override filter restrictions. Defaults to True. :type override: bool, optional :param target_dimension: The target dimensionality for data filtering. Defaults to 3. :type target_dimension: int, optional :param \*\*kwargs: Additional keyword arguments passed to the plot rendering. :returns: A panel containing the volume plot if data is appropriate, otherwise returns filter match results. :rtype: pn.pane.Plotly | None .. py:method:: to_volume_ds(dataset: xarray.Dataset, result_var: param.Parameter, width=600, height=600) -> panel.pane.Plotly | None Given a benchCfg generate a 3D surface plot :returns: A 3d volume plot as a holoview in a pane :rtype: pn.pane.Plotly .. py:class:: HistogramResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.holoview_results.holoview_result.HoloviewResult` .. py:method:: to_plot(result_var: param.Parameter | None = None, target_dimension: int = 2, **kwargs) -> panel.pane.Pane | None Generates a histogram plot from benchmark data. This method applies filters to ensure the data is appropriate for a histogram and then passes the filtered data to to_histogram_ds for rendering. :param result_var: The result variable to plot. If None, uses the default. :type result_var: Parameter, optional :param target_dimension: The target dimensionality for data filtering. Defaults to 2. :type target_dimension: int, optional :param \*\*kwargs: Additional keyword arguments passed to the plot rendering. :returns: A panel containing the histogram if data is appropriate, otherwise returns filter match results. :rtype: pn.pane.Pane | None .. py:method:: _make_histogram(dataset: xarray.Dataset, result_var: param.Parameter, **kwargs) Render a single histogram from a dataset (no over_time handling). .. py:method:: to_histogram_ds(dataset: xarray.Dataset, result_var: param.Parameter, **kwargs) Creates a histogram from the provided dataset. Given a filtered dataset, this method generates a histogram visualization showing the distribution of values for the result variable. When over_time is active with multiple time points, produces per-time-point and pooled-aggregate tabs. :param dataset: The dataset containing benchmark results. :type dataset: xr.Dataset :param result_var: The result variable to plot in the histogram. :type result_var: Parameter :param \*\*kwargs: Additional keyword arguments passed to the histogram plot options. :returns: A histogram visualization of the benchmark data distribution. :rtype: hvplot.element.Histogram .. py:class:: ExplorerResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.pane_result.PaneResult` .. py:method:: to_plot(**kwargs) -> panel.pane.Pane Produces a hvplot explorer instance to explore the generated dataset see: https://hvplot.holoviz.org/getting_started/explorer.html :returns: A dynamic pane for exploring a dataset :rtype: pn.pane.Pane .. py:class:: DataSetResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.bench_result_base.BenchResultBase` .. py:method:: to_plot(result_var: param.Parameter | None = None, hv_dataset=None, target_dimension: int = 0, container=None, subsampling_divisions: int | None = None, **kwargs) -> panel.pane.panel | None .. py:function:: hmap_canonical_input(dic: dict) -> tuple From a dictionary of kwargs, return a hashable representation (tuple) that is always the same for the same inputs and retains the order of the input arguments. e.g, {x=1,y=2} -> (1,2) and {y=2,x=1} -> (1,2). This is used so that keywords arguments can be hashed and converted the the tuple keys that are used for holomaps :param dic: dictionary with keyword arguments and values in any order :type dic: dict :returns: values of the dictionary always in the same order and hashable :rtype: tuple .. py:function:: get_nearest_coords(dataset: xarray.Dataset, collapse_list: bool = False, **kwargs) -> dict Find the nearest coordinates in an xarray dataset based on provided coordinate values. Given an xarray dataset and kwargs of key-value pairs of coordinate values, return a dictionary of the nearest coordinate name-value pair that was found in the dataset. :param dataset: The xarray dataset to search in :type dataset: xr.Dataset :param collapse_list: If True, when a coordinate value is a list, only the first item is returned. Defaults to False. :type collapse_list: bool, optional :param \*\*kwargs: Key-value pairs where keys are coordinate names and values are points to find the nearest match for :returns: Dictionary of coordinate name-value pairs with the nearest values found in the dataset :rtype: dict .. py:function:: make_namedtuple(class_name: str, **fields) -> collections.namedtuple Convenience method for making a named tuple :param class_name: name of the named tuple :type class_name: str :returns: a named tuple with the fields as values :rtype: namedtuple .. py:function:: gen_path(filename: str, folder: str = 'generic', suffix: str = '.dat') -> str Generate a path for a file in the cache directory. When called inside a benchmark sweep, files are placed in a per-job-key subdirectory so that cache overwrites can cleanly delete old media. Outside a sweep, falls back to UUID-based naming. :param filename: Base name for the file :type filename: str :param folder: Subfolder within cachedir. Defaults to "generic". :type folder: str, optional :param suffix: File extension. Defaults to ".dat". :type suffix: str, optional :returns: Absolute path to a file location :rtype: str .. py:function:: gen_image_path(image_name: str = 'img', filetype: str = '.png') -> str Generate a unique path for an image file in the cache directory. :param image_name: Base name for the image file. Defaults to "img". :type image_name: str, optional :param filetype: Image file extension. Defaults to ".png". :type filetype: str, optional :returns: Absolute path to a unique image file location :rtype: str .. py:function:: gen_video_path(video_name: str = 'vid', extension: str = '.mp4') -> str Generate a unique path for a video file in the cache directory. :param video_name: Base name for the video file. Defaults to "vid". :type video_name: str, optional :param extension: Video file extension. Defaults to ".mp4". :type extension: str, optional :returns: Absolute path to a unique video file location :rtype: str .. py:function:: gen_rerun_data_path(rrd_name: str = 'rrd', filetype: str = '.rrd') -> str Generate a unique path for a rerun data file in the cache directory. :param rrd_name: Base name for the rerun data file. Defaults to "rrd". :type rrd_name: str, optional :param filetype: File extension. Defaults to ".rrd". :type filetype: str, optional :returns: Absolute path to a unique rerun data file location :rtype: str .. py:function:: lerp(value: float, input_low: float, input_high: float, output_low: float, output_high: float) -> float Linear interpolation between two ranges. Maps a value from one range [input_low, input_high] to another range [output_low, output_high]. :param value: The input value to interpolate :type value: float :param input_low: The lower bound of the input range :type input_low: float :param input_high: The upper bound of the input range :type input_high: float :param output_low: The lower bound of the output range :type output_low: float :param output_high: The upper bound of the output range :type output_high: float :returns: The interpolated value in the output range :rtype: float .. py:function:: tabs_in_markdown(regular_str: str, spaces: int = 2) -> str Given a string with tabs in the form convert the to   which is a double space in markdown :param regular_str: A string with tabs in it :type regular_str: str :param spaces: the number of spaces per tab :type spaces: int :returns: A string with sets of   to represent the tabs in markdown :rtype: str .. py:function:: publish_file(filepath: str, remote: str, branch_name: str) -> str Publish a file to an orphan git branch: .. code-block:: python def publish_args(branch_name) -> tuple[str, str]: return ( "https://github.com/blooop/bencher.git", f"https://github.com/blooop/bencher/blob/{branch_name}") :param remote: A function the returns a tuple of the publishing urls. It must follow the signature def publish_args(branch_name) -> tuple[str, str]. The first url is the git repo name, the second url needs to match the format for viewable html pages on your git provider. The second url can use the argument branch_name to point to the file on a specified branch. :type remote: Callable :returns: the url of the published file :rtype: str .. py:function:: github_content(remote: str, branch_name: str, filename: str) .. py:function:: publish_and_view_rrd(file_path: str, remote: str, branch_name: str, content_callback: callable, version: str | None = None) .. py:function:: rrd_to_pane(url: str, width: int = 500, height: int = 600, version: str | None = None) Display an .rrd file from a URL using the hosted rerun web viewer. .. py:function:: rrd_file_to_pane(file_path, width: int = 300, height: int = 300, viewer_version: str | None = None, report_dir: str | pathlib.Path | None = None) Create a rerun viewer pane from an .rrd file path. Uses an HTML iframe to display the .rrd file. By default the viewer is loaded from the ``@rerun-io/web-viewer`` CDN at the installed ``rerun-sdk`` version. The viewer page and the ``.rrd`` data are both served from the Panel server's ``/rrd_static/`` route, keeping everything on a single origin (no CORS, no extra ports). The file must be located under ``cachedir/rrd/``. :param file_path: Path to the .rrd file (must be under ``cachedir/rrd/``). :param width: Width of the viewer iframe in pixels. :param height: Height of the viewer iframe in pixels. :param viewer_version: Rerun web-viewer version to load from CDN. Defaults to the installed ``rerun-sdk`` version. Set explicitly when the .rrd was recorded with a different SDK version. :param report_dir: When set, copies the .rrd and viewer HTML into this directory and uses relative URLs in the iframe. This makes the report portable — it works when served from any HTTP origin without a live Panel server. .. py:function:: run_file_server(directory=None, port=8001) Start a background HTTP file server (daemon thread). If *port* is already in use the existing server is assumed to be running and the function returns ``None`` instead of raising. Returns the ``ThreadingHTTPServer`` so callers can query ``server.server_address[1]`` for the actual port (useful when *port=0*). .. py:class:: RegressionResult Result of regression detection for a single variable. .. py:attribute:: variable :type: str .. py:attribute:: method :type: str .. py:attribute:: regressed :type: bool .. py:attribute:: current_value :type: float .. py:attribute:: baseline_value :type: float .. py:attribute:: change_percent :type: float .. py:attribute:: threshold :type: float .. py:attribute:: direction :type: str .. py:attribute:: details :type: str .. py:attribute:: band_lower :type: float | None :value: None .. py:attribute:: band_upper :type: float | None :value: None .. py:attribute:: percent_band_lower :type: float | None :value: None .. py:attribute:: percent_band_upper :type: float | None :value: None .. py:attribute:: historical :type: numpy.ndarray | None :value: None .. py:attribute:: current_samples :type: numpy.ndarray | None :value: None .. py:attribute:: historical_all :type: numpy.ndarray | None :value: None .. py:attribute:: historical_all_x :type: numpy.ndarray | None :value: None .. py:attribute:: historical_x :type: numpy.ndarray | None :value: None .. py:attribute:: current_x :type: numpy.ndarray | None :value: None .. py:method:: render_png(historical: numpy.ndarray | None = None, current: numpy.ndarray | float | None = None, path: str | pathlib.Path | None = None, figsize: tuple[float, float] = (8.0, 5.0), dpi: int = 100) -> str Render this result as a diagnostic PNG (see :func:`render_regression_png`). .. py:method:: render_overlay(historical: numpy.ndarray | None = None, current: numpy.ndarray | float | None = None) Build a :class:`holoviews.Overlay` of this result (see :func:`build_regression_overlay`). .. py:class:: RegressionReport Aggregates regression results for all variables in a benchmark. .. py:attribute:: results :type: list[RegressionResult] :value: [] .. py:property:: has_regressions :type: bool .. py:property:: regressed_variables :type: list[RegressionResult] .. py:method:: summary() -> str .. py:method:: to_markdown() -> str Return a nicely formatted Markdown summary of all regression results. .. py:method:: append_to_report(report) -> None Append a formatted regression summary to a :class:`BenchReport`. .. py:method:: prepend_to_result(report, bench_res) -> None Insert a formatted regression summary at the top of *bench_res*'s tab. .. py:exception:: RegressionError Bases: :py:obj:`Exception` Raised when regression detection finds regressions and regression_fail is True. .. py:class:: MethodCells Per-method rendering of a single regression result. Each detector has a different gate — percent ratio, MAD-sigma, absolute delta, hard limit — so the report cells must describe it in its own units. This bundle is the single source of truth consumed by both the built-in text summary and the markdown table, and is exposed as public API so downstream report builders can produce their own layouts (custom columns, non-markdown output, templated HTML, GitHub PR comments with status decoration, etc.) without reimplementing method dispatch and drifting when new detection methods are added. Example — building a minimal custom row from a RegressionResult:: from bencher import method_cells cells = method_cells(result) row = f"{result.variable}: {cells.change} (gate {cells.threshold})" .. attribute:: change Change column (markdown) — gated quantity in its own units. .. attribute:: baseline Baseline column (markdown) — em-dash for absolute (no historical baseline exists). .. attribute:: threshold Threshold column (markdown) — carries the gate's native units (``±T%``, ``Tσ``, ``±T``, or a direction-aware inequality). .. attribute:: summary_lead First clause of the summary line, before the details parenthesis. Captures the gated quantity in sentence form. .. attribute:: summary_standalone When True, the summary line skips the ``(baseline=…, current=…, threshold=…)`` tail because ``summary_lead`` already contains the relevant values. Used by the absolute method (no baseline, limit is in the lead). .. py:attribute:: change :type: str .. py:attribute:: baseline :type: str .. py:attribute:: threshold :type: str .. py:attribute:: summary_lead :type: str .. py:attribute:: summary_standalone :type: bool :value: False .. py:function:: method_cells(r: RegressionResult) -> MethodCells Build the per-method cell bundle for a :class:`RegressionResult`. Returns a :class:`MethodCells` with pre-rendered display strings for the result's change, baseline, and threshold, plus the summary lead clause. Dispatches on ``r.method`` so each gate describes itself in its native units. Safe to call on any ``RegressionResult`` — unknown methods fall back to the percentage-style rendering. Intended for consumers that want to embed regression results in a custom layout while staying consistent with how the built-in :meth:`RegressionReport.summary` and :meth:`RegressionReport.to_markdown` present each method. Notes on the ``absolute`` branch: ``baseline_value`` and ``threshold`` both hold the limit for this detector (see :func:`detect_absolute`); the code reads from ``threshold`` to make the intent ("this is the gate value") explicit. .. py:class:: PerfTracker Context-manager based phase timer. Usage:: tracker = PerfTracker() with tracker.phase("setup"): do_setup() with tracker.phase("compute"): do_compute() report = tracker.report() .. py:attribute:: _phases :type: list[PhaseTime] :value: [] .. py:method:: phase(name: str) Time a block and record it as a named phase. .. py:method:: report() -> PerfReport Build a PerfReport from all recorded phases. .. py:method:: log_summary() -> None Log the report summary at INFO level. .. py:class:: PerfReport Collection of phase timings from a benchmark run. .. py:attribute:: phases :type: list[PhaseTime] :value: [] .. py:property:: total_s :type: float .. py:property:: total_ms :type: float .. py:method:: get_phase(name: str) -> PhaseTime | None Return the first phase matching *name*, or None. .. py:method:: summary() -> str Human-readable summary of all phases. .. py:method:: to_dict() -> dict Return phases as {name: duration_ms} plus a total. .. py:function:: git_time_event(repo_path: str | None = None) -> str Return a time-event label combining wall-clock time and short commit hash. Example return value: ``"2024-06-15 14:59 abc1234"`` The SHA portion is git's canonical abbreviated hash (``rev-parse --short``), which is typically 7 characters but may be longer in large repositories to avoid ambiguity. Intended to be used with ``BenchRunCfg(over_time=True, time_event=...)`` so the over-time slider shows *when* and *which commit* produced the data. Wall-clock time is used instead of commit time so that multiple benchmark runs on the same commit produce distinct labels. For fork-safety in multithreaded environments (ROS 2, DDS, etc.), call this at module level before starting background threads:: _TIME_EVENT = bn.git_time_event() # safe: no threads yet Falls back to ``" unknown"`` if not inside a git repository or git is unavailable, keeping the label format consistent. .. py:class:: VarRange(lower_bound: int = 0, upper_bound: int = -1) A VarRange represents the bounded and unbounded ranges of integers. This class is used to define filters for various variable types. For example by defining cat_var = VarRange(0,0), calling matches(0) will return true, but any other integer will not match. You can also have unbounded ranges for example VarRange(2,None) will match to 2,3,4... up to infinity. for By default the lower and upper bounds are set to -1 so so that no matter what value is passed to matches() will return false. Matches only takes 0 and positive integers. .. py:attribute:: lower_bound :value: 0 .. py:attribute:: upper_bound :value: -1 .. py:method:: matches(val: int) -> bool Checks that a value is within the variable range. lower_bound and upper_bound are inclusive (lower_bound<=val<=upper_bound ) :param val: A positive integer representing a number of items :type val: int :returns: True if the items is within the range, False otherwise. :rtype: bool :raises ValueError: If val < 0 .. py:method:: matches_info(val: int, name: str) -> tuple[bool, str] Get matching info for a value with a descriptive name. :param val: A positive integer to check against the range :type val: int :param name: A descriptive name for the value being checked, used in the output string :type name: str :returns: A tuple containing: - bool: True if the value matches the range, False otherwise - str: A formatted string describing the match result :rtype: tuple[bool, str] .. py:method:: __str__() -> str .. py:class:: PlotFilter A class for representing the types of results a plot is able to represent. .. py:attribute:: float_range :type: VarRange .. py:attribute:: cat_range :type: VarRange .. py:attribute:: vector_len :type: VarRange .. py:attribute:: result_vars :type: VarRange .. py:attribute:: panel_range :type: VarRange .. py:attribute:: repeats_range :type: VarRange .. py:attribute:: input_range :type: VarRange .. py:method:: matches_result(plt_cnt_cfg: bencher.plotting.plt_cnt_cfg.PltCntCfg, plot_name: str, override: bool) -> PlotMatchesResult Checks if the result data signature matches the type of data the plot is able to display. :param plt_cnt_cfg: Configuration containing counts of different plot elements :type plt_cnt_cfg: PltCntCfg :param plot_name: Name of the plot being checked :type plot_name: str :param override: Whether to override filter matching rules :type override: bool :returns: Object containing match results and information :rtype: PlotMatchesResult .. py:class:: ParametrizedSweep(**params) Bases: :py:obj:`param.Parameterized` Parent class for all Sweep types that need a custom hash .. py:method:: param_hash(param_type: param.Parameterized, hash_value: bool = True) -> int :staticmethod: A custom hash function for parametrised types with options for hashing the value of the type and hashing metadata :param param_type: A parameter :type param_type: Parameterized :param hash_value: use the value as part of the hash. Defaults to True. :type hash_value: bool, optional :param # hash_meta: use metadata as part of the hash. Defaults to False. :type # hash_meta: bool, optional :returns: a hash :rtype: int .. py:method:: hash_persistent() -> str A hash function that avoids the PYTHONHASHSEED 'feature' which returns a different hash value each time the program is run .. py:method:: update_params_from_kwargs(**kwargs) -> None Given a dictionary of kwargs, set the parameters of the passed class 'self' to the values in the dictionary. .. py:method:: get_input_and_results(include_name: bool = False) -> tuple[dict, dict] :classmethod: Get dictionaries of input parameters and result parameters :param cls: A parametrised class :param include_name: Include the name parameter that all parametrised classes have. Default False :type include_name: bool :returns: A tuple containing the inputs and result parameters as dictionaries :rtype: tuple[dict, dict] .. py:method:: get_inputs_as_dict() -> dict Get the key:value pairs for all the input variables .. py:method:: get_results_values_as_dict(holomap=None) -> dict Get a dictionary of result variables with the name and the current value .. py:method:: get_inputs_only() -> list[param.Parameter] :classmethod: Return a list of input parameters :returns: A list of input parameters :rtype: list[param.Parameter] .. py:method:: filter_fn(item, p_name) :staticmethod: .. py:method:: get_input_defaults(override_defaults: list | None = None) -> list[tuple[param.Parameter, Any]] :classmethod: .. py:method:: get_input_defaults_override(**kwargs) -> dict[str, Any] :classmethod: .. py:method:: get_results_only() -> list[param.Parameter] :classmethod: Return a list of result parameters :returns: A list of result parameters :rtype: list[param.Parameter] .. py:method:: get_inputs_as_dims(compute_values=False, remove_dims: str | list[str] | None = None) -> list[holoviews.Dimension] :classmethod: .. py:method:: to_dynamic_map(callback=None, name=None, remove_dims: str | list[str] | None = None, result_var: str | None = None) -> holoviews.DynamicMap .. py:method:: to_gui(result_var: str | None = None, **kwargs) .. py:method:: to_holomap(callback, remove_dims: str | list[str] | None = None) -> holoviews.DynamicMap .. py:method:: __call__(**kwargs) -> dict Dispatch to benchmark() if overridden, otherwise use legacy path. :returns: a dictionary with all the result variables as named key value pairs. :rtype: dict .. py:method:: benchmark() Override this with your benchmark logic. When called, all sweep parameters (self.x, etc.) are already set. Set result variables (self.result, etc.) directly on self. No need to call update_params_from_kwargs or super().__call__(). .. py:method:: plot_hmap(**kwargs) .. py:method:: to_bench(run_cfg=None, report=None, name=None) Create a Bench instance from this ParametrizedSweep. .. py:method:: to_optimize(n_trials=100, run_cfg=None, **kwargs) Create a Bench and run optimization in one call. :param n_trials: Number of optuna trials. :param run_cfg: Optional BenchRunCfg. :param \*\*kwargs: Forwarded to ``Bench.optimize()``. :returns: OptimizeResult wrapping the completed study. .. py:method:: to_bench_runner(run_cfg=None, name=None) Create a BenchRunner instance from this ParametrizedSweep. Enables fluent chaining like: MyConfig().to_bench_runner().add(func).run(subsampling_divisions=2, max_subsampling_divisions=4) .. py:class:: ParametrizedSweepSingleton(**params) Bases: :py:obj:`bencher.variables.parametrised_sweep.ParametrizedSweep` A minimal per-subclass singleton for ParametrizedSweep. - Repeated construction returns the same instance for each subclass. - Ensures the Parametrized ``__init__`` chain runs only once. - ``init_singleton()`` returns a result that is truthy once per subclass and doubles as a context manager for automatic rollback on failure. - ``reset_singleton()`` explicitly clears singleton state for a subclass. - Thread-safe: all shared state is protected by ``_lock``. .. py:attribute:: _instances .. py:attribute:: _seen .. py:attribute:: _lock .. py:attribute:: _singleton_inited :value: True .. py:method:: init_singleton() -> _SingletonInitResult :classmethod: Mark *cls* as seen and return a ``_SingletonInitResult``. The result is **truthy** the first time a subclass calls this and **falsy** on every subsequent call — identical to the previous boolean return value. It can also be used as a **context manager**:: with self.init_singleton() as is_first: if is_first: self._fallible_setup() If the ``with`` block raises during a first-time init, the singleton bookkeeping is rolled back so the next construction can retry cleanly. .. py:method:: reset_singleton() -> None :classmethod: Clear singleton state for *cls*, allowing re-initialisation. .. py:class:: SampleOrder Bases: :py:obj:`strenum.StrEnum` Controls the sampling traversal order for plot_sweep. - INORDER: Traverse inputs in the natural Cartesian product order (right-most dimension varies fastest). - REVERSED: Traverse the same set of samples in the reverse order. Note: This only affects sampling order, not plotting or dataset dimension order. .. py:attribute:: INORDER .. py:attribute:: REVERSED .. py:data:: DEFAULT_CACHE_SIZE_BYTES :value: 0 .. py:class:: CacheDirStats Statistics for a single cache or media directory. .. py:attribute:: path :type: str .. py:attribute:: entries :type: int .. py:attribute:: size_bytes :type: int .. py:attribute:: size_limit_bytes :type: int | None :value: None .. py:method:: summary_line() -> str .. py:class:: CacheStats Aggregate cache statistics. .. py:attribute:: managed :type: list[CacheDirStats] .. py:attribute:: media :type: list[CacheDirStats] .. py:attribute:: total_bytes :type: int .. py:method:: summary() -> str .. py:function:: cache_stats(cachedir: str = 'cachedir') -> CacheStats Collect statistics for all managed caches and media directories. .. py:function:: print_cache_stats(cachedir: str = 'cachedir') -> None Print a human-readable cache statistics summary. .. py:function:: clear_all(cachedir: str = 'cachedir') -> None Remove the entire cache directory tree. .. py:function:: clear_media(cachedir: str = 'cachedir') -> tuple[int, int] Delete all files in media directories. Returns (files_deleted, bytes_freed). .. py:function:: clean_orphaned_media(cachedir: str = 'cachedir', dry_run: bool = True) -> tuple[list[str], int] Find and optionally delete per-job-key media dirs with no cache entry. Walks the media tree looking for job-key subdirectories. If the key is not present in the sample cache, the directory is an orphan (its cache entry was evicted by LRU or cleared). :param cachedir: Root cache directory. :param dry_run: If True, only report orphans without deleting. :returns: (orphan_dirs, total_bytes) — list of orphaned directory paths and their combined size. .. py:function:: cleanup_job_media(job_key: str, cachedir: str = 'cachedir') -> int Delete the per-job-key media directories for *job_key*. Called automatically before a cache entry is overwritten so that stale media files from the previous run are removed. Returns the number of directories removed. .. py:function:: ensure_cache_version(cachedir: str = 'cachedir') -> None Check the cache version file; clear everything on mismatch. Called automatically when a :class:`Bench` is instantiated. If the version file is missing or doesn't match ``CACHE_VERSION``, the entire cache tree is deleted so stale data from incompatible layouts doesn't linger. .. py:class:: BenchResult(bench_cfg) Bases: :py:obj:`bencher.results.rerun_result.RerunResult`, :py:obj:`bencher.results.volume_result.VolumeResult`, :py:obj:`bencher.results.holoview_results.distribution_result.box_whisker_result.BoxWhiskerResult`, :py:obj:`bencher.results.holoview_results.distribution_result.violin_result.ViolinResult`, :py:obj:`bencher.results.holoview_results.distribution_result.scatter_jitter_result.ScatterJitterResult`, :py:obj:`bencher.results.holoview_results.scatter_result.ScatterResult`, :py:obj:`bencher.results.holoview_results.line_result.LineResult`, :py:obj:`bencher.results.holoview_results.bar_result.BarResult`, :py:obj:`bencher.results.holoview_results.heatmap_result.HeatmapResult`, :py:obj:`bencher.results.holoview_results.curve_result.CurveResult`, :py:obj:`bencher.results.holoview_results.band_result.BandResult`, :py:obj:`bencher.results.holoview_results.surface_result.SurfaceResult`, :py:obj:`bencher.results.histogram_result.HistogramResult`, :py:obj:`bencher.results.holoview_results.holoview_result.HoloviewResult`, :py:obj:`bencher.results.video_summary.VideoSummaryResult`, :py:obj:`bencher.results.dataset_result.DataSetResult`, :py:obj:`bencher.results.optuna_result.OptunaResult` Contains the results of the benchmark and has methods to cast the results to various datatypes and graphical representations .. py:attribute:: timings :value: None .. py:method:: from_existing(original: BenchResult) -> BenchResult :classmethod: .. py:method:: to(result_type: BenchResult, result_var: param.Parameter | None = None, override: bool = True, reduce: bencher.results.bench_result_base.ReduceType | None = None, aggregate: bool | int | list[str] | None = None, agg_fn: Literal['mean', 'sum', 'max', 'min', 'median'] = 'mean', **kwargs: Any) -> BenchResult Return the current instance of BenchResult. :returns: The current instance of the benchmark result :rtype: BenchResult .. py:method:: default_plot_callbacks() -> list[callable] :staticmethod: Get the default list of plot callback functions. These callbacks are used by default in the to_auto method if no specific plot list is provided. :returns: A list of plotting callback functions :rtype: list[callable] .. py:method:: plotly_callbacks() -> list[callable] :staticmethod: Get the list of Plotly-specific callback functions. :returns: A list of Plotly-based visualization callback functions :rtype: list[callable] .. py:method:: plot() -> panel.panel Plots the benchresult using the plot callbacks defined by the bench run. This method uses the plot_callbacks defined in the bench_cfg to generate plots for the benchmark results. :returns: A panel representation of the results, or None if no plot_callbacks defined :rtype: pn.panel .. py:method:: to_auto(plot_list: list[callable] | None = None, remove_plots: list[callable] | None = None, default_container=pn.Column, override: bool = False, numeric_only: bool = False, **kwargs) -> list[panel.panel] Automatically generate plots based on the provided plot callbacks. :param plot_list: List of plot callback functions to use. Defaults to None. :type plot_list: list[callable], optional :param remove_plots: List of plot callback functions to exclude. Defaults to None. :type remove_plots: list[callable], optional :param default_container: Default container type for the plots. Defaults to pn.Column. :type default_container: type, optional :param override: Whether to override unsupported plots. Defaults to False. :type override: bool, optional :param numeric_only: When True, skip pane-type result callbacks (images, videos, rerun, etc.) that cannot be numerically aggregated. Defaults to False. :type numeric_only: bool, optional :param \*\*kwargs: Additional keyword arguments for plot configuration. :returns: A list of panel objects containing the generated plots. :rtype: list[pn.panel] .. py:method:: to_auto_plots(extra_panels: collections.abc.Sequence[collections.abc.Callable[[BenchResult], panel.viewable.Viewable] | panel.viewable.Viewable] | None = None, **kwargs) -> panel.panel Given the dataset result of a benchmark run, automatically deduce how to plot the data based on the types of variables that were sampled. :param extra_panels: Extra panel callables or static panels to inject after the sweep summary and before aggregate/auto plots. Each item is either a callable(BenchResult) -> panel, or a static panel object. :param \*\*kwargs: Additional keyword arguments for plot configuration. :returns: A panel containing plot results. :rtype: pn.panel .. py:method:: _scalar_aggregate_summary() -> panel.pane.Markdown Render a Markdown table for a fully-aggregated (scalar) result. .. py:class:: OptimizeResult Wraps an ``optuna.Study`` with bencher-friendly accessors. .. attribute:: study The underlying optuna study. .. attribute:: n_warm_start_trials Number of trials seeded from cache / prior results. .. attribute:: n_new_trials Number of new trials evaluated during optimization. .. attribute:: target_names Names of the optimization target variables. .. attribute:: bench_cfg Optional BenchCfg for rich report generation. .. py:attribute:: study :type: optuna.Study .. py:attribute:: n_warm_start_trials :type: int :value: 0 .. py:attribute:: n_new_trials :type: int :value: 0 .. py:attribute:: target_names :type: list[str] :value: [] .. py:attribute:: bench_cfg :type: bencher.bench_cfg.BenchCfg | None :value: None .. py:method:: _ensure_single_objective() -> None Raise if study is multi-objective. .. py:property:: best_params :type: dict[str, Any] Best parameters found (single-objective only). .. py:property:: best_value :type: float Best objective value (single-objective only). .. py:property:: best_trials :type: list[optuna.trial.FrozenTrial] Pareto-optimal trials (multi-objective). .. py:method:: summary() -> str Return a human-readable summary of the optimization. .. py:class:: PaneResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.bench_result_base.BenchResultBase` .. py:method:: to_video(result_var: param.Parameter | None = None, **kwargs) .. py:method:: to_panes(result_var: param.Parameter | None = None, hv_dataset=None, target_dimension: int = 0, container=None, subsampling_divisions: int | None = None, **kwargs) -> panel.pane.panel | None .. py:data:: VideoResult .. py:class:: ReduceType Bases: :py:obj:`enum.Enum` Generic enumeration. Derive from this class to define new enumerations. .. py:attribute:: AUTO .. py:attribute:: SQUEEZE .. py:attribute:: REDUCE .. py:attribute:: MINMAX .. py:attribute:: NONE .. py:class:: HoloviewResult(bench_cfg: bencher.bench_cfg.BenchCfg) Bases: :py:obj:`bencher.results.pane_result.PaneResult` .. py:method:: set_default_opts(width: int = 600, height: int = 600) -> dict :staticmethod: Set default options for HoloViews visualizations. :param width: Default width for visualizations. Defaults to 600. :type width: int, optional :param height: Default height for visualizations. Defaults to 600. :type height: int, optional :returns: Dictionary containing width, height, and tools settings. :rtype: dict .. py:method:: to_hv_type(hv_type: type, reduce: bencher.results.bench_result_base.ReduceType = ReduceType.AUTO, **kwargs) -> holoviews.Chart Convert the dataset to a specific HoloViews visualization type. :param hv_type: The HoloViews chart type to convert to (e.g., hv.Points, hv.Curve). :type hv_type: type :param reduce: How to reduce dataset dimensions. Defaults to ReduceType.AUTO. :type reduce: ReduceType, optional :param \*\*kwargs: Additional parameters to pass to the chart constructor. :returns: A HoloViews chart of the specified type. :rtype: hv.Chart .. py:method:: overlay_plots(plot_callback: callable) -> holoviews.Overlay | panel.Row | None Create an overlay of plots by applying a callback to each result variable. :param plot_callback: Function to apply to each result variable to create a plot. :type plot_callback: callable :returns: An overlay of plots or Row of plots, or None if no results. :rtype: hv.Overlay | pn.Row | None .. py:method:: layout_plots(plot_callback: callable) -> holoviews.Layout | None Create a layout of plots by applying a callback to each result variable. :param plot_callback: Function to apply to each result variable to create a plot. :type plot_callback: callable :returns: A layout of plots or None if no results. :rtype: hv.Layout | None .. py:method:: time_widget(title: str) -> dict Create widget configuration for time-based visualizations. :param title: Title for the widget. :type title: str :returns: Widget configuration dictionary with title. :rtype: dict .. py:method:: _use_holomap_for_time(dataset: xarray.Dataset) -> bool Check whether over_time should be rendered via an hv.HoloMap slider. Returns True when over_time is active and the dataset has >1 time points. .. py:method:: _apply_opts(plot, **opts_kwargs) :staticmethod: Apply .opts() to a plot, handling panel.pane.HoloViews wrappers. When hvplot is called with widget_location, it returns a panel pane whose underlying .object is the actual holoviews element. .. py:method:: _over_time_kdims() -> list :staticmethod: Return the kdim list for over_time HoloMaps. .. py:method:: _holomap_with_slider_bottom(hvobj, widgets=None) :staticmethod: Wrap a HoloViews object so any scrubber/slider appears below the plot. ``pn.pane.HoloViews(holomap, widget_location="bottom")`` does not embed correctly in static HTML (the widget is lost). Instead we let Panel auto-create the widget via ``pn.panel(hvobj)`` (which produces a ``Row(plot, widget_box)``), then rearrange into a ``Column(plot, widget_box)`` so the slider sits underneath. Force ``DiscreteSlider`` for the *over_time* dimension so that string-based ``TimeEvent`` coordinates get a slider instead of the default dropdown ``Select`` widget. Safe to call on any HoloViews object; if no widgets are produced the original object is returned unchanged. The slider defaults to the most recent (last) time point by setting the widget value in Python. Panel's embed system computes JSON patches relative to this default, so every other position gets a valid patch and the last position is the initial state. .. py:method:: _build_curve_overlay(dataset: xarray.Dataset, result_var: param.Parameter, **kwargs) -> holoviews.Overlay Build a Curve (+ optional Spread) overlay for a single time slice or aggregated data. When ``_std`` exists in the dataset the spread band is rendered automatically. This is used by both the curve renderer and the line renderer (for aggregated data that gained ``_std`` from ``_mean_over_time``). Performance: avoids ``to_dataframe()`` when there are no categorical groupby dimensions by constructing ``hv.Dataset`` directly from the xarray Dataset. The heavier DataFrame path is only used when manual groupby is required. .. py:method:: _mean_over_time(dataset, result_var_name) :staticmethod: Average a dataset across all time points. Always produces a ``_std`` variable so that downstream renderers (e.g. curve spread, error bars) can visualise the aggregation uncertainty. When a per-time-point ``_std`` already exists the pooled standard deviation is computed via the law of total variance; otherwise the standard deviation of the means across time points is used. .. py:method:: subsample_indices(n, max_points) :staticmethod: Return evenly-spaced indices into a length-*n* array. Always includes the first and last index. When *max_points* is ``None`` or >= *n*, returns ``range(n)`` (no subsampling). .. py:method:: _build_time_holomap(dataset, result_var_name, make_plot_fn) Build per-time-point HoloMap + optional aggregated plot. ``make_plot_fn`` receives a Dataset *without* the ``over_time`` dimension. The aggregated dataset produced by ``_mean_over_time`` always contains a ``_std`` variable; callbacks that are ``_std``-aware (e.g. delegating to ``_build_curve_overlay``) will automatically render spread bands on the aggregated tab. When ``bench_cfg.max_slider_points`` is set, only that many evenly-spaced time points are rendered for the slider (first and last always included). The aggregated tab still uses all data. When ``bench_cfg.show_aggregated_time_tab`` is ``False``, the aggregation is skipped entirely for faster rendering. .. py:method:: _build_time_holomap_raw(da, make_plot_fn) Build per-time-point HoloMap + optional aggregated plot for distributions. *make_plot_fn* receives a DataArray that **retains** the ``over_time`` dimension (a single-element slice for per-time-point entries, or the full array for the aggregated tab). Callers should flatten via ``.to_dataframe().reset_index()`` or equivalent. Respects ``bench_cfg.max_slider_points`` and ``bench_cfg.show_aggregated_time_tab``. .. py:method:: _build_tap_plot(plot: holoviews.Element, dataset: xarray.Dataset, result_var_plots: list[param.Parameter], container: type | list[type] | None = None, tap_container_direction: type | None = None) -> panel.Row Wrap a plot element with interactive PointerXY tap functionality. Sets up ``hv.streams.PointerXY`` and ``hv.streams.MouseLeave`` on the given *plot*, updating the supplied containers with the nearest data point values as the user hovers. :param plot: The base HoloViews element to attach tap streams to. :param dataset: The full xarray Dataset for value look-ups. :param result_var_plots: Result variables whose values are shown on tap. :param container: Panel container type(s) for displaying tapped values. :param tap_container_direction: Layout class (``pn.Row`` or ``pn.Column``) for the tap containers. Defaults to ``pn.Column``. :returns: A ``pn.Row`` containing the interactive plot and tap info panel. .. py:method:: hv_container_ds(dataset: xarray.Dataset, result_var: param.Parameter, container: holoviews.Chart | None = None, **kwargs) -> holoviews.Chart Convert an xarray Dataset to a HoloViews container for a specific result variable. :param dataset: The xarray Dataset containing the data. :type dataset: xr.Dataset :param result_var: The result variable to visualize. :type result_var: Parameter :param container: The HoloViews container type to use. Defaults to None. :type container: hv.Chart, optional :param \*\*kwargs: Additional options to pass to the chart's opts() method. :returns: A HoloViews chart containing the visualization. :rtype: hv.Chart .. py:method:: to_hv_container(container: panel.pane.panel, reduce_type: bencher.results.bench_result_base.ReduceType = ReduceType.AUTO, target_dimension: int = 2, result_var: param.Parameter | None = None, result_types: tuple | None = (ResultFloat, ), **kwargs) -> panel.pane.panel | None Convert the data to a HoloViews container with specified dimensions and options. :param container: The panel container type to use. :type container: pn.pane.panel :param reduce_type: How to reduce the dataset dimensions. Defaults to ReduceType.AUTO. :type reduce_type: ReduceType, optional :param target_dimension: Target dimension for the visualization. Defaults to 2. :type target_dimension: int, optional :param result_var: Specific result variable to visualize. Defaults to None. :type result_var: Parameter, optional :param result_types: Types of result variables to include. Defaults to (ResultFloat). :type result_types: tuple, optional :param \*\*kwargs: Additional visualization options. :returns: A panel containing the visualization, or None if no valid results. :rtype: pn.pane.panel | None .. py:method:: result_var_to_container(result_var: param.Parameter) -> type Determine the appropriate container type for a given result variable. :param result_var: The result variable to find a container for. :type result_var: Parameter :returns: The appropriate panel container type (PNG, Video, or Column). :rtype: type .. py:method:: setup_results_and_containers(result_var_plots: param.Parameter | list[param.Parameter], container: type | list[type] | None = None, **kwargs) -> tuple[list[param.Parameter], list[panel.pane.panel]] Set up appropriate containers for result variables. :param result_var_plots: Result variables to visualize. :type result_var_plots: Parameter | list[Parameter] :param container: Container types to use. Defaults to None. :type container: type | list[type], optional :param \*\*kwargs: Additional options to pass to the container constructors. :returns: Tuple containing: - List of result variables - List of initialized container instances :rtype: tuple[list[Parameter], list[pn.pane.panel]] .. py:method:: to_error_bar(result_var: param.Parameter | str | None = None, **kwargs) -> holoviews.Bars Convert the dataset to an ErrorBars visualization for a specific result variable. :param result_var: Result variable to visualize. Defaults to None. :type result_var: Parameter | str, optional :param \*\*kwargs: Additional options for dataset reduction. :returns: A HoloViews ErrorBars object showing error ranges. :rtype: hv.Bars .. py:method:: to_points(reduce: bencher.results.bench_result_base.ReduceType = ReduceType.AUTO) -> holoviews.Points Convert the dataset to a Points visualization with optional error bars. :param reduce: How to reduce the dataset dimensions. Defaults to ReduceType.AUTO. :type reduce: ReduceType, optional :returns: A HoloViews Points object, potentially with ErrorBars if reduction is applied. :rtype: hv.Points .. py:method:: to_nd_layout(hmap_name: str) -> holoviews.NdLayout Convert a HoloMap to an NdLayout for multi-dimensional visualization. :param hmap_name: Name of the HoloMap to convert. :type hmap_name: str :returns: A HoloViews NdLayout object with the visualization. :rtype: hv.NdLayout .. py:method:: to_holomap(name: str | None = None) -> holoviews.HoloMap Convert an NdLayout to a HoloMap for animated/interactive visualization. :param name: Name of the HoloMap to use. Defaults to None. :type name: str, optional :returns: A HoloViews HoloMap object with the visualization. :rtype: hv.HoloMap .. py:method:: to_holomap_list(hmap_names: list[str] | None = None) -> panel.Column Create a column of HoloMaps from multiple named maps. :param hmap_names: list of HoloMap names to include. If None, uses all result_hmaps. Defaults to None. :type hmap_names: list[str], optional :returns: A panel Column containing multiple HoloMaps. :rtype: pn.Column .. py:method:: get_nearest_holomap(name: str | None = None, **kwargs) -> holoviews.HoloMap Get the HoloMap element closest to the specified coordinates. :param name: Name of the HoloMap to use. Defaults to None. :type name: str, optional :param \*\*kwargs: Coordinate values to find nearest match for. :returns: The nearest matching HoloMap element. :rtype: hv.HoloMap .. py:method:: to_dynamic_map(name: str | None = None) -> holoviews.DynamicMap Create a DynamicMap from the HoloMap dictionary. Uses the values stored in the holomap dictionary to populate a dynamic map. This is much faster than passing the holomap to a holomap object as the values are calculated on the fly. :param name: Name of the HoloMap to use. Defaults to None. :type name: str, optional :returns: A HoloViews DynamicMap for interactive visualization. :rtype: hv.DynamicMap .. py:method:: to_grid(inputs: list[str] | None = None) -> holoviews.GridSpace Create a grid visualization from a HoloMap. :param inputs: Input variable names to use for the grid dimensions. If None, uses bench_cfg.inputs_as_str(). Defaults to None. :type inputs: list[str], optional :returns: A HoloViews GridSpace object showing the data as a grid. :rtype: hv.GridSpace .. py:class:: BenchReport(bench_name: str | None = None) Bases: :py:obj:`bencher.bench_plot_server.BenchPlotServer` A server for display plots of benchmark results .. py:attribute:: bench_name :value: None .. py:attribute:: pane .. py:attribute:: last_save_ms :type: float :value: 0.0 .. py:attribute:: bench_results :type: list[bencher.results.bench_result.BenchResult] :value: [] .. py:method:: clear() -> None Remove all tabs and results so the report can be reused between runs. Not safe to call while the report is being served to a live Panel session. .. py:method:: append_title(title: str, new_tab: bool = True) .. py:method:: append_markdown(markdown: str, name: str | None = None, width: int = 800, **kwargs) -> panel.pane.Markdown .. py:method:: append(pane: panel.panel, name: str | None = None) -> None .. py:method:: append_col(pane: panel.panel, name: str | None = None) -> None .. py:method:: _time_event_label(bench_res: bencher.results.bench_result.BenchResult) -> str | None :staticmethod: Extract a human-readable label for the latest time event from a result. .. py:method:: append_result(bench_res: bencher.results.bench_result.BenchResult, render_from: bencher.results.bench_result.BenchResult | None = None) -> None .. py:method:: append_to_result(bench_res: bencher.results.bench_result.BenchResult, pane: panel.panel) -> None Append *pane* to the tab that belongs to *bench_res*. .. py:method:: prepend_to_result(bench_res: bencher.results.bench_result.BenchResult, pane: panel.panel) -> None Insert *pane* at the beginning of the tab that belongs to *bench_res*. .. py:method:: append_tab(pane: panel.panel, name: str | None = None) -> None .. py:method:: save_index(directory: str = '', filename: str = 'index.html') -> pathlib.Path Saves the result to index.html in the root folder so that it can be displayed by github pages. :returns: save path :rtype: Path .. py:method:: save(directory: str | pathlib.Path = 'cachedir', filename: str | None = None, in_html_folder: bool = True, portable: bool = False, **kwargs) -> pathlib.Path Save the result to a html file. When the report contains multiple tabs, each tab is saved to its own embedded HTML file and the index page uses iframes to display them. This prevents HoloMap slider widgets from colliding across tabs. :param directory: base folder to save to. Defaults to "cachedir" which should be ignored by git. :type directory: str | Path, optional :param filename: The name of the html file. Defaults to the name of the benchmark :type filename: str, optional :param in_html_folder: Put the saved files in a html subfolder to help keep the results separate from source code. Defaults to True. :type in_html_folder: bool, optional :param portable: When True, base64-encode .rrd data directly into the viewer HTML so the report works from ``file://`` without any server. When False (default), .rrd files are copied as sidecar files and loaded via relative URLs — the report must be served over HTTP. :type portable: bool, optional :returns: the save path :rtype: Path .. py:method:: _write_iframe_index(index_path: pathlib.Path, tab_files: list) -> None :staticmethod: Write a lightweight HTML index with tab buttons and an iframe. .. py:method:: show(run_cfg: bencher.bench_cfg.BenchRunCfg | None = None) -> threading.Thread Launches a webserver with plots of the benchmark results, blocking :param run_cfg: Options for the webserve such as the port. Defaults to None. :type run_cfg: BenchRunCfg, optional .. py:method:: publish_gh_pages(github_user: str, repo_name: str, folder_name: str = 'report', branch_name: str = 'gh-pages') -> str .. py:method:: publish(remote_callback: Callable, branch_name: str | None = None, debug: bool = False) -> str Publish the results as an html file by committing it to the bench_results branch in the current repo. If you have set up your repo with github pages or equivalent then the html file will be served as a viewable webpage. This is an example of a callable to publish on github pages: .. code-block:: python def publish_args(branch_name) -> tuple[str, str]: return ( "https://github.com/blooop/bencher.git", f"https://github.com/blooop/bencher/blob/{branch_name}") :param remote: A function the returns a tuple of the publishing urls. It must follow the signature def publish_args(branch_name) -> tuple[str, str]. The first url is the git repo name, the second url needs to match the format for viewable html pages on your git provider. The second url can use the argument branch_name to point to the report on a specified branch. :type remote: Callable :returns: the url of the published report :rtype: str .. py:class:: GithubPagesCfg .. py:attribute:: github_user :type: str .. py:attribute:: repo_name :type: str .. py:attribute:: folder_name :type: str :value: 'report' .. py:attribute:: branch_name :type: str :value: 'gh-pages' .. py:class:: Publisher Bases: :py:obj:`Protocol` Generic publisher protocol for benchmark reports. Any object with a ``publish(report)`` method satisfies this protocol. Downstream projects implement their own publishers (GCS, S3, etc.) without modifying bencher. .. py:method:: publish(report: BenchReport) -> str | None Publish a report. Returns the published URL, or None. .. py:class:: Executors Bases: :py:obj:`strenum.StrEnum` Enumeration of available execution strategies for benchmark jobs. This enum defines the execution modes for running benchmark jobs and provides a factory method to create appropriate executors. .. py:attribute:: SERIAL .. py:attribute:: MULTIPROCESSING .. py:attribute:: SCOOP .. py:method:: factory(provider: Executors) -> concurrent.futures.Future | None :staticmethod: Create an executor instance based on the specified execution strategy. :param provider: The type of executor to create :type provider: Executors :returns: The executor instance, or None for serial execution :rtype: Future | None .. py:class:: SweepTimings Timing data for a single run_sweep() call. Each field records the wall-clock duration in milliseconds of a major phase inside ``Bench.run_sweep()`` or ``Bench.calculate_benchmark_results()``. .. py:attribute:: cache_check_ms :type: float :value: 0.0 .. py:attribute:: sample_cache_init_ms :type: float :value: 0.0 .. py:attribute:: dataset_setup_ms :type: float :value: 0.0 .. py:attribute:: job_submission_ms :type: float :value: 0.0 .. py:attribute:: job_execution_ms :type: float :value: 0.0 .. py:attribute:: history_merge_ms :type: float :value: 0.0 .. py:attribute:: post_setup_ms :type: float :value: 0.0 .. py:attribute:: render_ms :type: float :value: 0.0 .. py:attribute:: report_save_ms :type: float :value: 0.0 .. py:attribute:: total_ms :type: float :value: 0.0 .. py:method:: compute_total() -> float Compute total_ms as the sum of all phase timing fields. .. py:method:: summary() -> dict[str, float] Return all phase timings as a dict. .. py:class:: VideoWriter(filename: str = 'vid') .. py:attribute:: images :value: [] .. py:attribute:: image_files :value: [] .. py:attribute:: video_files :value: [] .. py:attribute:: filename :value: '/vid.mp4' .. py:method:: append(img) .. py:method:: write() -> str .. py:method:: create_label(label, width=None, height=16, color=(255, 255, 255)) :staticmethod: .. py:method:: label_image(path: pathlib.Path, label, padding=20, color=(255, 255, 255)) -> pathlib.Path :staticmethod: .. py:method:: convert_to_compatible_format(video_path: str) -> str :staticmethod: .. py:method:: write_video_raw(video_clip: moviepy.video.VideoClip, fps: int = 30) -> str .. py:method:: extract_frame(video_path: str, time: float | None = None, output_path: str | None = None) -> str :staticmethod: Extract a frame from a video at a specific time. :param video_path: Path to the video file :param time: Time in seconds to extract frame. If None, uses last frame :param output_path: Optional path where to save the image. If None, uses video name with _frame.png :returns: Path to the saved PNG image :rtype: str .. py:function:: add_image(np_array: numpy.ndarray, name: str = 'img') -> str Creates a file on disk from a numpy array and returns the created image path .. py:class:: ClassEnum Bases: :py:obj:`strenum.StrEnum` A string-based enum class that maps enum values to corresponding class instances. ClassEnum is a pattern to make it easier to create a factory method that converts from an enum value to a corresponding class instance. Subclasses should implement the to_class() method which takes an enum value and returns an instance of the corresponding class. This pattern is useful for configuration-driven class instantiation, allowing classes to be selected via string configuration values that match enum names. .. py:method:: to_class_generic(module_import: str, class_name: str) -> Any :classmethod: Create an instance of a class from its module path and class name. This utility method dynamically imports a module and instantiates a class from it. :param module_import: The module path to import (e.g., "bencher.class_enum") :type module_import: str :param class_name: The name of the class to instantiate :type class_name: str :returns: A new instance of the specified class :rtype: Any .. py:method:: to_class(enum_val: ClassEnum) -> Any :classmethod: :abstractmethod: Convert an enum value to its corresponding class instance. Subclasses must override this method to implement the mapping from enum values to class instances. :param enum_val: The enum value to convert to a class instance :type enum_val: ClassEnum :returns: An instance of the class corresponding to the enum value :rtype: Any :raises NotImplementedError: If this method is not overridden by a subclass .. py:class:: ExampleEnum Bases: :py:obj:`ClassEnum` An example implementation of ClassEnum. This enum demonstrates how to use ClassEnum to map enum values to class instances. Each enum value corresponds to a class name that can be instantiated. .. py:attribute:: Class1 .. py:attribute:: Class2 .. py:method:: to_class(enum_val: ExampleEnum) -> BaseClass :classmethod: Convert an ExampleEnum value to its corresponding class instance. :param enum_val: The enum value to convert :type enum_val: ExampleEnum :returns: An instance of either Class1 or Class2, depending on the enum value :rtype: BaseClass .. py:function:: create_bench(sweep: bencher.variables.parametrised_sweep.ParametrizedSweep, run_cfg: bencher.bench_cfg.BenchRunCfg | None = None, report: bencher.bench_report.BenchReport | None = None, name: str | None = None) -> bencher.bencher.Bench Create a Bench instance from a ParametrizedSweep. :param sweep: The ParametrizedSweep instance to benchmark. :param run_cfg: Optional benchmark run configuration. :param report: Optional existing report to append results to. :param name: Optional name for the benchmark. If None, derived from sweep's class name. :returns: A configured Bench instance. .. py:function:: create_bench_runner(sweep: bencher.variables.parametrised_sweep.ParametrizedSweep, run_cfg: bencher.bench_cfg.BenchRunCfg | None = None, name: str | None = None) -> bencher.bench_runner.BenchRunner Create a BenchRunner instance from a ParametrizedSweep. Enables fluent chaining like: MyConfig().to_bench_runner().add(func).run(subsampling_divisions=2, max_subsampling_divisions=4) :param sweep: The ParametrizedSweep instance to use as the benchmark class. :param run_cfg: Optional benchmark run configuration. Created if not provided. :param name: Optional name for the runner. If None, auto-generated. :returns: A configured BenchRunner instance. .. py:function:: run(target: Callable | type | bencher.variables.parametrised_sweep.ParametrizedSweep, *, subsampling_divisions=UNSET, repeats: int = 1, max_subsampling_divisions: int | None = None, max_repeats: int | None = None, run_cfg: bencher.bench_cfg.BenchRunCfg | None = None, show: bool | str | bencher.bench_cfg.ShowMode = True, save: bool = False, publish: bool = False, publisher: bencher.bench_report.Publisher | bencher.bench_report.GithubPagesCfg | Callable | None = None, grouped: bool = False, cache_samples: bool | None = None, over_time: bool | None = None, backend: str | None = None, optimise: int | bool = 0, sampling_context: contextlib.AbstractContextManager[Any] | None = None, **kwargs) -> list[bencher.bench_cfg.BenchCfg] Run a benchmark target with sensible defaults. Handles three cases: 1. Callable (e.g. ``bn.run(example_fn)``) — wraps in BenchRunner. 2. ParametrizedSweep subclass (e.g. ``bn.run(SimpleFloat)``) — instantiates, calls ``to_bench()`` + ``plot_sweep()``. 3. ParametrizedSweep instance (e.g. ``bn.run(SimpleFloat())``) — same as above without instantiation. :param target: A benchmark function, ParametrizedSweep class, or ParametrizedSweep instance. :param subsampling_divisions: Benchmark sampling resolution subsampling_divisions. Defaults to 2. :param repeats: Number of repeats. Defaults to 1. :param max_subsampling_divisions: Maximum subsampling_divisions for progressive runs. Defaults to None (single subsampling_divisions). :param max_repeats: Maximum repeats for progressive runs. Defaults to None (single repeat count). :param run_cfg: Optional explicit BenchRunCfg. Defaults to None. :param show: Where to view the report. Accepts ``True``/``ShowMode.LIVE`` (default — start a Panel server and block on ``input()`` until the user presses Enter), ``ShowMode.HTML`` (save an embedded HTML file and open it in the browser, then return), ``ShowMode.PUBLISHED`` (open the URL returned by publish — requires ``publish=True``), or ``False``/``ShowMode.NONE`` (display nothing). :param save: Save results to disk. Defaults to False. :param publish: Publish results. Defaults to False. :param publisher: An object conforming to the :class:`Publisher` protocol (i.e. has a ``publish(report)`` method). Passed to :class:`BenchRunner` and called after each progressive iteration when *publish* is ``True``. :param grouped: Produce a single HTML page with all benchmarks. Defaults to False. :param cache_samples: Use sample cache for previous results. None (default) auto-enables for progressive runs. Pass False to disable even for progressive runs. :param over_time: Enable time-series benchmarking. None preserves run_cfg value. :param backend: Visualization backend ('panel' or 'rerun'). None preserves run_cfg value. :param optimise: When > 0, appends optuna analysis plots (parameter importance, with/without repeats comparison, best parameters) from the sweep results to the report. Defaults to 0 (no optimisation analysis). :param sampling_context: An optional context manager that wraps only the sampling phase (``br.run(...)``). Its ``__exit__`` is guaranteed to run *before* the Panel/Bokeh server starts, so resources held by the context (DB pools, GPU handles, simulators, etc.) are released while nothing blocks. ``save`` and ``publish`` still execute inside the context (they happen during ``br.run(show=False, ...)``). Defaults to ``None`` (no wrapper, fully backward-compatible). **Anti-pattern** — wrapping the whole call keeps resources held during the interactive viewing session:: with gpu_context(): # held for the entire viewing session! bn.run(target, show=True) **Recommended** — use ``sampling_context`` so the context exits before the server starts:: bn.run(target, show=True, sampling_context=gpu_context()) :returns: A list of benchmark configuration objects with results. :rtype: list[BenchCfg] .. py:data:: _DEPRECATED_ALIASES .. py:function:: __getattr__(name: str)