bencher.variables.results

Result variable classes for benchmark outputs.

IMPORTANT — hash_persistent() contract:

Every Result* class MUST implement hash_persistent() using _hash_slots() which hashes ALL __slots__ by default. This is critical for the over_time history cache: BenchCfg.hash_persistent() includes result variable hashes in the cache key, so a non-deterministic hash means historical data can never be found.

The default behavior hashes every slot. If a slot holds a non-deterministic value (runtime objects, callbacks, etc.), add it to _hash_exclude on the class:

class MyResult(param.Parameter):
__slots__ = [“units”, “obj”] _hash_exclude = (“obj”,) # runtime object, not deterministic

def hash_persistent(self) -> str:
return _hash_slots(self)

WRONG — never do this (str(self) includes the memory address for param.Parameter):

def hash_persistent(self) -> str:: return hash_sha1(self)

Tests in test/test_hash_persistent.py auto-discover all Result* classes and verify: - Determinism: two equivalent instances produce the same hash - Slot coverage: every __slots__ entry is either hashed or in _hash_exclude Adding a new class without proper hashing will fail CI.

Attributes

`_PARAM_MODULES`
`_NAN`
`RESULT_SPECS`
`PANEL_TYPES`
`SCALAR_RESULT_TYPES`
`XARRAY_MULTIDIM_RESULT_TYPES`
`ALL_RESULT_TYPES`
`RESULT_KIND_ORDER`
`_MEDIA_RESULT_TYPES`
`_REFERENCE_MISSING_TYPES`
`_OBJECT_MISSING_TYPES`
`DATA_VAR_RESULT_TYPES`
`RESULT_SPEC_EXEMPT`

Classes

`OptDir`	StrEnum is a Python `enum.Enum` that inherits from `str`. The default
`ResultFloat`	A class to represent continuous float result variables and the desired optimisation direction.
`ResultBool`	A result type for binary outcomes (success/failure, pass/fail, reachable/unreachable).
`ResultVec`	A class to represent fixed size vector result variable
`ResultHmap`	Deprecated: use ResultContainer or ResultReference with a declared container instead.
`ResultPath`	A path to a file the benchmark produced.
`ResultVideo`	Parameter that can be set to a string specifying the path of a file.
`ResultImage`	Parameter that can be set to a string specifying the path of a file.
`ResultString`	Text the benchmark produced.
`ResultContainer`	Embeddable HTML/panel content the benchmark produced.
`ResultRerun`	Result type for rerun .rrd spatial visualizations.
`ResultReference`	Use this class to save arbitrary objects that are not picklable or native to panel.
`ResultDataSet`	An arbitrary picklable data payload stored for each benchmark sample.
`ResultKind`	Coarse, serializable classification of a result variable.
`ResultSpec`	Storage and classification contract for one Result* class.
`ResultVar`	Deprecated: use ResultFloat instead.

Functions

`_hash_slots`(instance)	Hash all __slots__ from the class hierarchy, excluding non-deterministic attributes.
`curve`(→ holoviews.Curve)
`result_spec`(→ ResultSpec \| None)	Spec for a result-variable instance, resolved most-derived-first.
`_spec_types`(→ tuple[type, Ellipsis])	The registry keys whose spec satisfies predicate, in registry order.
`result_kind`(→ str)	Classify a result variable into a coarse, serializable kind name used by
`result_missing_fill`(→ tuple[Any, type])	Return the `(fill_value, numpy_dtype)` used for missing entries of rv.
`_dataset_cell_is_missing`(→ bool)	Missingness for a `ResultDataSet` cell, across both sentinel generations.
`result_is_missing`(→ bool)	True when value is the missing/unrecorded sentinel for rv's storage.

Module Contents

bencher.variables.results._PARAM_MODULES

bencher.variables.results._hash_slots(instance)

Hash all __slots__ from the class hierarchy, excluding non-deterministic attributes.

Walks the MRO from the concrete class up to (but not including) param framework base classes, collecting __slots__ from each ancestor. This supports Result class inheritance (e.g. ResultBool extends ResultFloat). Attributes listed in _hash_exclude on any class in the hierarchy are skipped.

The class name is always included in the hash to prevent collisions between different Result* classes that share the same slot layout and values (e.g. ResultPath, ResultVideo, and ResultImage all have __slots__ = [“units”] with default units=”path”).

The Parameter name is also included: history columns and regression baselines are keyed by name, so two same-typed result vars with different names are different measurements and must not share a cache identity. Unbound instances hash name=None, which is deterministic.

class bencher.variables.results.OptDir

Bases: strenum.StrEnum

StrEnum is a Python enum.Enum that inherits from str. The default auto() behavior uses the member name as its value.

Example usage:

class Example(StrEnum):
    UPPER_CASE = auto()
    lower_case = auto()
    MixedCase = auto()

assert Example.UPPER_CASE == "UPPER_CASE"
assert Example.lower_case == "lower_case"
assert Example.MixedCase == "MixedCase"

minimize

maximize

none

class bencher.variables.results.ResultFloat(units='ul', direction: OptDir = OptDir.minimize, share_axis=True, max_time_events=None, default=float('nan'), meaning_version=1, **params)

Bases: param.Number

A class to represent continuous float result variables and the desired optimisation direction.

For boolean (success/failure) outcomes, use ResultBool instead — it locks bounds to [0, 1] and produces correct boolean-style plots.

__slots__ = ['units', 'direction', 'share_axis', 'max_time_events', 'meaning_version']

_hash_exclude = ('direction', 'share_axis', 'max_time_events')

units = 'ul'

meaning_version = 1

default

direction

share_axis = True

max_time_events = None

as_dim() → holoviews.Dimension

hash_persistent() → str: A hash function that avoids the PYTHONHASHSEED ‘feature’ which returns a different hash value each time the program is run

class bencher.variables.results.ResultBool(units='ratio', direction: OptDir = OptDir.minimize, default=float('nan'), **params)

Bases: ResultFloat

A result type for binary outcomes (success/failure, pass/fail, reachable/unreachable).

Bounds are locked to [0, 1] and plots use boolean-style rendering. For continuous scalar metrics (time, distance, score), use ResultFloat instead.

default

bounds = (0, 1)

_validate_bounds(val, bounds, inclusive_bounds)

class bencher.variables.results.ResultVec(size, units='ul', direction: OptDir = OptDir.minimize, max_time_events=None, default=float('nan'), **params)

Bases: param.List

A class to represent fixed size vector result variable

__slots__ = ['units', 'direction', 'size', 'max_time_events']

_hash_exclude = ('max_time_events',)

units = 'ul'

default

direction

size

max_time_events = None

hash_persistent() → str: A hash function that avoids the PYTHONHASHSEED ‘feature’ which returns a different hash value each time the program is run

index_name(idx: int) → str

given the index of the vector, return the column name that

Parameters:: idx (int) – index of the result vector
Returns:: column name of the vector for the xarray dataset
Return type:: str

index_names() → list[str]

Returns a list of all the xarray column names for the result vector

Returns:: column names
Return type:: list[str]

class bencher.variables.results.ResultHmap(*args, **kwargs)

Bases: param.Parameter

Deprecated: use ResultContainer or ResultReference with a declared container instead.

A class to represent a holomap return type. Its data lives out-of-band in bench_res.hmaps rather than in the canonical result dataset, so it cannot participate in the A6 grammar-of-ND-data migration; removal is scheduled for a later phase of that migration.

Note: this class has no __slots__, so _hash_slots hashes only the class name. Every ResultHmap instance produces the same hash. This is intentional — there are no configuration attributes that would differentiate instances. If a slot is added in the future, _hash_slots will automatically include it.

hash_persistent() → str: A hash function that avoids the PYTHONHASHSEED ‘feature’ which returns a different hash value each time the program is run

bencher.variables.results.curve(x_vals: list[float], y_vals: list[float], x_name: str, y_name: str, label: str | None = None, **kwargs) → holoviews.Curve

class bencher.variables.results.ResultPath(default=None, units='path', container: collections.abc.Callable[[Any], Any] | None = None, max_time_events=None, **params)

Bases: param.Filename

A path to a file the benchmark produced.

Renders as a download widget by default. Declare container= a callable taking the path and returning anything panel can display to render the file’s contents instead — a CSV as a chart, a JSON as a tree — and it wins over the download widget. See ResultDataSet for the contract the callback has to satisfy.

__slots__ = ['units', 'container', 'max_time_events']

_hash_exclude = ('container', 'max_time_events')

units = 'path'

container = None

max_time_events = None

hash_persistent() → str: A hash function that avoids the PYTHONHASHSEED ‘feature’ which returns a different hash value each time the program is run

to_container(): Returns a partial function for creating a FileDownload widget with embedding enabled. This function is used to create a panel container to represent the ResultPath object

class bencher.variables.results.ResultVideo(default=None, units='path', max_time_events=None, **params)

Bases: param.Filename

Parameter that can be set to a string specifying the path of a file.

The string should be specified in UNIX style, but it will be returned in the format of the user’s operating system.

The specified path can be absolute, or relative to either:

any of the paths specified in the search_paths attribute (if search_paths is not None);
any of the paths searched by resolve_path() (if search_paths is None).

__slots__ = ['units', 'max_time_events']

_hash_exclude = ('max_time_events',)

units = 'path'

max_time_events = None

hash_persistent() → str: A hash function that avoids the PYTHONHASHSEED ‘feature’ which returns a different hash value each time the program is run

class bencher.variables.results.ResultImage(default=None, units='path', max_time_events=None, **params)

Bases: param.Filename

Parameter that can be set to a string specifying the path of a file.

The string should be specified in UNIX style, but it will be returned in the format of the user’s operating system.

The specified path can be absolute, or relative to either:

any of the paths specified in the search_paths attribute (if search_paths is not None);
any of the paths searched by resolve_path() (if search_paths is None).

__slots__ = ['units', 'max_time_events']

_hash_exclude = ('max_time_events',)

units = 'path'

max_time_events = None

hash_persistent() → str: A hash function that avoids the PYTHONHASHSEED ‘feature’ which returns a different hash value each time the program is run

class bencher.variables.results.ResultString(default=None, units='str', container: collections.abc.Callable[[Any], Any] | None = None, max_time_events=None, **params)

Bases: param.String

Text the benchmark produced.

Renders as plain text by default. Declare container= a callable taking the string and returning anything panel can display to render it as something richer — Markdown, syntax-highlighted code, a parsed structure. See ResultDataSet for the contract the callback has to satisfy.

__slots__ = ['units', 'container', 'max_time_events']

_hash_exclude = ('container', 'max_time_events')

units = 'str'

container = None

max_time_events = None

hash_persistent() → str: A hash function that avoids the PYTHONHASHSEED ‘feature’ which returns a different hash value each time the program is run

class bencher.variables.results.ResultContainer(default=None, units='container', container: collections.abc.Callable[[Any], Any] | None = None, max_time_events=None, **params)

Bases: param.Parameter

Embeddable HTML/panel content the benchmark produced.

Handed to panel as-is by default. Declare container= a callable taking the stored value and returning anything panel can display to wrap or transform it first. See ResultDataSet for the contract the callback has to satisfy.

__slots__ = ['units', 'container', 'max_time_events']

_hash_exclude = ('container', 'max_time_events')

units = 'container'

container = None

max_time_events = None

hash_persistent() → str: A hash function that avoids the PYTHONHASHSEED ‘feature’ which returns a different hash value each time the program is run

class bencher.variables.results.ResultRerun(default=None, units='rerun', width=600, height=600, max_time_events=None, **params)

Bases: ResultContainer

Result type for rerun .rrd spatial visualizations.

Stores a path to an .rrd file (like ResultContainer) but carries viewer sizing metadata and provides a dedicated to_container() that renders the file with the rerun web viewer. A ComposableContainerRerun can also be assigned directly; result collection materializes it to one .rrd file before caching.

Usage in a ParametrizedSweep:

out_rerun = ResultRerun(width=600, height=600)

def benchmark(self):
    rr.log("boxes", rr.Boxes2D(half_sizes=[self.theta, 1]))
    self.out_rerun = bn.capture_rerun_window(width=600, height=600)

__slots__ = ['width', 'height']

_hash_exclude = ('width', 'height')

width = 600

height = 600

to_container(): Return a callable that renders an .rrd file path as a rerun viewer pane.

class bencher.variables.results.ResultReference(obj: Any | None = None, container: collections.abc.Callable[[Any], Any] | None = None, default: Any | None = None, units: str = 'container', max_time_events=None, **params)

Bases: param.Parameter

Use this class to save arbitrary objects that are not picklable or native to panel.

container is a callback taking the stored object and returning something panel can display. It can be attached to a single sample inside benchmark() or declared once on the class, exactly as for ResultDataSet:

plot = ResultReference(container=my_renderer)   # my_renderer(obj) -> pane

The callback receives only the object — no plot kwargs — so single-argument callables are safe, and one renderer works for both this and a ResultDataSet.

This is the documented same-process escape hatch: the stored object stays live, is stripped by both the result cache write and the collect/render split, and is never load-bearing for the core rendering algebra. Use ResultDataSet when the payload should survive process boundaries.

__slots__ = ['units', 'obj', 'container', 'max_time_events']

_hash_exclude = ('obj', 'container', 'max_time_events')

units = 'container'

obj = None

container = None

max_time_events = None

hash_persistent() → str: A hash function that avoids the PYTHONHASHSEED ‘feature’ which returns a different hash value each time the program is run

class bencher.variables.results.ResultDataSet(obj: Any | None = None, container: collections.abc.Callable[[Any], Any] | None = None, default: Any | None = None, units: str = 'dataset', max_time_events=None, **params)

Bases: param.Parameter

An arbitrary picklable data payload stored for each benchmark sample.

The payload may be a DataFrame, xarray object, mapping, sequence, custom dataclass, or any other object that can travel through the configured result cache. Bencher stores and retrieves it without interpreting its type.

Payloads are materialized into the cache’s content-addressed blob store at collect time (parquet for DataFrames, netCDF for xarray objects, pickle otherwise — see bencher.blob_store); the dataset cell stores the blob path, so any process sharing the cache filesystem can render any sample, including over_time history points.

container is an optional renderer taking the stored object and returning something Panel can display. Declare it once on the class and every sample renders through it, in result_vars order, alongside the other results:

payload = ResultDataSet(container=render_payload)

def benchmark(self):
    self.payload = ResultDataSet(measure())

Per-sample overrides are honoured too (ResultDataSet(data, container=...) inside benchmark()), and an explicit container= passed to a renderer beats both. The callback receives only the object — no plot kwargs — so single-argument callables are safe.

A declared renderer travels with BenchCfg into the result cache and through the collect/render split, so it has to be picklable: a module-level function or a callable object, not a lambda or a local function. Use ResultReference instead when the payload itself is not picklable.

__slots__ = ['units', 'obj', 'container', 'max_time_events']

_hash_exclude = ('obj', 'container', 'max_time_events')

units = 'dataset'

obj = None

container = None

max_time_events = None

hash_persistent() → str: A hash function that avoids the PYTHONHASHSEED ‘feature’ which returns a different hash value each time the program is run

class bencher.variables.results.ResultKind

Bases: strenum.StrEnum

Coarse, serializable classification of a result variable.

The values feed the A2 plot-selection signatures, so they must stay stable strings. Explicit values rather than auto(): strenum’s auto() yields the member name verbatim, which would silently change these to uppercase (plan 23 D4).

BOOL = 'bool'

FLOAT = 'float'

VEC = 'vec'

IMAGE = 'image'

VIDEO = 'video'

PATH = 'path'

STRING = 'string'

DATASET = 'dataset'

RERUN = 'rerun'

CONTAINER = 'container'

HMAP = 'hmap'

REFERENCE = 'reference'

class bencher.variables.results.ResultSpec

Storage and classification contract for one Result* class.

kind: Coarse kind name for plot-selection signatures (A2).

missing_fill: Value written for a missing/unrecorded sample.

fill_dtype: Numpy dtype of the backing array (float | object | int).

missing_sentinels: Exact-equality sentinel values accepted as “missing” on READ. This can be wider than {missing_fill} because missingness is not a pure function of the fill: ResultDataSet accepts both its cell generations ("NAN" blob paths from plan 22 onwards, -1 indices before) permanently. NaN/None missingness is dtype-generic and handled in result_is_missing(), not listed here (NaN has no useful equality semantics in a set).

is_scalar: Continuous/boolean scalar metric (regression, optimization).

is_panel: Rendered through the panel pathway rather than holoviews.

is_media: Cell values reference media files on disk that over_time aging must delete when entries age out.

is_data_var: Gets a single data variable (column) in the dataset. ResultVec expands to one column per element and ResultHmap is stored out-of-band, so neither is a data var.

multidim: Member of the xarray multidim store family (XARRAY_MULTIDIM_RESULT_TYPES).

reference_backed: Cell stores an object_index index (ResultReference only).

kind: ResultKind

missing_fill: Any

fill_dtype: type

missing_sentinels: frozenset

is_scalar: bool

is_panel: bool

is_media: bool

is_data_var: bool

multidim: bool

reference_backed: bool

bencher.variables.results._NAN

bencher.variables.results.RESULT_SPECS: dict[type, ResultSpec]

bencher.variables.results.result_spec(result_var) → ResultSpec | None

Spec for a result-variable instance, resolved most-derived-first.

Returns None for parameters that are not registered result types. Deprecated subclasses absent from the registry (ResultVar) resolve to their base class’s spec via isinstance.

bencher.variables.results._spec_types(predicate) → tuple[type, Ellipsis]: The registry keys whose spec satisfies predicate, in registry order.

bencher.variables.results.PANEL_TYPES

bencher.variables.results.SCALAR_RESULT_TYPES

bencher.variables.results.XARRAY_MULTIDIM_RESULT_TYPES

bencher.variables.results.ALL_RESULT_TYPES

bencher.variables.results.RESULT_KIND_ORDER

bencher.variables.results._MEDIA_RESULT_TYPES

bencher.variables.results.result_kind(result_var) → str: Classify a result variable into a coarse, serializable kind name used by plot-selection signatures (A2).

bencher.variables.results._REFERENCE_MISSING_TYPES

bencher.variables.results._OBJECT_MISSING_TYPES

bencher.variables.results.DATA_VAR_RESULT_TYPES

bencher.variables.results.result_missing_fill(rv) → tuple[Any, type]

Return the (fill_value, numpy_dtype) used for missing entries of rv.

Read from the ResultSpec registry; an unregistered parameter (or a future numeric result type before registration) falls back to the NaN family, matching the pre-registry behavior.

bencher.variables.results._dataset_cell_is_missing(value) → bool

Missingness for a ResultDataSet cell, across both sentinel generations.

"NAN" (blob-path cells, plan 22 onwards) and -1 (index-backed cells collected before it, including the float -1.0 an over_time concat can promote an int column to) are both missing, permanently — a mixed-generation history holds cells of each kind. NaN/None also count as missing so an unrepaired concat fill is never handed to load_blob or a dataset_list lookup as data.

bencher.variables.results.result_is_missing(rv, value) → bool

True when value is the missing/unrecorded sentinel for rv’s storage.

For NaN-backed (numeric) types, both NaN and None count as missing — the latter is treated as missing intentionally so a value that never reached the typed array (e.g. an absent object-index entry) is not mistaken for real data. Non-numeric values (strings, lists, …) are never missing for a numeric type: they cannot be the NaN sentinel, so no float coercion is attempted (the string "nan" is real data, not a missing marker). For the -1 / "NAN" sentinel types, missingness is exact equality with the sentinel.

ResultDataSet accepts BOTH its sentinel generations, permanently — see _dataset_cell_is_missing().

class bencher.variables.results.ResultVar(*args, **kwargs)

Bases: ResultFloat

Deprecated: use ResultFloat instead.

bencher.variables.results.RESULT_SPEC_EXEMPT: dict[type, str]