API REFERENCE/omnibias-difference

omnibias.difference.validation

0.1.0a2AlphaPython >=3.10Apache-2.0

Finite-difference stencils, confluence limits, polynomial sequences, and scoped derivative-extraction certificates.

import omnibias.difference.validation

Module contract & mathematical conventions

Data-driven refinement harness for omnibias-difference.

Pure-Python (core-only) probe utilities that turn each tier capability into an *instrumented* experiment, per the omnibias-empirical-validation gates:

• enclosure_soundness -- the verified-primitive rule that an enclosure must contain a **dense deterministic grid AND a random sample** of true values.
• high_precision_derivative / require_mpmath -- the mpmath high-precision oracle adapter.
• baseline_compare -- the best-in-class comparator against a named baseline (no baseline => no claim).
• Finding / FindingsLedger -- a structured ledger of the gaps, flaws, and bugs a probe exposes, written as JSON to a configurable output directory ($OMNIBIAS_RUNS_DIR or a per-user temp dir by default) so large artifacts stay out of the repo tree.

This module is import-light and **optional**: it is never imported by omnibias.difference itself, and mpmath is imported lazily so the core-only import guard stays clean.

Shared formulas do not imply universal floating-point identity. Verify dtype, shape, derivative-order support, and the stated numerical assumptions for the path you use.

Module source ↗Source revision 9a2d73df0573Running reference examples →

CLASS

BaselineComparison

omnibias.difference.validation.BaselineComparisonSource ↗

class BaselineComparison

Dataclass constructor fields; inherited fields, if any, are declared by the base class. A field(...) expression describes a default factory or field options, not a literal argument default.

A best-in-class comparison of a candidate against a named baseline.

Parameters

Name	Type & default	Contract
namedataclass field	strRequired	See the operation contract and implementation below.
candidatedataclass field	floatRequired	See the operation contract and implementation below.
baselinedataclass field	floatRequired	See the operation contract and implementation below.
lower_is_betterdataclass field	boolRequired	See the operation contract and implementation below.
winsdataclass field	boolRequired	See the operation contract and implementation below.
ratiodataclass field	floatRequired	See the operation contract and implementation below.

Regression example · baseline compare lower is better

This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_baseline_compare_lower_is_better() -> None:
    win = baseline_compare("fd", candidate=1e-12, baseline=1e-3)
    assert isinstance(win, BaselineComparison)
    assert win.wins
    assert win.ratio < 1.0
    lose = baseline_compare("fd", candidate=1e-1, baseline=1e-3)
    assert not lose.wins

python -m pytest packages/omnibias-difference/tests/test_validation_harness.py -k test_baseline_compare_lower_is_better -q

Fields & methods

namefield

name: str

Declared str field on BaselineComparison. A value must be provided when constructing the configuration, unless an inherited constructor supplies it.

View source ↗

candidatefield

candidate: float

Declared float field on BaselineComparison. A value must be provided when constructing the configuration, unless an inherited constructor supplies it.

View source ↗

baselinefield

baseline: float

Declared float field on BaselineComparison. A value must be provided when constructing the configuration, unless an inherited constructor supplies it.

View source ↗

lower_is_betterfield

lower_is_better: bool

Declared bool field on BaselineComparison. A value must be provided when constructing the configuration, unless an inherited constructor supplies it.

View source ↗

winsfield

wins: bool

Declared bool field on BaselineComparison. A value must be provided when constructing the configuration, unless an inherited constructor supplies it.

View source ↗

ratiofield

ratio: float

Declared float field on BaselineComparison. A value must be provided when constructing the configuration, unless an inherited constructor supplies it.

View source ↗

Read the implementation

class BaselineComparison:
    """A best-in-class comparison of a candidate against a named baseline."""

    name: str
    candidate: float
    baseline: float
    lower_is_better: bool
    wins: bool
    ratio: float

CLASS

Finding

omnibias.difference.validation.FindingSource ↗

class Finding

Dataclass constructor fields; inherited fields, if any, are declared by the base class. A field(...) expression describes a default factory or field options, not a literal argument default.

One gaps/flaws/bugs entry surfaced by a probe.

Parameters

Name	Type & default	Contract
workstreamdataclass field	strRequired	See the operation contract and implementation below.
severitydataclass field	strRequired	See the operation contract and implementation below.
summarydataclass field	strRequired	See the operation contract and implementation below.
detaildataclass field	strDefault: ''	See the operation contract and implementation below.
reprodataclass field	dict[str, Any]Default: field(default_factory=dict)	See the operation contract and implementation below.

Executed regression · finding rejects bad severity

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_finding_rejects_bad_severity() -> None:
    with pytest.raises(ValueError):
        Finding(workstream="w", severity="nope", summary="s")

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_finding_rejects_bad_severity" -q

Fields & methods

workstreamfield

workstream: str

Declared str field on Finding. A value must be provided when constructing the configuration, unless an inherited constructor supplies it.

View source ↗

severityfield

severity: str

Declared str field on Finding. A value must be provided when constructing the configuration, unless an inherited constructor supplies it.

View source ↗

summaryfield

summary: str

Declared str field on Finding. A value must be provided when constructing the configuration, unless an inherited constructor supplies it.

View source ↗

detailfield

detail: str = ''

Declared str field on Finding. Default expression: ''.

View source ↗

reprofield

repro: dict[str, Any] = field(default_factory=dict)

Declared dict[str, Any] field on Finding. Default expression: field(default_factory=dict).

View source ↗

__post_init__method

__post_init__(self) -> None

See the class contract above and the exact method implementation below.

Returns: None

Validation & failure cases

• When self.severity not in SEVERITIES, raises ValueError(f'severity must be one of {SEVERITIES}, got {self.severity!r}').

Executed regression · finding rejects bad severity

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_finding_rejects_bad_severity() -> None:
    with pytest.raises(ValueError):
        Finding(workstream="w", severity="nope", summary="s")

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_finding_rejects_bad_severity" -q

Method implementation

    def __post_init__(self) -> None:
        if self.severity not in SEVERITIES:
            raise ValueError(f"severity must be one of {SEVERITIES}, got {self.severity!r}")

View source ↗

to_dictmethod

to_dict(self) -> dict[str, Any]

JSON-ready mapping of the finding.

Returns: dict[str, Any]

Executed regression · ledger add counts and json

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_ledger_add_counts_and_json(tmp_path: Path) -> None:
    ledger = FindingsLedger("probe")
    ledger.add("W1", "gap", "missing constant", detail="no PI_IV", where="transcend")
    ledger.add("W1", "bug", "escape", value=1.5)
    assert len(ledger) == 2
    counts = ledger.counts()
    assert counts["gap"] == 1
    assert counts["bug"] == 1
    assert counts["info"] == 0

    path = ledger.write(str(tmp_path / "probe.json"))
    payload = json.loads(Path(path).read_text(encoding="utf-8"))
    assert payload["name"] == "probe"
    assert payload["counts"]["gap"] == 1
    assert len(payload["findings"]) == 2
    assert "missing constant" in ledger.summary()

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_ledger_add_counts_and_json" -q

Method implementation

    def to_dict(self) -> dict[str, Any]:
        """JSON-ready mapping of the finding."""
        return asdict(self)

View source ↗

Read the implementation

class Finding:
    """One gaps/flaws/bugs entry surfaced by a probe."""

    workstream: str
    severity: str
    summary: str
    detail: str = ""
    repro: dict[str, Any] = field(default_factory=dict)

    def __post_init__(self) -> None:
        if self.severity not in SEVERITIES:
            raise ValueError(f"severity must be one of {SEVERITIES}, got {self.severity!r}")

    def to_dict(self) -> dict[str, Any]:
        """JSON-ready mapping of the finding."""
        return asdict(self)

CLASS

FindingsLedger

omnibias.difference.validation.FindingsLedgerSource ↗

class FindingsLedger

A collection of Finding records with JSON serialisation.

Parameters

Name	Type & default	Contract
namepositional-or-keyword	strRequired	See the operation contract and implementation below.

Executed regression · ledger add counts and json

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_ledger_add_counts_and_json(tmp_path: Path) -> None:
    ledger = FindingsLedger("probe")
    ledger.add("W1", "gap", "missing constant", detail="no PI_IV", where="transcend")
    ledger.add("W1", "bug", "escape", value=1.5)
    assert len(ledger) == 2
    counts = ledger.counts()
    assert counts["gap"] == 1
    assert counts["bug"] == 1
    assert counts["info"] == 0

    path = ledger.write(str(tmp_path / "probe.json"))
    payload = json.loads(Path(path).read_text(encoding="utf-8"))
    assert payload["name"] == "probe"
    assert payload["counts"]["gap"] == 1
    assert len(payload["findings"]) == 2
    assert "missing constant" in ledger.summary()

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_ledger_add_counts_and_json" -q

Fields & methods

__init__method

__init__(self, name: str) -> None

See the class contract above and the exact method implementation below.

Parameters

Name	Type & default	Contract
namepositional-or-keyword	strRequired	See the operation contract and implementation below.

Returns: None

Executed regression · ledger add counts and json

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_ledger_add_counts_and_json(tmp_path: Path) -> None:
    ledger = FindingsLedger("probe")
    ledger.add("W1", "gap", "missing constant", detail="no PI_IV", where="transcend")
    ledger.add("W1", "bug", "escape", value=1.5)
    assert len(ledger) == 2
    counts = ledger.counts()
    assert counts["gap"] == 1
    assert counts["bug"] == 1
    assert counts["info"] == 0

    path = ledger.write(str(tmp_path / "probe.json"))
    payload = json.loads(Path(path).read_text(encoding="utf-8"))
    assert payload["name"] == "probe"
    assert payload["counts"]["gap"] == 1
    assert len(payload["findings"]) == 2
    assert "missing constant" in ledger.summary()

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_ledger_add_counts_and_json" -q

Method implementation

    def __init__(self, name: str) -> None:
        self.name = name
        self.findings: list[Finding] = []

View source ↗

addmethod

add(self, workstream: str, severity: str, summary: str, detail: str='', **repro: Any) -> Finding

Record a finding and return it.

Parameters

Name	Type & default	Contract
workstreampositional-or-keyword	strRequired	See the operation contract and implementation below.
severitypositional-or-keyword	strRequired	See the operation contract and implementation below.
summarypositional-or-keyword	strRequired	See the operation contract and implementation below.
detailpositional-or-keyword	strDefault: ''	See the operation contract and implementation below.
reprovariadic keyword	AnyRequired	See the operation contract and implementation below.

Returns: Finding

Executed regression · ledger add counts and json

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_ledger_add_counts_and_json(tmp_path: Path) -> None:
    ledger = FindingsLedger("probe")
    ledger.add("W1", "gap", "missing constant", detail="no PI_IV", where="transcend")
    ledger.add("W1", "bug", "escape", value=1.5)
    assert len(ledger) == 2
    counts = ledger.counts()
    assert counts["gap"] == 1
    assert counts["bug"] == 1
    assert counts["info"] == 0

    path = ledger.write(str(tmp_path / "probe.json"))
    payload = json.loads(Path(path).read_text(encoding="utf-8"))
    assert payload["name"] == "probe"
    assert payload["counts"]["gap"] == 1
    assert len(payload["findings"]) == 2
    assert "missing constant" in ledger.summary()

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_ledger_add_counts_and_json" -q

Method implementation

    def add(
        self, workstream: str, severity: str, summary: str, detail: str = "", **repro: Any
    ) -> Finding:
        """Record a finding and return it."""
        finding = Finding(
            workstream=workstream,
            severity=severity,
            summary=summary,
            detail=detail,
            repro=dict(repro),
        )
        self.findings.append(finding)
        return finding

View source ↗

extendmethod

extend(self, findings: Iterable[Finding]) -> None

Absorb findings from another probe.

Parameters

Name	Type & default	Contract
findingspositional-or-keyword	Iterable[Finding]Required	See the operation contract and implementation below.

Returns: None

Method implementation

    def extend(self, findings: Iterable[Finding]) -> None:
        """Absorb findings from another probe."""
        self.findings.extend(findings)

View source ↗

__len__method

__len__(self) -> int

See the class contract above and the exact method implementation below.

Returns: int

Executed regression · ledger add counts and json

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_ledger_add_counts_and_json(tmp_path: Path) -> None:
    ledger = FindingsLedger("probe")
    ledger.add("W1", "gap", "missing constant", detail="no PI_IV", where="transcend")
    ledger.add("W1", "bug", "escape", value=1.5)
    assert len(ledger) == 2
    counts = ledger.counts()
    assert counts["gap"] == 1
    assert counts["bug"] == 1
    assert counts["info"] == 0

    path = ledger.write(str(tmp_path / "probe.json"))
    payload = json.loads(Path(path).read_text(encoding="utf-8"))
    assert payload["name"] == "probe"
    assert payload["counts"]["gap"] == 1
    assert len(payload["findings"]) == 2
    assert "missing constant" in ledger.summary()

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_ledger_add_counts_and_json" -q

Method implementation

    def __len__(self) -> int:
        return len(self.findings)

View source ↗

__iter__method

__iter__(self) -> Iterator[Finding]

See the class contract above and the exact method implementation below.

Returns: Iterator[Finding]

Method implementation

    def __iter__(self) -> Iterator[Finding]:
        return iter(self.findings)

View source ↗

countsmethod

counts(self) -> dict[str, int]

Count of findings by severity (every severity present, possibly 0).

Returns: dict[str, int]

Executed regression · ledger add counts and json

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_ledger_add_counts_and_json(tmp_path: Path) -> None:
    ledger = FindingsLedger("probe")
    ledger.add("W1", "gap", "missing constant", detail="no PI_IV", where="transcend")
    ledger.add("W1", "bug", "escape", value=1.5)
    assert len(ledger) == 2
    counts = ledger.counts()
    assert counts["gap"] == 1
    assert counts["bug"] == 1
    assert counts["info"] == 0

    path = ledger.write(str(tmp_path / "probe.json"))
    payload = json.loads(Path(path).read_text(encoding="utf-8"))
    assert payload["name"] == "probe"
    assert payload["counts"]["gap"] == 1
    assert len(payload["findings"]) == 2
    assert "missing constant" in ledger.summary()

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_ledger_add_counts_and_json" -q

Method implementation

    def counts(self) -> dict[str, int]:
        """Count of findings by severity (every severity present, possibly ``0``)."""
        out = {severity: 0 for severity in SEVERITIES}
        for finding in self.findings:
            out[finding.severity] += 1
        return out

View source ↗

to_jsonmethod

to_json(self) -> str

Canonical, deterministic JSON of the ledger.

Returns: str

Executed regression · ledger add counts and json

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_ledger_add_counts_and_json(tmp_path: Path) -> None:
    ledger = FindingsLedger("probe")
    ledger.add("W1", "gap", "missing constant", detail="no PI_IV", where="transcend")
    ledger.add("W1", "bug", "escape", value=1.5)
    assert len(ledger) == 2
    counts = ledger.counts()
    assert counts["gap"] == 1
    assert counts["bug"] == 1
    assert counts["info"] == 0

    path = ledger.write(str(tmp_path / "probe.json"))
    payload = json.loads(Path(path).read_text(encoding="utf-8"))
    assert payload["name"] == "probe"
    assert payload["counts"]["gap"] == 1
    assert len(payload["findings"]) == 2
    assert "missing constant" in ledger.summary()

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_ledger_add_counts_and_json" -q

Method implementation

    def to_json(self) -> str:
        """Canonical, deterministic JSON of the ledger."""
        return json.dumps(
            {
                "name": self.name,
                "counts": self.counts(),
                "findings": [finding.to_dict() for finding in self.findings],
            },
            indent=2,
            sort_keys=True,
        )

View source ↗

writemethod

write(self, path: str | None=None) -> str

Write the ledger JSON (default: the scratch findings dir); return the path.

Parameters

Name	Type & default	Contract
pathpositional-or-keyword	str | NoneDefault: None	See the operation contract and implementation below.

Returns: str

Executed regression · ledger add counts and json

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_ledger_add_counts_and_json(tmp_path: Path) -> None:
    ledger = FindingsLedger("probe")
    ledger.add("W1", "gap", "missing constant", detail="no PI_IV", where="transcend")
    ledger.add("W1", "bug", "escape", value=1.5)
    assert len(ledger) == 2
    counts = ledger.counts()
    assert counts["gap"] == 1
    assert counts["bug"] == 1
    assert counts["info"] == 0

    path = ledger.write(str(tmp_path / "probe.json"))
    payload = json.loads(Path(path).read_text(encoding="utf-8"))
    assert payload["name"] == "probe"
    assert payload["counts"]["gap"] == 1
    assert len(payload["findings"]) == 2
    assert "missing constant" in ledger.summary()

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_ledger_add_counts_and_json" -q

Method implementation

    def write(self, path: str | None = None) -> str:
        """Write the ledger JSON (default: the scratch findings dir); return the path."""
        if path is None:
            directory = default_ledger_dir()
            os.makedirs(directory, exist_ok=True)
            path = os.path.join(directory, f"{self.name}.json")
        else:
            parent = os.path.dirname(path)
            if parent:
                os.makedirs(parent, exist_ok=True)
        with open(path, "w", encoding="utf-8") as handle:
            handle.write(self.to_json())
        return path

View source ↗

summarymethod

summary(self) -> str

A short human-readable digest for the CI smoke output.

Returns: str

Executed regression · ledger add counts and json

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_ledger_add_counts_and_json(tmp_path: Path) -> None:
    ledger = FindingsLedger("probe")
    ledger.add("W1", "gap", "missing constant", detail="no PI_IV", where="transcend")
    ledger.add("W1", "bug", "escape", value=1.5)
    assert len(ledger) == 2
    counts = ledger.counts()
    assert counts["gap"] == 1
    assert counts["bug"] == 1
    assert counts["info"] == 0

    path = ledger.write(str(tmp_path / "probe.json"))
    payload = json.loads(Path(path).read_text(encoding="utf-8"))
    assert payload["name"] == "probe"
    assert payload["counts"]["gap"] == 1
    assert len(payload["findings"]) == 2
    assert "missing constant" in ledger.summary()

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_ledger_add_counts_and_json" -q

Method implementation

    def summary(self) -> str:
        """A short human-readable digest for the CI smoke output."""
        counts = self.counts()
        head = f"[{self.name}] " + ", ".join(f"{k}={counts[k]}" for k in SEVERITIES)
        lines = [head]
        for finding in self.findings:
            lines.append(f"  - ({finding.severity}) [{finding.workstream}] {finding.summary}")
        return "\n".join(lines)

View source ↗

nameattribute

name: str = name

Public instance attribute assigned by __init__. The source assignment is name = name; it describes instance state, not a literal constructor default.

View source ↗

findingsattribute

findings: list[Finding] = []

Public instance attribute assigned by __init__. The source assignment is findings = []; it describes instance state, not a literal constructor default.

View source ↗

Read the implementation

class FindingsLedger:
    """A collection of :class:`Finding` records with JSON serialisation."""

    def __init__(self, name: str) -> None:
        self.name = name
        self.findings: list[Finding] = []

    def add(
        self, workstream: str, severity: str, summary: str, detail: str = "", **repro: Any
    ) -> Finding:
        """Record a finding and return it."""
        finding = Finding(
            workstream=workstream,
            severity=severity,
            summary=summary,
            detail=detail,
            repro=dict(repro),
        )
        self.findings.append(finding)
        return finding

    def extend(self, findings: Iterable[Finding]) -> None:
        """Absorb findings from another probe."""
        self.findings.extend(findings)

    def __len__(self) -> int:
        return len(self.findings)

    def __iter__(self) -> Iterator[Finding]:
        return iter(self.findings)

    def counts(self) -> dict[str, int]:
        """Count of findings by severity (every severity present, possibly ``0``)."""
        out = {severity: 0 for severity in SEVERITIES}
        for finding in self.findings:
            out[finding.severity] += 1
        return out

    def to_json(self) -> str:
        """Canonical, deterministic JSON of the ledger."""
        return json.dumps(
            {
                "name": self.name,
                "counts": self.counts(),
                "findings": [finding.to_dict() for finding in self.findings],
            },
            indent=2,
            sort_keys=True,
        )

    def write(self, path: str | None = None) -> str:
        """Write the ledger JSON (default: the scratch findings dir); return the path."""
        if path is None:
            directory = default_ledger_dir()
            os.makedirs(directory, exist_ok=True)
            path = os.path.join(directory, f"{self.name}.json")
        else:
            parent = os.path.dirname(path)
            if parent:
                os.makedirs(parent, exist_ok=True)
        with open(path, "w", encoding="utf-8") as handle:
            handle.write(self.to_json())
        return path

    def summary(self) -> str:
        """A short human-readable digest for the CI smoke output."""
        counts = self.counts()
        head = f"[{self.name}] " + ", ".join(f"{k}={counts[k]}" for k in SEVERITIES)
        lines = [head]
        for finding in self.findings:
            lines.append(f"  - ({finding.severity}) [{finding.workstream}] {finding.summary}")
        return "\n".join(lines)

Related APIsomnibias.difference.validation.Finding↗omnibias.difference.validation.default_ledger_dir↗

CONSTANT

SEVERITIES

omnibias.difference.validation.SEVERITIESSource ↗

SEVERITIES: tuple[str, ...] = ('info', 'gap', 'flaw', 'bug')

Severity tags, ascending: an info note, a missing capability (gap), a correctness/tightness weakness (flaw), or an outright bug.

Read the implementation

SEVERITIES: tuple[str, ...] = ("info", "gap", "flaw", "bug")

CLASS

SoundnessReport

omnibias.difference.validation.SoundnessReportSource ↗

class SoundnessReport

Dataclass constructor fields; inherited fields, if any, are declared by the base class. A field(...) expression describes a default factory or field options, not a literal argument default.

Result of a grid-and-random enclosure soundness check.

Parameters

Name	Type & default	Contract
sounddataclass field	boolRequired	See the operation contract and implementation below.
n_griddataclass field	intRequired	See the operation contract and implementation below.
n_randomdataclass field	intRequired	See the operation contract and implementation below.
max_escapedataclass field	floatRequired	See the operation contract and implementation below.
failuresdataclass field	tuple[tuple[float, float], ...]Required	See the operation contract and implementation below.

Executed regression · enclosure soundness detects containment and escape

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_enclosure_soundness_detects_containment_and_escape() -> None:
    box = Interval(0.0, 1.0)
    ok = enclosure_soundness(Interval(-0.1, 1.1), lambda x: x, box, grid=11, random_samples=10)
    assert ok.sound
    assert not ok.failures
    assert ok.max_escape == 0.0
    assert ok.n_points == 21

    bad = enclosure_soundness(Interval(0.0, 0.5), lambda x: x, box, grid=11, random_samples=0)
    assert not bad.sound
    assert bad.failures
    assert bad.max_escape > 0.0

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_enclosure_soundness_detects_containment_and_escape" -q

Fields & methods

soundfield

sound: bool

Declared bool field on SoundnessReport. A value must be provided when constructing the configuration, unless an inherited constructor supplies it.

View source ↗

n_gridfield

n_grid: int

Declared int field on SoundnessReport. A value must be provided when constructing the configuration, unless an inherited constructor supplies it.

View source ↗

n_randomfield

n_random: int

Declared int field on SoundnessReport. A value must be provided when constructing the configuration, unless an inherited constructor supplies it.

View source ↗

max_escapefield

max_escape: float

Declared float field on SoundnessReport. A value must be provided when constructing the configuration, unless an inherited constructor supplies it.

View source ↗

failuresfield

failures: tuple[tuple[float, float], ...]

Declared tuple[tuple[float, float], ...] field on SoundnessReport. A value must be provided when constructing the configuration, unless an inherited constructor supplies it.

View source ↗

n_pointsproperty

n_points(self) -> int

Total number of oracle points checked.

Returns: int

Executed regression · enclosure soundness detects containment and escape

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_enclosure_soundness_detects_containment_and_escape() -> None:
    box = Interval(0.0, 1.0)
    ok = enclosure_soundness(Interval(-0.1, 1.1), lambda x: x, box, grid=11, random_samples=10)
    assert ok.sound
    assert not ok.failures
    assert ok.max_escape == 0.0
    assert ok.n_points == 21

    bad = enclosure_soundness(Interval(0.0, 0.5), lambda x: x, box, grid=11, random_samples=0)
    assert not bad.sound
    assert bad.failures
    assert bad.max_escape > 0.0

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_enclosure_soundness_detects_containment_and_escape" -q

Method implementation

    def n_points(self) -> int:
        """Total number of oracle points checked."""
        return self.n_grid + self.n_random

View source ↗

Read the implementation

class SoundnessReport:
    """Result of a grid-and-random enclosure soundness check."""

    sound: bool
    n_grid: int
    n_random: int
    max_escape: float
    failures: tuple[tuple[float, float], ...]

    @property
    def n_points(self) -> int:
        """Total number of oracle points checked."""
        return self.n_grid + self.n_random

FUNCTION

baseline_compare

omnibias.difference.validation.baseline_compareSource ↗

baseline_compare(name: str, candidate: float, baseline: float, *, lower_is_better: bool=True, tol: float=0.0) -> BaselineComparison

Compare a candidate metric to a named baseline (lower_is_better by default).

Parameters

Name	Type & default	Contract
namepositional-or-keyword	strRequired	See the operation contract and implementation below.
candidatepositional-or-keyword	floatRequired	See the operation contract and implementation below.
baselinepositional-or-keyword	floatRequired	See the operation contract and implementation below.
lower_is_betterkeyword-only	boolDefault: True	See the operation contract and implementation below.
tolkeyword-only	floatDefault: 0.0	See the operation contract and implementation below.

ReturnsBaselineComparison

Executed regression · baseline compare lower is better

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_baseline_compare_lower_is_better() -> None:
    win = baseline_compare("fd", candidate=1e-12, baseline=1e-3)
    assert isinstance(win, BaselineComparison)
    assert win.wins
    assert win.ratio < 1.0
    lose = baseline_compare("fd", candidate=1e-1, baseline=1e-3)
    assert not lose.wins

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_baseline_compare_lower_is_better" -q

Read the implementation

def baseline_compare(
    name: str,
    candidate: float,
    baseline: float,
    *,
    lower_is_better: bool = True,
    tol: float = 0.0,
) -> BaselineComparison:
    """Compare a candidate metric to a named baseline (``lower_is_better`` by default)."""
    if lower_is_better:
        wins = candidate <= baseline + tol
    else:
        wins = candidate + tol >= baseline
    ratio = candidate / baseline if baseline != 0.0 else math.inf
    return BaselineComparison(
        name=name,
        candidate=candidate,
        baseline=baseline,
        lower_is_better=lower_is_better,
        wins=wins,
        ratio=ratio,
    )

Related APIsomnibias.difference.validation.BaselineComparison↗

FUNCTION

default_ledger_dir

omnibias.difference.validation.default_ledger_dirSource ↗

default_ledger_dir() -> str

Directory for findings JSON (kept out of the repo tree by default).

Override with the OMNIBIAS_RUNS_DIR environment variable; otherwise a per-user temporary directory is used so large artifacts never land in the working tree.

Returnsstr

Read the implementation

def default_ledger_dir() -> str:
    """Directory for findings JSON (kept out of the repo tree by default).

    Override with the ``OMNIBIAS_RUNS_DIR`` environment variable; otherwise a
    per-user temporary directory is used so large artifacts never land in the
    working tree.
    """
    base = os.environ.get("OMNIBIAS_RUNS_DIR")
    if base:
        return os.path.join(os.path.expanduser(base), "difference_refine")
    return os.path.join(tempfile.gettempdir(), "omnibias_runs", "difference_refine")

FUNCTION

enclosure_soundness

omnibias.difference.validation.enclosure_soundnessSource ↗

enclosure_soundness(enclosure: Interval, oracle: Callable[[float], float], box: Interval, *, grid: int=25, random_samples: int=40, seed: int=0) -> SoundnessReport

Check that enclosure contains oracle over a grid-and-random sample of box.

This is the delta -> 0 acceptance gate: the interval enclosure must contain a dense deterministic grid **and** a random sample of true values. A single escaping point makes the report sound=False -- a soundness bug, never something to be papered over by widening.

Parameters

Name	Type & default	Contract
enclosurepositional-or-keyword	IntervalRequired	See the operation contract and implementation below.
oraclepositional-or-keyword	Callable[[float], float]Required	See the operation contract and implementation below.
boxpositional-or-keyword	IntervalRequired	See the operation contract and implementation below.
gridkeyword-only	intDefault: 25	See the operation contract and implementation below.
random_sampleskeyword-only	intDefault: 40	See the operation contract and implementation below.
seedkeyword-only	intDefault: 0	See the operation contract and implementation below.

ReturnsSoundnessReport

Executed regression · enclosure soundness detects containment and escape

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_enclosure_soundness_detects_containment_and_escape() -> None:
    box = Interval(0.0, 1.0)
    ok = enclosure_soundness(Interval(-0.1, 1.1), lambda x: x, box, grid=11, random_samples=10)
    assert ok.sound
    assert not ok.failures
    assert ok.max_escape == 0.0
    assert ok.n_points == 21

    bad = enclosure_soundness(Interval(0.0, 0.5), lambda x: x, box, grid=11, random_samples=0)
    assert not bad.sound
    assert bad.failures
    assert bad.max_escape > 0.0

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_enclosure_soundness_detects_containment_and_escape" -q

Read the implementation

def enclosure_soundness(
    enclosure: Interval,
    oracle: Callable[[float], float],
    box: Interval,
    *,
    grid: int = 25,
    random_samples: int = 40,
    seed: int = 0,
) -> SoundnessReport:
    """Check that ``enclosure`` contains ``oracle`` over a grid-and-random sample of ``box``.

    This is the ``delta -> 0`` acceptance gate: the interval enclosure must
    contain a dense deterministic grid **and** a random sample of true values. A
    single escaping point makes the report ``sound=False`` -- a soundness bug,
    never something to be papered over by widening.
    """
    failures: list[tuple[float, float]] = []
    max_escape = 0.0
    for x in grid_and_random_points(box, grid=grid, random_samples=random_samples, seed=seed):
        value = oracle(x)
        if not (enclosure.lo <= value <= enclosure.hi):
            escape = max(enclosure.lo - value, value - enclosure.hi)
            max_escape = max(max_escape, escape)
            failures.append((x, value))
    return SoundnessReport(
        sound=not failures,
        n_grid=grid,
        n_random=random_samples,
        max_escape=max_escape,
        failures=tuple(failures),
    )

Related APIsomnibias.difference.validation.SoundnessReport↗omnibias.difference.validation.grid_and_random_points↗

FUNCTION

grid_and_random_points

omnibias.difference.validation.grid_and_random_pointsSource ↗

grid_and_random_points(box: Interval, *, grid: int=25, random_samples: int=40, seed: int=0) -> list[float]

A dense deterministic grid over box plus a seeded random sample.

Parameters

Name	Type & default	Contract
boxpositional-or-keyword	IntervalRequired	See the operation contract and implementation below.
gridkeyword-only	intDefault: 25	See the operation contract and implementation below.
random_sampleskeyword-only	intDefault: 40	See the operation contract and implementation below.
seedkeyword-only	intDefault: 0	See the operation contract and implementation below.

Returnslist[float]

Explicit validation & failure cases

• When grid < 0 or random_samples < 0, raises ValueError('grid and random_samples must be >= 0').

These are explicit source guards. Backend operations may also raise for invalid shapes, dtypes, or unsupported devices.

Executed regression · grid and random points count and determinism

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_grid_and_random_points_count_and_determinism() -> None:
    box = Interval(-1.0, 2.0)
    pts = grid_and_random_points(box, grid=10, random_samples=5, seed=0)
    assert len(pts) == 15
    assert all(box.lo <= p <= box.hi for p in pts)
    assert pts == grid_and_random_points(box, grid=10, random_samples=5, seed=0)

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_grid_and_random_points_count_and_determinism" -q

Read the implementation

def grid_and_random_points(
    box: Interval, *, grid: int = 25, random_samples: int = 40, seed: int = 0
) -> list[float]:
    """A dense deterministic grid over ``box`` plus a seeded random sample."""
    if grid < 0 or random_samples < 0:
        raise ValueError("grid and random_samples must be >= 0")
    lo, hi = box.lo, box.hi
    points: list[float] = []
    if grid == 1:
        points.append(0.5 * (lo + hi))
    elif grid > 1:
        step = (hi - lo) / (grid - 1)
        points.extend(lo + i * step for i in range(grid))
    rng = random.Random(seed)
    points.extend(rng.uniform(lo, hi) for _ in range(random_samples))
    return points

FUNCTION

high_precision_derivative

omnibias.difference.validation.high_precision_derivativeSource ↗

high_precision_derivative(func: Callable[[Any], Any], z0: float, order: int, *, dps: int=50) -> float

High-precision func^(order)(z0) via mpmath.taylor (the numerical oracle).

Parameters

Name	Type & default	Contract
funcpositional-or-keyword	Callable[[Any], Any]Required	See the operation contract and implementation below.
z0positional-or-keyword	floatRequired	See the operation contract and implementation below.
orderpositional-or-keyword	intRequired	See the operation contract and implementation below.
dpskeyword-only	intDefault: 50	See the operation contract and implementation below.

Returnsfloat

Explicit validation & failure cases

• When order < 0, raises ValueError(f'order must be >= 0, got {order}').

These are explicit source guards. Backend operations may also raise for invalid shapes, dtypes, or unsupported devices.

Executed regression · high precision derivative matches known

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_high_precision_derivative_matches_known() -> None:
    mpmath = pytest.importorskip("mpmath")
    # d^3/dz^3 tanh(0) = -2.
    got = high_precision_derivative(mpmath.tanh, 0.0, 3)
    assert abs(got - (-2.0)) < 1e-9

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_high_precision_derivative_matches_known" -q

Read the implementation

def high_precision_derivative(
    func: Callable[[Any], Any], z0: float, order: int, *, dps: int = 50
) -> float:
    """High-precision ``func^(order)(z0)`` via ``mpmath.taylor`` (the numerical oracle)."""
    if order < 0:
        raise ValueError(f"order must be >= 0, got {order}")
    mp = require_mpmath()
    with mp.workdps(dps):
        taylor = mp.taylor(func, z0, order)
        return float(taylor[order] * math.factorial(order))

Related APIsomnibias.difference.validation.require_mpmath↗

FUNCTION

require_mpmath

omnibias.difference.validation.require_mpmathSource ↗

require_mpmath() -> Any

Return the mpmath module or raise a helpful error (the probe oracle).

ReturnsAny

Executed regression · high precision derivative matches known

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_high_precision_derivative_matches_known() -> None:
    mpmath = pytest.importorskip("mpmath")
    # d^3/dz^3 tanh(0) = -2.
    got = high_precision_derivative(mpmath.tanh, 0.0, 3)
    assert abs(got - (-2.0)) < 1e-9

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_high_precision_derivative_matches_known" -q

Read the implementation

def require_mpmath() -> Any:
    """Return the ``mpmath`` module or raise a helpful error (the probe oracle)."""
    try:
        return importlib.import_module("mpmath")
    except ImportError as exc:  # pragma: no cover - the [test] extra ships mpmath
        raise RuntimeError(
            "this probe needs mpmath: pip install 'omnibias-difference[test]'"
        ) from exc

FUNCTION

seed_sweep

omnibias.difference.validation.seed_sweepSource ↗

seed_sweep(metric: Callable[[int], float], seeds: Iterable[int]) -> dict[str, float]

Aggregate a per-seed metric across seeds (the anti-overfitting rule).

Parameters

Name	Type & default	Contract
metricpositional-or-keyword	Callable[[int], float]Required	See the operation contract and implementation below.
seedspositional-or-keyword	Iterable[int]Required	See the operation contract and implementation below.

Returnsdict[str, float]

Explicit validation & failure cases

• When not values, raises ValueError('seeds must be non-empty').

These are explicit source guards. Backend operations may also raise for invalid shapes, dtypes, or unsupported devices.

Executed regression · seed sweep aggregates

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_seed_sweep_aggregates() -> None:
    stats = seed_sweep(float, range(5))
    assert stats["min"] == 0.0
    assert stats["max"] == 4.0
    assert stats["mean"] == 2.0
    assert stats["n"] == 5.0

python -m pytest "packages/omnibias-difference/tests/test_validation_harness.py::test_seed_sweep_aggregates" -q

Read the implementation

def seed_sweep(metric: Callable[[int], float], seeds: Iterable[int]) -> dict[str, float]:
    """Aggregate a per-seed metric across ``seeds`` (the anti-overfitting rule)."""
    values = [metric(seed) for seed in seeds]
    if not values:
        raise ValueError("seeds must be non-empty")
    return {
        "min": min(values),
        "max": max(values),
        "mean": sum(values) / len(values),
        "n": float(len(values)),
    }

← Package overviewRunnable learning examples →