omnibias.curvature.regularize

0.1.0a2AlphaPython >=3.10AGPL / commercial

Scoped parameter-curvature formulas, Fisher and Gauss–Newton constructions, natural steps, and sharpness utilities.

import omnibias.curvature.regularize

Module contract & mathematical conventions

The eps -> 0 rank / regularization collapse (JAX differentiable register).

Tikhonov-regularized solving collapses onto the Moore-Penrose / minimum-norm solution as the damping vanishes:

Taken naively the eps -> 0 limit **blows up** whenever A is rank-deficient or ill-conditioned. This module does the limit *stably* and pairs it with the rigorous conditioning certificate in omnibias.core.verified.conditioning:

regularized_solve -- (A + eps I)^{-1} b (the generic damped solve that omnibias.curvature.damped_solve and omnibias.curvature.mse_newton_step now delegate to).
min_norm_solve -- the collapse limit done right: a symmetric eigendecomposition with small-eigenvalue truncation yields A^+ b with no blow-up.
regularization_path -- the measured homotopy x(eps) over a grid (the data-driven curve behind an eps / rcond choice).
rank_collapse -- the high-level entry: pick a certified damping for a target condition number (or take the min-norm limit), solve, and attach a sealed ~omnibias.core.verified.conditioning.conditioning_certificate.

Collapse honesty

This eps -> 0 collapse is a **distinct** limit from the founding multi-bias delta -> 0 derivative collapse (omnibias.torch.unit) and from the beta -> inf feasibility penalty used in the discrete-optimization packages -- same spirit (a parameter driven to a limit that recovers a canonical object), different parameter, never conflated. Nothing here rides the sigma derivative tower: the omnibias value is the *stable collapse* plus the *rigorous certificate*. The solves themselves (jnp.linalg.solve / eigh) are ordinary numerical (LAPACK-class) operations; only the conditioning enclosure is verified. The input matrix may of course come from omnibias closed-form curvature (omnibias.curvature.mse_gauss_newton_fisher, omnibias.curvature.glm_fisher.glm_fisher, omnibias.curvature.mse_loss_hessian).

Shared formulas do not imply universal floating-point identity. Verify dtype, shape, derivative-order support, and the stated numerical assumptions for the path you use.

Module source ↗Source revision 9a2d73df0573Running reference examples →

CLASS

CollapseResult

omnibias.curvature.CollapseResultSource ↗

class CollapseResult

Dataclass constructor fields; inherited fields, if any, are declared by the base class. A field(...) expression describes a default factory or field options, not a literal argument default.

The outcome of a rank / regularization collapse.

Attributes

solution: The solved direction x (min-norm A^+ b or certified-damped (A + eps I)^{-1} b). effective_rank: The numerical rank of A at the rcond cutoff. eps: The damping actually used (0.0 on the pure min-norm path). certificate: The sealed conditioning certificate, or None when certify=False.

Parameters

Name	Type & default	Contract
`solution`dataclass field	`Array`Required	The solved direction `x` (min-norm `A^+ b` or certified-damped `(A + eps I)^{-1} b`).
`effective_rank`dataclass field	`int`Required	The numerical rank of `A` at the `rcond` cutoff.
`eps`dataclass field	`float`Required	The damping actually used (`0.0` on the pure min-norm path).
`certificate`dataclass field	`dict[str, Any] \| None`Required	The sealed conditioning certificate, or `None` when `certify=False`.

Regression example · rank collapse target condition meets target

This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_rank_collapse_target_condition_meets_target() -> None:
    rng = np.random.default_rng(6)
    A_np = _spd(rng, 5, cond=1e6)  # deliberately ill-conditioned
    A = jnp.asarray(A_np)
    b = jnp.asarray(rng.normal(size=5))
    res = rank_collapse(A, b, target_condition=100.0)
    assert isinstance(res, CollapseResult)
    assert res.eps > 0.0
    # the certified damping provably meets the target (numpy oracle re-check)
    assert np.linalg.cond(A_np + res.eps * np.eye(5)) <= 100.0 + 1e-6
    assert res.certificate is not None and verify_certificate_digest(res.certificate)
    assert res.certificate["payload"]["type"] == "conditioning"

python -m pytest packages/omnibias-curvature/tests/test_regularize.py -k test_rank_collapse_target_condition_meets_target -q

Fields & methods

solutionfield

solution: Array

The solved direction x (min-norm A^+ b or certified-damped (A + eps I)^{-1} b).

View source ↗

effective_rankfield

effective_rank: int

The numerical rank of A at the rcond cutoff.

View source ↗

epsfield

eps: float

The damping actually used (0.0 on the pure min-norm path).

View source ↗

certificatefield

certificate: dict[str, Any] | None

The sealed conditioning certificate, or None when certify=False.

View source ↗

Read the implementation

class CollapseResult:
    """The outcome of a rank / regularization collapse.

    Attributes
    ----------
    solution:
        The solved direction ``x`` (min-norm ``A^+ b`` or certified-damped
        ``(A + eps I)^{-1} b``).
    effective_rank:
        The numerical rank of ``A`` at the ``rcond`` cutoff.
    eps:
        The damping actually used (``0.0`` on the pure min-norm path).
    certificate:
        The sealed conditioning certificate, or ``None`` when ``certify=False``.
    """

    solution: Array
    effective_rank: int
    eps: float
    certificate: dict[str, Any] | None

Other public import paths

omnibias.curvature.CollapseResult
omnibias.curvature.regularize.CollapseResult

FUNCTION

min_norm_solve

omnibias.curvature.min_norm_solveSource ↗

min_norm_solve(matrix: Array, rhs: Array, *, rcond: float | None=None) -> Array

The eps -> 0 limit done right: the minimum-norm / pseudoinverse solution.

Computes A^+ b via a symmetric eigendecomposition with small-eigenvalue truncation (eigenvalues with |lambda| <= rcond * max|lambda| are treated as zero and dropped from the inverse), so the limit is taken *stably* instead of letting (A + eps I)^{-1} b blow up. matrix must be symmetric. rcond defaults to max(P) * eps(dtype).

Parameters

Name	Type & default	Contract
`matrix`positional-or-keyword	`Array`Required	See the operation contract and implementation below.
`rhs`positional-or-keyword	`Array`Required	See the operation contract and implementation below.
`rcond`keyword-only	`float \| None`Default: None	See the operation contract and implementation below.

ReturnsArray

Executed regression · min norm solve matches pinv full rank

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_min_norm_solve_matches_pinv_full_rank() -> None:
    rng = np.random.default_rng(2)
    for _ in range(10):
        p = rng.integers(2, 6)
        A = _spd(rng, p)
        b = rng.normal(size=p)
        x = min_norm_solve(jnp.asarray(A), jnp.asarray(b), rcond=1e-12)
        x0 = np.linalg.pinv(A, rcond=1e-12) @ b
        assert np.allclose(np.asarray(x), x0, rtol=1e-8, atol=1e-8)

python -m pytest "packages/omnibias-curvature/tests/test_regularize.py::test_min_norm_solve_matches_pinv_full_rank" -q

Regression example · min norm solve matches pinv rank deficient

This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_min_norm_solve_matches_pinv_rank_deficient() -> None:
    rng = np.random.default_rng(3)
    for _ in range(10):
        p = int(rng.integers(3, 6))
        r = p - 1
        A = _low_rank(rng, p, r)
        y = rng.normal(size=p)
        b = A @ y  # range-consistent
        x = min_norm_solve(jnp.asarray(A), jnp.asarray(b), rcond=1e-9)
        x0 = np.linalg.pinv(A, rcond=1e-9) @ b
        assert np.allclose(np.asarray(x), x0, rtol=1e-7, atol=1e-7)
        assert numerical_rank(jnp.asarray(A), rcond=1e-9) == r

python -m pytest packages/omnibias-curvature/tests/test_regularize.py -k test_min_norm_solve_matches_pinv_rank_deficient -q

Read the implementation

def min_norm_solve(matrix: Array, rhs: Array, *, rcond: float | None = None) -> Array:
    r"""The ``eps -> 0`` limit done right: the minimum-norm / pseudoinverse solution.

    Computes ``A^+ b`` via a symmetric eigendecomposition with small-eigenvalue
    truncation (eigenvalues with ``|lambda| <= rcond * max|lambda|`` are treated as
    zero and dropped from the inverse), so the limit is taken *stably* instead of
    letting ``(A + eps I)^{-1} b`` blow up. ``matrix`` must be symmetric. ``rcond``
    defaults to ``max(P) * eps(dtype)``.
    """
    _validate_square_system(matrix, rhs)
    w, vecs = jnp.linalg.eigh(matrix)
    rc = _default_rcond(matrix) if rcond is None else float(rcond)
    cutoff = rc * jnp.max(jnp.abs(w))
    keep = jnp.abs(w) > cutoff
    safe_w = jnp.where(keep, w, 1.0)
    inv_w = jnp.where(keep, 1.0 / safe_w, 0.0)
    out: Array = vecs @ (inv_w * (vecs.T @ rhs))
    return out

Other public import paths

omnibias.curvature.min_norm_solve
omnibias.curvature.regularize.min_norm_solve

FUNCTION

numerical_rank

omnibias.curvature.numerical_rankSource ↗

numerical_rank(matrix: Array, *, rcond: float | None=None) -> int

The numerical rank of a symmetric matrix at the rcond cutoff.

Parameters

Name	Type & default	Contract
`matrix`positional-or-keyword	`Array`Required	See the operation contract and implementation below.
`rcond`keyword-only	`float \| None`Default: None	See the operation contract and implementation below.

Returnsint

Executed regression · rank collapse certify false has no certificate

Full file, imports & fixtures ↗

def test_rank_collapse_certify_false_has_no_certificate() -> None:
    rng = np.random.default_rng(8)
    A = jnp.asarray(_spd(rng, 4))
    b = jnp.asarray(rng.normal(size=4))
    res = rank_collapse(A, b, target_condition=50.0, certify=False)
    assert res.certificate is None

python -m pytest "packages/omnibias-curvature/tests/test_regularize.py::test_rank_collapse_certify_false_has_no_certificate" -q

Regression example · min norm solve matches pinv rank deficient

This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_min_norm_solve_matches_pinv_rank_deficient() -> None:
    rng = np.random.default_rng(3)
    for _ in range(10):
        p = int(rng.integers(3, 6))
        r = p - 1
        A = _low_rank(rng, p, r)
        y = rng.normal(size=p)
        b = A @ y  # range-consistent
        x = min_norm_solve(jnp.asarray(A), jnp.asarray(b), rcond=1e-9)
        x0 = np.linalg.pinv(A, rcond=1e-9) @ b
        assert np.allclose(np.asarray(x), x0, rtol=1e-7, atol=1e-7)
        assert numerical_rank(jnp.asarray(A), rcond=1e-9) == r

python -m pytest packages/omnibias-curvature/tests/test_regularize.py -k test_min_norm_solve_matches_pinv_rank_deficient -q

Read the implementation

def numerical_rank(matrix: Array, *, rcond: float | None = None) -> int:
    r"""The numerical rank of a symmetric ``matrix`` at the ``rcond`` cutoff."""
    _validate_square_system(matrix, matrix[:, 0])
    w = jnp.linalg.eigvalsh(matrix)
    rc = _default_rcond(matrix) if rcond is None else float(rcond)
    cutoff = rc * jnp.max(jnp.abs(w))
    return int(jnp.sum(jnp.abs(w) > cutoff))

Other public import paths

omnibias.curvature.numerical_rank
omnibias.curvature.regularize.numerical_rank

FUNCTION

rank_collapse

omnibias.curvature.rank_collapseSource ↗

rank_collapse(matrix: Array, rhs: Array, *, target_condition: float | None=None, rcond: float | None=None, certify: bool=True) -> CollapseResult

High-level eps -> 0 collapse: min-norm limit or certified-damped solve.

Two modes:

target_condition is None -- take the collapse limit directly with min_norm_solve (eps = 0, A^+ b).
target_condition set -- pick the smallest damping eps that provably brings kappa(A + eps I) at or below the target (via omnibias.core.verified.conditioning.certified_damping) and solve (A + eps I)^{-1} b.

When certify is set the result carries a sealed ~omnibias.core.verified.conditioning.conditioning_certificate (lambda_min / lambda_max / kappa enclosures + the lambda_min > 0 pivot data). theorem_prover_verified is *not* asserted here -- it is earned only by driving the sealed certificate through the Lean bridge.

Parameters

Name	Type & default	Contract
`matrix`positional-or-keyword	`Array`Required	See the operation contract and implementation below.
`rhs`positional-or-keyword	`Array`Required	See the operation contract and implementation below.
`target_condition`keyword-only	`float \| None`Default: None	See the operation contract and implementation below.
`rcond`keyword-only	`float \| None`Default: None	See the operation contract and implementation below.
`certify`keyword-only	`bool`Default: True	See the operation contract and implementation below.

ReturnsCollapseResult

Executed regression · rank collapse certify false has no certificate

Full file, imports & fixtures ↗

def test_rank_collapse_certify_false_has_no_certificate() -> None:
    rng = np.random.default_rng(8)
    A = jnp.asarray(_spd(rng, 4))
    b = jnp.asarray(rng.normal(size=4))
    res = rank_collapse(A, b, target_condition=50.0, certify=False)
    assert res.certificate is None

python -m pytest "packages/omnibias-curvature/tests/test_regularize.py::test_rank_collapse_certify_false_has_no_certificate" -q

Regression example · rank collapse target condition meets target

This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_rank_collapse_target_condition_meets_target() -> None:
    rng = np.random.default_rng(6)
    A_np = _spd(rng, 5, cond=1e6)  # deliberately ill-conditioned
    A = jnp.asarray(A_np)
    b = jnp.asarray(rng.normal(size=5))
    res = rank_collapse(A, b, target_condition=100.0)
    assert isinstance(res, CollapseResult)
    assert res.eps > 0.0
    # the certified damping provably meets the target (numpy oracle re-check)
    assert np.linalg.cond(A_np + res.eps * np.eye(5)) <= 100.0 + 1e-6
    assert res.certificate is not None and verify_certificate_digest(res.certificate)
    assert res.certificate["payload"]["type"] == "conditioning"

python -m pytest packages/omnibias-curvature/tests/test_regularize.py -k test_rank_collapse_target_condition_meets_target -q

Read the implementation

def rank_collapse(
    matrix: Array,
    rhs: Array,
    *,
    target_condition: float | None = None,
    rcond: float | None = None,
    certify: bool = True,
) -> CollapseResult:
    r"""High-level ``eps -> 0`` collapse: min-norm limit or certified-damped solve.

    Two modes:

    * ``target_condition is None`` -- take the collapse limit directly with
      :func:`min_norm_solve` (``eps = 0``, ``A^+ b``).
    * ``target_condition`` set -- pick the smallest damping ``eps`` that provably
      brings ``kappa(A + eps I)`` at or below the target (via
      :func:`omnibias.core.verified.conditioning.certified_damping`) and solve
      ``(A + eps I)^{-1} b``.

    When ``certify`` is set the result carries a sealed
    :func:`~omnibias.core.verified.conditioning.conditioning_certificate`
    (``lambda_min`` / ``lambda_max`` / ``kappa`` enclosures + the ``lambda_min > 0``
    pivot data). ``theorem_prover_verified`` is *not* asserted here -- it is earned
    only by driving the sealed certificate through the Lean bridge.
    """
    _validate_square_system(matrix, rhs)
    if target_condition is None:
        eps = 0.0
        solution = min_norm_solve(matrix, rhs, rcond=rcond)
    else:
        rows = _to_float_rows(matrix)
        eps = certified_damping(rows, target_condition=float(target_condition))
        solution = regularized_solve(matrix, rhs, eps=eps)
    rank = numerical_rank(matrix, rcond=rcond)
    certificate: dict[str, Any] | None = None
    if certify:
        rows = _to_float_rows(matrix)
        certificate = conditioning_certificate(
            rows,
            target_condition=target_condition,
            eps=(eps if target_condition is not None else None),
        )
    return CollapseResult(
        solution=solution, effective_rank=rank, eps=float(eps), certificate=certificate
    )

Other public import paths

omnibias.curvature.rank_collapse
omnibias.curvature.regularize.rank_collapse

FUNCTION

regularization_path

omnibias.curvature.regularization_pathSource ↗

regularization_path(matrix: Array, rhs: Array, eps_grid: Array) -> Array

Stacked Tikhonov solutions x(eps) over eps_grid (the collapse homotopy).

Returns a (len(eps_grid), P) array whose k-th row is (A + eps_grid[k] I)^{-1} b -- the measured curve that a data-driven eps or rcond choice is read off from.

Parameters

Name	Type & default	Contract
`matrix`positional-or-keyword	`Array`Required	See the operation contract and implementation below.
`rhs`positional-or-keyword	`Array`Required	See the operation contract and implementation below.
`eps_grid`positional-or-keyword	`Array`Required	See the operation contract and implementation below.

ReturnsArray

Explicit validation & failure cases

When grid.ndim != 1, raises ValueError(f'eps_grid must be 1-D, got shape {tuple(grid.shape)}').

These are explicit source guards. Backend operations may also raise for invalid shapes, dtypes, or unsupported devices.

Executed regression · regularization path rows match regularized solve

Full file, imports & fixtures ↗

def test_regularization_path_rows_match_regularized_solve() -> None:
    rng = np.random.default_rng(5)
    p = 4
    A = jnp.asarray(_spd(rng, p))
    b = jnp.asarray(rng.normal(size=p))
    grid = jnp.asarray([1e-1, 1e-2, 1e-3, 1e-4])
    path = regularization_path(A, b, grid)
    assert path.shape == (4, p)
    for k, eps in enumerate([1e-1, 1e-2, 1e-3, 1e-4]):
        assert jnp.allclose(path[k], regularized_solve(A, b, eps=eps), rtol=1e-12, atol=1e-12)

python -m pytest "packages/omnibias-curvature/tests/test_regularize.py::test_regularization_path_rows_match_regularized_solve" -q

Read the implementation

def regularization_path(matrix: Array, rhs: Array, eps_grid: Array) -> Array:
    r"""Stacked Tikhonov solutions ``x(eps)`` over ``eps_grid`` (the collapse homotopy).

    Returns a ``(len(eps_grid), P)`` array whose ``k``-th row is
    ``(A + eps_grid[k] I)^{-1} b`` -- the measured curve that a data-driven ``eps``
    or ``rcond`` choice is read off from.
    """
    p = _validate_square_system(matrix, rhs)
    grid = jnp.asarray(eps_grid)
    if grid.ndim != 1:
        raise ValueError(f"eps_grid must be 1-D, got shape {tuple(grid.shape)}")
    eye = jnp.eye(p, dtype=matrix.dtype)

    def solve_one(e: Array) -> Array:
        row: Array = jnp.linalg.solve(matrix + e * eye, rhs)
        return row

    out: Array = jax.vmap(solve_one)(grid)
    return out

Other public import paths

omnibias.curvature.regularization_path
omnibias.curvature.regularize.regularization_path

FUNCTION

regularized_solve

omnibias.curvature.regularized_solveSource ↗

regularized_solve(matrix: Array, rhs: Array, *, eps: float=0.001) -> Array

The Tikhonov-regularized solve (A + eps I)^{-1} b.

The generic damped solve at the heart of the eps -> 0 collapse: for eps > 0 it is well-posed even when A is singular. matrix is a (P, P) symmetric matrix and rhs a (P,) vector. omnibias.curvature.damped_solve and omnibias.curvature.mse_newton_step delegate here, so they are bit-identical to this routine at the same damping.

Parameters

Name	Type & default	Contract
`matrix`positional-or-keyword	`Array`Required	See the operation contract and implementation below.
`rhs`positional-or-keyword	`Array`Required	See the operation contract and implementation below.
`eps`keyword-only	`float`Default: 0.001	See the operation contract and implementation below.

ReturnsArray

Explicit validation & failure cases

When eps < 0.0, raises ValueError(f'eps must be >= 0, got {eps}').

These are explicit source guards. Backend operations may also raise for invalid shapes, dtypes, or unsupported devices.

Executed regression · rank collapse certify false has no certificate

Full file, imports & fixtures ↗

def test_rank_collapse_certify_false_has_no_certificate() -> None:
    rng = np.random.default_rng(8)
    A = jnp.asarray(_spd(rng, 4))
    b = jnp.asarray(rng.normal(size=4))
    res = rank_collapse(A, b, target_condition=50.0, certify=False)
    assert res.certificate is None

python -m pytest "packages/omnibias-curvature/tests/test_regularize.py::test_rank_collapse_certify_false_has_no_certificate" -q

Regression example · regularized solve normal equations

This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_regularized_solve_normal_equations() -> None:
    rng = np.random.default_rng(0)
    p = 6
    A = jnp.asarray(_spd(rng, p))
    b = jnp.asarray(rng.normal(size=p))
    eps = 1e-2
    x = regularized_solve(A, b, eps=eps)
    resid = (A + eps * jnp.eye(p)) @ x - b
    assert float(jnp.max(jnp.abs(resid))) < 1e-10

python -m pytest packages/omnibias-curvature/tests/test_regularize.py -k test_regularized_solve_normal_equations -q

Read the implementation

def regularized_solve(matrix: Array, rhs: Array, *, eps: float = 1e-3) -> Array:
    r"""The Tikhonov-regularized solve ``(A + eps I)^{-1} b``.

    The generic damped solve at the heart of the ``eps -> 0`` collapse: for
    ``eps > 0`` it is well-posed even when ``A`` is singular. ``matrix`` is a
    ``(P, P)`` symmetric matrix and ``rhs`` a ``(P,)`` vector.
    :func:`omnibias.curvature.damped_solve` and
    :func:`omnibias.curvature.mse_newton_step` delegate here, so they are
    bit-identical to this routine at the same damping.
    """
    p = _validate_square_system(matrix, rhs)
    if eps < 0.0:
        raise ValueError(f"eps must be >= 0, got {eps}")
    damped = matrix + eps * jnp.eye(p, dtype=matrix.dtype)
    out: Array = jnp.linalg.solve(damped, rhs)
    return out

Other public import paths

omnibias.curvature.regularized_solve
omnibias.curvature.regularize.regularized_solve

← Package overview Runnable learning examples →

omnibias.curvature.regularize

CollapseResult​

Parameters

Fields & methods

min_norm_solve​

Parameters

numerical_rank​

Parameters

rank_collapse​

Parameters

regularization_path​

Parameters

regularized_solve​

Parameters

CollapseResult

min_norm_solve

numerical_rank

rank_collapse

regularization_path

regularized_solve