omnibias.difference.torch

0.1.0a2AlphaPython >=3.10Apache-2.0

Finite-difference stencils, confluence limits, polynomial sequences, and scoped derivative-extraction certificates.

import omnibias.difference.torch

Module contract & mathematical conventions

PyTorch twin of the founding finite-difference stencil operator.

Batched, differentiable finite_difference_tower / collapse_to_derivative built on the **shared pure-Python** stencil numbers in omnibias.difference._core.stencil, so this operator and its JAX twin (omnibias.difference.jax) are algebraically corresponding by construction (only the base activation's libm evaluation differs across backends, by a few ulp).

finite_difference_tower is the literal multi-bias sum sum_k s_k sigma(z + b_k) (numerical); collapse_to_derivative is the closed-form sigma^(order)(z) from the backend fastpath. Their residual -> 0 as delta -> 0 -- the founding bias collapse of many biases into the derivative sigma^(K-1).

Numerical scope: shared coefficients and formulas do not establish universal floating-point identity across backends, devices, compiler transforms, or reduction orders. Compare the chosen path with dtype-appropriate tolerances; even float64 is not a general bitwise-equality guarantee. Algebraic initialization identities likewise need a numerical check for the selected coefficients and reduction order.

Shared formulas do not imply universal floating-point identity. Verify dtype, shape, derivative-order support, and the stated numerical assumptions for the path you use.

Module source ↗Source revision 9a2d73df0573Running reference examples →

FUNCTION

collapse_to_derivative

omnibias.difference.torch.collapse_to_derivativeSource ↗

collapse_to_derivative(name: str, z: Tensor, order: int) -> Tensor

Closed-form sigma^(order)(z) via the backend fastpath (the exact limit).

Parameters

Name	Type & default	Contract
`name`positional-or-keyword	`str`Required	See the operation contract and implementation below.
`z`positional-or-keyword	`Tensor`Required	See the operation contract and implementation below.
`order`positional-or-keyword	`int`Required	See the operation contract and implementation below.

ReturnsTensor

Explicit validation & failure cases

When order < 0, raises ValueError(f'order must be >= 0, got {order}').
When spec.fastpath is None, raises NotImplementedError(f'activation {name!r} has no closed-form derivative kernel').

These are explicit source guards. Backend operations may also raise for invalid shapes, dtypes, or unsupported devices.

Executed regression · collapse to derivative parity

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_collapse_to_derivative_parity(name: str, order: int) -> None:
    zt = torch.tensor(_XS, dtype=torch.float64)
    zj = jnp.asarray(_XS)
    ct = DT.collapse_to_derivative(name, zt, order).detach().numpy()
    cj = np.asarray(DJ.collapse_to_derivative(name, zj, order))
    scale = 1.0 + float(np.max(np.abs(ct)))
    assert np.max(np.abs(ct - cj)) < 1e-8 * scale

python -m pytest "packages/omnibias-difference/tests/test_parity.py::test_collapse_to_derivative_parity[0-gaussian]" -q

Read the implementation

def collapse_to_derivative(name: str, z: Tensor, order: int) -> Tensor:
    """Closed-form ``sigma^(order)(z)`` via the backend fastpath (the exact limit)."""
    if order < 0:
        raise ValueError(f"order must be >= 0, got {order}")
    spec = get_activation(name)
    if spec.fastpath is None:
        raise NotImplementedError(f"activation {name!r} has no closed-form derivative kernel")
    out: Tensor = spec.fastpath(z, order)
    return out

FUNCTION

finite_difference_residual

omnibias.difference.torch.finite_difference_residualSource ↗

finite_difference_residual(name: str, z: Tensor, order: int, delta: float, stencil: Stencil='central') -> Tensor

finite_difference_tower - collapse_to_derivative; -> 0 as delta -> 0.

Parameters

Name	Type & default	Contract
`name`positional-or-keyword	`str`Required	See the operation contract and implementation below.
`z`positional-or-keyword	`Tensor`Required	See the operation contract and implementation below.
`order`positional-or-keyword	`int`Required	See the operation contract and implementation below.
`delta`positional-or-keyword	`float`Required	See the operation contract and implementation below.
`stencil`positional-or-keyword	`Stencil`Default: 'central'	See the operation contract and implementation below.

ReturnsTensor

Executed regression · residual parity

Full file, imports & fixtures ↗

def test_residual_parity(name: str) -> None:
    delta = 1e-2
    zt = torch.tensor(_XS, dtype=torch.float64)
    zj = jnp.asarray(_XS)
    rt = DT.finite_difference_residual(name, zt, 2, delta, "central").detach().numpy()
    rj = np.asarray(DJ.finite_difference_residual(name, zj, 2, delta, "central"))
    sign_mag = sum(abs(s) for s in stencil_signs(2, delta, "central"))
    assert np.max(np.abs(rt - rj)) < 1e-12 * sign_mag + 1e-12

python -m pytest "packages/omnibias-difference/tests/test_parity.py::test_residual_parity[gaussian]" -q

Read the implementation

def finite_difference_residual(
    name: str, z: Tensor, order: int, delta: float, stencil: Stencil = "central"
) -> Tensor:
    """``finite_difference_tower - collapse_to_derivative``; ``-> 0`` as ``delta -> 0``."""
    return finite_difference_tower(name, z, order, delta, stencil) - collapse_to_derivative(
        name, z, order
    )

FUNCTION

finite_difference_tower

omnibias.difference.torch.finite_difference_towerSource ↗

finite_difference_tower(name: str, z: Tensor, order: int, delta: float, stencil: Stencil='central') -> Tensor

Evaluate the numerical multi-bias stencil sum_k s_k sigma(z + b_k).

Signs and offsets come from the shared pure-Python stencil implementation and are materialized with z's dtype and device. Finite-gap evaluation remains a numerical difference calculation; cancellation and activation rounding can dominate as the gap shrinks. Matching formulas between Torch and JAX do not guarantee bitwise identity. Compare against an independent derivative or reference stencil using a stated tolerance.

Parameters

Name	Type & default	Contract
`name`positional-or-keyword	`str`Required	See the operation contract and implementation below.
`z`positional-or-keyword	`Tensor`Required	See the operation contract and implementation below.
`order`positional-or-keyword	`int`Required	See the operation contract and implementation below.
`delta`positional-or-keyword	`float`Required	See the operation contract and implementation below.
`stencil`positional-or-keyword	`Stencil`Default: 'central'	See the operation contract and implementation below.

ReturnsTensor

Executed regression · residual parity

Full file, imports & fixtures ↗

def test_residual_parity(name: str) -> None:
    delta = 1e-2
    zt = torch.tensor(_XS, dtype=torch.float64)
    zj = jnp.asarray(_XS)
    rt = DT.finite_difference_residual(name, zt, 2, delta, "central").detach().numpy()
    rj = np.asarray(DJ.finite_difference_residual(name, zj, 2, delta, "central"))
    sign_mag = sum(abs(s) for s in stencil_signs(2, delta, "central"))
    assert np.max(np.abs(rt - rj)) < 1e-12 * sign_mag + 1e-12

python -m pytest "packages/omnibias-difference/tests/test_parity.py::test_residual_parity[gaussian]" -q

Regression example · finite difference tower parity

This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_finite_difference_tower_parity(name: str, order: int, stencil: str) -> None:
    delta = 1e-2
    zt = torch.tensor(_XS, dtype=torch.float64)
    zj = jnp.asarray(_XS)
    ft = DT.finite_difference_tower(name, zt, order, delta, stencil).detach().numpy()
    fj = np.asarray(DJ.finite_difference_tower(name, zj, order, delta, stencil))
    sign_mag = sum(abs(s) for s in stencil_signs(order, delta, stencil))
    tol = 1e-12 * sign_mag + 1e-12  # calibrated to the 1/delta^order amplification
    assert np.max(np.abs(ft - fj)) < tol

python -m pytest packages/omnibias-difference/tests/test_parity.py -k test_finite_difference_tower_parity -q

Read the implementation

def finite_difference_tower(
    name: str, z: Tensor, order: int, delta: float, stencil: Stencil = "central"
) -> Tensor:
    """Numerical multi-bias finite-difference stencil ``sum_k s_k sigma(z + b_k)``.

    The signs and offsets come from the shared pure-Python
    :mod:`omnibias.difference._core.stencil`, materialised into tensors matching
    ``z``'s dtype / device, so the value is bit-identical to the JAX twin up to the
    base activation's per-backend rounding.
    """
    spec = get_activation(name)
    signs = torch.tensor(stencil_signs(order, delta, stencil), dtype=z.dtype, device=z.device)
    offsets = torch.tensor(stencil_offsets(order, delta, stencil), dtype=z.dtype, device=z.device)
    values = spec.forward(z.unsqueeze(-1) + offsets)  # (..., K)
    return (values * signs).sum(dim=-1)

← Package overview Runnable learning examples →

omnibias.difference.torch

collapse_to_derivative​

Parameters

finite_difference_residual​

Parameters

finite_difference_tower​

Parameters

collapse_to_derivative

finite_difference_residual

finite_difference_tower