omnibias.torch.identity_init

0.5.0rc1Release candidatePython >=3.10Apache-2.0

PyTorch activation derivatives, trainable operator layers, network jets, and direct differential-operator contractions.

import omnibias.torch.identity_init

Module contract & mathematical conventions

Lemma 1 (identity nesting) helpers.

Lemma 1 of the multi-bias paper says: for any base activation sigma and any signs s_1, ..., s_K summing to one, tying b_1 = ... = b_K = b gives sum_k s_k * sigma(z + b_k) = sigma(z + b) as a real-arithmetic identity.

These helpers build init tensors that satisfy the lemma so a fresh OMBU returns its base activation at step zero, no matter what K is.

Numerical scope: shared coefficients and formulas do not establish universal floating-point identity across backends, devices, compiler transforms, or reduction orders. Compare the chosen path with dtype-appropriate tolerances; even float64 is not a general bitwise-equality guarantee. Algebraic initialization identities likewise need a numerical check for the selected coefficients and reduction order.

Shared formulas do not imply universal floating-point identity. Verify dtype, shape, derivative-order support, and the stated numerical assumptions for the path you use.

Module source ↗Source revision 9a2d73df0573Running reference examples →

Imports & aliases

These names share a definition with another public module. Follow a name for its full contract, parameters, and examples.

Import from this module	Canonical reference
`identity_signs`	`omnibias.torch.identity_init.identity_signs`

FUNCTION

identity_init_biases

omnibias.torch.identity_init.identity_init_biasesSource ↗

identity_init_biases(num_channels: int, K: int, bias_value: float=0.0, dtype: torch.dtype | None=None) -> Tensor

Tied biases for identity nesting.

Returns a (num_channels, K) tensor with every entry equal to bias_value. Combined with identity_init_signs, makes the OMBU forward equal sigma(z + bias_value) bit-identically.

dtype defaults to torch.get_default_dtype() so callers that have set a non-default global dtype (e.g. float64 for scientific work) get matching parameters by default.

Parameters

Name	Type & default	Contract
`num_channels`positional-or-keyword	`int`Required	See the operation contract and implementation below.
`K`positional-or-keyword	`int`Required	See the operation contract and implementation below.
`bias_value`positional-or-keyword	`float`Default: 0.0	See the operation contract and implementation below.
`dtype`positional-or-keyword	`torch.dtype \| None`Default: None	See the operation contract and implementation below.

ReturnsTensor

Explicit validation & failure cases

When num_channels < 1, raises ValueError(f'num_channels must be >= 1, got {num_channels}').
When K < 1, raises ValueError(f'K must be >= 1, got {K}').

These are explicit source guards. Backend operations may also raise for invalid shapes, dtypes, or unsupported devices.

Executed regression · cmblinear forward shape

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_cmblinear_forward_shape() -> None:
    layer = cmbLinear(5, 3, op="identity", base="tanh")
    out = layer(torch.randn(2, 5))
    assert out.shape == (2, 3)

python -m pytest "packages/omnibias-torch/tests/test_blocks.py::test_cmblinear_forward_shape" -q

Regression example · identity init biases follows default dtype

This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_identity_init_biases_follows_default_dtype(torch_default_float64) -> None:
    biases = identity_init_biases(num_channels=4, K=2)
    assert biases.dtype == torch.float64

python -m pytest packages/omnibias-torch/tests/test_audit_regressions.py -k test_identity_init_biases_follows_default_dtype -q

Read the implementation

def identity_init_biases(
    num_channels: int,
    K: int,
    bias_value: float = 0.0,
    dtype: torch.dtype | None = None,
) -> Tensor:
    """Tied biases for identity nesting.

    Returns a ``(num_channels, K)`` tensor with every entry equal to
    ``bias_value``. Combined with :func:`identity_init_signs`, makes the
    OMBU forward equal ``sigma(z + bias_value)`` bit-identically.

    ``dtype`` defaults to :func:`torch.get_default_dtype()` so callers
    that have set a non-default global dtype (e.g. ``float64`` for
    scientific work) get matching parameters by default.
    """
    if num_channels < 1:
        raise ValueError(f"num_channels must be >= 1, got {num_channels}")
    if K < 1:
        raise ValueError(f"K must be >= 1, got {K}")
    if dtype is None:
        dtype = torch.get_default_dtype()
    return torch.full((num_channels, K), bias_value, dtype=dtype)

FUNCTION

identity_init_signs

omnibias.torch.identity_init.identity_init_signsSource ↗

identity_init_signs(num_channels: int, K: int, dtype: torch.dtype | None=None) -> Tensor

Signs that sum to one, broadcast across channels.

dtype defaults to torch.get_default_dtype().

Parameters

Name	Type & default	Contract
`num_channels`positional-or-keyword	`int`Required	See the operation contract and implementation below.
`K`positional-or-keyword	`int`Required	See the operation contract and implementation below.
`dtype`positional-or-keyword	`torch.dtype \| None`Default: None	See the operation contract and implementation below.

ReturnsTensor

Executed regression · cmblinear forward shape

Full file, imports & fixtures ↗

def test_cmblinear_forward_shape() -> None:
    layer = cmbLinear(5, 3, op="identity", base="tanh")
    out = layer(torch.randn(2, 5))
    assert out.shape == (2, 3)

python -m pytest "packages/omnibias-torch/tests/test_blocks.py::test_cmblinear_forward_shape" -q

Regression example · identity init signs follows default dtype

This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_identity_init_signs_follows_default_dtype(torch_default_float64) -> None:
    signs = identity_init_signs(num_channels=4, K=2)
    assert signs.dtype == torch.float64

python -m pytest packages/omnibias-torch/tests/test_audit_regressions.py -k test_identity_init_signs_follows_default_dtype -q

Read the implementation

def identity_init_signs(
    num_channels: int,
    K: int,
    dtype: torch.dtype | None = None,
) -> Tensor:
    """Signs that sum to one, broadcast across channels.

    ``dtype`` defaults to :func:`torch.get_default_dtype()`.
    """
    base = identity_signs(K, dtype=dtype)
    return base.unsqueeze(0).expand(num_channels, K).contiguous()

FUNCTION

verify_identity_init

omnibias.torch.identity_init.verify_identity_initSource ↗

verify_identity_init(biases: Tensor, signs: Tensor, atol: float=0.0) -> bool

Cheap sanity check: returns True if the init satisfies Lemma 1.

Tests two conditions per channel:

1. All biases are equal within atol. 2. The signs sum to one within atol.

Parameters

Name	Type & default	Contract
`biases`positional-or-keyword	`Tensor`Required	See the operation contract and implementation below.
`signs`positional-or-keyword	`Tensor`Required	See the operation contract and implementation below.
`atol`positional-or-keyword	`float`Default: 0.0	See the operation contract and implementation below.

Returnsbool

Explicit validation & failure cases

When biases.shape != signs.shape, raises ValueError(f'biases and signs must have the same shape, got {biases.shape} vs {signs.shape}').

These are explicit source guards. Backend operations may also raise for invalid shapes, dtypes, or unsupported devices.

Executed regression · verify identity init

Full file, imports & fixtures ↗

def test_verify_identity_init() -> None:
    b = identity_init_biases(num_channels=4, K=3, bias_value=0.0)
    s = identity_init_signs(num_channels=4, K=3)
    assert verify_identity_init(b, s, atol=0.0)

python -m pytest "packages/omnibias-torch/tests/test_identity_nesting.py::test_verify_identity_init" -q

Read the implementation

def verify_identity_init(
    biases: Tensor,
    signs: Tensor,
    atol: float = 0.0,
) -> bool:
    """Cheap sanity check: returns True if the init satisfies Lemma 1.

    Tests two conditions per channel:

    1. All biases are equal within ``atol``.
    2. The signs sum to one within ``atol``.
    """
    if biases.shape != signs.shape:
        raise ValueError(
            f"biases and signs must have the same shape, got {biases.shape} vs {signs.shape}"
        )
    bias_constancy = (biases - biases[..., :1]).abs().max().item()
    sign_sum_err = (signs.sum(dim=-1) - 1.0).abs().max().item()
    return bias_constancy <= atol and sign_sum_err <= atol

← Package overview Runnable learning examples →

omnibias.torch.identity_init

Imports & aliases​

identity_init_biases​

Parameters

identity_init_signs​

Parameters

verify_identity_init​

Parameters

Imports & aliases

identity_init_biases

identity_init_signs

verify_identity_init