omnibias.keras.unit
Keras 3 operator layers and activation paths implemented with keras.ops for TensorFlow, JAX, or Torch.
import omnibias.keras.unitModule contract & mathematical conventions
OperatorMultiBiasUnit (OMBU) for the Keras backend.
A K-bias multi-bias unit applied per channel:
f_K(z; b, s) = sum_{k=1}^{K} s_k * sigma(z + b_k)
with K learnable biases and K signs per channel on top of a fixed base
activation sigma. Mirrors omnibias.torch.unit.OperatorMultiBiasUnit
as a keras.layers.Layer written against keras.ops.
Two operating regimes:
1. Identity-nested (Lemma 1): tied biases + signs summing to one reduce the unit by the stated algebraic identity to sigma(z + b) (the default init).
2. Bias-collapse derivative tower: analytic_derivative evaluates the closed-form sigma^(order)(z + bias_mean) via the activation fast-path kernel.
Numerical scope: shared coefficients and formulas do not establish universal floating-point identity across backends, devices, compiler transforms, or reduction orders. Compare the chosen path with dtype-appropriate tolerances; even float64 is not a general bitwise-equality guarantee. Algebraic initialization identities likewise need a numerical check for the selected coefficients and reduction order.
Shared formulas do not imply universal floating-point identity. Verify dtype, shape, derivative-order support, and the stated numerical assumptions for the path you use.
OperatorMultiBiasUnit
omnibias.keras.OMBUSource ↗class OperatorMultiBiasUnit(layers.Layer)The trainable scalar-operator primitive (Keras backend).
Parameters
num_channels : int Number of independent channels (the last input dim).
K : int, default 2 Number of bias terms per channel.
base : str or ActivationSpec, default "sigmoid" Base activation, by name or spec.
init_bias : float, default 0.0 Initial value for every bias entry (Lemma 1 identity nesting).
init_delta : float, default 0.0 If non-zero, place biases on the central-difference stencil.
learnable_biases : bool, default True
learnable_signs : bool, default False
biases_init, signs_init : array-like, optional Explicit (num_channels, K) init values.
Parameters
| Name | Type & default | Contract |
|---|---|---|
num_channelspositional-or-keyword | intRequired | int Number of independent channels (the last input dim). |
Kpositional-or-keyword | intDefault: 2 | int, default 2 Number of bias terms per channel. |
basepositional-or-keyword | str | ActivationSpec[Any]Default: 'sigmoid' | str or ActivationSpec, default "sigmoid" Base activation, by name or spec. |
init_biaspositional-or-keyword | floatDefault: 0.0 | float, default 0.0 Initial value for every bias entry (Lemma 1 identity nesting). |
init_deltapositional-or-keyword | floatDefault: 0.0 | float, default 0.0 If non-zero, place biases on the central-difference stencil. |
learnable_biasespositional-or-keyword | boolDefault: True | bool, default True |
learnable_signspositional-or-keyword | boolDefault: False | bool, default False |
biases_initpositional-or-keyword | AnyDefault: None | Explicit (num_channels, K) init values. |
signs_initpositional-or-keyword | AnyDefault: None | See the operation contract and implementation below. |
kwargsvariadic keyword | AnyRequired | See the operation contract and implementation below. |
Executed regression · bad num channels raises
Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.
def test_bad_num_channels_raises() -> None:
with pytest.raises(ValueError):
OMBU(num_channels=0, K=2)python -m pytest "packages/omnibias-keras/tests/test_unit.py::test_bad_num_channels_raises" -qRegression example · analytic derivative no fastpath raises
This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.
def test_analytic_derivative_no_fastpath_raises() -> None:
ombu = OMBU(num_channels=2, K=2, base="tanh")
# tanh has a fastpath; use an artificial order check instead
with pytest.raises(ValueError):
ombu.analytic_derivative(_t(np.zeros((4, 2))), order=-1)python -m pytest packages/omnibias-keras/tests/test_unit.py -k test_analytic_derivative_no_fastpath_raises -qFields & methods
__init__method
__init__(self, num_channels: int, K: int=2, base: str | ActivationSpec[Any]='sigmoid', init_bias: float=0.0, init_delta: float=0.0, learnable_biases: bool=True, learnable_signs: bool=False, biases_init: Any=None, signs_init: Any=None, **kwargs: Any) -> NoneSee the class contract above and the exact method implementation below.
Parameters
| Name | Type & default | Contract |
|---|---|---|
num_channelspositional-or-keyword | intRequired | See the operation contract and implementation below. |
Kpositional-or-keyword | intDefault: 2 | See the operation contract and implementation below. |
basepositional-or-keyword | str | ActivationSpec[Any]Default: 'sigmoid' | See the operation contract and implementation below. |
init_biaspositional-or-keyword | floatDefault: 0.0 | See the operation contract and implementation below. |
init_deltapositional-or-keyword | floatDefault: 0.0 | See the operation contract and implementation below. |
learnable_biasespositional-or-keyword | boolDefault: True | See the operation contract and implementation below. |
learnable_signspositional-or-keyword | boolDefault: False | See the operation contract and implementation below. |
biases_initpositional-or-keyword | AnyDefault: None | See the operation contract and implementation below. |
signs_initpositional-or-keyword | AnyDefault: None | See the operation contract and implementation below. |
kwargsvariadic keyword | AnyRequired | See the operation contract and implementation below. |
Returns: None
Validation & failure cases
- When
num_channels < 1, raisesValueError(f'num_channels must be >= 1, got {num_channels}'). - When
K < 1, raisesValueError(f'K must be >= 1, got {K}'). - When
init_delta < 0, raisesValueError(f'init_delta must be >= 0, got {init_delta}'). - When
biases_np.shape != (num_channels, K), raisesValueError(f'biases_init shape {biases_np.shape} does not match ({num_channels}, {K}).'). - When
signs_np.shape != (num_channels, K), raisesValueError(f'signs_init shape {signs_np.shape} does not match ({num_channels}, {K}) (or ({K},) for broadcast).').
Executed regression · bad num channels raises
Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.
def test_bad_num_channels_raises() -> None:
with pytest.raises(ValueError):
OMBU(num_channels=0, K=2)python -m pytest "packages/omnibias-keras/tests/test_unit.py::test_bad_num_channels_raises" -qMethod implementation
def __init__(
self,
num_channels: int,
K: int = 2,
base: str | ActivationSpec[Any] = "sigmoid",
init_bias: float = 0.0,
init_delta: float = 0.0,
learnable_biases: bool = True,
learnable_signs: bool = False,
biases_init: Any = None,
signs_init: Any = None,
**kwargs: Any,
) -> None:
super().__init__(**kwargs)
if num_channels < 1:
raise ValueError(f"num_channels must be >= 1, got {num_channels}")
if K < 1:
raise ValueError(f"K must be >= 1, got {K}")
if init_delta < 0:
raise ValueError(f"init_delta must be >= 0, got {init_delta}")
self.num_channels = num_channels
self.K = K
self._base_name = base if isinstance(base, str) else base.name
self.spec = base if isinstance(base, ActivationSpec) else get_activation(base)
self.init_bias = init_bias
self.init_delta = init_delta
self.learnable_biases = learnable_biases
self.learnable_signs = learnable_signs
dtype = self.variable_dtype
if biases_init is None:
biases_np = identity_init_biases(num_channels, K, bias_value=init_bias, dtype=dtype)
if init_delta > 0 and K > 1:
offsets = central_bias_offsets(K, init_delta, dtype=dtype)[None, :]
biases_np = biases_np + offsets
else:
biases_np = np.asarray(biases_init, dtype=dtype)
if biases_np.shape != (num_channels, K):
raise ValueError(
f"biases_init shape {biases_np.shape} does not match ({num_channels}, {K})."
)
if signs_init is None:
signs_np = identity_init_signs(num_channels, K, dtype=dtype)
else:
signs_np = np.asarray(signs_init, dtype=dtype)
if signs_np.shape == (K,):
signs_np = np.broadcast_to(signs_np, (num_channels, K)).copy()
elif signs_np.shape != (num_channels, K):
raise ValueError(
f"signs_init shape {signs_np.shape} does not match "
f"({num_channels}, {K}) (or ({K},) for broadcast)."
)
self.biases = self.add_weight(
name="biases",
shape=(num_channels, K),
initializer="zeros",
trainable=learnable_biases,
dtype=dtype,
)
self.signs = self.add_weight(
name="signs",
shape=(num_channels, K),
initializer="zeros",
trainable=learnable_signs,
dtype=dtype,
)
self.biases.assign(biases_np)
self.signs.assign(signs_np)
self.built = Truecallmethod
call(self, z: Any) -> AnyLiteral multi-bias forward sum_k s_k * sigma(z + b_k).
Parameters
| Name | Type & default | Contract |
|---|---|---|
zpositional-or-keyword | AnyRequired | See the operation contract and implementation below. |
Returns: Any
Executed regression · cmbconv1d shape
Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.
def test_cmbconv1d_shape() -> None:
x = np.random.default_rng(6).normal(size=(4, 10, 3))
layer = cmbConv1D(filters=5, kernel_size=3, op="identity", base="tanh")
out = _np(layer(_t(x)))
assert out.shape[0] == 4 and out.shape[-1] == 5python -m pytest "packages/omnibias-keras/tests/test_blocks.py::test_cmbconv1d_shape" -qMethod implementation
def call(self, z: Any) -> Any:
"""Literal multi-bias forward ``sum_k s_k * sigma(z + b_k)``."""
return multibias_literal_forward(z, self.biases, self.signs, self.spec.forward)analytic_derivativemethod
analytic_derivative(self, z: Any, order: int | None=None) -> AnyClosed-form sigma^(order)(z + bias_mean) (defaults order K-1).
This is the **mean-bias fast-path**: it evaluates the closed-form
order-th derivative once, at the *mean* of the bias columns. It
equals the literal multi-bias stencil sum_k s_k sigma(z + b_k) only
under the identity-nesting precondition (init_delta = 0 so all biases
share a value). For untied biases that have drifted apart it is the
single-point derivative at mean(biases), not the literal stencil --
see omnibias.keras.blocks.operator.OperatorBlock.
Parameters
| Name | Type & default | Contract |
|---|---|---|
zpositional-or-keyword | AnyRequired | See the operation contract and implementation below. |
orderpositional-or-keyword | int | NoneDefault: None | See the operation contract and implementation below. |
Returns: Any
Validation & failure cases
- When
order < 0, raisesValueError(f'order must be >= 0, got {order}'). - When
self.spec.fastpath is None, raisesNotImplementedError(f'Activation {self.spec.name!r} has no closed-form derivative kernel.').
Executed regression · cmbconv2d shape
Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.
def test_cmbconv2d_shape() -> None:
x = np.random.default_rng(7).normal(size=(4, 8, 8, 2))
layer = cmbConv2D(filters=3, kernel_size=3, op="laplacian", base="gaussian")
out = _np(layer(_t(x)))
assert out.shape[0] == 4 and out.shape[-1] == 3python -m pytest "packages/omnibias-keras/tests/test_blocks.py::test_cmbconv2d_shape" -qMethod implementation
def analytic_derivative(self, z: Any, order: int | None = None) -> Any:
"""Closed-form ``sigma^(order)(z + bias_mean)`` (defaults order ``K-1``).
This is the **mean-bias fast-path**: it evaluates the closed-form
``order``-th derivative once, at the *mean* of the bias columns. It
equals the literal multi-bias stencil ``sum_k s_k sigma(z + b_k)`` only
under the identity-nesting precondition (``init_delta = 0`` so all biases
share a value). For untied biases that have drifted apart it is the
single-point derivative at ``mean(biases)``, not the literal stencil --
see :class:`omnibias.keras.blocks.operator.OperatorBlock`.
"""
if order is None:
order = self.K - 1
if order < 0:
raise ValueError(f"order must be >= 0, got {order}")
if self.spec.fastpath is None:
raise NotImplementedError(
f"Activation {self.spec.name!r} has no closed-form derivative kernel."
)
b_mean = ops.mean(self.biases, axis=-1)
return self.spec.fastpath(z + b_mean, order)analytic_integralmethod
analytic_integral(self, z: Any, *, strict_window: bool=False, normalize: bool=False, small_width_threshold: float=0.0001) -> AnyClosed-form sign-weighted antiderivative sum.
Parameters
| Name | Type & default | Contract |
|---|---|---|
zpositional-or-keyword | AnyRequired | See the operation contract and implementation below. |
strict_windowkeyword-only | boolDefault: False | See the operation contract and implementation below. |
normalizekeyword-only | boolDefault: False | See the operation contract and implementation below. |
small_width_thresholdkeyword-only | floatDefault: 0.0001 | See the operation contract and implementation below. |
Returns: Any
Validation & failure cases
- When
self.K != 2, raisesValueError('strict integral windows require K=2.').
Executed regression · integral block runs
Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.
def test_integral_block_runs() -> None:
z = np.random.default_rng(4).normal(size=(8, 4))
block = OperatorBlock(channels=4, op="integral", base="gaussian", init_delta=1.0)
out = _np(block(_t(z)))
assert out.shape == (8, 4)
assert np.all(np.isfinite(out))python -m pytest "packages/omnibias-keras/tests/test_blocks.py::test_integral_block_runs" -qMethod implementation
def analytic_integral(
self,
z: Any,
*,
strict_window: bool = False,
normalize: bool = False,
small_width_threshold: float = 1e-4,
) -> Any:
"""Closed-form sign-weighted antiderivative sum."""
if strict_window:
if self.K != 2:
raise ValueError("strict integral windows require K=2.")
center = self.biases[:, 0]
width = ops.softplus(self.biases[:, 1])
use_taylor = not self.learnable_signs
return multibias_integral_window_forward(
z,
center,
width,
self.signs,
self.spec,
normalize=normalize,
small_width_threshold=small_width_threshold,
use_small_width_taylor=use_taylor,
)
return multibias_integral_forward(z, self.biases, self.signs, self.spec)bias_spreadproperty
bias_spread(self) -> AnySee the class contract above and the exact method implementation below.
Returns: Any
Method implementation
def bias_spread(self) -> Any:
return ops.max(self.biases, axis=-1) - ops.min(self.biases, axis=-1)is_identity_nestedproperty
is_identity_nested(self) -> boolSee the class contract above and the exact method implementation below.
Returns: bool
Executed regression · identity nesting recovers base
Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.
def test_identity_nesting_recovers_base() -> None:
"""A fresh OMBU (tied biases, signs sum to one) equals sigma(z)."""
z = np.random.default_rng(0).normal(size=(16, 4))
for base in ("sigmoid", "tanh", "gaussian"):
ombu = OMBU(num_channels=4, K=3, base=base, init_bias=0.0)
out = _np(ombu(_t(z)))
ref = _np(get_activation(base).forward(_t(z)))
np.testing.assert_allclose(out, ref, rtol=1e-10, atol=1e-12)
assert ombu.is_identity_nestedpython -m pytest "packages/omnibias-keras/tests/test_unit.py::test_identity_nesting_recovers_base" -qMethod implementation
def is_identity_nested(self) -> bool:
biases = np.asarray(ops.convert_to_numpy(self.biases.value))
signs = np.asarray(ops.convert_to_numpy(self.signs.value))
tied = bool(np.all(biases.max(axis=-1) - biases.min(axis=-1) == 0))
sums_to_one = bool(np.all(np.abs(signs.sum(axis=-1) - 1.0) < 1e-6))
return tied and sums_to_oneget_configmethod
get_config(self) -> dict[str, Any]See the class contract above and the exact method implementation below.
Returns: dict[str, Any]
Method implementation
def get_config(self) -> dict[str, Any]:
config = super().get_config()
config.update(
{
"num_channels": self.num_channels,
"K": self.K,
"base": self._base_name,
"init_bias": self.init_bias,
"init_delta": self.init_delta,
"learnable_biases": self.learnable_biases,
"learnable_signs": self.learnable_signs,
}
)
return confignum_channelsattribute
num_channels: int = num_channelsNumber of independent channels (the last input dim).
specattribute
spec = base if isinstance(base, ActivationSpec) else get_activation(base)Public instance attribute assigned by __init__. The source assignment is spec = base if isinstance(base, ActivationSpec) else get_activation(base); it describes instance state, not a literal constructor default.
init_biasattribute
init_bias: float = init_biasInitial value for every bias entry (Lemma 1 identity nesting).
init_deltaattribute
init_delta: float = init_deltaIf non-zero, place biases on the central-difference stencil.
biasesattribute
biases = self.add_weight(name='biases', shape=(num_channels, K), initializer='zeros', trainable=learnable_biases, dtype=dtype)Public instance attribute assigned by __init__. The source assignment is biases = self.add_weight(name='biases', shape=(num_channels, K), initializer='zeros', trainable=learnable_biases, dtype=dtype); it describes instance state, not a literal constructor default.
signsattribute
signs = self.add_weight(name='signs', shape=(num_channels, K), initializer='zeros', trainable=learnable_signs, dtype=dtype)Public instance attribute assigned by __init__. The source assignment is signs = self.add_weight(name='signs', shape=(num_channels, K), initializer='zeros', trainable=learnable_signs, dtype=dtype); it describes instance state, not a literal constructor default.
builtattribute
built = TruePublic instance attribute assigned by __init__. The source assignment is built = True; it describes instance state, not a literal constructor default.
Read the implementation
class OperatorMultiBiasUnit(layers.Layer):
"""The trainable scalar-operator primitive (Keras backend).
Parameters
----------
num_channels : int
Number of independent channels (the last input dim).
K : int, default 2
Number of bias terms per channel.
base : str or :class:`ActivationSpec`, default ``"sigmoid"``
Base activation, by name or spec.
init_bias : float, default 0.0
Initial value for every bias entry (Lemma 1 identity nesting).
init_delta : float, default 0.0
If non-zero, place biases on the central-difference stencil.
learnable_biases : bool, default True
learnable_signs : bool, default False
biases_init, signs_init : array-like, optional
Explicit ``(num_channels, K)`` init values.
"""
def __init__(
self,
num_channels: int,
K: int = 2,
base: str | ActivationSpec[Any] = "sigmoid",
init_bias: float = 0.0,
init_delta: float = 0.0,
learnable_biases: bool = True,
learnable_signs: bool = False,
biases_init: Any = None,
signs_init: Any = None,
**kwargs: Any,
) -> None:
super().__init__(**kwargs)
if num_channels < 1:
raise ValueError(f"num_channels must be >= 1, got {num_channels}")
if K < 1:
raise ValueError(f"K must be >= 1, got {K}")
if init_delta < 0:
raise ValueError(f"init_delta must be >= 0, got {init_delta}")
self.num_channels = num_channels
self.K = K
self._base_name = base if isinstance(base, str) else base.name
self.spec = base if isinstance(base, ActivationSpec) else get_activation(base)
self.init_bias = init_bias
self.init_delta = init_delta
self.learnable_biases = learnable_biases
self.learnable_signs = learnable_signs
dtype = self.variable_dtype
if biases_init is None:
biases_np = identity_init_biases(num_channels, K, bias_value=init_bias, dtype=dtype)
if init_delta > 0 and K > 1:
offsets = central_bias_offsets(K, init_delta, dtype=dtype)[None, :]
biases_np = biases_np + offsets
else:
biases_np = np.asarray(biases_init, dtype=dtype)
if biases_np.shape != (num_channels, K):
raise ValueError(
f"biases_init shape {biases_np.shape} does not match ({num_channels}, {K})."
)
if signs_init is None:
signs_np = identity_init_signs(num_channels, K, dtype=dtype)
else:
signs_np = np.asarray(signs_init, dtype=dtype)
if signs_np.shape == (K,):
signs_np = np.broadcast_to(signs_np, (num_channels, K)).copy()
elif signs_np.shape != (num_channels, K):
raise ValueError(
f"signs_init shape {signs_np.shape} does not match "
f"({num_channels}, {K}) (or ({K},) for broadcast)."
)
self.biases = self.add_weight(
name="biases",
shape=(num_channels, K),
initializer="zeros",
trainable=learnable_biases,
dtype=dtype,
)
self.signs = self.add_weight(
name="signs",
shape=(num_channels, K),
initializer="zeros",
trainable=learnable_signs,
dtype=dtype,
)
self.biases.assign(biases_np)
self.signs.assign(signs_np)
self.built = True
# ----- forward passes -----
def call(self, z: Any) -> Any:
"""Literal multi-bias forward ``sum_k s_k * sigma(z + b_k)``."""
return multibias_literal_forward(z, self.biases, self.signs, self.spec.forward)
def analytic_derivative(self, z: Any, order: int | None = None) -> Any:
"""Closed-form ``sigma^(order)(z + bias_mean)`` (defaults order ``K-1``).
This is the **mean-bias fast-path**: it evaluates the closed-form
``order``-th derivative once, at the *mean* of the bias columns. It
equals the literal multi-bias stencil ``sum_k s_k sigma(z + b_k)`` only
under the identity-nesting precondition (``init_delta = 0`` so all biases
share a value). For untied biases that have drifted apart it is the
single-point derivative at ``mean(biases)``, not the literal stencil --
see :class:`omnibias.keras.blocks.operator.OperatorBlock`.
"""
if order is None:
order = self.K - 1
if order < 0:
raise ValueError(f"order must be >= 0, got {order}")
if self.spec.fastpath is None:
raise NotImplementedError(
f"Activation {self.spec.name!r} has no closed-form derivative kernel."
)
b_mean = ops.mean(self.biases, axis=-1)
return self.spec.fastpath(z + b_mean, order)
def analytic_integral(
self,
z: Any,
*,
strict_window: bool = False,
normalize: bool = False,
small_width_threshold: float = 1e-4,
) -> Any:
"""Closed-form sign-weighted antiderivative sum."""
if strict_window:
if self.K != 2:
raise ValueError("strict integral windows require K=2.")
center = self.biases[:, 0]
width = ops.softplus(self.biases[:, 1])
use_taylor = not self.learnable_signs
return multibias_integral_window_forward(
z,
center,
width,
self.signs,
self.spec,
normalize=normalize,
small_width_threshold=small_width_threshold,
use_small_width_taylor=use_taylor,
)
return multibias_integral_forward(z, self.biases, self.signs, self.spec)
# ----- introspection -----
@property
def bias_spread(self) -> Any:
return ops.max(self.biases, axis=-1) - ops.min(self.biases, axis=-1)
@property
def is_identity_nested(self) -> bool:
biases = np.asarray(ops.convert_to_numpy(self.biases.value))
signs = np.asarray(ops.convert_to_numpy(self.signs.value))
tied = bool(np.all(biases.max(axis=-1) - biases.min(axis=-1) == 0))
sums_to_one = bool(np.all(np.abs(signs.sum(axis=-1) - 1.0) < 1e-6))
return tied and sums_to_one
def get_config(self) -> dict[str, Any]:
config = super().get_config()
config.update(
{
"num_channels": self.num_channels,
"K": self.K,
"base": self._base_name,
"init_bias": self.init_bias,
"init_delta": self.init_delta,
"learnable_biases": self.learnable_biases,
"learnable_signs": self.learnable_signs,
}
)
return configOther public import paths
omnibias.keras.OMBUomnibias.keras.OperatorMultiBiasUnitomnibias.keras.unit.OperatorMultiBiasUnit