omnibias.keras.growable

0.0.2a1AlphaPython >=3.10Apache-2.0

Keras 3 operator layers and activation paths implemented with keras.ops for TensorFlow, JAX, or Torch.

import omnibias.keras.growable

Module contract & mathematical conventions

GrowableOperatorMultiBiasUnit (Keras backend).

An OMBU whose bias arity K can grow during training without changing its output shape. Mirrors omnibias.torch.growable.GrowableOperatorMultiBiasUnit.

Two growth strategies preserve the unit's output at the moment of growth:

"pair" (default; any base activation). Adds two biases at an existing location with signs (+eta, -eta) (sum zero, output unchanged).
"saturate" (only for activations that vanish as z -> -inf: sigmoid, softplus, gaussian, exp, relu, huber). Adds one bias at -big so the new term contributes ~0.

Biases / signs are pre-allocated at K_max; the forward only sums over the first active_K columns.

Shared formulas do not imply universal floating-point identity. Verify dtype, shape, derivative-order support, and the stated numerical assumptions for the path you use.

Module source ↗Source revision 9a2d73df0573Running reference examples →

CONSTANT

GrowStrategy

omnibias.keras.GrowStrategySource ↗

GrowStrategy = Literal['pair', 'saturate']

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

GrowStrategy = Literal["pair", "saturate"]

Other public import paths

omnibias.keras.GrowStrategy
omnibias.keras.growable.GrowStrategy

CLASS

GrowableOperatorMultiBiasUnit

omnibias.keras.GrowableOMBUSource ↗

class GrowableOperatorMultiBiasUnit(layers.Layer)

A multi-bias unit whose arity K can grow during training.

Parameters

num_channels : int init_K : int, default 1 K_max : int, default 8 base : str or ActivationSpec, default "sigmoid" init_bias : float, default 0.0 learnable_biases : bool, default True learnable_signs : bool, default True pair_eta : float, default 0.05 saturate_big : float, default 25.0

Parameters

Name	Type & default	Contract
`num_channels`positional-or-keyword	`int`Required	int
`init_K`positional-or-keyword	`int`Default: 1	int, default 1
`K_max`positional-or-keyword	`int`Default: 8	int, default 8
`base`positional-or-keyword	`str \| ActivationSpec[Any]`Default: 'sigmoid'	str or `ActivationSpec`, default `"sigmoid"`
`init_bias`positional-or-keyword	`float`Default: 0.0	float, default 0.0
`learnable_biases`positional-or-keyword	`bool`Default: True	bool, default True
`learnable_signs`positional-or-keyword	`bool`Default: True	bool, default True
`pair_eta`positional-or-keyword	`float`Default: 0.05	float, default 0.05
`saturate_big`positional-or-keyword	`float`Default: 25.0	float, default 25.0
`kwargs`variadic keyword	`Any`Required	See the operation contract and implementation below.

Executed regression · grow beyond kmax raises

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_grow_beyond_kmax_raises() -> None:
    unit = GrowableOMBU(num_channels=2, init_K=1, K_max=2, base="sigmoid")
    with pytest.raises(RuntimeError):
        unit.grow("pair")

python -m pytest "packages/omnibias-keras/tests/test_growable.py::test_grow_beyond_kmax_raises" -q

Regression example · saturate growth friendly only

This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_saturate_growth_friendly_only() -> None:
    unit = GrowableOMBU(num_channels=2, init_K=1, K_max=4, base="sigmoid")
    unit.grow("saturate")
    assert unit.active_K == 2

    bad = GrowableOMBU(num_channels=2, init_K=1, K_max=4, base="tanh")
    with pytest.raises(ValueError):
        bad.grow("saturate")

python -m pytest packages/omnibias-keras/tests/test_growable.py -k test_saturate_growth_friendly_only -q

Fields & methods

__init__method

__init__(self, num_channels: int, init_K: int=1, K_max: int=8, base: str | ActivationSpec[Any]='sigmoid', init_bias: float=0.0, learnable_biases: bool=True, learnable_signs: bool=True, pair_eta: float=0.05, saturate_big: float=25.0, **kwargs: Any) -> None

See the class contract above and the exact method implementation below.

Parameters

Name	Type & default	Contract
`num_channels`positional-or-keyword	`int`Required	See the operation contract and implementation below.
`init_K`positional-or-keyword	`int`Default: 1	See the operation contract and implementation below.
`K_max`positional-or-keyword	`int`Default: 8	See the operation contract and implementation below.
`base`positional-or-keyword	`str \| ActivationSpec[Any]`Default: 'sigmoid'	See the operation contract and implementation below.
`init_bias`positional-or-keyword	`float`Default: 0.0	See the operation contract and implementation below.
`learnable_biases`positional-or-keyword	`bool`Default: True	See the operation contract and implementation below.
`learnable_signs`positional-or-keyword	`bool`Default: True	See the operation contract and implementation below.
`pair_eta`positional-or-keyword	`float`Default: 0.05	See the operation contract and implementation below.
`saturate_big`positional-or-keyword	`float`Default: 25.0	See the operation contract and implementation below.
`kwargs`variadic keyword	`Any`Required	See the operation contract and implementation below.

Returns: None

Validation & failure cases

When num_channels < 1, raises ValueError(f'num_channels must be >= 1, got {num_channels}').
When K_max < 1, raises ValueError(f'K_max must be >= 1, got {K_max}').
When not 1 <= init_K <= K_max, raises ValueError(f'init_K must be in [1, K_max={K_max}], got {init_K}').
When pair_eta <= 0, raises ValueError(f'pair_eta must be > 0, got {pair_eta}').
When saturate_big <= 0, raises ValueError(f'saturate_big must be > 0, got {saturate_big}').

Executed regression · grow beyond kmax raises

Full file, imports & fixtures ↗

def test_grow_beyond_kmax_raises() -> None:
    unit = GrowableOMBU(num_channels=2, init_K=1, K_max=2, base="sigmoid")
    with pytest.raises(RuntimeError):
        unit.grow("pair")

python -m pytest "packages/omnibias-keras/tests/test_growable.py::test_grow_beyond_kmax_raises" -q

Method implementation

    def __init__(
        self,
        num_channels: int,
        init_K: int = 1,
        K_max: int = 8,
        base: str | ActivationSpec[Any] = "sigmoid",
        init_bias: float = 0.0,
        learnable_biases: bool = True,
        learnable_signs: bool = True,
        pair_eta: float = 0.05,
        saturate_big: float = 25.0,
        **kwargs: Any,
    ) -> None:
        super().__init__(**kwargs)
        if num_channels < 1:
            raise ValueError(f"num_channels must be >= 1, got {num_channels}")
        if K_max < 1:
            raise ValueError(f"K_max must be >= 1, got {K_max}")
        if not (1 <= init_K <= K_max):
            raise ValueError(f"init_K must be in [1, K_max={K_max}], got {init_K}")
        if pair_eta <= 0:
            raise ValueError(f"pair_eta must be > 0, got {pair_eta}")
        if saturate_big <= 0:
            raise ValueError(f"saturate_big must be > 0, got {saturate_big}")

        self.num_channels = num_channels
        self.K_max = K_max
        self._base_name = base if isinstance(base, str) else base.name
        self.spec = base if isinstance(base, ActivationSpec) else get_activation(base)
        self.pair_eta = float(pair_eta)
        self.saturate_big = float(saturate_big)
        self.init_bias = init_bias
        self.learnable_biases = learnable_biases
        self.learnable_signs = learnable_signs

        dtype = self.variable_dtype
        biases_np = np.full((num_channels, K_max), float(init_bias), dtype=dtype)
        signs_np = np.zeros((num_channels, K_max), dtype=dtype)
        signs_np[:, :init_K] = identity_signs(init_K, dtype=dtype)[None, :]

        self.biases = self.add_weight(
            name="biases",
            shape=(num_channels, K_max),
            initializer="zeros",
            trainable=learnable_biases,
            dtype=dtype,
        )
        self.signs = self.add_weight(
            name="signs",
            shape=(num_channels, K_max),
            initializer="zeros",
            trainable=learnable_signs,
            dtype=dtype,
        )
        self.biases.assign(biases_np)
        self.signs.assign(signs_np)
        self._active_K = int(init_K)
        self.built = True

Name	Type & default	Contract
`strategy`positional-or-keyword	`GrowStrategy`Default: 'pair'	See the operation contract and implementation below.
`channel`keyword-only	`int \| None`Default: None	See the operation contract and implementation below.
`anchor_value`keyword-only	`float \| np.ndarray \| None`Default: None	See the operation contract and implementation below.

omnibias.keras.growable

GrowStrategy​

GrowableOperatorMultiBiasUnit​

Parameters

Fields & methods

Parameters

Parameters

Parameters

GrowStrategy

GrowableOperatorMultiBiasUnit