omnibias.keras.activations.piecewise

0.0.2a1AlphaPython >=3.10Apache-2.0

Keras 3 operator layers and activation paths implemented with keras.ops for TensorFlow, JAX, or Torch.

import omnibias.keras.activations.piecewise

Module contract & mathematical conventions

Piecewise activations with almost-everywhere (regular-part) derivative towers.

Keras (keras.ops) twin of omnibias.torch.activations.piecewise; see that module for the derivation of every tower. Registered into the Keras registry so all three backends expose the same names. Higher-order towers use the a.e. / regular-part convention (singular delta dropped; H(0) = 0, sign(0) = 0). Smooth surrogates live in omnibias.keras.activations.tempered.

Shared formulas do not imply universal floating-point identity. Verify dtype, shape, derivative-order support, and the stated numerical assumptions for the path you use.

Module source ↗Source revision 9a2d73df0573Running reference examples →

CONSTANT

ABS

omnibias.keras.activations.piecewise.ABSSource ↗

ABS = register_activation(ActivationSpec(name='abs', forward=_abs_forward, derivative=_abs_derivative, fastpath=_abs_fastpath, integral=_abs_integral, riccati_polynomial=None, noise_model='none', operator_role='Absolute value; derivative sign(z) (sign(0)=0), n>=2 -> 0 (a.e.).'))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

ABS = register_activation(
    ActivationSpec(
        name="abs",
        forward=_abs_forward,
        derivative=_abs_derivative,
        fastpath=_abs_fastpath,
        integral=_abs_integral,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="Absolute value; derivative sign(z) (sign(0)=0), n>=2 -> 0 (a.e.).",
    )
)

CONSTANT

CELU

omnibias.keras.activations.piecewise.CELUSource ↗

CELU = register_activation(make_celu_spec(_DEFAULT_CELU_ALPHA))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

CELU = register_activation(make_celu_spec(_DEFAULT_CELU_ALPHA))

CONSTANT

ELU

omnibias.keras.activations.piecewise.ELUSource ↗

ELU = register_activation(make_elu_spec(_DEFAULT_ELU_ALPHA))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

ELU = register_activation(make_elu_spec(_DEFAULT_ELU_ALPHA))

CONSTANT

HARDSHRINK

omnibias.keras.activations.piecewise.HARDSHRINKSource ↗

HARDSHRINK = register_activation(make_hardshrink_spec())

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

HARDSHRINK = register_activation(make_hardshrink_spec())

CONSTANT

HARDSIGMOID

omnibias.keras.activations.piecewise.HARDSIGMOIDSource ↗

HARDSIGMOID = register_activation(ActivationSpec(name='hardsigmoid', forward=_hardsigmoid_forward, derivative=_hardsigmoid_derivative, fastpath=_hardsigmoid_fastpath, riccati_polynomial=None, noise_model='none', operator_role='Hard sigmoid: clamp(z/6 + 1/2, 0, 1); a.e. regular-part tower (n>=2 -> 0).', limit_pos_inf=1.0, limit_neg_inf=0.0))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

HARDSIGMOID = register_activation(
    ActivationSpec(
        name="hardsigmoid",
        forward=_hardsigmoid_forward,
        derivative=_hardsigmoid_derivative,
        fastpath=_hardsigmoid_fastpath,
        riccati_polynomial=None,
        noise_model="none",
        operator_role=(
            "Hard sigmoid: clamp(z/6 + 1/2, 0, 1); a.e. regular-part tower (n>=2 -> 0)."
        ),
        limit_pos_inf=1.0,
        limit_neg_inf=0.0,
    )
)

CONSTANT

HARDSWISH

omnibias.keras.activations.piecewise.HARDSWISHSource ↗

HARDSWISH = register_activation(ActivationSpec(name='hardswish', forward=_hardswish_forward, derivative=_hardswish_derivative, fastpath=_hardswish_fastpath, riccati_polynomial=None, noise_model='none', operator_role='Hard swish: z * hardsigmoid(z); a.e. regular-part tower (quadratic middle gives n=2 -> 1/3, n>=3 -> 0).', limit_neg_inf=0.0))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

HARDSWISH = register_activation(
    ActivationSpec(
        name="hardswish",
        forward=_hardswish_forward,
        derivative=_hardswish_derivative,
        fastpath=_hardswish_fastpath,
        riccati_polynomial=None,
        noise_model="none",
        operator_role=(
            "Hard swish: z * hardsigmoid(z); a.e. regular-part tower "
            "(quadratic middle gives n=2 -> 1/3, n>=3 -> 0)."
        ),
        limit_neg_inf=0.0,
    )
)

CONSTANT

HARDTANH

omnibias.keras.activations.piecewise.HARDTANHSource ↗

HARDTANH = register_activation(make_hardtanh_spec())

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

HARDTANH = register_activation(make_hardtanh_spec())

CONSTANT

LEAKY_RELU

omnibias.keras.activations.piecewise.LEAKY_RELUSource ↗

LEAKY_RELU = register_activation(make_leaky_relu_spec(_DEFAULT_LEAKY_SLOPE))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

LEAKY_RELU = register_activation(make_leaky_relu_spec(_DEFAULT_LEAKY_SLOPE))

CONSTANT

PRELU

omnibias.keras.activations.piecewise.PRELUSource ↗

PRELU = register_activation(make_leaky_relu_spec(_DEFAULT_PRELU_INIT, name='prelu'))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

PRELU = register_activation(make_leaky_relu_spec(_DEFAULT_PRELU_INIT, name="prelu"))

CONSTANT

RELU6

omnibias.keras.activations.piecewise.RELU6Source ↗

RELU6 = register_activation(ActivationSpec(name='relu6', forward=_relu6_forward, derivative=_relu6_derivative, fastpath=_relu6_fastpath, riccati_polynomial=None, noise_model='none', operator_role='ReLU6: clamp(z, 0, 6); a.e. regular-part tower (n>=2 -> 0).', limit_pos_inf=6.0, limit_neg_inf=0.0))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

RELU6 = register_activation(
    ActivationSpec(
        name="relu6",
        forward=_relu6_forward,
        derivative=_relu6_derivative,
        fastpath=_relu6_fastpath,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="ReLU6: clamp(z, 0, 6); a.e. regular-part tower (n>=2 -> 0).",
        limit_pos_inf=6.0,
        limit_neg_inf=0.0,
    )
)

CONSTANT

SELU

omnibias.keras.activations.piecewise.SELUSource ↗

SELU = register_activation(ActivationSpec(name='selu', forward=_selu_forward, derivative=_selu_derivative, fastpath=_selu_fastpath, riccati_polynomial=None, noise_model='none', operator_role='SELU: scale * ELU_{alpha}; self-normalizing; exact per-piece tower.', limit_neg_inf=-_SELU_SCALE * _SELU_ALPHA))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

SELU = register_activation(
    ActivationSpec(
        name="selu",
        forward=_selu_forward,
        derivative=_selu_derivative,
        fastpath=_selu_fastpath,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="SELU: scale * ELU_{alpha}; self-normalizing; exact per-piece tower.",
        limit_neg_inf=-_SELU_SCALE * _SELU_ALPHA,
    )
)

CONSTANT

SIGN

omnibias.keras.activations.piecewise.SIGNSource ↗

SIGN = register_activation(ActivationSpec(name='sign', forward=_sign_forward, derivative=_sign_derivative, fastpath=_sign_fastpath, riccati_polynomial=None, noise_model='none', operator_role='Sign (sign(0)=0); a.e. tower is zero for all n>=1 (the singular delta at 0 is dropped). Smooth twin: soft_sign (tanh(beta z)).', limit_pos_inf=1.0, limit_neg_inf=-1.0))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

SIGN = register_activation(
    ActivationSpec(
        name="sign",
        forward=_sign_forward,
        derivative=_sign_derivative,
        fastpath=_sign_fastpath,
        riccati_polynomial=None,
        noise_model="none",
        operator_role=(
            "Sign (sign(0)=0); a.e. tower is zero for all n>=1 (the singular "
            "delta at 0 is dropped). Smooth twin: soft_sign (tanh(beta z))."
        ),
        limit_pos_inf=1.0,
        limit_neg_inf=-1.0,
    )
)

CONSTANT

SOFTSHRINK

omnibias.keras.activations.piecewise.SOFTSHRINKSource ↗

SOFTSHRINK = register_activation(make_softshrink_spec())

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

SOFTSHRINK = register_activation(make_softshrink_spec())

CONSTANT

SOFTSIGN

omnibias.keras.activations.piecewise.SOFTSIGNSource ↗

SOFTSIGN = register_activation(ActivationSpec(name='softsign', forward=_softsign_forward, derivative=_softsign_derivative, fastpath=_softsign_fastpath, riccati_polynomial=None, noise_model='none', operator_role='Softsign: z/(1+|z|); exact rational per-arm tower.', limit_pos_inf=1.0, limit_neg_inf=-1.0))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

SOFTSIGN = register_activation(
    ActivationSpec(
        name="softsign",
        forward=_softsign_forward,
        derivative=_softsign_derivative,
        fastpath=_softsign_fastpath,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="Softsign: z/(1+|z|); exact rational per-arm tower.",
        limit_pos_inf=1.0,
        limit_neg_inf=-1.0,
    )
)

CONSTANT

STEP

omnibias.keras.activations.piecewise.STEPSource ↗

STEP = register_activation(ActivationSpec(name='step', forward=_step_forward, derivative=_sign_derivative, fastpath=_step_fastpath, riccati_polynomial=None, noise_model='none', operator_role='Heaviside step (H(0)=0); a.e. tower is zero for all n>=1. Smooth twin: soft_step (sigmoid(beta z)).', aliases=('heaviside',), limit_pos_inf=1.0, limit_neg_inf=0.0))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

STEP = register_activation(
    ActivationSpec(
        name="step",
        forward=_step_forward,
        derivative=_sign_derivative,
        fastpath=_step_fastpath,
        riccati_polynomial=None,
        noise_model="none",
        operator_role=(
            "Heaviside step (H(0)=0); a.e. tower is zero for all n>=1. Smooth "
            "twin: soft_step (sigmoid(beta z))."
        ),
        aliases=("heaviside",),
        limit_pos_inf=1.0,
        limit_neg_inf=0.0,
    )
)

CONSTANT

THRESHOLD

omnibias.keras.activations.piecewise.THRESHOLDSource ↗

THRESHOLD = register_activation(make_threshold_spec())

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

THRESHOLD = register_activation(make_threshold_spec())

FUNCTION

make_celu_spec

omnibias.keras.activations.piecewise.make_celu_specSource ↗

make_celu_spec(alpha: float=_DEFAULT_CELU_ALPHA, *, name: str='celu') -> ActivationSpec[Any]

CELU spec with fixed alpha (exact on each open piece, all orders).

Parameters

Name	Type & default	Contract
`alpha`positional-or-keyword	`float`Default: _DEFAULT_CELU_ALPHA	See the operation contract and implementation below.
`name`keyword-only	`str`Default: 'celu'	See the operation contract and implementation below.

ReturnsActivationSpec[Any]

Explicit validation & failure cases

When alpha == 0, raises ValueError('CELU alpha must be nonzero.').

These are explicit source guards. Backend operations may also raise for invalid shapes, dtypes, or unsupported devices.

Read the implementation

def make_celu_spec(alpha: float = _DEFAULT_CELU_ALPHA, *, name: str = "celu") -> ActivationSpec[Any]:
    """CELU spec with fixed ``alpha`` (exact on each open piece, all orders)."""
    if alpha == 0:
        raise ValueError("CELU alpha must be nonzero.")

    def fwd(z: Any) -> Any:
        return _celu_forward(z, alpha)

    def deriv(z: Any) -> Any:
        return _celu_derivative(z, alpha)

    def fp(z: Any, n: int) -> Any:
        return _celu_fastpath(z, n, alpha)

    return ActivationSpec(
        name=name,
        forward=fwd,
        derivative=deriv,
        fastpath=fp,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="CELU: z for z>0, alpha*(exp(z/alpha)-1) otherwise; exact per-piece tower.",
        limit_neg_inf=-alpha,
    )

FUNCTION

make_elu_spec

omnibias.keras.activations.piecewise.make_elu_specSource ↗

make_elu_spec(alpha: float=_DEFAULT_ELU_ALPHA, *, name: str='elu') -> ActivationSpec[Any]

ELU spec with fixed alpha (exact on each open piece, all orders).

Parameters

Name	Type & default	Contract
`alpha`positional-or-keyword	`float`Default: _DEFAULT_ELU_ALPHA	See the operation contract and implementation below.
`name`keyword-only	`str`Default: 'elu'	See the operation contract and implementation below.

ReturnsActivationSpec[Any]

Read the implementation

def make_elu_spec(alpha: float = _DEFAULT_ELU_ALPHA, *, name: str = "elu") -> ActivationSpec[Any]:
    """ELU spec with fixed ``alpha`` (exact on each open piece, all orders)."""

    def fwd(z: Any) -> Any:
        return _elu_forward(z, alpha)

    def deriv(z: Any) -> Any:
        return _elu_derivative(z, alpha)

    def integ(z: Any) -> Any:
        return _elu_integral(z, alpha)

    def fp(z: Any, n: int) -> Any:
        return _elu_fastpath(z, n, alpha)

    return ActivationSpec(
        name=name,
        forward=fwd,
        derivative=deriv,
        fastpath=fp,
        integral=integ,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="ELU: z for z>0, alpha*(exp(z)-1) otherwise; exact per-piece tower.",
        limit_neg_inf=-alpha,
    )

FUNCTION

make_hardshrink_spec

omnibias.keras.activations.piecewise.make_hardshrink_specSource ↗

make_hardshrink_spec(lambd: float=_DEFAULT_SHRINK_LAMBD, *, name: str='hardshrink') -> ActivationSpec[Any]

Hard-shrinkage spec with threshold lambd (discontinuous at +/-lambd).

Parameters

Name	Type & default	Contract
`lambd`positional-or-keyword	`float`Default: _DEFAULT_SHRINK_LAMBD	See the operation contract and implementation below.
`name`keyword-only	`str`Default: 'hardshrink'	See the operation contract and implementation below.

ReturnsActivationSpec[Any]

Read the implementation

def make_hardshrink_spec(
    lambd: float = _DEFAULT_SHRINK_LAMBD, *, name: str = "hardshrink"
) -> ActivationSpec[Any]:
    """Hard-shrinkage spec with threshold ``lambd`` (discontinuous at +/-lambd)."""

    def fwd(z: Any) -> Any:
        return _hardshrink_forward(z, lambd)

    def deriv(z: Any) -> Any:
        return _hardshrink_derivative(z, lambd)

    def fp(z: Any, n: int) -> Any:
        return _ae_linear_fastpath(z, n, fwd, deriv)

    return ActivationSpec(
        name=name,
        forward=fwd,
        derivative=deriv,
        fastpath=fp,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="Hard-shrink; a.e. regular-part tower (n>=2 -> 0).",
    )

FUNCTION

make_hardtanh_spec

omnibias.keras.activations.piecewise.make_hardtanh_specSource ↗

make_hardtanh_spec(min_val: float=_DEFAULT_HARDTANH_MIN, max_val: float=_DEFAULT_HARDTANH_MAX, *, name: str='hardtanh') -> ActivationSpec[Any]

Hardtanh spec clamping to [min_val, max_val] (a.e. all-orders tower).

Parameters

Name	Type & default	Contract
`min_val`positional-or-keyword	`float`Default: _DEFAULT_HARDTANH_MIN	See the operation contract and implementation below.
`max_val`positional-or-keyword	`float`Default: _DEFAULT_HARDTANH_MAX	See the operation contract and implementation below.
`name`keyword-only	`str`Default: 'hardtanh'	See the operation contract and implementation below.

ReturnsActivationSpec[Any]

Read the implementation

def make_hardtanh_spec(
    min_val: float = _DEFAULT_HARDTANH_MIN,
    max_val: float = _DEFAULT_HARDTANH_MAX,
    *,
    name: str = "hardtanh",
) -> ActivationSpec[Any]:
    """Hardtanh spec clamping to ``[min_val, max_val]`` (a.e. all-orders tower)."""

    def fwd(z: Any) -> Any:
        return _hardtanh_forward(z, min_val, max_val)

    def deriv(z: Any) -> Any:
        return _hardtanh_derivative(z, min_val, max_val)

    def fp(z: Any, n: int) -> Any:
        return _ae_linear_fastpath(z, n, fwd, deriv)

    return ActivationSpec(
        name=name,
        forward=fwd,
        derivative=deriv,
        fastpath=fp,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="Hardtanh: clamp(z, min, max); a.e. regular-part tower (n>=2 -> 0).",
        limit_pos_inf=max_val,
        limit_neg_inf=min_val,
    )

FUNCTION

make_leaky_relu_spec

omnibias.keras.activations.piecewise.make_leaky_relu_specSource ↗

make_leaky_relu_spec(slope: float=_DEFAULT_LEAKY_SLOPE, *, name: str='leaky_relu') -> ActivationSpec[Any]

Leaky-ReLU spec with fixed negative slope (a.e. all-orders tower).

Parameters

Name	Type & default	Contract
`slope`positional-or-keyword	`float`Default: _DEFAULT_LEAKY_SLOPE	See the operation contract and implementation below.
`name`keyword-only	`str`Default: 'leaky_relu'	See the operation contract and implementation below.

ReturnsActivationSpec[Any]

Read the implementation

def make_leaky_relu_spec(
    slope: float = _DEFAULT_LEAKY_SLOPE, *, name: str = "leaky_relu"
) -> ActivationSpec[Any]:
    """Leaky-ReLU spec with fixed negative ``slope`` (a.e. all-orders tower)."""

    def fwd(z: Any) -> Any:
        return _leaky_relu_forward(z, slope)

    def deriv(z: Any) -> Any:
        return _leaky_relu_derivative(z, slope)

    def integ(z: Any) -> Any:
        return _leaky_relu_integral(z, slope)

    def fp(z: Any, n: int) -> Any:
        return _ae_linear_fastpath(z, n, fwd, deriv)

    return ActivationSpec(
        name=name,
        forward=fwd,
        derivative=deriv,
        fastpath=fp,
        integral=integ,
        riccati_polynomial=None,
        noise_model="none",
        operator_role=(
            "Leaky ReLU: z for z>0, slope*z otherwise; a.e. regular-part tower "
            "(n>=2 -> 0)."
        ),
    )

FUNCTION

make_softshrink_spec

omnibias.keras.activations.piecewise.make_softshrink_specSource ↗

make_softshrink_spec(lambd: float=_DEFAULT_SHRINK_LAMBD, *, name: str='softshrink') -> ActivationSpec[Any]

Soft-shrinkage spec (proximal of L1) with threshold lambd.

Parameters

Name	Type & default	Contract
`lambd`positional-or-keyword	`float`Default: _DEFAULT_SHRINK_LAMBD	See the operation contract and implementation below.
`name`keyword-only	`str`Default: 'softshrink'	See the operation contract and implementation below.

ReturnsActivationSpec[Any]

Read the implementation

def make_softshrink_spec(
    lambd: float = _DEFAULT_SHRINK_LAMBD, *, name: str = "softshrink"
) -> ActivationSpec[Any]:
    """Soft-shrinkage spec (proximal of L1) with threshold ``lambd``."""

    def fwd(z: Any) -> Any:
        return _softshrink_forward(z, lambd)

    def deriv(z: Any) -> Any:
        return _softshrink_derivative(z, lambd)

    def fp(z: Any, n: int) -> Any:
        return _ae_linear_fastpath(z, n, fwd, deriv)

    return ActivationSpec(
        name=name,
        forward=fwd,
        derivative=deriv,
        fastpath=fp,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="Soft-shrink (L1 proximal); a.e. regular-part tower (n>=2 -> 0).",
    )

FUNCTION

make_threshold_spec

omnibias.keras.activations.piecewise.make_threshold_specSource ↗

make_threshold_spec(threshold: float=_DEFAULT_THRESHOLD, value: float=_DEFAULT_THRESHOLD_VALUE, *, name: str='threshold') -> ActivationSpec[Any]

Threshold spec: identity above threshold, constant value below.

Parameters

Name	Type & default	Contract
`threshold`positional-or-keyword	`float`Default: _DEFAULT_THRESHOLD	See the operation contract and implementation below.
`value`positional-or-keyword	`float`Default: _DEFAULT_THRESHOLD_VALUE	See the operation contract and implementation below.
`name`keyword-only	`str`Default: 'threshold'	See the operation contract and implementation below.

ReturnsActivationSpec[Any]

Read the implementation

def make_threshold_spec(
    threshold: float = _DEFAULT_THRESHOLD,
    value: float = _DEFAULT_THRESHOLD_VALUE,
    *,
    name: str = "threshold",
) -> ActivationSpec[Any]:
    """Threshold spec: identity above ``threshold``, constant ``value`` below."""

    def fwd(z: Any) -> Any:
        return _threshold_forward(z, threshold, value)

    def deriv(z: Any) -> Any:
        return _threshold_derivative(z, threshold)

    def fp(z: Any, n: int) -> Any:
        return _ae_linear_fastpath(z, n, fwd, deriv)

    return ActivationSpec(
        name=name,
        forward=fwd,
        derivative=deriv,
        fastpath=fp,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="Threshold gate; a.e. regular-part tower (n>=2 -> 0).",
        limit_neg_inf=value,
    )

← Package overview Runnable learning examples →

omnibias.keras.activations.piecewise

ABS​

CELU​

ELU​

HARDSHRINK​

HARDSIGMOID​

HARDSWISH​

HARDTANH​

LEAKY_RELU​

PRELU​

RELU6​

SELU​

SIGN​

SOFTSHRINK​

SOFTSIGN​

STEP​

THRESHOLD​

make_celu_spec​

Parameters

make_elu_spec​

Parameters

make_hardshrink_spec​

Parameters

make_hardtanh_spec​

Parameters

make_leaky_relu_spec​

Parameters

make_softshrink_spec​

Parameters

make_threshold_spec​

Parameters

ABS

CELU

ELU

HARDSHRINK

HARDSIGMOID

HARDSWISH

HARDTANH

LEAKY_RELU

PRELU

RELU6

SELU

SIGN

SOFTSHRINK

SOFTSIGN

STEP

THRESHOLD

make_celu_spec

make_elu_spec

make_hardshrink_spec

make_hardtanh_spec

make_leaky_relu_spec

make_softshrink_spec

make_threshold_spec