omnibias.torch.activations.piecewise

0.5.0rc1Release candidatePython >=3.10Apache-2.0

PyTorch activation derivatives, trainable operator layers, network jets, and direct differential-operator contractions.

import omnibias.torch.activations.piecewise

Module contract & mathematical conventions

Piecewise activations with almost-everywhere (regular-part) derivative towers.

This module covers the "hard" non-smooth activation family: ReLU relatives, clamps, shrinkage operators, and the sign / step primitives. Each carries a **closed-form all-orders** fast path on the *almost-everywhere* (a.e.) / regular-part convention:

on every open linear piece the higher-order tower is exactly zero;
on every open smooth piece (elu/selu/celu exponential arms, hardswish quadratic middle, softsign rational arms) the tower is the exact classical derivative;
the **singular part** (Dirac deltas and their derivatives) living on the measure-zero breakpoint set is *dropped*, and boundary values follow the PyTorch convention (H(0) = 0, sign(0) = 0).

For a differentiable surrogate whose singular bump *emerges* as a temperature grows -- soft_relu, soft_step, soft_sign -- see omnibias.torch.activations.tempered.

Shared formulas do not imply universal floating-point identity. Verify dtype, shape, derivative-order support, and the stated numerical assumptions for the path you use.

Module source ↗Source revision 9a2d73df0573Running reference examples →

CONSTANT

ABS

omnibias.torch.activations.piecewise.ABSSource ↗

ABS = register_activation(ActivationSpec(name='abs', forward=_abs_forward, derivative=_abs_derivative, fastpath=_abs_fastpath, integral=_abs_integral, riccati_polynomial=None, noise_model='none', operator_role='Absolute value; derivative sign(z) (sign(0)=0), n>=2 -> 0 (a.e.).'))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

ABS = register_activation(
    ActivationSpec(
        name="abs",
        forward=_abs_forward,
        derivative=_abs_derivative,
        fastpath=_abs_fastpath,
        integral=_abs_integral,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="Absolute value; derivative sign(z) (sign(0)=0), n>=2 -> 0 (a.e.).",
    )
)

CONSTANT

CELU

omnibias.torch.activations.piecewise.CELUSource ↗

CELU = register_activation(make_celu_spec(_DEFAULT_CELU_ALPHA))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

CELU = register_activation(make_celu_spec(_DEFAULT_CELU_ALPHA))

CONSTANT

ELU

omnibias.torch.activations.piecewise.ELUSource ↗

ELU = register_activation(make_elu_spec(_DEFAULT_ELU_ALPHA))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

ELU = register_activation(make_elu_spec(_DEFAULT_ELU_ALPHA))

CONSTANT

HARDSHRINK

omnibias.torch.activations.piecewise.HARDSHRINKSource ↗

HARDSHRINK = register_activation(make_hardshrink_spec())

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

HARDSHRINK = register_activation(make_hardshrink_spec())

CONSTANT

HARDSIGMOID

omnibias.torch.activations.piecewise.HARDSIGMOIDSource ↗

HARDSIGMOID = register_activation(ActivationSpec(name='hardsigmoid', forward=_hardsigmoid_forward, derivative=_hardsigmoid_derivative, fastpath=_hardsigmoid_fastpath, riccati_polynomial=None, noise_model='none', operator_role='Hard sigmoid: clamp(z/6 + 1/2, 0, 1); a.e. regular-part tower (n>=2 -> 0).', limit_pos_inf=1.0, limit_neg_inf=0.0))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

HARDSIGMOID = register_activation(
    ActivationSpec(
        name="hardsigmoid",
        forward=_hardsigmoid_forward,
        derivative=_hardsigmoid_derivative,
        fastpath=_hardsigmoid_fastpath,
        riccati_polynomial=None,
        noise_model="none",
        operator_role=(
            "Hard sigmoid: clamp(z/6 + 1/2, 0, 1); a.e. regular-part tower (n>=2 -> 0)."
        ),
        limit_pos_inf=1.0,
        limit_neg_inf=0.0,
    )
)

CONSTANT

HARDSWISH

omnibias.torch.activations.piecewise.HARDSWISHSource ↗

HARDSWISH = register_activation(ActivationSpec(name='hardswish', forward=_hardswish_forward, derivative=_hardswish_derivative, fastpath=_hardswish_fastpath, riccati_polynomial=None, noise_model='none', operator_role='Hard swish: z * hardsigmoid(z); a.e. regular-part tower (quadratic middle gives n=2 -> 1/3, n>=3 -> 0).', limit_neg_inf=0.0))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

HARDSWISH = register_activation(
    ActivationSpec(
        name="hardswish",
        forward=_hardswish_forward,
        derivative=_hardswish_derivative,
        fastpath=_hardswish_fastpath,
        riccati_polynomial=None,
        noise_model="none",
        operator_role=(
            "Hard swish: z * hardsigmoid(z); a.e. regular-part tower "
            "(quadratic middle gives n=2 -> 1/3, n>=3 -> 0)."
        ),
        limit_neg_inf=0.0,
    )
)

CONSTANT

HARDTANH

omnibias.torch.activations.piecewise.HARDTANHSource ↗

HARDTANH = register_activation(make_hardtanh_spec())

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

HARDTANH = register_activation(make_hardtanh_spec())

CONSTANT

LEAKY_RELU

omnibias.torch.activations.piecewise.LEAKY_RELUSource ↗

LEAKY_RELU = register_activation(make_leaky_relu_spec(_DEFAULT_LEAKY_SLOPE))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

LEAKY_RELU = register_activation(make_leaky_relu_spec(_DEFAULT_LEAKY_SLOPE))

CONSTANT

PRELU

omnibias.torch.activations.piecewise.PRELUSource ↗

PRELU = register_activation(make_leaky_relu_spec(_DEFAULT_PRELU_INIT, name='prelu'))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

PRELU = register_activation(make_leaky_relu_spec(_DEFAULT_PRELU_INIT, name="prelu"))

CONSTANT

RELU6

omnibias.torch.activations.piecewise.RELU6Source ↗

RELU6 = register_activation(ActivationSpec(name='relu6', forward=_relu6_forward, derivative=_relu6_derivative, fastpath=_relu6_fastpath, riccati_polynomial=None, noise_model='none', operator_role='ReLU6: clamp(z, 0, 6); a.e. regular-part tower (n>=2 -> 0).', limit_pos_inf=6.0, limit_neg_inf=0.0))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

RELU6 = register_activation(
    ActivationSpec(
        name="relu6",
        forward=_relu6_forward,
        derivative=_relu6_derivative,
        fastpath=_relu6_fastpath,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="ReLU6: clamp(z, 0, 6); a.e. regular-part tower (n>=2 -> 0).",
        limit_pos_inf=6.0,
        limit_neg_inf=0.0,
    )
)

CONSTANT

SELU

omnibias.torch.activations.piecewise.SELUSource ↗

SELU = register_activation(ActivationSpec(name='selu', forward=_selu_forward, derivative=_selu_derivative, fastpath=_selu_fastpath, riccati_polynomial=None, noise_model='none', operator_role='SELU: scale * ELU_{alpha}; self-normalizing; exact per-piece tower.', limit_neg_inf=-_SELU_SCALE * _SELU_ALPHA))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

SELU = register_activation(
    ActivationSpec(
        name="selu",
        forward=_selu_forward,
        derivative=_selu_derivative,
        fastpath=_selu_fastpath,
        riccati_polynomial=None,
        noise_model="none",
        operator_role=(
            "SELU: scale * ELU_{alpha}; self-normalizing; exact per-piece tower."
        ),
        limit_neg_inf=-_SELU_SCALE * _SELU_ALPHA,
    )
)

CONSTANT

SIGN

omnibias.torch.activations.piecewise.SIGNSource ↗

SIGN = register_activation(ActivationSpec(name='sign', forward=_sign_forward, derivative=_sign_derivative, fastpath=_sign_fastpath, riccati_polynomial=None, noise_model='none', operator_role='Sign (sign(0)=0); a.e. tower is zero for all n>=1 (the singular delta at 0 is dropped). Smooth twin: soft_sign (tanh(beta z)).', limit_pos_inf=1.0, limit_neg_inf=-1.0))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

SIGN = register_activation(
    ActivationSpec(
        name="sign",
        forward=_sign_forward,
        derivative=_sign_derivative,
        fastpath=_sign_fastpath,
        riccati_polynomial=None,
        noise_model="none",
        operator_role=(
            "Sign (sign(0)=0); a.e. tower is zero for all n>=1 (the singular "
            "delta at 0 is dropped). Smooth twin: soft_sign (tanh(beta z))."
        ),
        limit_pos_inf=1.0,
        limit_neg_inf=-1.0,
    )
)

CONSTANT

SOFTSHRINK

omnibias.torch.activations.piecewise.SOFTSHRINKSource ↗

SOFTSHRINK = register_activation(make_softshrink_spec())

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

SOFTSHRINK = register_activation(make_softshrink_spec())

CONSTANT

SOFTSIGN

omnibias.torch.activations.piecewise.SOFTSIGNSource ↗

SOFTSIGN = register_activation(ActivationSpec(name='softsign', forward=_softsign_forward, derivative=_softsign_derivative, fastpath=_softsign_fastpath, riccati_polynomial=None, noise_model='none', operator_role='Softsign: z/(1+|z|); exact rational per-arm tower.', limit_pos_inf=1.0, limit_neg_inf=-1.0))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

SOFTSIGN = register_activation(
    ActivationSpec(
        name="softsign",
        forward=_softsign_forward,
        derivative=_softsign_derivative,
        fastpath=_softsign_fastpath,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="Softsign: z/(1+|z|); exact rational per-arm tower.",
        limit_pos_inf=1.0,
        limit_neg_inf=-1.0,
    )
)

CONSTANT

STEP

omnibias.torch.activations.piecewise.STEPSource ↗

STEP = register_activation(ActivationSpec(name='step', forward=_step_forward, derivative=_sign_derivative, fastpath=_step_fastpath, riccati_polynomial=None, noise_model='none', operator_role='Heaviside step (H(0)=0); a.e. tower is zero for all n>=1. Smooth twin: soft_step (sigmoid(beta z)).', aliases=('heaviside',), limit_pos_inf=1.0, limit_neg_inf=0.0))

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

STEP = register_activation(
    ActivationSpec(
        name="step",
        forward=_step_forward,
        derivative=_sign_derivative,
        fastpath=_step_fastpath,
        riccati_polynomial=None,
        noise_model="none",
        operator_role=(
            "Heaviside step (H(0)=0); a.e. tower is zero for all n>=1. Smooth "
            "twin: soft_step (sigmoid(beta z))."
        ),
        aliases=("heaviside",),
        limit_pos_inf=1.0,
        limit_neg_inf=0.0,
    )
)

CONSTANT

THRESHOLD

omnibias.torch.activations.piecewise.THRESHOLDSource ↗

THRESHOLD = register_activation(make_threshold_spec())

This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.

Read the implementation

THRESHOLD = register_activation(make_threshold_spec())

FUNCTION

make_celu_spec

omnibias.torch.activations.piecewise.make_celu_specSource ↗

make_celu_spec(alpha: float=_DEFAULT_CELU_ALPHA, *, name: str='celu') -> ActivationSpec[Tensor]

CELU spec with fixed alpha (exact on each open piece, all orders).

Parameters

Name	Type & default	Contract
`alpha`positional-or-keyword	`float`Default: _DEFAULT_CELU_ALPHA	See the operation contract and implementation below.
`name`keyword-only	`str`Default: 'celu'	See the operation contract and implementation below.

ReturnsActivationSpec[Tensor]

Explicit validation & failure cases

When alpha == 0, raises ValueError('CELU alpha must be nonzero.').

These are explicit source guards. Backend operations may also raise for invalid shapes, dtypes, or unsupported devices.

Read the implementation

def make_celu_spec(
    alpha: float = _DEFAULT_CELU_ALPHA, *, name: str = "celu"
) -> ActivationSpec[Tensor]:
    """CELU spec with fixed ``alpha`` (exact on each open piece, all orders)."""
    if alpha == 0:
        raise ValueError("CELU alpha must be nonzero.")

    def fwd(z: Tensor) -> Tensor:
        return _celu_forward(z, alpha)

    def deriv(z: Tensor) -> Tensor:
        return _celu_derivative(z, alpha)

    def fp(z: Tensor, n: int) -> Tensor:
        return _celu_fastpath(z, n, alpha)

    return ActivationSpec(
        name=name,
        forward=fwd,
        derivative=deriv,
        fastpath=fp,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="CELU: z for z>0, alpha*(exp(z/alpha)-1) otherwise; exact per-piece tower.",
        limit_neg_inf=-alpha,
    )

FUNCTION

make_elu_spec

omnibias.torch.activations.piecewise.make_elu_specSource ↗

make_elu_spec(alpha: float=_DEFAULT_ELU_ALPHA, *, name: str='elu') -> ActivationSpec[Tensor]

ELU spec with fixed alpha (exact on each open piece, all orders).

Parameters

Name	Type & default	Contract
`alpha`positional-or-keyword	`float`Default: _DEFAULT_ELU_ALPHA	See the operation contract and implementation below.
`name`keyword-only	`str`Default: 'elu'	See the operation contract and implementation below.

ReturnsActivationSpec[Tensor]

Read the implementation

def make_elu_spec(
    alpha: float = _DEFAULT_ELU_ALPHA, *, name: str = "elu"
) -> ActivationSpec[Tensor]:
    """ELU spec with fixed ``alpha`` (exact on each open piece, all orders)."""

    def fwd(z: Tensor) -> Tensor:
        return _elu_forward(z, alpha)

    def deriv(z: Tensor) -> Tensor:
        return _elu_derivative(z, alpha)

    def integ(z: Tensor) -> Tensor:
        return _elu_integral(z, alpha)

    def fp(z: Tensor, n: int) -> Tensor:
        return _elu_fastpath(z, n, alpha)

    return ActivationSpec(
        name=name,
        forward=fwd,
        derivative=deriv,
        fastpath=fp,
        integral=integ,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="ELU: z for z>0, alpha*(exp(z)-1) otherwise; exact per-piece tower.",
        limit_neg_inf=-alpha,
    )

FUNCTION

make_hardshrink_spec

omnibias.torch.activations.piecewise.make_hardshrink_specSource ↗

make_hardshrink_spec(lambd: float=_DEFAULT_SHRINK_LAMBD, *, name: str='hardshrink') -> ActivationSpec[Tensor]

Hard-shrinkage spec with threshold lambd (discontinuous at +/-lambd).

Parameters

Name	Type & default	Contract
`lambd`positional-or-keyword	`float`Default: _DEFAULT_SHRINK_LAMBD	See the operation contract and implementation below.
`name`keyword-only	`str`Default: 'hardshrink'	See the operation contract and implementation below.

ReturnsActivationSpec[Tensor]

Read the implementation

def make_hardshrink_spec(
    lambd: float = _DEFAULT_SHRINK_LAMBD, *, name: str = "hardshrink"
) -> ActivationSpec[Tensor]:
    """Hard-shrinkage spec with threshold ``lambd`` (discontinuous at +/-lambd)."""

    def fwd(z: Tensor) -> Tensor:
        return _hardshrink_forward(z, lambd)

    def deriv(z: Tensor) -> Tensor:
        return _hardshrink_derivative(z, lambd)

    def fp(z: Tensor, n: int) -> Tensor:
        return _ae_linear_fastpath(z, n, fwd, deriv)

    return ActivationSpec(
        name=name,
        forward=fwd,
        derivative=deriv,
        fastpath=fp,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="Hard-shrink; a.e. regular-part tower (n>=2 -> 0).",
    )

FUNCTION

make_hardtanh_spec

omnibias.torch.activations.piecewise.make_hardtanh_specSource ↗

make_hardtanh_spec(min_val: float=_DEFAULT_HARDTANH_MIN, max_val: float=_DEFAULT_HARDTANH_MAX, *, name: str='hardtanh') -> ActivationSpec[Tensor]

Hardtanh spec clamping to [min_val, max_val] (a.e. all-orders tower).

Parameters

Name	Type & default	Contract
`min_val`positional-or-keyword	`float`Default: _DEFAULT_HARDTANH_MIN	See the operation contract and implementation below.
`max_val`positional-or-keyword	`float`Default: _DEFAULT_HARDTANH_MAX	See the operation contract and implementation below.
`name`keyword-only	`str`Default: 'hardtanh'	See the operation contract and implementation below.

ReturnsActivationSpec[Tensor]

Read the implementation

def make_hardtanh_spec(
    min_val: float = _DEFAULT_HARDTANH_MIN,
    max_val: float = _DEFAULT_HARDTANH_MAX,
    *,
    name: str = "hardtanh",
) -> ActivationSpec[Tensor]:
    """Hardtanh spec clamping to ``[min_val, max_val]`` (a.e. all-orders tower)."""

    def fwd(z: Tensor) -> Tensor:
        return _hardtanh_forward(z, min_val, max_val)

    def deriv(z: Tensor) -> Tensor:
        return _hardtanh_derivative(z, min_val, max_val)

    def fp(z: Tensor, n: int) -> Tensor:
        return _ae_linear_fastpath(z, n, fwd, deriv)

    return ActivationSpec(
        name=name,
        forward=fwd,
        derivative=deriv,
        fastpath=fp,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="Hardtanh: clamp(z, min, max); a.e. regular-part tower (n>=2 -> 0).",
        limit_pos_inf=max_val,
        limit_neg_inf=min_val,
    )

FUNCTION

make_leaky_relu_spec

omnibias.torch.activations.piecewise.make_leaky_relu_specSource ↗

make_leaky_relu_spec(slope: float=_DEFAULT_LEAKY_SLOPE, *, name: str='leaky_relu') -> ActivationSpec[Tensor]

Leaky-ReLU spec with fixed negative slope (a.e. all-orders tower).

Parameters

Name	Type & default	Contract
`slope`positional-or-keyword	`float`Default: _DEFAULT_LEAKY_SLOPE	See the operation contract and implementation below.
`name`keyword-only	`str`Default: 'leaky_relu'	See the operation contract and implementation below.

ReturnsActivationSpec[Tensor]

Executed regression · soft leaky relu converges

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_soft_leaky_relu_converges() -> None:
    alpha = 0.1
    leaky = get_activation("leaky_relu")  # slope 0.01; build matching alpha spec
    from omnibias.torch.activations.piecewise import make_leaky_relu_spec

    hard = make_leaky_relu_spec(alpha)
    z = torch.tensor([-3.0, -1.0, -0.3, 0.3, 1.0, 3.0], dtype=torch.float64)
    err = (make_soft_leaky_relu_spec(alpha, 128.0).forward(z) - hard.forward(z)).abs().max()
    assert err.item() < 1e-2
    # derivative arms: alpha on z<0, 1 on z>0.
    d = make_soft_leaky_relu_spec(alpha, 128.0).fastpath(z, 1)
    expected = torch.where(z > 0, torch.ones_like(z), torch.full_like(z, alpha))
    torch.testing.assert_close(d, expected, rtol=0, atol=1e-2)
    assert leaky.name == "leaky_relu"

python -m pytest "packages/omnibias-torch/tests/test_tempered.py::test_soft_leaky_relu_converges" -q

Read the implementation

def make_leaky_relu_spec(
    slope: float = _DEFAULT_LEAKY_SLOPE, *, name: str = "leaky_relu"
) -> ActivationSpec[Tensor]:
    """Leaky-ReLU spec with fixed negative ``slope`` (a.e. all-orders tower)."""

    def fwd(z: Tensor) -> Tensor:
        return _leaky_relu_forward(z, slope)

    def deriv(z: Tensor) -> Tensor:
        return _leaky_relu_derivative(z, slope)

    def integ(z: Tensor) -> Tensor:
        return _leaky_relu_integral(z, slope)

    def fp(z: Tensor, n: int) -> Tensor:
        return _ae_linear_fastpath(z, n, fwd, deriv)

    return ActivationSpec(
        name=name,
        forward=fwd,
        derivative=deriv,
        fastpath=fp,
        integral=integ,
        riccati_polynomial=None,
        noise_model="none",
        operator_role=(
            "Leaky ReLU: z for z>0, slope*z otherwise; a.e. regular-part tower "
            "(n>=2 -> 0)."
        ),
    )

FUNCTION

make_softshrink_spec

omnibias.torch.activations.piecewise.make_softshrink_specSource ↗

make_softshrink_spec(lambd: float=_DEFAULT_SHRINK_LAMBD, *, name: str='softshrink') -> ActivationSpec[Tensor]

Soft-shrinkage spec (proximal of L1) with threshold lambd.

Parameters

Name	Type & default	Contract
`lambd`positional-or-keyword	`float`Default: _DEFAULT_SHRINK_LAMBD	See the operation contract and implementation below.
`name`keyword-only	`str`Default: 'softshrink'	See the operation contract and implementation below.

ReturnsActivationSpec[Tensor]

Read the implementation

def make_softshrink_spec(
    lambd: float = _DEFAULT_SHRINK_LAMBD, *, name: str = "softshrink"
) -> ActivationSpec[Tensor]:
    """Soft-shrinkage spec (proximal of L1) with threshold ``lambd``."""

    def fwd(z: Tensor) -> Tensor:
        return _softshrink_forward(z, lambd)

    def deriv(z: Tensor) -> Tensor:
        return _softshrink_derivative(z, lambd)

    def fp(z: Tensor, n: int) -> Tensor:
        return _ae_linear_fastpath(z, n, fwd, deriv)

    return ActivationSpec(
        name=name,
        forward=fwd,
        derivative=deriv,
        fastpath=fp,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="Soft-shrink (L1 proximal); a.e. regular-part tower (n>=2 -> 0).",
    )

FUNCTION

make_threshold_spec

omnibias.torch.activations.piecewise.make_threshold_specSource ↗

make_threshold_spec(threshold: float=_DEFAULT_THRESHOLD, value: float=_DEFAULT_THRESHOLD_VALUE, *, name: str='threshold') -> ActivationSpec[Tensor]

Threshold spec: identity above threshold, constant value below.

Parameters

Name	Type & default	Contract
`threshold`positional-or-keyword	`float`Default: _DEFAULT_THRESHOLD	See the operation contract and implementation below.
`value`positional-or-keyword	`float`Default: _DEFAULT_THRESHOLD_VALUE	See the operation contract and implementation below.
`name`keyword-only	`str`Default: 'threshold'	See the operation contract and implementation below.

ReturnsActivationSpec[Tensor]

Read the implementation

def make_threshold_spec(
    threshold: float = _DEFAULT_THRESHOLD,
    value: float = _DEFAULT_THRESHOLD_VALUE,
    *,
    name: str = "threshold",
) -> ActivationSpec[Tensor]:
    """Threshold spec: identity above ``threshold``, constant ``value`` below."""

    def fwd(z: Tensor) -> Tensor:
        return _threshold_forward(z, threshold, value)

    def deriv(z: Tensor) -> Tensor:
        return _threshold_derivative(z, threshold)

    def fp(z: Tensor, n: int) -> Tensor:
        return _ae_linear_fastpath(z, n, fwd, deriv)

    return ActivationSpec(
        name=name,
        forward=fwd,
        derivative=deriv,
        fastpath=fp,
        riccati_polynomial=None,
        noise_model="none",
        operator_role="Threshold gate; a.e. regular-part tower (n>=2 -> 0).",
        limit_neg_inf=value,
    )

← Package overview Runnable learning examples →

omnibias.torch.activations.piecewise

ABS​

CELU​

ELU​

HARDSHRINK​

HARDSIGMOID​

HARDSWISH​

HARDTANH​

LEAKY_RELU​

PRELU​

RELU6​

SELU​

SIGN​

SOFTSHRINK​

SOFTSIGN​

STEP​

THRESHOLD​

make_celu_spec​

Parameters

make_elu_spec​

Parameters

make_hardshrink_spec​

Parameters

make_hardtanh_spec​

Parameters

make_leaky_relu_spec​

Parameters

make_softshrink_spec​

Parameters

make_threshold_spec​

Parameters

ABS

CELU

ELU

HARDSHRINK

HARDSIGMOID

HARDSWISH

HARDTANH

LEAKY_RELU

PRELU

RELU6

SELU

SIGN

SOFTSHRINK

SOFTSIGN

STEP

THRESHOLD

make_celu_spec

make_elu_spec

make_hardshrink_spec

make_hardtanh_spec

make_leaky_relu_spec

make_softshrink_spec

make_threshold_spec