omnibias.torch.activations.proximal
PyTorch activation derivatives, trainable operator layers, network jets, and direct differential-operator contractions.
import omnibias.torch.activations.proximalModule contract & mathematical conventions
Proximal-class activations: huber, arctan, log1pu2.
These are the *new design-rule* activations introduced by the multi-bias operator framing. Each one is chosen so that its K=2 bias-collapse limit (i.e. its first derivative) is a classical proximal / robust-statistics primitive:
============ ======================================== ===========================
Activation K=2 collapse output Operator role
============ ======================================== ===========================
huber clip(z, -tau, tau) LASSO ISTA soft-shrink step
arctan 1 / (1 + z^2) Cauchy IRLS weight (robust)
log1pu2 2 z / (1 + z^2) Redescending M-estimator
============ ======================================== ===========================
arctan and log1pu2 are analytic (fast path n in {0, 1, 2}).
huber is piecewise-polynomial: its fast path is closed form to all
orders on the almost-everywhere / regular-part convention (n = 2 is the
indicator of the quadratic region, n >= 3 is zero), dropping the singular
part at the +/-tau kinks.
Shared formulas do not imply universal floating-point identity. Verify dtype, shape, derivative-order support, and the stated numerical assumptions for the path you use.
ARCTAN
omnibias.torch.activations.proximal.ARCTANSource ↗ARCTAN = register_activation(ActivationSpec(name='arctan', forward=_arctan_forward, derivative=_arctan_derivative, fastpath=_arctan_fastpath, integral=_arctan_integral, riccati_polynomial=None, noise_model='cauchy', operator_role='K=2 collapse -> 1/(1+z^2); Cauchy IRLS weight, natural for heavy-tailed regression residuals.', aliases=('atan',), limit_pos_inf=math.pi / 2.0, limit_neg_inf=-math.pi / 2.0))This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.
Read the implementation
ARCTAN = register_activation(
ActivationSpec(
name="arctan",
forward=_arctan_forward,
derivative=_arctan_derivative,
fastpath=_arctan_fastpath,
integral=_arctan_integral,
riccati_polynomial=None,
noise_model="cauchy",
operator_role=(
"K=2 collapse -> 1/(1+z^2); Cauchy IRLS weight, "
"natural for heavy-tailed regression residuals."
),
aliases=("atan",),
limit_pos_inf=math.pi / 2.0,
limit_neg_inf=-math.pi / 2.0,
)
)HUBER
omnibias.torch.activations.proximal.HUBERSource ↗HUBER = register_activation(make_huber_spec(_DEFAULT_HUBER_TAU))This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.
Read the implementation
HUBER = register_activation(make_huber_spec(_DEFAULT_HUBER_TAU))LOG1PU2
omnibias.torch.activations.proximal.LOG1PU2Source ↗LOG1PU2 = register_activation(ActivationSpec(name='log1pu2', forward=_log1pu2_forward, derivative=_log1pu2_derivative, fastpath=_log1pu2_fastpath, integral=_log1pu2_integral, riccati_polynomial=None, noise_model='redescending_m', operator_role='K=2 collapse -> 2 z / (1 + z^2); redescending M-estimator (Black-Anandan family), down-weights large residuals to zero.', aliases=('logcosh_dual',)))This exported constant is defined by the expression above. Its exact value and annotation come from the pinned source; use it with the module contract.
Read the implementation
LOG1PU2 = register_activation(
ActivationSpec(
name="log1pu2",
forward=_log1pu2_forward,
derivative=_log1pu2_derivative,
fastpath=_log1pu2_fastpath,
integral=_log1pu2_integral,
riccati_polynomial=None,
noise_model="redescending_m",
operator_role=(
"K=2 collapse -> 2 z / (1 + z^2); redescending M-estimator "
"(Black-Anandan family), down-weights large residuals to zero."
),
aliases=("logcosh_dual",),
)
)make_huber_spec
omnibias.torch.activations.proximal.make_huber_specSource ↗make_huber_spec(tau: float=_DEFAULT_HUBER_TAU, *, register: bool=False) -> ActivationSpec[Tensor]Build a Huber ActivationSpec with custom threshold tau.
The default-tau spec is registered as "huber" at module load.
Pass register=True to add a non-default spec to the global
registry (it will be stored under f"huber_tau{tau:g}").
Parameters
| Name | Type & default | Contract |
|---|---|---|
taupositional-or-keyword | floatDefault: _DEFAULT_HUBER_TAU | See the operation contract and implementation below. |
registerkeyword-only | boolDefault: False | See the operation contract and implementation below. |
ActivationSpec[Tensor]Explicit validation & failure cases
- When
tau <= 0, raisesValueError(f'Huber tau must be > 0, got {tau}.').
These are explicit source guards. Backend operations may also raise for invalid shapes, dtypes, or unsupported devices.
Read the implementation
def make_huber_spec(
tau: float = _DEFAULT_HUBER_TAU, *, register: bool = False
) -> ActivationSpec[Tensor]:
"""Build a Huber :class:`ActivationSpec` with custom threshold ``tau``.
The default-``tau`` spec is registered as ``"huber"`` at module load.
Pass ``register=True`` to add a non-default spec to the global
registry (it will be stored under ``f"huber_tau{tau:g}"``).
"""
if tau <= 0:
raise ValueError(f"Huber tau must be > 0, got {tau}.")
def fwd(z: Tensor) -> Tensor:
return _huber_forward(z, tau)
def deriv(z: Tensor) -> Tensor:
return _huber_derivative(z, tau)
def integ(z: Tensor) -> Tensor:
return _huber_integral(z, tau)
def fp(z: Tensor, n: int) -> Tensor:
return _huber_fastpath(z, n, tau)
name = "huber" if math.isclose(tau, _DEFAULT_HUBER_TAU) else f"huber_tau{tau:g}"
spec = ActivationSpec(
name=name,
forward=fwd,
derivative=deriv,
fastpath=fp,
integral=integ,
riccati_polynomial=None,
noise_model="laplace_smoothed",
operator_role=(
"K=2 collapse -> clip(z, -tau, tau); proximal of L1 / ISTA soft-shrink step for LASSO."
),
)
if register:
register_activation(spec)
return spec