API REFERENCE/omnibias-struct

omnibias.struct.torch

0.1.0a2AlphaPython >=3.10AGPL / commercial

Differentiable dynamic programs for sequences and parses, structured distributions, and finite-temperature bounds.

import omnibias.struct.torch

Module contract & mathematical conventions

Torch backend for omnibias-struct: soft-DP layers + closed-form lse/softmax jets.

Shared formulas do not imply universal floating-point identity. Verify dtype, shape, derivative-order support, and the stated numerical assumptions for the path you use.

Module source ↗Source revision 9a2d73df0573Running reference examples →

alignModule ↗attentionModule ↗curvatureModule ↗distributionsModule ↗dtwModule ↗eisnerModule ↗jetsModule ↗monotonicModule ↗mttModule ↗parseModule ↗planModule ↗selectModule ↗semiringModule ↗soft_dpModule ↗tropicalModule ↗

Imports & aliases

These names share a definition with another public module. Follow a name for its full contract, parameters, and examples.

Import from this module	Canonical reference
certified_argmax	omnibias.struct.torch.certified_argmax
chain_directional_curvature	omnibias.struct.torch.chain_directional_curvature
chain_hessian	omnibias.struct.torch.chain_hessian
chain_lse_jet	omnibias.struct.torch.chain_lse_jet
chain_sharpness	omnibias.struct.torch.chain_sharpness
chain_value_and_hvp	omnibias.struct.torch.chain_value_and_hvp
cky_lse_jet	omnibias.struct.torch.cky_lse_jet
dag_lse_jet	omnibias.struct.torch.dag_lse_jet
dp_value_jet	omnibias.struct.torch.dp_value_jet
eisner_lse_jet	omnibias.struct.torch.eisner_lse_jet
eisner_marginals	omnibias.struct.torch.eisner_marginals
gibbs_covariance	omnibias.struct.torch.gibbs_covariance
gibbs_cumulants_directional	omnibias.struct.torch.gibbs_cumulants_directional
gibbs_mean	omnibias.struct.torch.gibbs_mean
gumbel_relaxed_sample	omnibias.struct.torch.gumbel_relaxed_sample
hypergraph_lse_jet	omnibias.struct.torch.hypergraph_lse_jet
inside_outside	omnibias.struct.torch.inside_outside
lse2_jet	omnibias.struct.torch.lse2_jet
matrix_tree_marginals	omnibias.struct.torch.matrix_tree_marginals
path_entropy	omnibias.struct.torch.path_entropy
sample_paths	omnibias.struct.torch.sample_paths
semiring_marginals	omnibias.struct.torch.semiring_marginals
semiring_value	omnibias.struct.torch.semiring_value
soft_align	omnibias.struct.torch.soft_align
soft_align_batched	omnibias.struct.torch.soft_align_batched
soft_align_marginals	omnibias.struct.torch.soft_align_marginals
soft_argmax	omnibias.struct.torch.soft_argmax
soft_ctc	omnibias.struct.torch.soft_ctc
soft_ctc_batched	omnibias.struct.torch.soft_ctc_batched
soft_ctc_marginals	omnibias.struct.torch.soft_ctc_marginals
soft_dtw	omnibias.struct.torch.soft_dtw
soft_dtw_batched	omnibias.struct.torch.soft_dtw_batched
soft_dtw_marginals	omnibias.struct.torch.soft_dtw_marginals
soft_eisner	omnibias.struct.torch.soft_eisner
soft_eisner_batched	omnibias.struct.torch.soft_eisner_batched
soft_gotoh	omnibias.struct.torch.soft_gotoh
soft_gotoh_marginals	omnibias.struct.torch.soft_gotoh_marginals
soft_inside	omnibias.struct.torch.soft_inside
soft_inside_batched	omnibias.struct.torch.soft_inside_batched
soft_local_align	omnibias.struct.torch.soft_local_align
soft_local_align_marginals	omnibias.struct.torch.soft_local_align_marginals
soft_mas	omnibias.struct.torch.soft_mas
soft_mas_batched	omnibias.struct.torch.soft_mas_batched
soft_mas_marginals	omnibias.struct.torch.soft_mas_marginals
soft_matrix_tree	omnibias.struct.torch.soft_matrix_tree
soft_max_value	omnibias.struct.torch.soft_max_value
soft_shortest_path	omnibias.struct.torch.soft_shortest_path
soft_shortest_path_batched	omnibias.struct.torch.soft_shortest_path_batched
soft_shortest_path_marginals	omnibias.struct.torch.soft_shortest_path_marginals
soft_value_iteration	omnibias.struct.torch.soft_value_iteration
soft_value_iteration_batched	omnibias.struct.torch.soft_value_iteration_batched
soft_viterbi	omnibias.struct.torch.soft_viterbi
soft_viterbi_batched	omnibias.struct.torch.soft_viterbi_batched
soft_viterbi_marginals	omnibias.struct.torch.soft_viterbi_marginals
soft_viterbi_transition_marginals	omnibias.struct.torch.soft_viterbi_transition_marginals
span_marginals	omnibias.struct.torch.span_marginals
structured_attention	omnibias.struct.torch.structured_attention
structured_attention_batched	omnibias.struct.torch.structured_attention_batched
topk_free_energy	omnibias.struct.torch.topk_free_energy
topk_paths	omnibias.struct.torch.topk_paths

FUNCTION

logsumexp_beta

omnibias.struct.torch.logsumexp_betaSource ↗

logsumexp_beta(a: Tensor, beta: float=1.0, *, axis: int=-1) -> Tensor

Stable lse_beta(a) = beta^-1 log sum_i exp(beta a_i) along axis.

Parameters

Name	Type & default	Contract
apositional-or-keyword	TensorRequired	See the operation contract and implementation below.
betapositional-or-keyword	floatDefault: 1.0	See the operation contract and implementation below.
axiskeyword-only	intDefault: -1	See the operation contract and implementation below.

ReturnsTensor

Executed regression · alignment soft ge hard

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_alignment_soft_ge_hard(kind: str) -> None:
    pytest.importorskip("torch")
    assert probe_align_gap(kind, 0, BETA, "torch").ok

python -m pytest "packages/omnibias-struct/tests/test_refinement_families.py::test_alignment_soft_ge_hard[affine]" -q

Regression example · primitives and jets torch jax parity

This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_primitives_and_jets_torch_jax_parity() -> None:
    pytest.importorskip("torch")
    jax = pytest.importorskip("jax")
    jax.config.update("jax_enable_x64", True)
    import jax.numpy as jnp
    import torch
    from omnibias.struct.jax import logsumexp_beta as lj
    from omnibias.struct.jax import pairwise_lse_jet as jj
    from omnibias.struct.torch import logsumexp_beta as lt
    from omnibias.struct.torch import pairwise_lse_jet as jt

    vt = lt(torch.tensor(A_1D), BETA).item()
    vj = float(lj(jnp.asarray(A_1D), BETA))
    assert abs(vt - vj) < ATOL
    f64 = torch.float64
    tj = np.asarray(
        jt(torch.tensor(0.3, dtype=f64), torch.tensor(-0.4, dtype=f64), torch.tensor(1.0, dtype=f64), BETA, order=3).detach()
    )
    jjj = np.asarray(jj(jnp.asarray(0.3), jnp.asarray(-0.4), jnp.asarray(1.0), BETA, order=3))
    assert np.allclose(tj, jjj, rtol=RTOL, atol=ATOL)

python -m pytest packages/omnibias-struct/tests/test_logsumexp.py -k test_primitives_and_jets_torch_jax_parity -q

Read the implementation

def logsumexp_beta(a: Tensor, beta: float = 1.0, *, axis: int = -1) -> Tensor:
    r"""Stable ``lse_beta(a) = beta^-1 log sum_i exp(beta a_i)`` along ``axis``."""
    scaled = beta * a
    max_a = scaled.amax(dim=axis, keepdim=True)
    log_sum = max_a.squeeze(axis) + torch.log(torch.exp(scaled - max_a).sum(dim=axis))
    return log_sum / beta

Other public import paths

• omnibias.struct.torch.logsumexp_beta
• omnibias.struct.torch.select.logsumexp_beta

FUNCTION

logsumexp_beta_hessian

omnibias.struct.torch.logsumexp_beta_hessianSource ↗

logsumexp_beta_hessian(a: Tensor, beta: float=1.0, *, axis: int=-1) -> Tensor

Closed-form Hessian of lse_beta: beta (diag(p) - p p^T) ((..., n, n)).

Parameters

Name	Type & default	Contract
apositional-or-keyword	TensorRequired	See the operation contract and implementation below.
betapositional-or-keyword	floatDefault: 1.0	See the operation contract and implementation below.
axiskeyword-only	intDefault: -1	See the operation contract and implementation below.

ReturnsTensor

Executed regression · torch primitives vs reference and autodiff

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_torch_primitives_vs_reference_and_autodiff() -> None:
    torch = pytest.importorskip("torch")
    from omnibias.struct.torch import (
        logsumexp_beta,
        logsumexp_beta_hessian,
        logsumexp_beta_jacobian,
        softmax_beta,
    )

    a = torch.tensor(A_1D, requires_grad=True)
    val = logsumexp_beta(a, BETA)
    assert np.isclose(val.item(), _ref_lse(A_1D, BETA), rtol=RTOL, atol=ATOL)
    assert np.allclose(softmax_beta(a.detach(), BETA).numpy(), _ref_softmax(A_1D, BETA), rtol=RTOL, atol=ATOL)
    val.backward()
    jac = logsumexp_beta_jacobian(a.detach(), BETA).numpy()
    assert np.allclose(a.grad.numpy(), jac, rtol=RTOL, atol=ATOL)  # grad lse == softmax
    hess = logsumexp_beta_hessian(a.detach(), BETA).numpy()
    p = _ref_softmax(A_1D, BETA)
    assert np.allclose(hess, BETA * (np.diag(p) - np.outer(p, p)), rtol=RTOL, atol=ATOL)

python -m pytest "packages/omnibias-struct/tests/test_logsumexp.py::test_torch_primitives_vs_reference_and_autodiff" -q

Read the implementation

def logsumexp_beta_hessian(a: Tensor, beta: float = 1.0, *, axis: int = -1) -> Tensor:
    r"""Closed-form Hessian of ``lse_beta``: ``beta (diag(p) - p p^T)`` (``(..., n, n)``)."""
    p = softmax_beta(a, beta, axis=axis)
    outer = p.unsqueeze(-1) * p.unsqueeze(-2)
    diag = torch.diag_embed(p)
    return beta * (diag - outer)

Related APIsomnibias.struct.torch.softmax_beta↗

FUNCTION

logsumexp_beta_jacobian

omnibias.struct.torch.logsumexp_beta_jacobianSource ↗

logsumexp_beta_jacobian(a: Tensor, beta: float=1.0, *, axis: int=-1) -> Tensor

Closed-form gradient of lse_beta w.r.t. a: softmax(beta a).

Parameters

Name	Type & default	Contract
apositional-or-keyword	TensorRequired	See the operation contract and implementation below.
betapositional-or-keyword	floatDefault: 1.0	See the operation contract and implementation below.
axiskeyword-only	intDefault: -1	See the operation contract and implementation below.

ReturnsTensor

Executed regression · torch primitives vs reference and autodiff

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_torch_primitives_vs_reference_and_autodiff() -> None:
    torch = pytest.importorskip("torch")
    from omnibias.struct.torch import (
        logsumexp_beta,
        logsumexp_beta_hessian,
        logsumexp_beta_jacobian,
        softmax_beta,
    )

    a = torch.tensor(A_1D, requires_grad=True)
    val = logsumexp_beta(a, BETA)
    assert np.isclose(val.item(), _ref_lse(A_1D, BETA), rtol=RTOL, atol=ATOL)
    assert np.allclose(softmax_beta(a.detach(), BETA).numpy(), _ref_softmax(A_1D, BETA), rtol=RTOL, atol=ATOL)
    val.backward()
    jac = logsumexp_beta_jacobian(a.detach(), BETA).numpy()
    assert np.allclose(a.grad.numpy(), jac, rtol=RTOL, atol=ATOL)  # grad lse == softmax
    hess = logsumexp_beta_hessian(a.detach(), BETA).numpy()
    p = _ref_softmax(A_1D, BETA)
    assert np.allclose(hess, BETA * (np.diag(p) - np.outer(p, p)), rtol=RTOL, atol=ATOL)

python -m pytest "packages/omnibias-struct/tests/test_logsumexp.py::test_torch_primitives_vs_reference_and_autodiff" -q

Read the implementation

def logsumexp_beta_jacobian(a: Tensor, beta: float = 1.0, *, axis: int = -1) -> Tensor:
    r"""Closed-form gradient of ``lse_beta`` w.r.t. ``a``: ``softmax(beta a)``."""
    return softmax_beta(a, beta, axis=axis)

Related APIsomnibias.struct.torch.softmax_beta↗

FUNCTION

pairwise_lse

omnibias.struct.torch.pairwise_lseSource ↗

pairwise_lse(a: Tensor, b: Tensor, beta: float=1.0) -> Tensor

Elementwise soft-max combine lse_beta(a, b) = beta^-1 log(e^{beta a} + e^{beta b}).

Equal to a + beta^-1 softplus(beta (b - a)); computed by the stable symmetric log-sum-exp of the stacked pair.

Parameters

Name	Type & default	Contract
apositional-or-keyword	TensorRequired	See the operation contract and implementation below.
bpositional-or-keyword	TensorRequired	See the operation contract and implementation below.
betapositional-or-keyword	floatDefault: 1.0	See the operation contract and implementation below.

ReturnsTensor

Read the implementation

def pairwise_lse(a: Tensor, b: Tensor, beta: float = 1.0) -> Tensor:
    r"""Elementwise soft-max combine ``lse_beta(a, b) = beta^-1 log(e^{beta a} + e^{beta b})``.

    Equal to ``a + beta^-1 softplus(beta (b - a))``; computed by the stable symmetric
    log-sum-exp of the stacked pair.
    """
    stacked = torch.stack([a, b], dim=-1)
    return logsumexp_beta(stacked, beta, axis=-1)

Related APIsomnibias.struct.torch.logsumexp_beta↗

FUNCTION

pairwise_lse_jet

omnibias.struct.torch.pairwise_lse_jetSource ↗

pairwise_lse_jet(a0: Tensor, b0: Tensor, db: Tensor, beta: float=1.0, order: int=1) -> Tensor

Taylor jet (order order) of t -> lse_beta(a0, b0 + t db) from the tower.

Writing lse_beta(a0, b0 + t db) = a0 + beta^-1 softplus(u(t)) with u(t) = beta (b0 - a0) + (beta db) t, the jet is omnibias.torch.jet.compose_jet of the linear u-jet with the closed-form softplus derivative tower softplus^(k)(u_0) (softplus^(k) = sigma^(k-1) from omnibias.core) -- exact, no autodiff or finite differences. Returns coefficients c_k = f^(k)(0) / k! along a new leading axis of length order + 1.

Parameters

Name	Type & default	Contract
a0positional-or-keyword	TensorRequired	See the operation contract and implementation below.
b0positional-or-keyword	TensorRequired	See the operation contract and implementation below.
dbpositional-or-keyword	TensorRequired	See the operation contract and implementation below.
betapositional-or-keyword	floatDefault: 1.0	See the operation contract and implementation below.
orderpositional-or-keyword	intDefault: 1	See the operation contract and implementation below.

ReturnsTensor

Explicit validation & failure cases

• When order < 0, raises ValueError(f'order must be >= 0, got {order}').
• When fastpath is None, raises NotImplementedError('softplus is missing its closed-form fastpath tower').

These are explicit source guards. Backend operations may also raise for invalid shapes, dtypes, or unsupported devices.

Executed regression · lse2 jet generalizes pairwise lse jet torch

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_lse2_jet_generalizes_pairwise_lse_jet_torch() -> None:
    torch = pytest.importorskip("torch")
    from omnibias.struct.torch import pairwise_lse_jet
    from omnibias.struct.torch.jets import lse2_jet

    a0 = torch.tensor(0.3)
    b0 = torch.tensor(-0.4)
    db = torch.tensor(1.1)
    beta = 2.0
    order = 3
    # pairwise_lse_jet fixes a constant and moves b linearly: build the same two jets.
    a_jet = torch.stack([a0] + [torch.zeros(()) for _ in range(order)])
    b_jet = torch.stack([b0, db] + [torch.zeros(()) for _ in range(order - 1)])
    got = lse2_jet(a_jet, b_jet, beta, order)
    ref = pairwise_lse_jet(a0, b0, db, beta, order)
    assert np.max(np.abs(got.numpy() - ref.numpy())) < 1e-12

python -m pytest "packages/omnibias-struct/tests/test_jets.py::test_lse2_jet_generalizes_pairwise_lse_jet_torch" -q

Regression example · primitives and jets torch jax parity

This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_primitives_and_jets_torch_jax_parity() -> None:
    pytest.importorskip("torch")
    jax = pytest.importorskip("jax")
    jax.config.update("jax_enable_x64", True)
    import jax.numpy as jnp
    import torch
    from omnibias.struct.jax import logsumexp_beta as lj
    from omnibias.struct.jax import pairwise_lse_jet as jj
    from omnibias.struct.torch import logsumexp_beta as lt
    from omnibias.struct.torch import pairwise_lse_jet as jt

    vt = lt(torch.tensor(A_1D), BETA).item()
    vj = float(lj(jnp.asarray(A_1D), BETA))
    assert abs(vt - vj) < ATOL
    f64 = torch.float64
    tj = np.asarray(
        jt(torch.tensor(0.3, dtype=f64), torch.tensor(-0.4, dtype=f64), torch.tensor(1.0, dtype=f64), BETA, order=3).detach()
    )
    jjj = np.asarray(jj(jnp.asarray(0.3), jnp.asarray(-0.4), jnp.asarray(1.0), BETA, order=3))
    assert np.allclose(tj, jjj, rtol=RTOL, atol=ATOL)

python -m pytest packages/omnibias-struct/tests/test_logsumexp.py -k test_primitives_and_jets_torch_jax_parity -q

Read the implementation

def pairwise_lse_jet(
    a0: Tensor,
    b0: Tensor,
    db: Tensor,
    beta: float = 1.0,
    order: int = 1,
) -> Tensor:
    r"""Taylor jet (order ``order``) of ``t -> lse_beta(a0, b0 + t db)`` from the tower.

    Writing ``lse_beta(a0, b0 + t db) = a0 + beta^-1 softplus(u(t))`` with
    ``u(t) = beta (b0 - a0) + (beta db) t``, the jet is
    :func:`omnibias.torch.jet.compose_jet` of the linear ``u``-jet with the closed-form
    ``softplus`` derivative tower ``softplus^(k)(u_0)`` (``softplus^(k) = sigma^(k-1)``
    from :mod:`omnibias.core`) -- exact, no autodiff or finite differences. Returns
    coefficients ``c_k = f^(k)(0) / k!`` along a new leading axis of length ``order + 1``.
    """
    if order < 0:
        raise ValueError(f"order must be >= 0, got {order}")
    spec = get_activation("softplus")
    fastpath = spec.fastpath
    if fastpath is None:  # defensive: softplus always ships its closed-form tower
        raise NotImplementedError("softplus is missing its closed-form fastpath tower")
    a0 = torch.as_tensor(a0)
    b0 = torch.as_tensor(b0)
    db = torch.as_tensor(db)
    u0 = beta * (b0 - a0)
    zero = torch.zeros_like(u0)
    u_rows = [u0]
    if order >= 1:
        u_rows.append((beta * db) + zero)
    u_rows.extend(zero for _ in range(order - 1))
    u_jet = torch.stack(u_rows[: order + 1], dim=0)
    tower_rows = [spec.forward(u0)]
    for k in range(1, order + 1):
        tower_rows.append(fastpath(u0, k))
    sigma_tower = torch.stack(tower_rows, dim=0)
    sp_jet = compose_jet(u_jet, sigma_tower) / beta
    out0 = (sp_jet[0] + a0).unsqueeze(0)
    return torch.cat([out0, sp_jet[1:]], dim=0)

Related APIsomnibias.torch.get_activation↗omnibias.torch.compose_jet↗

FUNCTION

softmax_beta

omnibias.struct.torch.softmax_betaSource ↗

softmax_beta(a: Tensor, beta: float=1.0, *, axis: int=-1) -> Tensor

Stable softmax(beta a) along axis -- the gradient of logsumexp_beta.

Parameters

Name	Type & default	Contract
apositional-or-keyword	TensorRequired	See the operation contract and implementation below.
betapositional-or-keyword	floatDefault: 1.0	See the operation contract and implementation below.
axiskeyword-only	intDefault: -1	See the operation contract and implementation below.

ReturnsTensor

Executed regression · structured attention collapses to softmax without transitions

Recorded coverage confirms this test executed the API body. This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_structured_attention_collapses_to_softmax_without_transitions() -> None:
    rng = np.random.default_rng(4)
    scores = rng.standard_normal((5, 4))
    zero = np.zeros((4, 4))
    beta = 2.0
    attn = tattn(torch.tensor(scores), torch.tensor(zero), beta)
    plain = tsoftmax(torch.tensor(scores), beta, axis=1)
    assert torch.max(torch.abs(attn - plain)).item() < 1e-9  # zero transitions -> plain softmax
    assert torch.allclose(attn.sum(dim=1), torch.ones(5), atol=1e-9)

python -m pytest "packages/omnibias-struct/tests/test_plan_attn.py::test_structured_attention_collapses_to_softmax_without_transitions" -q

Regression example · torch primitives vs reference and autodiff

This example belongs to the pinned upstream test suite. It may use fixtures and imports from its full file. Run it from that checkout with its test dependencies installed.

Full file, imports & fixtures ↗

def test_torch_primitives_vs_reference_and_autodiff() -> None:
    torch = pytest.importorskip("torch")
    from omnibias.struct.torch import (
        logsumexp_beta,
        logsumexp_beta_hessian,
        logsumexp_beta_jacobian,
        softmax_beta,
    )

    a = torch.tensor(A_1D, requires_grad=True)
    val = logsumexp_beta(a, BETA)
    assert np.isclose(val.item(), _ref_lse(A_1D, BETA), rtol=RTOL, atol=ATOL)
    assert np.allclose(softmax_beta(a.detach(), BETA).numpy(), _ref_softmax(A_1D, BETA), rtol=RTOL, atol=ATOL)
    val.backward()
    jac = logsumexp_beta_jacobian(a.detach(), BETA).numpy()
    assert np.allclose(a.grad.numpy(), jac, rtol=RTOL, atol=ATOL)  # grad lse == softmax
    hess = logsumexp_beta_hessian(a.detach(), BETA).numpy()
    p = _ref_softmax(A_1D, BETA)
    assert np.allclose(hess, BETA * (np.diag(p) - np.outer(p, p)), rtol=RTOL, atol=ATOL)

python -m pytest packages/omnibias-struct/tests/test_logsumexp.py -k test_torch_primitives_vs_reference_and_autodiff -q

Read the implementation

def softmax_beta(a: Tensor, beta: float = 1.0, *, axis: int = -1) -> Tensor:
    r"""Stable ``softmax(beta a)`` along ``axis`` -- the gradient of :func:`logsumexp_beta`."""
    scaled = beta * a
    max_a = scaled.amax(dim=axis, keepdim=True)
    exp_a = torch.exp(scaled - max_a)
    return exp_a / exp_a.sum(dim=axis, keepdim=True)

Other public import paths

• omnibias.struct.torch.softmax_beta
• omnibias.struct.torch.select.softmax_beta

← Package overviewRunnable learning examples →