laguerre_expansion

Bases: transformation

The laguerre expansion function.

Applies Laguerre polynomial expansion to input data.

Notes

In mathematics, the Laguerre polynomials, named after Edmond Laguerre, are the nontrivial solutions of Laguerre's differential equation:

\[
    \begin{equation}
    x y'' + (\alpha + 1 -x) y' + d y = 0,
    \end{equation}
\]

where \(y = y(x)\) is a function of variable \(x\). Notations \(y'\) and \(y''\) denote first- and second-order derivatives of function \(y\) with respect to variable \(x\). Term \(d \in N\) is a non-negative integer and \(\alpha \in R\) is a hyper-parameter.

The closed-form of the Laguerre polynomials can be represented as follows:

\[
    \begin{equation}
    P^{(\alpha)}_n(x) = \frac{e^x}{n!} \frac{\mathrm{d}^n}{\mathrm{d} x^n} (e^{-x} x^n) = \frac{x^{-\alpha}}{n!} \left( \frac{\mathrm{d}}{\mathrm{d} x} - 1 \right)^n x^{n + \alpha},
    \end{equation}
\]

where \(\frac{\mathrm{d}}{\mathrm{d} x}\) denotes the derivative operator.

In practice, the Laguerre polynomials can be recursively defined as follows, which will be used for defining the data expansion function below. Specifically, when \(\alpha = 0\), the above Laguerre polynomials are also known as simple Laguerre polynomials.

Base cases \(n=0\) and \(n=1\):

\[
    \begin{equation}
    P^{(\alpha)}_0(x) = 1 \text{, and } P^{(\alpha)}_1(x) = 1 + \alpha -  x.
    \end{equation}
\]

High-order cases with degree \(n \ge 2\):

\[
    \begin{equation}
    P^{(\alpha)}_n(x) = \frac{(2n-1+\alpha-x) P^{(\alpha)}_{n-1}(x) - (n-1+\alpha) P^{(\alpha)}_{n-2}(x) }{n}
    \end{equation}
\]

The recursive-form representations of the Laguerre polynomials can be used to define the data expansion function as follows:

\[
    \begin{equation}
    \kappa(\mathbf{x} | d, \alpha) = \left[ P^{(\alpha)}_1(\mathbf{x}), P^{(\alpha)}_2(\mathbf{x}), \cdots, P^{(\alpha)}_d(\mathbf{x}) \right] \in R^D,
    \end{equation}
\]

where \(d\) and \(\alpha\) are the function hyper-parameters and the output dimension \(D = md\).

Attributes:

Name	Type	Description
d	int	The degree of Laguerre polynomial expansion.
alpha	float	Parameter controlling the Laguerre polynomial.

Methods:

Name	Description
calculate_D	Calculates the output dimension after expansion.
forward	Performs Laguerre polynomial expansion on the input tensor.

Source code in tinybig/expansion/orthogonal_polynomial_expansion.py

class laguerre_expansion(transformation):
    r"""
        The laguerre expansion function.

        Applies Laguerre polynomial expansion to input data.

        Notes
        ----------

        In mathematics, the Laguerre polynomials, named after Edmond Laguerre, are the nontrivial solutions of Laguerre's differential equation:

        $$
            \begin{equation}
            x y'' + (\alpha + 1 -x) y' + d y = 0,
            \end{equation}
        $$

        where $y = y(x)$ is a function of variable $x$. Notations $y'$ and $y''$ denote first- and second-order derivatives of function $y$ with respect to variable $x$. Term $d \in N$ is a non-negative integer and $\alpha \in R$ is a hyper-parameter.

        The closed-form of the Laguerre polynomials can be represented as follows:

        $$
            \begin{equation}
            P^{(\alpha)}_n(x) = \frac{e^x}{n!} \frac{\mathrm{d}^n}{\mathrm{d} x^n} (e^{-x} x^n) = \frac{x^{-\alpha}}{n!} \left( \frac{\mathrm{d}}{\mathrm{d} x} - 1 \right)^n x^{n + \alpha},
            \end{equation}
        $$

        where $\frac{\mathrm{d}}{\mathrm{d} x}$ denotes the derivative operator.

        In practice, the Laguerre polynomials can be recursively defined as follows, which will be used for defining the data expansion function below. Specifically, when $\alpha = 0$, the above Laguerre polynomials are also known as simple Laguerre polynomials.

        __Base cases $n=0$ and $n=1$:__

        $$
            \begin{equation}
            P^{(\alpha)}_0(x) = 1 \text{, and } P^{(\alpha)}_1(x) = 1 + \alpha -  x.
            \end{equation}
        $$

        __High-order cases with degree $n \ge 2$:__

        $$
            \begin{equation}
            P^{(\alpha)}_n(x) = \frac{(2n-1+\alpha-x) P^{(\alpha)}_{n-1}(x) - (n-1+\alpha) P^{(\alpha)}_{n-2}(x) }{n}
            \end{equation}
        $$

        The recursive-form representations of the Laguerre polynomials can be used to define the data expansion function as follows:

        $$
            \begin{equation}
            \kappa(\mathbf{x} | d, \alpha) = \left[ P^{(\alpha)}_1(\mathbf{x}), P^{(\alpha)}_2(\mathbf{x}), \cdots, P^{(\alpha)}_d(\mathbf{x}) \right] \in R^D,
            \end{equation}
        $$

        where $d$ and $\alpha$ are the function hyper-parameters and the output dimension $D = md$.

        Attributes
        ----------
        d : int
            The degree of Laguerre polynomial expansion.
        alpha : float
            Parameter controlling the Laguerre polynomial.

        Methods
        -------
        calculate_D(m: int)
            Calculates the output dimension after expansion.
        forward(x: torch.Tensor, device='cpu', *args, **kwargs)
            Performs Laguerre polynomial expansion on the input tensor.
    """
    def __init__(self, name='laguerre_polynomial_expansion', d: int = 2, alpha: float = 1.0, *args, **kwargs):
        """
            Initializes the Laguerre polynomial expansion transformation.

            Parameters
            ----------
            name : str, optional
                Name of the transformation. Defaults to 'laguerre_polynomial_expansion'.
            d : int, optional
                The maximum order of Laguerre polynomials for expansion. Defaults to 2.
            alpha : float, optional
                The alpha parameter for generalized Laguerre polynomials. Defaults to 1.0.
            *args : tuple
                Additional positional arguments.
            **kwargs : dict
                Additional keyword arguments.
        """
        super().__init__(name=name, *args, **kwargs)
        self.d = d
        self.alpha = alpha

    def calculate_D(self, m: int):
        """
            Calculates the output dimension after Laguerre polynomial expansion.

            Parameters
            ----------
            m : int
                Input dimension.

            Returns
            -------
            int
                Output dimension after expansion.
        """
        return m * self.d

    def forward(self, x: torch.Tensor, device='cpu', *args, **kwargs):
        """
            Performs Laguerre polynomial expansion on the input tensor.

            Parameters
            ----------
            x : torch.Tensor
                Input tensor of shape `(batch_size, input_dim)`.
            device : str, optional
                Device for computation ('cpu', 'cuda'). Defaults to 'cpu'.
            *args : tuple
                Additional positional arguments.
            **kwargs : dict
                Additional keyword arguments.

            Returns
            -------
            torch.Tensor
                Expanded tensor of shape `(batch_size, expanded_dim)`.

            Raises
            ------
            AssertionError
                If the output tensor shape does not match the expected dimensions.
        """
        b, m = x.shape
        x = self.pre_process(x=x, device=device)

        # base case: order 0
        expansion = torch.ones(size=[x.size(0), x.size(1), self.d + 1]).to(device)
        # base case: order 1
        if self.d > 0:
            expansion[:, :, 1] = x + 1.0 + self.alpha
        # high-order cases
        for n in range(2, self.d + 1):
            expansion[:, :, n] = (2*n-1+self.alpha-x)/n * expansion[:, :, n-1].clone() - (n-1+self.alpha)/n * expansion[:, :, n-2].clone()

        expansion = expansion[:, :, 1:].contiguous().view(x.size(0), -1)

        assert expansion.shape == (b, self.calculate_D(m=m))
        return self.post_process(x=expansion, device=device)

__init__(name='laguerre_polynomial_expansion', d=2, alpha=1.0, *args, **kwargs)

Initializes the Laguerre polynomial expansion transformation.

Parameters:

Name	Type	Description	Default
name	str	Name of the transformation. Defaults to 'laguerre_polynomial_expansion'.	'laguerre_polynomial_expansion'
d	int	The maximum order of Laguerre polynomials for expansion. Defaults to 2.	2
alpha	float	The alpha parameter for generalized Laguerre polynomials. Defaults to 1.0.	1.0
*args	tuple	Additional positional arguments.	()
**kwargs	dict	Additional keyword arguments.	{}

Source code in tinybig/expansion/orthogonal_polynomial_expansion.py

def __init__(self, name='laguerre_polynomial_expansion', d: int = 2, alpha: float = 1.0, *args, **kwargs):
    """
        Initializes the Laguerre polynomial expansion transformation.

        Parameters
        ----------
        name : str, optional
            Name of the transformation. Defaults to 'laguerre_polynomial_expansion'.
        d : int, optional
            The maximum order of Laguerre polynomials for expansion. Defaults to 2.
        alpha : float, optional
            The alpha parameter for generalized Laguerre polynomials. Defaults to 1.0.
        *args : tuple
            Additional positional arguments.
        **kwargs : dict
            Additional keyword arguments.
    """
    super().__init__(name=name, *args, **kwargs)
    self.d = d
    self.alpha = alpha

calculate_D(m)

Calculates the output dimension after Laguerre polynomial expansion.

Parameters:

Name	Type	Description	Default
m	int	Input dimension.	required

Returns:

Type	Description
int	Output dimension after expansion.

Source code in tinybig/expansion/orthogonal_polynomial_expansion.py

def calculate_D(self, m: int):
    """
        Calculates the output dimension after Laguerre polynomial expansion.

        Parameters
        ----------
        m : int
            Input dimension.

        Returns
        -------
        int
            Output dimension after expansion.
    """
    return m * self.d

forward(x, device='cpu', *args, **kwargs)

Performs Laguerre polynomial expansion on the input tensor.

Parameters:

Name	Type	Description	Default
x	Tensor	Input tensor of shape (batch_size, input_dim).	required
device	str	Device for computation ('cpu', 'cuda'). Defaults to 'cpu'.	'cpu'
*args	tuple	Additional positional arguments.	()
**kwargs	dict	Additional keyword arguments.	{}

Returns:

Type	Description
Tensor	Expanded tensor of shape (batch_size, expanded_dim).

Raises:

Type	Description
AssertionError	If the output tensor shape does not match the expected dimensions.

Source code in tinybig/expansion/orthogonal_polynomial_expansion.py

def forward(self, x: torch.Tensor, device='cpu', *args, **kwargs):
    """
        Performs Laguerre polynomial expansion on the input tensor.

        Parameters
        ----------
        x : torch.Tensor
            Input tensor of shape `(batch_size, input_dim)`.
        device : str, optional
            Device for computation ('cpu', 'cuda'). Defaults to 'cpu'.
        *args : tuple
            Additional positional arguments.
        **kwargs : dict
            Additional keyword arguments.

        Returns
        -------
        torch.Tensor
            Expanded tensor of shape `(batch_size, expanded_dim)`.

        Raises
        ------
        AssertionError
            If the output tensor shape does not match the expected dimensions.
    """
    b, m = x.shape
    x = self.pre_process(x=x, device=device)

    # base case: order 0
    expansion = torch.ones(size=[x.size(0), x.size(1), self.d + 1]).to(device)
    # base case: order 1
    if self.d > 0:
        expansion[:, :, 1] = x + 1.0 + self.alpha
    # high-order cases
    for n in range(2, self.d + 1):
        expansion[:, :, n] = (2*n-1+self.alpha-x)/n * expansion[:, :, n-1].clone() - (n-1+self.alpha)/n * expansion[:, :, n-2].clone()

    expansion = expansion[:, :, 1:].contiguous().view(x.size(0), -1)

    assert expansion.shape == (b, self.calculate_D(m=m))
    return self.post_process(x=expansion, device=device)