Skip to content

constant_remainder

Bases: remainder

The constant remainder function.

It calculates the constant remainder, and returns remainder term of certain length. This class inherits from the base remainder class (i.e., the remainder class in the module directory).

...

Notes

The constant remainder function \(\pi: {R}^m \to {R}^n\) just projects all inputs to a constant vector, i.e., $$ \begin{equation} \pi(\mathbf{x}) = \mathbf{c} = c \times \mathbf{1}^n \in {R}^n, \end{equation} $$ where \(\mathbf{c} = c \times \mathbf{1}^n\) is a constant vector of length \(n\).

By default, the remainder term will also be processed with the optional activation functions.

Attributes:

Name Type Description
name str, default = 'constant_remainder'

Name of the remainder function.
c float, default = 1.0

The constant value of the remainder function

Methods:

Name Description
__init__

It initializes the constant remainder function.
forward

It implements the abstract forward method declared in the base remainder class.
Source code in tinybig/remainder/basic_remainder.py 
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
class constant_remainder(remainder):
    r"""
    The constant remainder function.

    It calculates the constant remainder, and returns remainder term of certain length.
    This class inherits from the base remainder class (i.e., the remainder class in the module directory).

    ...

    Notes
    ----------
    The constant remainder function $\pi: {R}^m \to {R}^n$ just projects all inputs to a constant vector, i.e.,
    $$
        \begin{equation}
            \pi(\mathbf{x}) = \mathbf{c} = c \times \mathbf{1}^n \in {R}^n,
        \end{equation}
    $$
    where $\mathbf{c} = c \times \mathbf{1}^n$ is a constant vector of length $n$.

    By default, the remainder term will also be processed with the optional activation functions.

    Attributes
    ----------
    name: str, default = 'constant_remainder'
        Name of the remainder function.
    c: float, default = 1.0
        The constant value of the remainder function

    Methods
    ----------
    __init__
        It initializes the constant remainder function.

    forward
        It implements the abstract forward method declared in the base remainder class.
    """
    def __init__(self, name='constant_remainder', c=1.0, *args, **kwargs):
        """
        The initialization method of the constant remainder function.

        It initializes a constant remainder function object.
        This method will also call the initialization method of the base class as well.

        Parameters
        ----------
        name: str, default = 'constant_remainder'
            Name of the constant remainder function.
        c: float, default = 1.0
            The constant value of the remainder function

        Returns
        ----------
        remainder
            The constant remainder function object.
        """
        super().__init__(name=name, *args, **kwargs)
        self.c = c

    def forward(self, n: int, device='cpu', *args, **kwargs):
        r"""
        The forward method of the constant remainder function.

        It calculates the constant remainder term of length $n$, which can be represented as follows
        $$
            \begin{equation}
                \pi(\mathbf{x}) = \mathbf{c} \in {R}^n,
            \end{equation}
        $$

        By default, the remainder term will also be processed with the optional activation functions.

        Parameters
        ----------
        n: int
            The dimension of the output space.
        device: str, default = 'cpu'
            Device to calculate the remainder function.

        Returns
        ----------
        torch.Tensor
            The remainder term of length $n$.
        """
        x = self.c * torch.ones(n, device=device)
        return self.activation(x=x, device=device)

__init__(name='constant_remainder', c=1.0, *args, **kwargs)

The initialization method of the constant remainder function.

It initializes a constant remainder function object. This method will also call the initialization method of the base class as well.

Parameters:

Name Type Description Default
name

Name of the constant remainder function.

 'constant_remainder'
c

The constant value of the remainder function

 1.0

Returns:

Type Description
remainder

The constant remainder function object.
Source code in tinybig/remainder/basic_remainder.py 
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
def __init__(self, name='constant_remainder', c=1.0, *args, **kwargs):
    """
    The initialization method of the constant remainder function.

    It initializes a constant remainder function object.
    This method will also call the initialization method of the base class as well.

    Parameters
    ----------
    name: str, default = 'constant_remainder'
        Name of the constant remainder function.
    c: float, default = 1.0
        The constant value of the remainder function

    Returns
    ----------
    remainder
        The constant remainder function object.
    """
    super().__init__(name=name, *args, **kwargs)
    self.c = c

forward(n, device='cpu', *args, **kwargs)

The forward method of the constant remainder function.

It calculates the constant remainder term of length \(n\), which can be represented as follows $$ \begin{equation} \pi(\mathbf{x}) = \mathbf{c} \in {R}^n, \end{equation} $$

By default, the remainder term will also be processed with the optional activation functions.

Parameters:

Name Type Description Default
n int

The dimension of the output space.

 required
device

Device to calculate the remainder function.

 'cpu'

Returns:

Type Description
Tensor

The remainder term of length \(n\).
Source code in tinybig/remainder/basic_remainder.py 
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
def forward(self, n: int, device='cpu', *args, **kwargs):
    r"""
    The forward method of the constant remainder function.

    It calculates the constant remainder term of length $n$, which can be represented as follows
    $$
        \begin{equation}
            \pi(\mathbf{x}) = \mathbf{c} \in {R}^n,
        \end{equation}
    $$

    By default, the remainder term will also be processed with the optional activation functions.

    Parameters
    ----------
    n: int
        The dimension of the output space.
    device: str, default = 'cpu'
        Device to calculate the remainder function.

    Returns
    ----------
    torch.Tensor
        The remainder term of length $n$.
    """
    x = self.c * torch.ones(n, device=device)
    return self.activation(x=x, device=device)