constant_remainder
Bases: remainder
The constant remainder function.
It calculates the constant remainder, and returns remainder term of certain length. This class inherits from the base remainder class (i.e., the remainder class in the module directory).
...
Notes
The constant remainder function \(\pi: {R}^m \to {R}^n\) just projects all inputs to a constant vector, i.e., $$ \begin{equation} \pi(\mathbf{x}) = \mathbf{c} = c \times \mathbf{1}^n \in {R}^n, \end{equation} $$ where \(\mathbf{c} = c \times \mathbf{1}^n\) is a constant vector of length \(n\).
By default, the remainder term will also be processed with the optional activation functions.
Attributes:
Name | Type | Description |
---|---|---|
name |
str, default = 'constant_remainder'
|
Name of the remainder function. |
c |
float, default = 1.0
|
The constant value of the remainder function |
Methods:
Name | Description |
---|---|
__init__ |
It initializes the constant remainder function. |
forward |
It implements the abstract forward method declared in the base remainder class. |
Source code in tinybig/remainder/basic_remainder.py
__init__(name='constant_remainder', c=1.0, *args, **kwargs)
The initialization method of the constant remainder function.
It initializes a constant remainder function object. This method will also call the initialization method of the base class as well.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name |
Name of the constant remainder function. |
'constant_remainder'
|
|
c |
The constant value of the remainder function |
1.0
|
Returns:
Type | Description |
---|---|
object
|
The constant remainder function object. |
Source code in tinybig/remainder/basic_remainder.py
forward(n, device='cpu', *args, **kwargs)
The forward method of the constant remainder function.
It calculates the constant remainder term of length \(n\), which can be represented as follows $$ \begin{equation} \pi(\mathbf{x}) = \mathbf{c} \in {R}^n, \end{equation} $$
By default, the remainder term will also be processed with the optional activation functions.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
n |
int
|
The dimension of the output space. |
required |
device |
Device to calculate the remainder function. |
'cpu'
|
Returns:
Type | Description |
---|---|
Tensor
|
The remainder term of length \(n\). |