nested_expansion
Bases: transformation
The nested data expansion function.
It performs the data expansion of multiple expansion functions, and conctatnates their expansions to define wider expansions of the input data vector.
...
Notes
Formally, given the \(n\) different expansion functions (1) \(\kappa_1: {R}^{m} \to {R}^{d_1}\), (2) \(\kappa_2: {R}^{d_1} \to {R}^{d_2}\), \(\cdots\), (n) \(\kappa_n: {R}^{d_{n-1}} \to {R}^{D}\), we can represent their nested data expansion function \(\kappa: {R}^{m} \to {R}^D\) as follows: $$ \begin{equation} \kappa(\mathbf{x}) = \kappa_{n} \left( \kappa_{n-1} \left( \cdots \kappa_2 \left( \kappa_{1} \left( \mathbf{x} \right) \right) \right) \right) \in {R}^{D}. \end{equation} $$ where the expansion output dimension \(D\) is determined by the last expansion function \(\kappa_n\).
Attributes:
Name | Type | Description |
---|---|---|
name |
str, default = 'nested_expansion'
|
The name of the nested expansion function. |
expansion_functions |
list
|
The list of expansion functions to be nested. |
Methods:
Name | Description |
---|---|
__init__ |
It performs the initialization of the expansion function. |
calculate_D |
It calculates the expansion space dimension D based on the input dimension parameter m. |
forward |
It implements the abstract forward method declared in the base expansion class. |
Source code in tinybig/expansion/nested_expansion.py
22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 |
|
__init__(name='nested_expansion', expansion_functions=None, expansion_function_configs=None, *args, **kwargs)
The initialization method of the nested expansion function.
It initializes an nested expansion object based on the input function name. This method will also call the initialization method of the base class as well.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name |
The name of the nested expansion function. |
'nested_expansion'
|
|
expansion_functions |
list
|
The list of data expansion functions to be nested. |
None
|
expansion_function_configs |
list | dict
|
The list or dictionary of the expansion function configurations. |
None
|
Source code in tinybig/expansion/nested_expansion.py
calculate_D(m)
The expansion dimension calculation method.
It calculates the intermediate expansion space dimension based on the input dimension parameter m. For the nested expansion function defined based on \(n\) functions (1) \(\kappa_1: {R}^{m} \to {R}^{d_1}\), (2) \(\kappa_2: {R}^{d_1} \to {R}^{d_2}\), \(\cdots\), (n) \(\kappa_n: {R}^{d_{n-1}} \to {R}^{D}\), the expansion space dimension will be determined by the last expansion function \(\kappa_n\).
Parameters:
Name | Type | Description | Default |
---|---|---|---|
m |
int
|
The dimension of the input space. |
required |
Returns:
Type | Description |
---|---|
int
|
The dimension of the expansion space. |
Source code in tinybig/expansion/nested_expansion.py
forward(x, device='cpu', *args, **kwargs)
The forward method of the nested expansion function.
For the nested expansion function defined based on \(n\) functions (1) \(\kappa_1: {R}^{m} \to {R}^{d_1}\), (2) \(\kappa_2: {R}^{d_1} \to {R}^{d_2}\), \(\cdots\), (n) \(\kappa_n: {R}^{d_{n-1}} \to {R}^{D}\), it performs the nested expansion of the input data and returns the expansion result as $$ \begin{equation} \kappa(\mathbf{x}) = \kappa_{n} \left( \kappa_{n-1} \left( \cdots \kappa_2 \left( \kappa_{1} \left( \mathbf{x} \right) \right) \right) \right) \in {R}^{D}. \end{equation} $$
Parameters:
Name | Type | Description | Default |
---|---|---|---|
x |
Tensor
|
The input data vector. |
required |
device |
The device to perform the data expansion. |
'cpu'
|
Returns:
Type | Description |
---|---|
Tensor
|
The expanded data vector of the input. |