Skip to content

extended_expansion

Bases: transformation

The extended data expansion function.

It performs the data expansion of multiple expansion functions, and conctatnates their expansions to define wider expansions of the input data vector.

...

Notes

Formally, given the \(n\) different expansion functions (1) \(\kappa_1: {R}^{m} \to {R}^{d_1}\), (2) \(\kappa_2: {R}^{m} \to {R}^{d_2}\), \(\cdots\), (n) \(\kappa_n: {R}^{m} \to {R}^{d_{n}}\), we can represent their extended data expansion function \(\kappa: {R}^{m} \to {R}^D\) as follows: $$ \begin{equation} \kappa(\mathbf{x}) = \left[ \kappa_1\left( \mathbf{x} \right), \kappa_2\left( \mathbf{x} \right), \cdots, \kappa_n\left( \mathbf{x} \right) \right] \in {R}^{D}, \end{equation} $$ where the expansion output dimension \(D = \sum_{i=1}^n d_i\).

Attributes:

Name Type Description
name str, default = 'extended_expansion'

The name of the extended expansion function.
expansion_functions list

The list of expansion functions to be extended.

Methods:

Name Description
__init__

It performs the initialization of the expansion function.
calculate_D

It calculates the expansion space dimension D based on the input dimension parameter m.
forward

It implements the abstract forward method declared in the base expansion class.
Source code in tinybig/expansion/extended_expansion.py 
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
class extended_expansion(transformation):
    r"""
    The extended data expansion function.

    It performs the data expansion of multiple expansion functions, and conctatnates their expansions to define
    wider expansions of the input data vector.

    ...

    Notes
    ----------
    Formally, given the $n$ different expansion functions (1) $\kappa_1: {R}^{m} \to {R}^{d_1}$,
    (2) $\kappa_2: {R}^{m} \to {R}^{d_2}$, $\cdots$, (n) $\kappa_n: {R}^{m} \to {R}^{d_{n}}$,
    we can represent their extended data expansion function $\kappa: {R}^{m} \to {R}^D$ as follows:
    $$
        \begin{equation}
        \kappa(\mathbf{x}) = \left[ \kappa_1\left( \mathbf{x} \right), \kappa_2\left( \mathbf{x} \right), \cdots, \kappa_n\left( \mathbf{x} \right) \right] \in {R}^{D},
        \end{equation}
    $$
    where the expansion output dimension $D = \sum_{i=1}^n d_i$.

    Attributes
    ----------
    name: str, default = 'extended_expansion'
        The name of the extended expansion function.
    expansion_functions: list
        The list of expansion functions to be extended.

    Methods
    ----------
    __init__
        It performs the initialization of the expansion function.

    calculate_D
        It calculates the expansion space dimension D based on the input dimension parameter m.

    forward
        It implements the abstract forward method declared in the base expansion class.

    """
    def __init__(
            self,
            name='extended_expansion',
            composition_functions: list = None,
            composition_function_configs: list | dict = None,
            *args, **kwargs
    ):
        r"""
        The initialization method of the extended expansion function.

        It initializes an extended expansion object based on the input function name.
        This method will also call the initialization method of the base class as well.

        Parameters
        ----------
        name: str, default = 'extended_expansion'
            The name of the extended expansion function.
        composition_functions: list, default = None
            The list of data expansion functions to be extended.
        composition_function_configs: list | dict, default = None
            The list or dictionary of the expansion function configurations.

        Returns
        ----------
        transformation
            The extended data expansion function.
        """
        super().__init__(name=name, *args, **kwargs)
        self.composition_functions = []
        if composition_functions is not None and composition_functions != []:
            self.composition_functions = composition_functions
        elif composition_function_configs is not None:
            for function_config in composition_function_configs:
                function_class = function_config['function_class']
                function_parameters = function_config['function_parameters']
                self.composition_functions.append(config.get_obj_from_str(function_class)(**function_parameters))

    def calculate_D(self, m: int):
        r"""
        The expansion dimension calculation method.

        It calculates the intermediate expansion space dimension based on the input dimension parameter m.
        For the extended expansion function defined based on $n$ functions (1) $\kappa_1: {R}^{m} \to {R}^{d_1}$,
        (2) $\kappa_2: {R}^{m} \to {R}^{d_2}$, $\cdots$, (n) $\kappa_n: {R}^{m} \to {R}^{d_{n}}$,
        the expansion space dimension will be
        $$ D = \sum_{i=1}^n d_i. $$

        Parameters
        ----------
        m: int
            The dimension of the input space.

        Returns
        -------
        int
            The dimension of the expansion space.
        """
        D = 0
        for func in self.composition_functions:
            D += func.calculate_D(m=m)
        return D

    def forward(self, x: torch.Tensor, device='cpu', *args, **kwargs):
        r"""
        The forward method of the extended expansion function.

        For the extended expansion function defined based on $n$ functions (1) $\kappa_1: {R}^{m} \to {R}^{d_1}$,
        (2) $\kappa_2: {R}^{m} \to {R}^{d_2}$, $\cdots$, (n) $\kappa_n: {R}^{m} \to {R}^{d_{n}}$,
        it performs the extended expansion of the input data and returns the expansion result as
        $$
            \begin{equation}
                \kappa(\mathbf{x}) = \left[ \kappa_1\left( \mathbf{x} \right), \kappa_2\left( \mathbf{x} \right), \cdots, \kappa_n\left( \mathbf{x} \right) \right] \in {R}^{D}.
            \end{equation}
        $$


        Parameters
        ----------
        x: torch.Tensor
            The input data vector.
        device: str, default = 'cpu'
            The device to perform the data expansion.

        Returns
        ----------
        torch.Tensor
            The expanded data vector of the input.
        """
        b, m = x.shape
        x = self.pre_process(x=x)

        expansion = []
        for func in self.composition_functions:
            expansion.append(func(x=x, device=device))
        expansion = torch.cat(expansion, dim=1)

        assert expansion.shape == (b, self.calculate_D(m=m))
        return self.post_process(x=expansion, device=device)

__init__(name='extended_expansion', composition_functions=None, composition_function_configs=None, *args, **kwargs)

The initialization method of the extended expansion function.

It initializes an extended expansion object based on the input function name. This method will also call the initialization method of the base class as well.

Parameters:

Name Type Description Default
name

The name of the extended expansion function.

 'extended_expansion'
composition_functions list

The list of data expansion functions to be extended.

 None
composition_function_configs list | dict

The list or dictionary of the expansion function configurations.

 None

Returns:

Type Description
transformation

The extended data expansion function.
Source code in tinybig/expansion/extended_expansion.py 
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
def __init__(
        self,
        name='extended_expansion',
        composition_functions: list = None,
        composition_function_configs: list | dict = None,
        *args, **kwargs
):
    r"""
    The initialization method of the extended expansion function.

    It initializes an extended expansion object based on the input function name.
    This method will also call the initialization method of the base class as well.

    Parameters
    ----------
    name: str, default = 'extended_expansion'
        The name of the extended expansion function.
    composition_functions: list, default = None
        The list of data expansion functions to be extended.
    composition_function_configs: list | dict, default = None
        The list or dictionary of the expansion function configurations.

    Returns
    ----------
    transformation
        The extended data expansion function.
    """
    super().__init__(name=name, *args, **kwargs)
    self.composition_functions = []
    if composition_functions is not None and composition_functions != []:
        self.composition_functions = composition_functions
    elif composition_function_configs is not None:
        for function_config in composition_function_configs:
            function_class = function_config['function_class']
            function_parameters = function_config['function_parameters']
            self.composition_functions.append(config.get_obj_from_str(function_class)(**function_parameters))

calculate_D(m)

The expansion dimension calculation method.

It calculates the intermediate expansion space dimension based on the input dimension parameter m. For the extended expansion function defined based on \(n\) functions (1) \(\kappa_1: {R}^{m} \to {R}^{d_1}\), (2) \(\kappa_2: {R}^{m} \to {R}^{d_2}\), \(\cdots\), (n) \(\kappa_n: {R}^{m} \to {R}^{d_{n}}\), the expansion space dimension will be $$ D = \sum_{i=1}^n d_i. $$

Parameters:

Name Type Description Default
m int

The dimension of the input space.

 required

Returns:

Type Description
int

The dimension of the expansion space.
Source code in tinybig/expansion/extended_expansion.py 
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
def calculate_D(self, m: int):
    r"""
    The expansion dimension calculation method.

    It calculates the intermediate expansion space dimension based on the input dimension parameter m.
    For the extended expansion function defined based on $n$ functions (1) $\kappa_1: {R}^{m} \to {R}^{d_1}$,
    (2) $\kappa_2: {R}^{m} \to {R}^{d_2}$, $\cdots$, (n) $\kappa_n: {R}^{m} \to {R}^{d_{n}}$,
    the expansion space dimension will be
    $$ D = \sum_{i=1}^n d_i. $$

    Parameters
    ----------
    m: int
        The dimension of the input space.

    Returns
    -------
    int
        The dimension of the expansion space.
    """
    D = 0
    for func in self.composition_functions:
        D += func.calculate_D(m=m)
    return D

forward(x, device='cpu', *args, **kwargs)

The forward method of the extended expansion function.

For the extended expansion function defined based on \(n\) functions (1) \(\kappa_1: {R}^{m} \to {R}^{d_1}\), (2) \(\kappa_2: {R}^{m} \to {R}^{d_2}\), \(\cdots\), (n) \(\kappa_n: {R}^{m} \to {R}^{d_{n}}\), it performs the extended expansion of the input data and returns the expansion result as $$ \begin{equation} \kappa(\mathbf{x}) = \left[ \kappa_1\left( \mathbf{x} \right), \kappa_2\left( \mathbf{x} \right), \cdots, \kappa_n\left( \mathbf{x} \right) \right] \in {R}^{D}. \end{equation} $$

Parameters:

Name Type Description Default
x Tensor

The input data vector.

 required
device

The device to perform the data expansion.

 'cpu'

Returns:

Type Description
Tensor

The expanded data vector of the input.
Source code in tinybig/expansion/extended_expansion.py 
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
def forward(self, x: torch.Tensor, device='cpu', *args, **kwargs):
    r"""
    The forward method of the extended expansion function.

    For the extended expansion function defined based on $n$ functions (1) $\kappa_1: {R}^{m} \to {R}^{d_1}$,
    (2) $\kappa_2: {R}^{m} \to {R}^{d_2}$, $\cdots$, (n) $\kappa_n: {R}^{m} \to {R}^{d_{n}}$,
    it performs the extended expansion of the input data and returns the expansion result as
    $$
        \begin{equation}
            \kappa(\mathbf{x}) = \left[ \kappa_1\left( \mathbf{x} \right), \kappa_2\left( \mathbf{x} \right), \cdots, \kappa_n\left( \mathbf{x} \right) \right] \in {R}^{D}.
        \end{equation}
    $$


    Parameters
    ----------
    x: torch.Tensor
        The input data vector.
    device: str, default = 'cpu'
        The device to perform the data expansion.

    Returns
    ----------
    torch.Tensor
        The expanded data vector of the input.
    """
    b, m = x.shape
    x = self.pre_process(x=x)

    expansion = []
    for func in self.composition_functions:
        expansion.append(func(x=x, device=device))
    expansion = torch.cat(expansion, dim=1)

    assert expansion.shape == (b, self.calculate_D(m=m))
    return self.post_process(x=expansion, device=device)