Skip to content

gaussian_rbf_expansion

Bases: transformation

The gaussian rbf data expansion function.

It performs the gaussian rbf expansion of the input vector, and returns the expansion result. The class inherits from the base expansion class (i.e., the transformation class in the module directory).

...

Notes

For input vector \(\mathbf{x} \in R^m\), its gaussian rbf expansion with \(d\) fixed points can be represented as follows: $$ \begin{equation} \kappa(\mathbf{x}) = {\varphi} (\mathbf{x} | \mathbf{c}) = \left[ \varphi (\mathbf{x} | c_1), \varphi (\mathbf{x} | c_2), \cdots, \varphi (\mathbf{x} | c_d) \right] \in {R}^D, \end{equation} $$ where the sub-vector element \({\varphi} (x | \mathbf{c})\) can be defined as follows: $$ \begin{equation} {\varphi} (x | \mathbf{c}) = \left[ \varphi (x | c_1), \varphi (x | c_2), \cdots, \varphi (x | c_d) \right] \in {R}^d. \end{equation} $$ and value \(\varphi (x | c)\) is defined as: $$ \begin{equation} \varphi (x | c) = \exp(-(\epsilon (x - c) )^2). \end{equation} $$

For gaussian rbf expansion, its output expansion dimensions will be \(D = md\).

By default, the input and output can also be processed with the optional pre- or post-processing functions in the gaussian rbf expansion function.

Attributes:

Name Type Description
name str, default = 'gaussian_rbf_expansion'

Name of the expansion function.
base_range tuple | list, default = (-1, 1)

Input value range.
num_interval int, default = 10

Number of intervals partitioned by the fixed points.
epsilon float, default = 1.0

The rbf function parameter.
base Tensor

The partition of value range into intervals, i.e., the vector \(\mathbf{c}\) in the above equation.

Methods:

Name Description
__init__

It performs the initialization of the expansion function.
calculate_D

It calculates the expansion space dimension D based on the input dimension parameter m.
forward

It implements the abstract forward method declared in the base expansion class.
Source code in tinybig/expansion/rbf_expansion.py 
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
class gaussian_rbf_expansion(transformation):
    r"""
    The gaussian rbf data expansion function.

    It performs the gaussian rbf expansion of the input vector, and returns the expansion result.
    The class inherits from the base expansion class (i.e., the transformation class in the module directory).

    ...

    Notes
    ----------
    For input vector $\mathbf{x} \in R^m$, its gaussian rbf expansion with $d$ fixed points can be represented as follows:
    $$
    \begin{equation}
        \kappa(\mathbf{x}) = {\varphi} (\mathbf{x} | \mathbf{c}) = \left[ \varphi (\mathbf{x} | c_1), \varphi (\mathbf{x} | c_2), \cdots, \varphi (\mathbf{x} | c_d) \right] \in {R}^D,
    \end{equation}
    $$
    where the sub-vector element ${\varphi} (x | \mathbf{c})$ can be defined as follows:
    $$
        \begin{equation}
            {\varphi} (x | \mathbf{c}) = \left[ \varphi (x | c_1), \varphi (x | c_2), \cdots, \varphi (x | c_d) \right] \in {R}^d.
        \end{equation}
    $$
    and value $\varphi (x | c)$ is defined as:
    $$
        \begin{equation}
            \varphi (x | c) = \exp(-(\epsilon (x - c) )^2).
        \end{equation}
    $$

    For gaussian rbf expansion, its output expansion dimensions will be $D = md$.

    By default, the input and output can also be processed with the optional pre- or post-processing functions
    in the gaussian rbf expansion function.

    Attributes
    ----------
    name: str, default = 'gaussian_rbf_expansion'
        Name of the expansion function.
    base_range: tuple | list, default = (-1, 1)
        Input value range.
    num_interval: int, default = 10
        Number of intervals partitioned by the fixed points.
    epsilon: float, default = 1.0
        The rbf function parameter.
    base: torch.Tensor
        The partition of value range into intervals, i.e., the vector $\mathbf{c}$ in the above equation.

    Methods
    ----------
    __init__
        It performs the initialization of the expansion function.

    calculate_D
        It calculates the expansion space dimension D based on the input dimension parameter m.

    forward
        It implements the abstract forward method declared in the base expansion class.

    """
    def __init__(self, name='gaussian_rbf_expansion', base_range=(-1, 1), num_interval=10, epsilon=1.0, base=None, *args, **kwargs):
        r"""
        The initialization method of the gaussian rbf expansion function.

        It initializes a gaussian rbf expansion object based on the input function name.
        This method will also call the initialization method of the base class as well.

        Parameters
        ----------
        name: str, default = 'gaussian_rbf_expansion'
            The name of the gaussian rbf expansion function.
        base_range: tuple | list, default = (-1, 1)
            Input value range.
        num_interval: int, default = 10
            Number of intervals partitioned by the fixed points.
        epsilon: float, default = 1.0
            The rbf function parameter.
        base: Tensor, default = None
            The partition of value range into intervals, i.e., the vector $\mathbf{c}$ in the above equation.

        Returns
        ----------
        transformation
            The gaussian rbf expansion function.
        """
        super().__init__(name=name, *args, **kwargs)
        self.base_range = base_range
        self.num_interval = num_interval
        self.epsilon = epsilon
        self.base = base

    def calculate_D(self, m: int):
        r"""
        The expansion dimension calculation method.

        It calculates the intermediate expansion space dimension based on the input dimension parameter m.
        For the gaussian rbf expansion function, the expansion space dimension will be
        $$ D = m d, $$
        where $d$ denotes the number of intervals of the input value range.

        Parameters
        ----------
        m: int
            The dimension of the input space.

        Returns
        -------
        int
            The dimension of the expansion space.
        """
        return m * self.num_interval

    def initialize_base(self, device='cpu', base_range=None, num_interval=None):
        r"""
        The fixed point base initialization method.

        It initializes the fixed point base tensor, which partitions the value range into equal-length intervals.
        The initialized base tensor corresponds to the fixed point vector $\mathbf{c}$ mentioned in the above equation.

        Parameters
        ----------
        device: str, default = 'cpu'
            The device to host the base tensor.
        base_range: tuple | list, default = None
            Input value range.
        num_interval: int, default = None
            Number of intervals partitioned by the fixed points.

        Returns
        -------
        torch.Tensor
            The fixed point base tensor.
        """
        base_range = base_range if base_range is not None else self.base_range
        num_interval = num_interval if num_interval is not None else self.num_interval
        self.base = torch.Tensor(torch.linspace(base_range[0], base_range[1], num_interval)).to(device)

    def forward(self, x: torch.Tensor, device='cpu', *args, **kwargs):
        r"""
        The forward method of the data expansion function.

        It performs the gaussian rbf data expansion of the input data and returns the expansion result as
        $$
            \begin{equation}
                \kappa(\mathbf{x}) = {\varphi} (\mathbf{x} | \mathbf{c}) = \left[ \varphi (\mathbf{x} | c_1), \varphi (\mathbf{x} | c_2), \cdots, \varphi (\mathbf{x} | c_d) \right] \in {R}^D,
            \end{equation}
        $$
        where vector $\mathbf{c}$ is the fixed point base tensor initialized above.


        Parameters
        ----------
        x: torch.Tensor
            The input data vector.
        device: str, default = 'cpu'
            The device to perform the data expansion.

        Returns
        ----------
        torch.Tensor
            The expanded data vector of the input.
        """
        b, m = x.shape
        x = self.pre_process(x=x, device=device)

        if self.base is None:
            self.initialize_base(device=device, *args, **kwargs)
        assert x.dim() == 2
        expansion = torch.exp(-((x[..., None] - self.base) * self.epsilon) ** 2).view(x.size(0), -1)

        assert expansion.shape == (b, self.calculate_D(m=m))
        return self.post_process(x=expansion, device=device)

__init__(name='gaussian_rbf_expansion', base_range=(-1, 1), num_interval=10, epsilon=1.0, base=None, *args, **kwargs)

The initialization method of the gaussian rbf expansion function.

It initializes a gaussian rbf expansion object based on the input function name. This method will also call the initialization method of the base class as well.

Parameters:

Name Type Description Default
name

The name of the gaussian rbf expansion function.

 'gaussian_rbf_expansion'
base_range

Input value range.

 (-1, 1)
num_interval

Number of intervals partitioned by the fixed points.

 10
epsilon

The rbf function parameter.

 1.0
base

The partition of value range into intervals, i.e., the vector \(\mathbf{c}\) in the above equation.

 None

Returns:

Type Description
transformation

The gaussian rbf expansion function.
Source code in tinybig/expansion/rbf_expansion.py 
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
def __init__(self, name='gaussian_rbf_expansion', base_range=(-1, 1), num_interval=10, epsilon=1.0, base=None, *args, **kwargs):
    r"""
    The initialization method of the gaussian rbf expansion function.

    It initializes a gaussian rbf expansion object based on the input function name.
    This method will also call the initialization method of the base class as well.

    Parameters
    ----------
    name: str, default = 'gaussian_rbf_expansion'
        The name of the gaussian rbf expansion function.
    base_range: tuple | list, default = (-1, 1)
        Input value range.
    num_interval: int, default = 10
        Number of intervals partitioned by the fixed points.
    epsilon: float, default = 1.0
        The rbf function parameter.
    base: Tensor, default = None
        The partition of value range into intervals, i.e., the vector $\mathbf{c}$ in the above equation.

    Returns
    ----------
    transformation
        The gaussian rbf expansion function.
    """
    super().__init__(name=name, *args, **kwargs)
    self.base_range = base_range
    self.num_interval = num_interval
    self.epsilon = epsilon
    self.base = base

calculate_D(m)

The expansion dimension calculation method.

It calculates the intermediate expansion space dimension based on the input dimension parameter m. For the gaussian rbf expansion function, the expansion space dimension will be $$ D = m d, $$ where \(d\) denotes the number of intervals of the input value range.

Parameters:

Name Type Description Default
m int

The dimension of the input space.

 required

Returns:

Type Description
int

The dimension of the expansion space.
Source code in tinybig/expansion/rbf_expansion.py 
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
def calculate_D(self, m: int):
    r"""
    The expansion dimension calculation method.

    It calculates the intermediate expansion space dimension based on the input dimension parameter m.
    For the gaussian rbf expansion function, the expansion space dimension will be
    $$ D = m d, $$
    where $d$ denotes the number of intervals of the input value range.

    Parameters
    ----------
    m: int
        The dimension of the input space.

    Returns
    -------
    int
        The dimension of the expansion space.
    """
    return m * self.num_interval

forward(x, device='cpu', *args, **kwargs)

The forward method of the data expansion function.

It performs the gaussian rbf data expansion of the input data and returns the expansion result as $$ \begin{equation} \kappa(\mathbf{x}) = {\varphi} (\mathbf{x} | \mathbf{c}) = \left[ \varphi (\mathbf{x} | c_1), \varphi (\mathbf{x} | c_2), \cdots, \varphi (\mathbf{x} | c_d) \right] \in {R}^D, \end{equation} $$ where vector \(\mathbf{c}\) is the fixed point base tensor initialized above.

Parameters:

Name Type Description Default
x Tensor

The input data vector.

 required
device

The device to perform the data expansion.

 'cpu'

Returns:

Type Description
Tensor

The expanded data vector of the input.
Source code in tinybig/expansion/rbf_expansion.py 
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
def forward(self, x: torch.Tensor, device='cpu', *args, **kwargs):
    r"""
    The forward method of the data expansion function.

    It performs the gaussian rbf data expansion of the input data and returns the expansion result as
    $$
        \begin{equation}
            \kappa(\mathbf{x}) = {\varphi} (\mathbf{x} | \mathbf{c}) = \left[ \varphi (\mathbf{x} | c_1), \varphi (\mathbf{x} | c_2), \cdots, \varphi (\mathbf{x} | c_d) \right] \in {R}^D,
        \end{equation}
    $$
    where vector $\mathbf{c}$ is the fixed point base tensor initialized above.


    Parameters
    ----------
    x: torch.Tensor
        The input data vector.
    device: str, default = 'cpu'
        The device to perform the data expansion.

    Returns
    ----------
    torch.Tensor
        The expanded data vector of the input.
    """
    b, m = x.shape
    x = self.pre_process(x=x, device=device)

    if self.base is None:
        self.initialize_base(device=device, *args, **kwargs)
    assert x.dim() == 2
    expansion = torch.exp(-((x[..., None] - self.base) * self.epsilon) ** 2).view(x.size(0), -1)

    assert expansion.shape == (b, self.calculate_D(m=m))
    return self.post_process(x=expansion, device=device)

initialize_base(device='cpu', base_range=None, num_interval=None)

The fixed point base initialization method.

It initializes the fixed point base tensor, which partitions the value range into equal-length intervals. The initialized base tensor corresponds to the fixed point vector \(\mathbf{c}\) mentioned in the above equation.

Parameters:

Name Type Description Default
device

The device to host the base tensor.

 'cpu'
base_range

Input value range.

 None
num_interval

Number of intervals partitioned by the fixed points.

 None

Returns:

Type Description
Tensor

The fixed point base tensor.
Source code in tinybig/expansion/rbf_expansion.py 
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
def initialize_base(self, device='cpu', base_range=None, num_interval=None):
    r"""
    The fixed point base initialization method.

    It initializes the fixed point base tensor, which partitions the value range into equal-length intervals.
    The initialized base tensor corresponds to the fixed point vector $\mathbf{c}$ mentioned in the above equation.

    Parameters
    ----------
    device: str, default = 'cpu'
        The device to host the base tensor.
    base_range: tuple | list, default = None
        Input value range.
    num_interval: int, default = None
        Number of intervals partitioned by the fixed points.

    Returns
    -------
    torch.Tensor
        The fixed point base tensor.
    """
    base_range = base_range if base_range is not None else self.base_range
    num_interval = num_interval if num_interval is not None else self.num_interval
    self.base = torch.Tensor(torch.linspace(base_range[0], base_range[1], num_interval)).to(device)