Skip to content

bspline_expansion

Bases: transformation

The bspline data expansion function.

It performs the bspline expansion of the input vector, and returns the expansion result. The class inherits from the base expansion class (i.e., the transformation class in the module directory).

...

Notes

For input vector \(\mathbf{x} \in R^m\), its bspline expansion of degree \(d\) will be $$ \begin{equation} \kappa (\mathbf{x} | d) = \left[ B_{0,d}(\mathbf{x}), B_{1,d}(\mathbf{x}), \cdots, B_{t-1,d}(\mathbf{x}) \right] \in {R}^D, \end{equation} $$ where \(B_{i,d}(\mathbf{x})\) denotes the bspline expansion polynomials of \(\mathbf{x}\) of degree \(d\) within the range \([x_i, x_{i+d+1})\). The output dimension will be \(D = m (t + d)\).

As to the specific representations of bspline polynomials, they can be defined recursively based on the lower-degree terms according to the following equations:

(1) Base B-splines with degree \(d = 0\): $$ \begin{equation} { B_{0,0}(x), B_{1,0}(x), \cdots, B_{t-1,0}(x) }, \end{equation} $$ where $$ \begin{equation} B_{i,0}(x) = \begin{cases} 1, &\text{if } x_i \le x < x_{i+1};\\ 0, &\text{otherwise}. \end{cases} \end{equation} $$

(2) Higher-degree B-splines with \(d > 0\): $$ \begin{equation} { B_{0,d}(x), B_{1,d}(x), \cdots, B_{t-1,d}(x) }, \end{equation} $$ where $$ \begin{equation} B_{i,d}(x) = \frac{x - x_i}{x_{i+d} - x_i} B_{i, d-1}(x) + \frac{x_{i+d+1} - x}{x_{i+d+1} - x_{i+1}} B_{i+1, d-1}(x). \end{equation} $$ According to the representations, term \(B_{i,d}(x)\) recursively defined above will have non-zero outputs if and only if the inputs lie within the value range \(x_i \le x < x_{i+d+1}\).

By default, the input and output can also be processed with the optional pre- or post-processing functions in the bspline expansion function.

Attributes:

Name Type Description
name str, default = 'bspline_expansion'

Name of the expansion function.
grid_range tuple, default = (-1, 1)

The input value range for defining the grid.
t int, default = 5

The interval number divided by the knots in bspline.
d int, default = 3

The degree of the bspline expansion.
grid torch.Tensor, default = None

The grid representing relationships between lower-order bspline polynomials with high-order ones.

Methods:

Name Description
__init__

It performs the initialization of the expansion function.
calculate_D

It calculates the expansion space dimension D based on the input dimension parameter m.
initialize_grid

It initializes the grid defining the relationships between lower-order bspline polynomials with high-order ones.
forward

It implements the abstract forward method declared in the base expansion class.
Source code in tinybig/expansion/recursive_expansion.py 
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
class bspline_expansion(transformation):
    r"""
    The bspline data expansion function.

    It performs the bspline expansion of the input vector, and returns the expansion result.
    The class inherits from the base expansion class (i.e., the transformation class in the module directory).

    ...

    Notes
    ----------
    For input vector $\mathbf{x} \in R^m$, its bspline expansion of degree $d$ will be
    $$
        \begin{equation}
            \kappa (\mathbf{x} | d) = \left[ B_{0,d}(\mathbf{x}), B_{1,d}(\mathbf{x}), \cdots, B_{t-1,d}(\mathbf{x}) \right] \in {R}^D,
        \end{equation}
    $$
    where $B_{i,d}(\mathbf{x})$ denotes the bspline expansion polynomials of $\mathbf{x}$ of degree $d$
    within the range $[x_i, x_{i+d+1})$. The output dimension will be $D = m (t + d)$.

    As to the specific representations of bspline polynomials, they can be defined recursively based on the
    lower-degree terms according to the following equations:

    (1) **Base B-splines with degree $d = 0$:**
    $$
        \begin{equation}
            \{ B\_{0,0}(x), B\_{1,0}(x), \cdots, B\_{t-1,0}(x) \},
        \end{equation}
    $$
        where
    $$
        \begin{equation}
            B\_{i,0}(x) = \begin{cases}
            1, &\text{if } x_i \le x < x_{i+1};\\\\
            0, &\text{otherwise}.
            \end{cases}
        \end{equation}
    $$

    (2) **Higher-degree B-splines with $d > 0$:**
    $$
        \begin{equation}
            \{ B\_{0,d}(x), B\_{1,d}(x), \cdots, B\_{t-1,d}(x) \},
        \end{equation}
    $$
    where
    $$
        \begin{equation}
            B\_{i,d}(x) = \frac{x - x_i}{x_{i+d} - x_i} B\_{i, d-1}(x) + \frac{x_{i+d+1} - x}{x_{i+d+1} - x_{i+1}} B\_{i+1, d-1}(x).
        \end{equation}
    $$
    According to the representations, term $B_{i,d}(x)$ recursively defined above will have non-zero outputs
    if and only if the inputs lie within the value range $x_i \le x < x_{i+d+1}$.

    By default, the input and output can also be processed with the optional pre- or post-processing functions
    in the bspline expansion function.

    Attributes
    ----------
    name: str, default = 'bspline_expansion'
        Name of the expansion function.
    grid_range: tuple, default = (-1, 1)
        The input value range for defining the grid.
    t: int, default = 5
        The interval number divided by the knots in bspline.
    d: int, default = 3
        The degree of the bspline expansion.
    grid: torch.Tensor, default = None
        The grid representing relationships between lower-order bspline polynomials with high-order ones.

    Methods
    ----------
    __init__
        It performs the initialization of the expansion function.

    calculate_D
        It calculates the expansion space dimension D based on the input dimension parameter m.

    initialize_grid
        It initializes the grid defining the relationships between lower-order bspline polynomials with high-order ones.

    forward
        It implements the abstract forward method declared in the base expansion class.

    """
    def __init__(self, name: str = 'bspline_expansion', grid_range=(-1, 1), t: int = 5, d: int = 3, *args, **kwargs):
        r"""
        The initialization method of bspline expansion function.

        It initializes a bspline expansion object based on the input function name and parameters.
        This method will also call the initialization method of the base class as well.

        Parameters
        ----------
        name: str, default = 'bspline_expansion'
            The name of the bspline expansion function.
        grid_range: tuple, default = (-1, 1)
            The input value range for defining the grid.
        t: int, default = 5
            The interval number divided by the knots in bspline.
        d: int, default = 3
            The degree of the bspline expansion.

        Returns
        -------
        transformation
            The bspline expansion function object.
        """
        super().__init__(name=name, *args, **kwargs)
        self.grid_range = grid_range
        self.t = t
        self.d = d
        self.grid = None

    def calculate_D(self, m: int):
        r"""
        The expansion dimension calculation method.

        It calculates the intermediate expansion space dimension based on the input dimension parameter m.
        For the bspline expansion function, the expansion space dimension is determined by m, and parameters t and d,
        which can be represented as:

        $$ D = m (t + d). $$

        Parameters
        ----------
        m: int
            The dimension of the input space.

        Returns
        -------
        int
            The dimension of the expansion space.
        """
        return m * (self.t + self.d)

    def initialize_grid(self, m: int, device='cpu', grid_range: tuple | list = None, t: int = None, d: int = None):
        """
        The grid initialization method.

        It initializes the grid defining the relationships between lower-order bspline polynomials with high-order ones.
        The grid enables faster calculation of the high-order bspline polynomials, which is defined by the parameters
        grid value range "grid_range", interval number $t$ and polynomial degree $d$.

        The grid is defined as a tensor of shape (m, t+d).

        Parameters
        ----------
        m: int
            The dimension of the input space.
        device: str, default = 'cpu'
            The device to host the grid.
        grid_range: tuple | list, default = None
            The customized grid value range.
        t: int, default = None
            The interval number divided by the knots in bspline.
        d: int, default = None
            The degree of the bspline expansion.

        Returns
        -------
        torch.Tensor
            The grid tensor of shape (m, t+d) denoting the lower-order and high-order bspline polynomial relationships.
        """
        grid_range = grid_range if grid_range is not None else self.grid_range
        t = t if t is not None else self.t
        d = d if d is not None else self.d

        h = (grid_range[1] - grid_range[0]) / t
        self.grid = torch.Tensor((torch.arange(-d, t + d + 1) * h + grid_range[0])
                                 .expand(m, -1).contiguous()).to(device)

    def forward(self, x: torch.Tensor, device='cpu', *args, **kwargs):
        r"""
        The forward method of the data expansion function.

        It performs the bspline data expansion of the input data and returns the expansion result
        according to the following equation:
        $$
        \begin{equation}
            \kappa (\mathbf{x} | d) = \left[ B_{0,d}(\mathbf{x}), B_{1,d}(\mathbf{x}), \cdots, B_{t-1,d}(\mathbf{x}) \right] \in {R}^D,
        \end{equation}
        $$


        Parameters
        ----------
        x: torch.Tensor
            The input data vector.
        device: str, default = 'cpu'
            The device to perform the data expansion.

        Returns
        ----------
        torch.Tensor
            The expanded data vector of the input.
        """
        b, m = x.shape
        x = self.pre_process(x=x, device=device)
        if self.grid is None:
            self.initialize_grid(m=x.size(1), device=device, *args, **kwargs)
        assert x.dim() == 2
        x = x.unsqueeze(-1)
        bases = ((x >= self.grid[:, :-1]) & (x < self.grid[:, 1:])).to(x.dtype)
        for k in range(1, self.d + 1):
            bases = (((x - self.grid[:, : -(k + 1)]) / (self.grid[:, k:-1] - self.grid[:, : -(k + 1)]) * bases[:, :, :-1]) +
                     ((self.grid[:, k + 1:] - x) / (self.grid[:, k + 1:] - self.grid[:, 1:(-k)]) * bases[:, :, 1:]))
        expansion = bases.contiguous().view(x.size(0), -1)

        assert expansion.shape == (b, self.calculate_D(m=m))
        return self.post_process(x=expansion, device=device)

__init__(name='bspline_expansion', grid_range=(-1, 1), t=5, d=3, *args, **kwargs)

The initialization method of bspline expansion function.

It initializes a bspline expansion object based on the input function name and parameters. This method will also call the initialization method of the base class as well.

Parameters:

Name Type Description Default
name str

The name of the bspline expansion function.

 'bspline_expansion'
grid_range

The input value range for defining the grid.

 (-1, 1)
t int

The interval number divided by the knots in bspline.

 5
d int

The degree of the bspline expansion.

 3

Returns:

Type Description
transformation

The bspline expansion function object.
Source code in tinybig/expansion/recursive_expansion.py 
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
def __init__(self, name: str = 'bspline_expansion', grid_range=(-1, 1), t: int = 5, d: int = 3, *args, **kwargs):
    r"""
    The initialization method of bspline expansion function.

    It initializes a bspline expansion object based on the input function name and parameters.
    This method will also call the initialization method of the base class as well.

    Parameters
    ----------
    name: str, default = 'bspline_expansion'
        The name of the bspline expansion function.
    grid_range: tuple, default = (-1, 1)
        The input value range for defining the grid.
    t: int, default = 5
        The interval number divided by the knots in bspline.
    d: int, default = 3
        The degree of the bspline expansion.

    Returns
    -------
    transformation
        The bspline expansion function object.
    """
    super().__init__(name=name, *args, **kwargs)
    self.grid_range = grid_range
    self.t = t
    self.d = d
    self.grid = None

calculate_D(m)

The expansion dimension calculation method.

It calculates the intermediate expansion space dimension based on the input dimension parameter m. For the bspline expansion function, the expansion space dimension is determined by m, and parameters t and d, which can be represented as:

\[ D = m (t + d). \]

Parameters:

Name Type Description Default
m int

The dimension of the input space.

 required

Returns:

Type Description
int

The dimension of the expansion space.
Source code in tinybig/expansion/recursive_expansion.py 
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
def calculate_D(self, m: int):
    r"""
    The expansion dimension calculation method.

    It calculates the intermediate expansion space dimension based on the input dimension parameter m.
    For the bspline expansion function, the expansion space dimension is determined by m, and parameters t and d,
    which can be represented as:

    $$ D = m (t + d). $$

    Parameters
    ----------
    m: int
        The dimension of the input space.

    Returns
    -------
    int
        The dimension of the expansion space.
    """
    return m * (self.t + self.d)

forward(x, device='cpu', *args, **kwargs)

The forward method of the data expansion function.

It performs the bspline data expansion of the input data and returns the expansion result according to the following equation: $$ \begin{equation} \kappa (\mathbf{x} | d) = \left[ B_{0,d}(\mathbf{x}), B_{1,d}(\mathbf{x}), \cdots, B_{t-1,d}(\mathbf{x}) \right] \in {R}^D, \end{equation} $$

Parameters:

Name Type Description Default
x Tensor

The input data vector.

 required
device

The device to perform the data expansion.

 'cpu'

Returns:

Type Description
Tensor

The expanded data vector of the input.
Source code in tinybig/expansion/recursive_expansion.py 
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
def forward(self, x: torch.Tensor, device='cpu', *args, **kwargs):
    r"""
    The forward method of the data expansion function.

    It performs the bspline data expansion of the input data and returns the expansion result
    according to the following equation:
    $$
    \begin{equation}
        \kappa (\mathbf{x} | d) = \left[ B_{0,d}(\mathbf{x}), B_{1,d}(\mathbf{x}), \cdots, B_{t-1,d}(\mathbf{x}) \right] \in {R}^D,
    \end{equation}
    $$


    Parameters
    ----------
    x: torch.Tensor
        The input data vector.
    device: str, default = 'cpu'
        The device to perform the data expansion.

    Returns
    ----------
    torch.Tensor
        The expanded data vector of the input.
    """
    b, m = x.shape
    x = self.pre_process(x=x, device=device)
    if self.grid is None:
        self.initialize_grid(m=x.size(1), device=device, *args, **kwargs)
    assert x.dim() == 2
    x = x.unsqueeze(-1)
    bases = ((x >= self.grid[:, :-1]) & (x < self.grid[:, 1:])).to(x.dtype)
    for k in range(1, self.d + 1):
        bases = (((x - self.grid[:, : -(k + 1)]) / (self.grid[:, k:-1] - self.grid[:, : -(k + 1)]) * bases[:, :, :-1]) +
                 ((self.grid[:, k + 1:] - x) / (self.grid[:, k + 1:] - self.grid[:, 1:(-k)]) * bases[:, :, 1:]))
    expansion = bases.contiguous().view(x.size(0), -1)

    assert expansion.shape == (b, self.calculate_D(m=m))
    return self.post_process(x=expansion, device=device)

initialize_grid(m, device='cpu', grid_range=None, t=None, d=None)

The grid initialization method.

It initializes the grid defining the relationships between lower-order bspline polynomials with high-order ones. The grid enables faster calculation of the high-order bspline polynomials, which is defined by the parameters grid value range "grid_range", interval number \(t\) and polynomial degree \(d\).

The grid is defined as a tensor of shape (m, t+d).

Parameters:

Name Type Description Default
m int

The dimension of the input space.

 required
device

The device to host the grid.

 'cpu'
grid_range tuple | list

The customized grid value range.

 None
t int

The interval number divided by the knots in bspline.

 None
d int

The degree of the bspline expansion.

 None

Returns:

Type Description
Tensor

The grid tensor of shape (m, t+d) denoting the lower-order and high-order bspline polynomial relationships.
Source code in tinybig/expansion/recursive_expansion.py 
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
def initialize_grid(self, m: int, device='cpu', grid_range: tuple | list = None, t: int = None, d: int = None):
    """
    The grid initialization method.

    It initializes the grid defining the relationships between lower-order bspline polynomials with high-order ones.
    The grid enables faster calculation of the high-order bspline polynomials, which is defined by the parameters
    grid value range "grid_range", interval number $t$ and polynomial degree $d$.

    The grid is defined as a tensor of shape (m, t+d).

    Parameters
    ----------
    m: int
        The dimension of the input space.
    device: str, default = 'cpu'
        The device to host the grid.
    grid_range: tuple | list, default = None
        The customized grid value range.
    t: int, default = None
        The interval number divided by the knots in bspline.
    d: int, default = None
        The degree of the bspline expansion.

    Returns
    -------
    torch.Tensor
        The grid tensor of shape (m, t+d) denoting the lower-order and high-order bspline polynomial relationships.
    """
    grid_range = grid_range if grid_range is not None else self.grid_range
    t = t if t is not None else self.t
    d = d if d is not None else self.d

    h = (grid_range[1] - grid_range[0]) / t
    self.grid = torch.Tensor((torch.arange(-d, t + d + 1) * h + grid_range[0])
                             .expand(m, -1).contiguous()).to(device)