Skip to content

svm

Bases: rpn

Support Vector Machine (SVM) model implemented as the RPN model.

This class defines an SVM model with customizable kernel functions, parameter reconciliation, and additional processing capabilities such as residual connections and multi-channel configurations.

Parameters:

Name Type Description Default
dims list[int] | tuple[int]

A list or tuple of integers representing the dimensions of each layer. Must contain at least two dimensions.

 required
name str

The name of the SVM model. Default is 'rpn_svm'.

 'rpn_svm'
kernel str

The kernel type to use. Options include 'linear', 'gaussian_rbf', and 'inverse_quadratic_rbf'. Default is 'linear'.

 'linear'
base_range tuple

The base range for kernel expansion. Default is (-1, 1).

 (-1, 1)
num_interval int

The number of intervals for kernel expansion. Default is 10.

 10
epsilon float

Parameter for kernel approximation. Default is 1.0.

 1.0
enable_bias bool

If True, enables bias in the layers. Default is False.

 False
with_lorr bool

If True, enables low-rank parameterized reconciliation. Default is False.

 False
r int

Rank parameter for low-rank reconciliation. Default is 3.

 3
with_residual bool

If True, adds residual connections to the layers. Default is False.

 False
channel_num int

The number of channels for each layer. Default is 1.

 1
width int

The number of parallel heads in each layer. Default is 1.

 1
device str

Device to perform computations ('cpu' or 'cuda'). Default is 'cpu'.

 'cpu'
*args optional

Additional positional arguments for the superclass.

 ()
**kwargs optional

Additional keyword arguments for the superclass.

 {}

Raises:

Type Description
ValueError

If dims contains fewer than two dimensions.
Warning

If more than two layers are defined, as SVMs typically use only two layers.

Methods:

Name Description
__init__

Initializes the SVM model with the specified parameters.
Source code in tinybig/model/rpn_svm.py 
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
class svm(rpn):
    """
    Support Vector Machine (SVM) model implemented as the RPN model.

    This class defines an SVM model with customizable kernel functions, parameter reconciliation,
    and additional processing capabilities such as residual connections and multi-channel configurations.

    Parameters
    ----------
    dims : list[int] | tuple[int]
        A list or tuple of integers representing the dimensions of each layer.
        Must contain at least two dimensions.
    name : str, optional
        The name of the SVM model. Default is 'rpn_svm'.
    kernel : str, optional
        The kernel type to use. Options include 'linear', 'gaussian_rbf', and 'inverse_quadratic_rbf'.
        Default is 'linear'.
    base_range : tuple, optional
        The base range for kernel expansion. Default is (-1, 1).
    num_interval : int, optional
        The number of intervals for kernel expansion. Default is 10.
    epsilon : float, optional
        Parameter for kernel approximation. Default is 1.0.
    enable_bias : bool, optional
        If True, enables bias in the layers. Default is False.
    with_lorr : bool, optional
        If True, enables low-rank parameterized reconciliation. Default is False.
    r : int, optional
        Rank parameter for low-rank reconciliation. Default is 3.
    with_residual : bool, optional
        If True, adds residual connections to the layers. Default is False.
    channel_num : int, optional
        The number of channels for each layer. Default is 1.
    width : int, optional
        The number of parallel heads in each layer. Default is 1.
    device : str, optional
        Device to perform computations ('cpu' or 'cuda'). Default is 'cpu'.
    *args : optional
        Additional positional arguments for the superclass.
    **kwargs : optional
        Additional keyword arguments for the superclass.

    Raises
    ------
    ValueError
        If `dims` contains fewer than two dimensions.
    Warning
        If more than two layers are defined, as SVMs typically use only two layers.

    Methods
    -------
    __init__(dims, name, kernel, base_range, num_interval, epsilon, enable_bias, ...)
        Initializes the SVM model with the specified parameters.
    """
    def __init__(
        self,
        dims: list[int] | tuple[int],
        name: str = 'rpn_svm',
        kernel: str = 'linear',
        base_range: tuple = (-1, 1),
        num_interval: int = 10,
        epsilon: float = 1.0,
        enable_bias: bool = False,
        # optional parameters
        with_lorr: bool = False,
        r: int = 3,
        with_residual: bool = False,
        channel_num: int = 1,
        width: int = 1,
        # other parameters
        device: str = 'cpu', *args, **kwargs
    ):
        """
        Initialize the SVM model as a recursive parameterized network (RPN).

        Parameters
        ----------
        dims : list[int] | tuple[int]
            A list or tuple of integers representing the dimensions of each layer.
            Must contain at least two dimensions.
        name : str, optional
            The name of the SVM model. Default is 'rpn_svm'.
        kernel : str, optional
            The kernel type to use. Options include 'linear', 'gaussian_rbf', and 'inverse_quadratic_rbf'.
            Default is 'linear'.
        base_range : tuple, optional
            The base range for kernel expansion. Default is (-1, 1).
        num_interval : int, optional
            The number of intervals for kernel expansion. Default is 10.
        epsilon : float, optional
            Parameter for kernel approximation. Default is 1.0.
        enable_bias : bool, optional
            If True, enables bias in the layers. Default is False.
        with_lorr : bool, optional
            If True, enables low-rank parameterized reconciliation. Default is False.
        r : int, optional
            Rank parameter for low-rank reconciliation. Default is 3.
        with_residual : bool, optional
            If True, adds residual connections to the layers. Default is False.
        channel_num : int, optional
            The number of channels for each layer. Default is 1.
        width : int, optional
            The number of parallel heads in each layer. Default is 1.
        device : str, optional
            Device to perform computations ('cpu' or 'cuda'). Default is 'cpu'.
        *args : optional
            Additional positional arguments for the superclass.
        **kwargs : optional
            Additional keyword arguments for the superclass.

        Raises
        ------
        ValueError
            If `dims` contains fewer than two dimensions.
        Warning
            If more than two layers are defined, as SVMs typically use only two layers.
        """
        if len(dims) < 2:
            raise ValueError("At least two dim values is needed for defining the model...")

        if len(dims) > 2:
            warnings.warn("Regular SVMs should have only two layers...")

        layers = []
        for m, n in zip(dims, dims[1:]):
            layers.append(
                svm_layer(
                    m=m, n=n,
                    enable_bias=enable_bias,
                    kernel=kernel,
                    base_range=base_range,
                    num_interval=num_interval,
                    epsilon=epsilon,
                    with_lorr=with_lorr, r=r,
                    with_residual=with_residual,
                    device=device,
                    channel_num=channel_num,
                    width=width,
                )
            )
        super().__init__(name=name, layers=layers, device=device, *args, **kwargs)

__init__(dims, name='rpn_svm', kernel='linear', base_range=(-1, 1), num_interval=10, epsilon=1.0, enable_bias=False, with_lorr=False, r=3, with_residual=False, channel_num=1, width=1, device='cpu', *args, **kwargs)

Initialize the SVM model as a recursive parameterized network (RPN).

Parameters:

Name Type Description Default
dims list[int] | tuple[int]

A list or tuple of integers representing the dimensions of each layer. Must contain at least two dimensions.

 required
name str

The name of the SVM model. Default is 'rpn_svm'.

 'rpn_svm'
kernel str

The kernel type to use. Options include 'linear', 'gaussian_rbf', and 'inverse_quadratic_rbf'. Default is 'linear'.

 'linear'
base_range tuple

The base range for kernel expansion. Default is (-1, 1).

 (-1, 1)
num_interval int

The number of intervals for kernel expansion. Default is 10.

 10
epsilon float

Parameter for kernel approximation. Default is 1.0.

 1.0
enable_bias bool

If True, enables bias in the layers. Default is False.

 False
with_lorr bool

If True, enables low-rank parameterized reconciliation. Default is False.

 False
r int

Rank parameter for low-rank reconciliation. Default is 3.

 3
with_residual bool

If True, adds residual connections to the layers. Default is False.

 False
channel_num int

The number of channels for each layer. Default is 1.

 1
width int

The number of parallel heads in each layer. Default is 1.

 1
device str

Device to perform computations ('cpu' or 'cuda'). Default is 'cpu'.

 'cpu'
*args optional

Additional positional arguments for the superclass.

 ()
**kwargs optional

Additional keyword arguments for the superclass.

 {}

Raises:

Type Description
ValueError

If dims contains fewer than two dimensions.
Warning

If more than two layers are defined, as SVMs typically use only two layers.
Source code in tinybig/model/rpn_svm.py 
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
def __init__(
    self,
    dims: list[int] | tuple[int],
    name: str = 'rpn_svm',
    kernel: str = 'linear',
    base_range: tuple = (-1, 1),
    num_interval: int = 10,
    epsilon: float = 1.0,
    enable_bias: bool = False,
    # optional parameters
    with_lorr: bool = False,
    r: int = 3,
    with_residual: bool = False,
    channel_num: int = 1,
    width: int = 1,
    # other parameters
    device: str = 'cpu', *args, **kwargs
):
    """
    Initialize the SVM model as a recursive parameterized network (RPN).

    Parameters
    ----------
    dims : list[int] | tuple[int]
        A list or tuple of integers representing the dimensions of each layer.
        Must contain at least two dimensions.
    name : str, optional
        The name of the SVM model. Default is 'rpn_svm'.
    kernel : str, optional
        The kernel type to use. Options include 'linear', 'gaussian_rbf', and 'inverse_quadratic_rbf'.
        Default is 'linear'.
    base_range : tuple, optional
        The base range for kernel expansion. Default is (-1, 1).
    num_interval : int, optional
        The number of intervals for kernel expansion. Default is 10.
    epsilon : float, optional
        Parameter for kernel approximation. Default is 1.0.
    enable_bias : bool, optional
        If True, enables bias in the layers. Default is False.
    with_lorr : bool, optional
        If True, enables low-rank parameterized reconciliation. Default is False.
    r : int, optional
        Rank parameter for low-rank reconciliation. Default is 3.
    with_residual : bool, optional
        If True, adds residual connections to the layers. Default is False.
    channel_num : int, optional
        The number of channels for each layer. Default is 1.
    width : int, optional
        The number of parallel heads in each layer. Default is 1.
    device : str, optional
        Device to perform computations ('cpu' or 'cuda'). Default is 'cpu'.
    *args : optional
        Additional positional arguments for the superclass.
    **kwargs : optional
        Additional keyword arguments for the superclass.

    Raises
    ------
    ValueError
        If `dims` contains fewer than two dimensions.
    Warning
        If more than two layers are defined, as SVMs typically use only two layers.
    """
    if len(dims) < 2:
        raise ValueError("At least two dim values is needed for defining the model...")

    if len(dims) > 2:
        warnings.warn("Regular SVMs should have only two layers...")

    layers = []
    for m, n in zip(dims, dims[1:]):
        layers.append(
            svm_layer(
                m=m, n=n,
                enable_bias=enable_bias,
                kernel=kernel,
                base_range=base_range,
                num_interval=num_interval,
                epsilon=epsilon,
                with_lorr=with_lorr, r=r,
                with_residual=with_residual,
                device=device,
                channel_num=channel_num,
                width=width,
            )
        )
    super().__init__(name=name, layers=layers, device=device, *args, **kwargs)