Skip to content

numerical_kernel_based_interdependence

Bases: interdependence

A numerical kernel-based interdependence function.

This class computes the interdependence matrix using a specified numerical kernel function.

Notes

Formally, given a data batch \(\mathbf{X} \in R^{b \times m}\), we define the numerical metric-based interdependence function as: $$ \begin{equation} \xi(\mathbf{X}) = \mathbf{A} \in R^{m \times m'} \text{, where } \mathbf{A}(i, j) = \text{kernel} \left(\mathbf{X}(:, i), \mathbf{X}(:, j)\right). \end{equation} $$ By convention, the resulting matrix \(\mathbf{A}\) is square, with dimensions \(m' = m\).

Attributes:

Name Type Description
kernel Callable

The kernel function used to compute the interdependence matrix.

Methods:

Name Description
__init__

Initializes the kernel-based interdependence function.
calculate_A

Computes the interdependence matrix using the specified kernel.
Source code in tinybig/interdependence/numerical_kernel_interdependence.py 
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
class numerical_kernel_based_interdependence(interdependence):
    r"""
        A numerical kernel-based interdependence function.

        This class computes the interdependence matrix using a specified numerical kernel function.

        Notes
        ----------
        Formally, given a data batch $\mathbf{X} \in R^{b \times m}$, we define the numerical metric-based interdependence function as:
        $$
            \begin{equation}
            \xi(\mathbf{X}) = \mathbf{A} \in R^{m \times m'} \text{, where } \mathbf{A}(i, j) = \text{kernel} \left(\mathbf{X}(:, i), \mathbf{X}(:, j)\right).
            \end{equation}
        $$
        By convention, the resulting matrix $\mathbf{A}$ is square, with dimensions $m' = m$.


        Attributes
        ----------
        kernel : Callable
            The kernel function used to compute the interdependence matrix.

        Methods
        -------
        __init__(...)
            Initializes the kernel-based interdependence function.
        calculate_A(x=None, w=None, device='cpu', *args, **kwargs)
            Computes the interdependence matrix using the specified kernel.
    """

    def __init__(
        self,
        b: int, m: int, kernel: Callable,
        interdependence_type: str = 'attribute',
        name: str = 'kernel_based_interdependence',
        require_data: bool = True,
        require_parameters: bool = False,
        device: str = 'cpu', *args, **kwargs
    ):
        """
            Initializes the kernel-based interdependence function.

            Parameters
            ----------
            b : int
                Number of rows in the input tensor.
            m : int
                Number of columns in the input tensor.
            kernel : Callable
                The kernel function used to compute the interdependence matrix.
            interdependence_type : str, optional
                Type of interdependence ('attribute', 'instance', etc.). Defaults to 'attribute'.
            name : str, optional
                Name of the interdependence function. Defaults to 'kernel_based_interdependence'.
            require_data : bool, optional
                If True, requires input data for matrix computation. Defaults to True.
            require_parameters : bool, optional
                If True, requires parameters for matrix computation. Defaults to False.
            device : str, optional
                Device for computation (e.g., 'cpu' or 'cuda'). Defaults to 'cpu'.
            *args : tuple
                Additional positional arguments for the parent class.
            **kwargs : dict
                Additional keyword arguments for the parent class.

            Raises
            ------
            ValueError
                If no kernel function is provided.
        """
        super().__init__(b=b, m=m, name=name, interdependence_type=interdependence_type, require_parameters=require_parameters, require_data=require_data, device=device, *args, **kwargs)

        if kernel is None:
            raise ValueError('the kernel is required for the kernel based interdependence function')
        self.kernel = kernel

    def calculate_A(self, x: torch.Tensor = None, w: torch.nn.Parameter = None, device: str = 'cpu', *args, **kwargs):
        """
            Computes the interdependence matrix using the specified kernel function.

            Parameters
            ----------
            x : torch.Tensor, optional
                Input tensor of shape `(batch_size, num_features)`. Required for computation. Defaults to None.
            w : torch.nn.Parameter, optional
                Parameter tensor. Defaults to None.
            device : str, optional
                Device for computation (e.g., 'cpu' or 'cuda'). Defaults to 'cpu'.
            *args : tuple
                Additional positional arguments for interdependence functions.
            **kwargs : dict
                Additional keyword arguments for interdependence functions.

            Returns
            -------
            torch.Tensor
                The computed interdependence matrix.

            Raises
            ------
            AssertionError
                If `x` is not provided or has an incorrect shape.
            ValueError
                If required data or parameters are missing.
        """
        if not self.require_data and not self.require_parameters and self.A is not None:
            return self.A
        else:
            assert x is not None and x.ndim == 2
            x = self.pre_process(x=x, device=device)
            A = self.kernel(x)
            A = self.post_process(x=A, device=device)

            if self.interdependence_type in ['column', 'right', 'attribute', 'attribute_interdependence']:
                assert A.shape == (self.m, self.calculate_m_prime())
            elif self.interdependence_type in ['row', 'left', 'instance', 'instance_interdependence']:
                assert A.shape == (self.b, self.calculate_b_prime())

            if not self.require_data and not self.require_parameters and self.A is None:
                self.A = A

            return A

__init__(b, m, kernel, interdependence_type='attribute', name='kernel_based_interdependence', require_data=True, require_parameters=False, device='cpu', *args, **kwargs)

Initializes the kernel-based interdependence function.

Parameters:

Name Type Description Default
b int

Number of rows in the input tensor.

 required
m int

Number of columns in the input tensor.

 required
kernel Callable

The kernel function used to compute the interdependence matrix.

 required
interdependence_type str

Type of interdependence ('attribute', 'instance', etc.). Defaults to 'attribute'.

 'attribute'
name str

Name of the interdependence function. Defaults to 'kernel_based_interdependence'.

 'kernel_based_interdependence'
require_data bool

If True, requires input data for matrix computation. Defaults to True.

 True
require_parameters bool

If True, requires parameters for matrix computation. Defaults to False.

 False
device str

Device for computation (e.g., 'cpu' or 'cuda'). Defaults to 'cpu'.

 'cpu'
*args tuple

Additional positional arguments for the parent class.

 ()
**kwargs dict

Additional keyword arguments for the parent class.

 {}

Raises:

Type Description
ValueError

If no kernel function is provided.
Source code in tinybig/interdependence/numerical_kernel_interdependence.py 
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
def __init__(
    self,
    b: int, m: int, kernel: Callable,
    interdependence_type: str = 'attribute',
    name: str = 'kernel_based_interdependence',
    require_data: bool = True,
    require_parameters: bool = False,
    device: str = 'cpu', *args, **kwargs
):
    """
        Initializes the kernel-based interdependence function.

        Parameters
        ----------
        b : int
            Number of rows in the input tensor.
        m : int
            Number of columns in the input tensor.
        kernel : Callable
            The kernel function used to compute the interdependence matrix.
        interdependence_type : str, optional
            Type of interdependence ('attribute', 'instance', etc.). Defaults to 'attribute'.
        name : str, optional
            Name of the interdependence function. Defaults to 'kernel_based_interdependence'.
        require_data : bool, optional
            If True, requires input data for matrix computation. Defaults to True.
        require_parameters : bool, optional
            If True, requires parameters for matrix computation. Defaults to False.
        device : str, optional
            Device for computation (e.g., 'cpu' or 'cuda'). Defaults to 'cpu'.
        *args : tuple
            Additional positional arguments for the parent class.
        **kwargs : dict
            Additional keyword arguments for the parent class.

        Raises
        ------
        ValueError
            If no kernel function is provided.
    """
    super().__init__(b=b, m=m, name=name, interdependence_type=interdependence_type, require_parameters=require_parameters, require_data=require_data, device=device, *args, **kwargs)

    if kernel is None:
        raise ValueError('the kernel is required for the kernel based interdependence function')
    self.kernel = kernel

calculate_A(x=None, w=None, device='cpu', *args, **kwargs)

Computes the interdependence matrix using the specified kernel function.

Parameters:

Name Type Description Default
x Tensor

Input tensor of shape (batch_size, num_features). Required for computation. Defaults to None.

 None
w Parameter

Parameter tensor. Defaults to None.

 None
device str

Device for computation (e.g., 'cpu' or 'cuda'). Defaults to 'cpu'.

 'cpu'
*args tuple

Additional positional arguments for interdependence functions.

 ()
**kwargs dict

Additional keyword arguments for interdependence functions.

 {}

Returns:

Type Description
Tensor

The computed interdependence matrix.

Raises:

Type Description
AssertionError

If x is not provided or has an incorrect shape.
ValueError

If required data or parameters are missing.
Source code in tinybig/interdependence/numerical_kernel_interdependence.py 
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
def calculate_A(self, x: torch.Tensor = None, w: torch.nn.Parameter = None, device: str = 'cpu', *args, **kwargs):
    """
        Computes the interdependence matrix using the specified kernel function.

        Parameters
        ----------
        x : torch.Tensor, optional
            Input tensor of shape `(batch_size, num_features)`. Required for computation. Defaults to None.
        w : torch.nn.Parameter, optional
            Parameter tensor. Defaults to None.
        device : str, optional
            Device for computation (e.g., 'cpu' or 'cuda'). Defaults to 'cpu'.
        *args : tuple
            Additional positional arguments for interdependence functions.
        **kwargs : dict
            Additional keyword arguments for interdependence functions.

        Returns
        -------
        torch.Tensor
            The computed interdependence matrix.

        Raises
        ------
        AssertionError
            If `x` is not provided or has an incorrect shape.
        ValueError
            If required data or parameters are missing.
    """
    if not self.require_data and not self.require_parameters and self.A is not None:
        return self.A
    else:
        assert x is not None and x.ndim == 2
        x = self.pre_process(x=x, device=device)
        A = self.kernel(x)
        A = self.post_process(x=A, device=device)

        if self.interdependence_type in ['column', 'right', 'attribute', 'attribute_interdependence']:
            assert A.shape == (self.m, self.calculate_m_prime())
        elif self.interdependence_type in ['row', 'left', 'instance', 'instance_interdependence']:
            assert A.shape == (self.b, self.calculate_b_prime())

        if not self.require_data and not self.require_parameters and self.A is None:
            self.A = A

        return A