Skip to content

chain_interdependence

Bases: interdependence

A chain-based interdependence function.

This class computes the interdependence matrix using a chain structure.

Notes

Formally, given a data instance \(\mathbf{x} \in R^{m}\) with sequential chain-structured interdependence relationships among the attributes of length \(m\), we can define the corresponding unidirectional chain interdependence function as follows:

\[
    \begin{equation}
    \xi(\mathbf{x}) = \mathbf{A}  \in R^{m \times m'},
    \end{equation}
\]

where \(\mathbf{A}\) is the composed attribute interdependence matrix. By default, the output dimension \(m'\) equals the input instance dimension, {\ie} \(m' = m\).

In many cases, we sum this interdependence matrix with an identity matrix to denote self-dependency:

\[
    \begin{equation}
    \xi(\mathbf{x}) = \mathbf{A} + \mathbf{I}  \in R^{m \times m}.
    \end{equation}
\]

Here, \(\mathbf{I} \in R^{m \times m}\) is a square diagonal identity matrix of size \(m \times m\), allowing the function to model both interdependence and self-dependence with a single dependency function. This self-dependence can also be defined using the linear remainder term in {\our}, both of which contribute to defining sequential interdependence relationships.

Attributes:

Name Type Description
chain chain

The chain structure representing the interdependence.
normalization bool

Whether to normalize the interdependence matrix.
normalization_mode str

The mode of normalization ('row', 'column', etc.).
self_dependence bool

Whether nodes are self-dependent.
self_scaling float

Scaling factor for self-dependence.

Methods:

Name Description
__init__

Initializes the chain-based interdependence function.
is_bi_directional

Checks if the chain is bidirectional.
calculate_A

Computes the interdependence matrix.
Source code in tinybig/interdependence/topological_interdependence.py 
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
class chain_interdependence(interdependence):
    r"""
        A chain-based interdependence function.

        This class computes the interdependence matrix using a chain structure.

        Notes
        ----------
        Formally, given a data instance $\mathbf{x} \in R^{m}$ with sequential chain-structured interdependence relationships among the attributes of length $m$,
        we can define the corresponding unidirectional chain interdependence function as follows:

        $$
            \begin{equation}
            \xi(\mathbf{x}) = \mathbf{A}  \in R^{m \times m'},
            \end{equation}
        $$

        where $\mathbf{A}$ is the composed attribute interdependence matrix. By default, the output dimension $m'$ equals the input instance dimension, {\ie} $m' = m$.

        In many cases, we sum this interdependence matrix with an identity matrix to denote self-dependency:

        $$
            \begin{equation}
            \xi(\mathbf{x}) = \mathbf{A} + \mathbf{I}  \in R^{m \times m}.
            \end{equation}
        $$

        Here, $\mathbf{I} \in R^{m \times m}$ is a square diagonal identity matrix of size $m \times m$, allowing the function to model both interdependence and self-dependence with a single dependency function. This self-dependence can also be defined using the linear remainder term in {\our}, both of which contribute to defining sequential interdependence relationships.


        Attributes
        ----------
        chain : chain_structure
            The chain structure representing the interdependence.
        normalization : bool
            Whether to normalize the interdependence matrix.
        normalization_mode : str
            The mode of normalization ('row', 'column', etc.).
        self_dependence : bool
            Whether nodes are self-dependent.
        self_scaling : float
            Scaling factor for self-dependence.

        Methods
        -------
        __init__(...)
            Initializes the chain-based interdependence function.
        is_bi_directional()
            Checks if the chain is bidirectional.
        calculate_A(...)
            Computes the interdependence matrix.
    """
    def __init__(
        self,
        b: int, m: int,
        interdependence_type: str = 'instance',
        name: str = 'chain_interdependence',
        chain: chain_structure = None,
        chain_length: int = None, bi_directional: bool = False,
        normalization: bool = False, normalization_mode: str = 'row',
        self_dependence: bool = True, self_scaling: float = 1.0,
        require_data: bool = False, require_parameters: bool = False,
        device: str = 'cpu', *args, **kwargs
    ):
        """
            Initializes the chain-based interdependence function.

            Parameters
            ----------
            b : int
                Number of rows in the input tensor.
            m : int
                Number of columns in the input tensor.
            interdependence_type : str, optional
                Type of interdependence ('instance', 'attribute', etc.). Defaults to 'instance'.
            name : str, optional
                Name of the interdependence function. Defaults to 'chain_interdependence'.
            chain : chain_structure, optional
                Predefined chain structure. Defaults to None.
            chain_length : int, optional
                Length of the chain structure. Required if `chain` is not provided. Defaults to None.
            bi_directional : bool, optional
                Whether the chain is bidirectional. Defaults to False.
            normalization : bool, optional
                Whether to normalize the interdependence matrix. Defaults to False.
            normalization_mode : str, optional
                The mode of normalization ('row', 'column', etc.). Defaults to 'row'.
            self_dependence : bool, optional
                Whether nodes are self-dependent. Defaults to True.
            self_scaling : float, optional
                Scaling factor for self-dependence. Defaults to 1.0.
            require_data : bool, optional
                Whether the interdependence function requires data. Defaults to False.
            require_parameters : bool, optional
                Whether the interdependence function requires parameters. Defaults to False.
            device : str, optional
                Device for computation ('cpu', 'cuda'). Defaults to 'cpu'.
            *args : tuple
                Additional positional arguments for the parent class.
            **kwargs : dict
                Additional keyword arguments for the parent class.

            Raises
            ------
            ValueError
                If neither `chain` nor `chain_length` is provided.
        """
        super().__init__(b=b, m=m, name=name, interdependence_type=interdependence_type, require_data=require_data, require_parameters=require_parameters, device=device, *args, **kwargs)

        if chain is not None:
            self.chain = chain
        elif chain_length is not None:
            self.chain = chain_structure(length=chain_length, bi_directional=bi_directional)
        else:
            raise ValueError('Either chain structure of chain length must be provided...')

        self.node_id_index_map = None
        self.node_index_id_map = None

        self.normalization = normalization
        self.normalization_mode = normalization_mode
        self.self_dependence = self_dependence
        self.self_scaling = self_scaling

    def is_bi_directional(self):
        """
            Checks if the chain is bidirectional.

            Returns
            -------
            bool
                True if the chain is bidirectional, False otherwise.
        """
        return not self.chain.is_directed()

    def calculate_A(self, x: torch.Tensor = None, w: torch.nn.Parameter = None, device: str = 'cpu', *args, **kwargs):
        """
            Computes the interdependence matrix using the chain structure.

            Parameters
            ----------
            x : torch.Tensor, optional
                Input tensor of shape `(batch_size, num_features)`. Defaults to None.
            w : torch.nn.Parameter, optional
                Parameter tensor. Defaults to None.
            device : str, optional
                Device for computation ('cpu', 'cuda'). Defaults to 'cpu'.
            *args : tuple
                Additional positional arguments.
            **kwargs : dict
                Additional keyword arguments.

            Returns
            -------
            torch.Tensor
                The computed interdependence matrix.

            Raises
            ------
            AssertionError
                If the computed matrix shape is invalid.
        """
        if not self.require_data and not self.require_parameters and self.A is not None:
            return self.A
        else:
            adj, mappings = self.chain.to_matrix(
                self_dependence=self.self_dependence,
                self_scaling=self.self_scaling,
                normalization=self.normalization,
                normalization_mode=self.normalization_mode,
                device=device
            )

            self.node_id_index_map = mappings['node_id_index_map']
            self.node_index_id_map = mappings['node_index_id_map']

            A = self.post_process(x=adj, device=device)

            if self.interdependence_type in ['column', 'right', 'attribute', 'attribute_interdependence']:
                assert A.shape == (self.m, self.calculate_m_prime())
            elif self.interdependence_type in ['row', 'left', 'instance', 'instance_interdependence']:
                assert A.shape == (self.b, self.calculate_b_prime())

            if not self.require_data and not self.require_parameters and self.A is None:
                self.A = A
            return A

__init__(b, m, interdependence_type='instance', name='chain_interdependence', chain=None, chain_length=None, bi_directional=False, normalization=False, normalization_mode='row', self_dependence=True, self_scaling=1.0, require_data=False, require_parameters=False, device='cpu', *args, **kwargs)

Initializes the chain-based interdependence function.

Parameters:

Name Type Description Default
b int

Number of rows in the input tensor.

 required
m int

Number of columns in the input tensor.

 required
interdependence_type str

Type of interdependence ('instance', 'attribute', etc.). Defaults to 'instance'.

 'instance'
name str

Name of the interdependence function. Defaults to 'chain_interdependence'.

 'chain_interdependence'
chain chain

Predefined chain structure. Defaults to None.

 None
chain_length int

Length of the chain structure. Required if chain is not provided. Defaults to None.

 None
bi_directional bool

Whether the chain is bidirectional. Defaults to False.

 False
normalization bool

Whether to normalize the interdependence matrix. Defaults to False.

 False
normalization_mode str

The mode of normalization ('row', 'column', etc.). Defaults to 'row'.

 'row'
self_dependence bool

Whether nodes are self-dependent. Defaults to True.

 True
self_scaling float

Scaling factor for self-dependence. Defaults to 1.0.

 1.0
require_data bool

Whether the interdependence function requires data. Defaults to False.

 False
require_parameters bool

Whether the interdependence function requires parameters. Defaults to False.

 False
device str

Device for computation ('cpu', 'cuda'). Defaults to 'cpu'.

 'cpu'
*args tuple

Additional positional arguments for the parent class.

 ()
**kwargs dict

Additional keyword arguments for the parent class.

 {}

Raises:

Type Description
ValueError

If neither chain nor chain_length is provided.
Source code in tinybig/interdependence/topological_interdependence.py 
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
def __init__(
    self,
    b: int, m: int,
    interdependence_type: str = 'instance',
    name: str = 'chain_interdependence',
    chain: chain_structure = None,
    chain_length: int = None, bi_directional: bool = False,
    normalization: bool = False, normalization_mode: str = 'row',
    self_dependence: bool = True, self_scaling: float = 1.0,
    require_data: bool = False, require_parameters: bool = False,
    device: str = 'cpu', *args, **kwargs
):
    """
        Initializes the chain-based interdependence function.

        Parameters
        ----------
        b : int
            Number of rows in the input tensor.
        m : int
            Number of columns in the input tensor.
        interdependence_type : str, optional
            Type of interdependence ('instance', 'attribute', etc.). Defaults to 'instance'.
        name : str, optional
            Name of the interdependence function. Defaults to 'chain_interdependence'.
        chain : chain_structure, optional
            Predefined chain structure. Defaults to None.
        chain_length : int, optional
            Length of the chain structure. Required if `chain` is not provided. Defaults to None.
        bi_directional : bool, optional
            Whether the chain is bidirectional. Defaults to False.
        normalization : bool, optional
            Whether to normalize the interdependence matrix. Defaults to False.
        normalization_mode : str, optional
            The mode of normalization ('row', 'column', etc.). Defaults to 'row'.
        self_dependence : bool, optional
            Whether nodes are self-dependent. Defaults to True.
        self_scaling : float, optional
            Scaling factor for self-dependence. Defaults to 1.0.
        require_data : bool, optional
            Whether the interdependence function requires data. Defaults to False.
        require_parameters : bool, optional
            Whether the interdependence function requires parameters. Defaults to False.
        device : str, optional
            Device for computation ('cpu', 'cuda'). Defaults to 'cpu'.
        *args : tuple
            Additional positional arguments for the parent class.
        **kwargs : dict
            Additional keyword arguments for the parent class.

        Raises
        ------
        ValueError
            If neither `chain` nor `chain_length` is provided.
    """
    super().__init__(b=b, m=m, name=name, interdependence_type=interdependence_type, require_data=require_data, require_parameters=require_parameters, device=device, *args, **kwargs)

    if chain is not None:
        self.chain = chain
    elif chain_length is not None:
        self.chain = chain_structure(length=chain_length, bi_directional=bi_directional)
    else:
        raise ValueError('Either chain structure of chain length must be provided...')

    self.node_id_index_map = None
    self.node_index_id_map = None

    self.normalization = normalization
    self.normalization_mode = normalization_mode
    self.self_dependence = self_dependence
    self.self_scaling = self_scaling

calculate_A(x=None, w=None, device='cpu', *args, **kwargs)

Computes the interdependence matrix using the chain structure.

Parameters:

Name Type Description Default
x Tensor

Input tensor of shape (batch_size, num_features). Defaults to None.

 None
w Parameter

Parameter tensor. Defaults to None.

 None
device str

Device for computation ('cpu', 'cuda'). Defaults to 'cpu'.

 'cpu'
*args tuple

Additional positional arguments.

 ()
**kwargs dict

Additional keyword arguments.

 {}

Returns:

Type Description
Tensor

The computed interdependence matrix.

Raises:

Type Description
AssertionError

If the computed matrix shape is invalid.
Source code in tinybig/interdependence/topological_interdependence.py 
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
def calculate_A(self, x: torch.Tensor = None, w: torch.nn.Parameter = None, device: str = 'cpu', *args, **kwargs):
    """
        Computes the interdependence matrix using the chain structure.

        Parameters
        ----------
        x : torch.Tensor, optional
            Input tensor of shape `(batch_size, num_features)`. Defaults to None.
        w : torch.nn.Parameter, optional
            Parameter tensor. Defaults to None.
        device : str, optional
            Device for computation ('cpu', 'cuda'). Defaults to 'cpu'.
        *args : tuple
            Additional positional arguments.
        **kwargs : dict
            Additional keyword arguments.

        Returns
        -------
        torch.Tensor
            The computed interdependence matrix.

        Raises
        ------
        AssertionError
            If the computed matrix shape is invalid.
    """
    if not self.require_data and not self.require_parameters and self.A is not None:
        return self.A
    else:
        adj, mappings = self.chain.to_matrix(
            self_dependence=self.self_dependence,
            self_scaling=self.self_scaling,
            normalization=self.normalization,
            normalization_mode=self.normalization_mode,
            device=device
        )

        self.node_id_index_map = mappings['node_id_index_map']
        self.node_index_id_map = mappings['node_index_id_map']

        A = self.post_process(x=adj, device=device)

        if self.interdependence_type in ['column', 'right', 'attribute', 'attribute_interdependence']:
            assert A.shape == (self.m, self.calculate_m_prime())
        elif self.interdependence_type in ['row', 'left', 'instance', 'instance_interdependence']:
            assert A.shape == (self.b, self.calculate_b_prime())

        if not self.require_data and not self.require_parameters and self.A is None:
            self.A = A
        return A

is_bi_directional()

Checks if the chain is bidirectional.

Returns:

Type Description
bool

True if the chain is bidirectional, False otherwise.
Source code in tinybig/interdependence/topological_interdependence.py 
158
159
160
161
162
163
164
165
166
167
def is_bi_directional(self):
    """
        Checks if the chain is bidirectional.

        Returns
        -------
        bool
            True if the chain is bidirectional, False otherwise.
    """
    return not self.chain.is_directed()