identity_interdependence

Bases: constant_interdependence

A class for identity interdependence.

This class defines an identity interdependence matrix, preserving the input dimensions.

Notes

Formally, based on the (optional) input data batch \(\mathbf{X} \in {R}^{b \times m}\), we define the one interdependence function as: $$ \begin{equation} \xi(\mathbf{X}) = \mathbf{I} \in {R}^{m \times m}, \text{ or } \xi(\mathbf{X}) = \mathbf{I} \in {R}^{b \times b}, \end{equation} $$ where \(\mathbf{I}\) denotes the identity interdependence matrix.

Methods:

Name	Description
__init__	Initializes the identity interdependence function.

Source code in tinybig/interdependence/basic_interdependence.py

class identity_interdependence(constant_interdependence):
    r"""
        A class for identity interdependence.

        This class defines an identity interdependence matrix, preserving the input dimensions.

        Notes
        -------
        Formally, based on the (optional) input data batch $\mathbf{X} \in {R}^{b \times m}$, we define the one interdependence function as:
        $$
            \begin{equation}
            \xi(\mathbf{X}) = \mathbf{I} \in {R}^{m \times m}, \text{ or } \xi(\mathbf{X}) = \mathbf{I} \in {R}^{b \times b},
            \end{equation}
        $$
        where $\mathbf{I}$ denotes the identity interdependence matrix.

        Methods
        -------
        __init__(b, m, b_prime=None, m_prime=None, ...)
            Initializes the identity interdependence function.
    """
    def __init__(
        self,
        b: int, m: int,
        b_prime: int = None, m_prime: int = None,
        name: str = 'identity_interdependence',
        interdependence_type: str = 'attribute',
        device: str = 'cpu',
        *args, **kwargs
    ):
        """
            Initializes the identity interdependence function.

            This class sets the interdependence matrix to be an identity matrix, preserving the
            dimensions and structure of the input tensor.

            Parameters
            ----------
            b : int
                Number of rows in the input tensor.
            m : int
                Number of columns in the input tensor.
            b_prime : int, optional
                Number of rows in the output tensor for row-based interdependence. Defaults to `b`.
            m_prime : int, optional
                Number of columns in the output tensor for column-based interdependence. Defaults to `m`.
            name : str, optional
                Name of the interdependence function. Defaults to 'identity_interdependence'.
            interdependence_type : str, optional
                Type of interdependence ('attribute', 'instance', etc.). Defaults to 'attribute'.
            device : str, optional
                Device for computation. Defaults to 'cpu'.
            *args : tuple
                Additional positional arguments for the parent `constant_interdependence` class.
            **kwargs : dict
                Additional keyword arguments for the parent `constant_interdependence` class.

            Raises
            ------
            ValueError
                If the interdependence type is not supported.
            Warning
                If `b != b_prime` or `m != m_prime`, indicating that the interdependence is not strictly identity.
        """
        b_prime = b_prime if b_prime is not None else b
        m_prime = m_prime if m_prime is not None else m

        if interdependence_type in ['row', 'left', 'instance', 'instance_interdependence']:
            assert b_prime is not None
            A = torch.eye(b, b_prime, device=device)
            if b != b_prime:
                warnings.warn("b and b_prime are different, this function will change the row dimensions of the inputs and cannot guarantee identity interdependence...")
        elif interdependence_type in ['column', 'right', 'attribute', 'attribute_interdependence']:
            assert m_prime is not None
            A = torch.eye(m, m_prime, device=device)
            if m != m_prime:
                warnings.warn("m and m_prime are different, this function will change the column dimensions of the inputs and cannot guarantee identity interdependence...")

        else:
            raise ValueError(f'Interdependence type {interdependence_type} is not supported')
        super().__init__(b=b, m=m, A=A, name=name, interdependence_type=interdependence_type, device=device, *args, **kwargs)

__init__(b, m, b_prime=None, m_prime=None, name='identity_interdependence', interdependence_type='attribute', device='cpu', *args, **kwargs)

Initializes the identity interdependence function.

This class sets the interdependence matrix to be an identity matrix, preserving the dimensions and structure of the input tensor.

Parameters:

Name	Type	Description	Default
b	int	Number of rows in the input tensor.	required
m	int	Number of columns in the input tensor.	required
b_prime	int	Number of rows in the output tensor for row-based interdependence. Defaults to b.	None
m_prime	int	Number of columns in the output tensor for column-based interdependence. Defaults to m.	None
name	str	Name of the interdependence function. Defaults to 'identity_interdependence'.	'identity_interdependence'
interdependence_type	str	Type of interdependence ('attribute', 'instance', etc.). Defaults to 'attribute'.	'attribute'
device	str	Device for computation. Defaults to 'cpu'.	'cpu'
*args	tuple	Additional positional arguments for the parent constant_interdependence class.	()
**kwargs	dict	Additional keyword arguments for the parent constant_interdependence class.	{}

Raises:

Type	Description
ValueError	If the interdependence type is not supported.
Warning	If b != b_prime or m != m_prime, indicating that the interdependence is not strictly identity.

Source code in tinybig/interdependence/basic_interdependence.py

def __init__(
    self,
    b: int, m: int,
    b_prime: int = None, m_prime: int = None,
    name: str = 'identity_interdependence',
    interdependence_type: str = 'attribute',
    device: str = 'cpu',
    *args, **kwargs
):
    """
        Initializes the identity interdependence function.

        This class sets the interdependence matrix to be an identity matrix, preserving the
        dimensions and structure of the input tensor.

        Parameters
        ----------
        b : int
            Number of rows in the input tensor.
        m : int
            Number of columns in the input tensor.
        b_prime : int, optional
            Number of rows in the output tensor for row-based interdependence. Defaults to `b`.
        m_prime : int, optional
            Number of columns in the output tensor for column-based interdependence. Defaults to `m`.
        name : str, optional
            Name of the interdependence function. Defaults to 'identity_interdependence'.
        interdependence_type : str, optional
            Type of interdependence ('attribute', 'instance', etc.). Defaults to 'attribute'.
        device : str, optional
            Device for computation. Defaults to 'cpu'.
        *args : tuple
            Additional positional arguments for the parent `constant_interdependence` class.
        **kwargs : dict
            Additional keyword arguments for the parent `constant_interdependence` class.

        Raises
        ------
        ValueError
            If the interdependence type is not supported.
        Warning
            If `b != b_prime` or `m != m_prime`, indicating that the interdependence is not strictly identity.
    """
    b_prime = b_prime if b_prime is not None else b
    m_prime = m_prime if m_prime is not None else m

    if interdependence_type in ['row', 'left', 'instance', 'instance_interdependence']:
        assert b_prime is not None
        A = torch.eye(b, b_prime, device=device)
        if b != b_prime:
            warnings.warn("b and b_prime are different, this function will change the row dimensions of the inputs and cannot guarantee identity interdependence...")
    elif interdependence_type in ['column', 'right', 'attribute', 'attribute_interdependence']:
        assert m_prime is not None
        A = torch.eye(m, m_prime, device=device)
        if m != m_prime:
            warnings.warn("m and m_prime are different, this function will change the column dimensions of the inputs and cannot guarantee identity interdependence...")

    else:
        raise ValueError(f'Interdependence type {interdependence_type} is not supported')
    super().__init__(b=b, m=m, A=A, name=name, interdependence_type=interdependence_type, device=device, *args, **kwargs)