mutual_information_interdependence

Bases: statistical_kernel_based_interdependence

A statistical kernel-based interdependence class using the mutual information kernel.

Notes

Mutual Information based Kernel:

\[
    \begin{equation}
    \text{kernel}(\mathbf{x}, \mathbf{y}) = \frac{1}{2} \log \left( \frac{ det(\Sigma_x) det( \Sigma_y) }{det \left( \Sigma \right)} \right).
    \end{equation}
\]

where \(\Sigma_{x}\) and \(\Sigma_{y}\) denote the variance matrix of \(\mathbf{x}\) and \(\mathbf{y}\) respectively.

Notation \(\Sigma\) is the co-variance matrix of the joint variables \(\mathbf{x}\) and \(\mathbf{y}\), which can be represented as follows:

\[
\begin{equation}
    \Sigma = \begin{bmatrix} \Sigma_x & \Sigma_{x,y} \\ \Sigma_{y,x} & \Sigma_y \end{bmatrix}.
\end{equation}
\]

Attributes:

Name	Type	Description
kernel	Callable	The mutual information kernel function.

Methods:

Name	Description
__init__	Initializes the mutual information interdependence function.

Source code in tinybig/interdependence/statistical_kernel_interdependence.py

class mutual_information_interdependence(statistical_kernel_based_interdependence):
    r"""
        A statistical kernel-based interdependence class using the mutual information kernel.

        Notes
        ----------
        __Mutual Information based Kernel:__

        $$
            \begin{equation}
            \text{kernel}(\mathbf{x}, \mathbf{y}) = \frac{1}{2} \log \left( \frac{ det(\Sigma_x) det( \Sigma_y) }{det \left( \Sigma \right)} \right).
            \end{equation}
        $$

        where $\Sigma_{x}$ and $\Sigma_{y}$ denote the variance matrix of $\mathbf{x}$ and $\mathbf{y}$ respectively.

        Notation $\Sigma$ is the co-variance matrix of the joint variables $\mathbf{x}$ and $\mathbf{y}$, which can be represented as follows:

        $$
        \begin{equation}
            \Sigma = \begin{bmatrix} \Sigma_x & \Sigma_{x,y} \\ \Sigma_{y,x} & \Sigma_y \end{bmatrix}.
        \end{equation}
        $$

        Attributes
        ----------
        kernel : Callable
            The mutual information kernel function.

        Methods
        -------
        __init__(...)
            Initializes the mutual information interdependence function.
    """
    def __init__(self, name: str = 'mutual_information_interdependence', *args, **kwargs):
        """
            Initializes the mutual information interdependence function.

            Parameters
            ----------
            name : str, optional
                Name of the interdependence function. Defaults to 'mutual_information_interdependence'.
            *args : tuple
                Additional positional arguments for the parent class.
            **kwargs : dict
                Additional keyword arguments for the parent class.
        """
        super().__init__(kernel=batch_mutual_information_kernel, name=name, *args, **kwargs)

__init__(name='mutual_information_interdependence', *args, **kwargs)

Initializes the mutual information interdependence function.

Parameters:

Name	Type	Description	Default
name	str	Name of the interdependence function. Defaults to 'mutual_information_interdependence'.	'mutual_information_interdependence'
*args	tuple	Additional positional arguments for the parent class.	()
**kwargs	dict	Additional keyword arguments for the parent class.	{}

Source code in tinybig/interdependence/statistical_kernel_interdependence.py

def __init__(self, name: str = 'mutual_information_interdependence', *args, **kwargs):
    """
        Initializes the mutual information interdependence function.

        Parameters
        ----------
        name : str, optional
            Name of the interdependence function. Defaults to 'mutual_information_interdependence'.
        *args : tuple
            Additional positional arguments for the parent class.
        **kwargs : dict
            Additional keyword arguments for the parent class.
    """
    super().__init__(kernel=batch_mutual_information_kernel, name=name, *args, **kwargs)