combinatorial_probabilistic_compression

Bases: combinatorial_compression

A combinatorial probabilistic compression class for dimensionality reduction.

This class extends combinatorial_compression by enabling probabilistic sampling of feature combinations based on a defined metric or distribution function.

Notes

Based on the combinatorial compression function, a corresponding multivariate distribution can also be applied to compute the log-likelihood of the tuples: $$ \begin{equation} \kappa(\mathbf{x}) = \log P\left( {\kappa(\mathbf{x} | k) \choose d} | \boldsymbol{\theta} \right) \in {R}^d, \end{equation} $$ which reduce the output dimension to be \(\sum_{i=1}^d k \times 1\).

Methods:

Name	Description
__init__	Initializes the combinatorial probabilistic compression instance.

Source code in tinybig/compression/combinatorial_compression.py

class combinatorial_probabilistic_compression(combinatorial_compression):
    r"""
        A combinatorial probabilistic compression class for dimensionality reduction.

        This class extends `combinatorial_compression` by enabling probabilistic sampling
        of feature combinations based on a defined metric or distribution function.

        Notes
        ----------
        Based on the combinatorial compression function, a corresponding multivariate distribution can also be applied to compute the log-likelihood of the tuples:
        $$
            \begin{equation}
            \kappa(\mathbf{x}) = \log P\left( {\kappa(\mathbf{x} | k) \choose d} | \boldsymbol{\theta} \right) \in {R}^d,
            \end{equation}
        $$
        which reduce the output dimension to be $\sum_{i=1}^d k \times 1$.

        Methods
        -------
        __init__(name='combinatorial_probabilistic_compression', d=1, k=1, ...)
            Initializes the combinatorial probabilistic compression instance.
    """
    def __init__(
        self,
        name: str = 'combinatorial_probabilistic_compression',
        d: int = 1, k: int = 1,
        metric: Callable[[torch.Tensor], torch.Tensor] = None,
        with_replacement: bool = False,
        require_normalization: bool = True,
        *args, **kwargs
    ):
        """
            Initializes the combinatorial probabilistic compression instance.

            Parameters
            ----------
            name : str, optional
                Name of the transformation. Defaults to 'combinatorial_probabilistic_compression'.
            d : int, optional
                Maximum order of combinations to generate. Defaults to 1.
            k : int, optional
                Number of combinations to retain per order. Defaults to 1.
            metric : Callable, optional
                Metric function to apply to the input tensor before sampling. Defaults to None.
            with_replacement : bool, optional
                If True, allows combinations to be generated with replacement. Defaults to False.
            require_normalization : bool, optional
                If True, normalizes the input tensor before sampling. Defaults to True.
            *args : tuple
                Additional positional arguments for the parent `combinatorial_compression` class.
            **kwargs : dict
                Additional keyword arguments for the parent `combinatorial_compression` class.
        """
        super().__init__(
            name=name,
            d=d, k=k,
            metric=metric,
            simply_sampling=False,
            log_prob=True,
            with_replacement=with_replacement,
            require_normalization=require_normalization,
            *args, **kwargs
        )

__init__(name='combinatorial_probabilistic_compression', d=1, k=1, metric=None, with_replacement=False, require_normalization=True, *args, **kwargs)

Initializes the combinatorial probabilistic compression instance.

Parameters:

Name	Type	Description	Default
name	str	Name of the transformation. Defaults to 'combinatorial_probabilistic_compression'.	'combinatorial_probabilistic_compression'
d	int	Maximum order of combinations to generate. Defaults to 1.	1
k	int	Number of combinations to retain per order. Defaults to 1.	1
metric	Callable	Metric function to apply to the input tensor before sampling. Defaults to None.	None
with_replacement	bool	If True, allows combinations to be generated with replacement. Defaults to False.	False
require_normalization	bool	If True, normalizes the input tensor before sampling. Defaults to True.	True
*args	tuple	Additional positional arguments for the parent combinatorial_compression class.	()
**kwargs	dict	Additional keyword arguments for the parent combinatorial_compression class.	{}

Source code in tinybig/compression/combinatorial_compression.py

def __init__(
    self,
    name: str = 'combinatorial_probabilistic_compression',
    d: int = 1, k: int = 1,
    metric: Callable[[torch.Tensor], torch.Tensor] = None,
    with_replacement: bool = False,
    require_normalization: bool = True,
    *args, **kwargs
):
    """
        Initializes the combinatorial probabilistic compression instance.

        Parameters
        ----------
        name : str, optional
            Name of the transformation. Defaults to 'combinatorial_probabilistic_compression'.
        d : int, optional
            Maximum order of combinations to generate. Defaults to 1.
        k : int, optional
            Number of combinations to retain per order. Defaults to 1.
        metric : Callable, optional
            Metric function to apply to the input tensor before sampling. Defaults to None.
        with_replacement : bool, optional
            If True, allows combinations to be generated with replacement. Defaults to False.
        require_normalization : bool, optional
            If True, normalizes the input tensor before sampling. Defaults to True.
        *args : tuple
            Additional positional arguments for the parent `combinatorial_compression` class.
        **kwargs : dict
            Additional keyword arguments for the parent `combinatorial_compression` class.
    """
    super().__init__(
        name=name,
        d=d, k=k,
        metric=metric,
        simply_sampling=False,
        log_prob=True,
        with_replacement=with_replacement,
        require_normalization=require_normalization,
        *args, **kwargs
    )