parameterized_weighted_summation_fusion

Bases: weighted_summation_fusion

A fusion mechanism that combines inputs using a parameterized weighted summation.

It inherits from the weighted summation fusion class, and the inputs are treated with a learnable parameter denoting their importance weights.

Attributes:

Name	Type	Description
weights	None	Weights are learnable parameters in this class.

Methods:

Name	Description
__init__	Initializes the parameterized weighted summation fusion function.

Source code in tinybig/fusion/basic_fusion.py

class parameterized_weighted_summation_fusion(weighted_summation_fusion):
    """
        A fusion mechanism that combines inputs using a parameterized weighted summation.

        It inherits from the weighted summation fusion class, and the inputs are treated with a learnable parameter denoting their importance weights.

        Attributes
        ----------
        weights : None
            Weights are learnable parameters in this class.

        Methods
        -------
        __init__(...)
            Initializes the parameterized weighted summation fusion function.
    """
    def __init__(self, dims: list[int] | tuple[int] = None, name: str = "parameterized_weighted_summation_fusion", require_parameters: bool = True, *args, **kwargs):
        """
            Initializes the parameterized weighted summation fusion function.

            Parameters
            ----------
            dims : list[int] | tuple[int], optional
                Dimensions of the input tensors. Defaults to None.
            name : str, optional
                Name of the fusion function. Defaults to "parameterized_weighted_summation_fusion".
            require_parameters : bool, optional
                Whether parameters are required. Defaults to True.
            *args : tuple
                Additional positional arguments for the parent class.
            **kwargs : dict
                Additional keyword arguments for the parent class.
        """
        super().__init__(dims=dims, weights=None, name=name, require_parameters=True, *args, **kwargs)

__init__(dims=None, name='parameterized_weighted_summation_fusion', require_parameters=True, *args, **kwargs)

Initializes the parameterized weighted summation fusion function.

Parameters:

Name	Type	Description	Default
dims	list[int] | tuple[int]	Dimensions of the input tensors. Defaults to None.	None
name	str	Name of the fusion function. Defaults to "parameterized_weighted_summation_fusion".	'parameterized_weighted_summation_fusion'
require_parameters	bool	Whether parameters are required. Defaults to True.	True
*args	tuple	Additional positional arguments for the parent class.	()
**kwargs	dict	Additional keyword arguments for the parent class.	{}

Source code in tinybig/fusion/basic_fusion.py

def __init__(self, dims: list[int] | tuple[int] = None, name: str = "parameterized_weighted_summation_fusion", require_parameters: bool = True, *args, **kwargs):
    """
        Initializes the parameterized weighted summation fusion function.

        Parameters
        ----------
        dims : list[int] | tuple[int], optional
            Dimensions of the input tensors. Defaults to None.
        name : str, optional
            Name of the fusion function. Defaults to "parameterized_weighted_summation_fusion".
        require_parameters : bool, optional
            Whether parameters are required. Defaults to True.
        *args : tuple
            Additional positional arguments for the parent class.
        **kwargs : dict
            Additional keyword arguments for the parent class.
    """
    super().__init__(dims=dims, weights=None, name=name, require_parameters=True, *args, **kwargs)