Skip to content

feature_selection

Bases: object

Base class for feature selection.

This class provides an abstract base for implementing feature selection algorithms. It supports incremental feature selection with stopping criteria based on thresholds.

Attributes:

Name Type Description
name str

The name of the feature selection method.
n_feature int

The number of features to select.
incremental bool

Whether to perform incremental feature selection.
incremental_stop_threshold float

The threshold for stopping incremental feature selection.
t_threshold int

A time or iteration threshold for incremental selection.

Methods:

Name Description
get_n_feature

Get the number of features to select.
set_n_feature

Set the number of features to select.
forward

Apply feature selection and return the reduced features.
fit_transform

Fit the feature selection model and return the reduced features.
fit

Abstract method to fit the feature selection model to the input data.
transform

Abstract method to transform the input data using the fitted model.
Source code in tinybig/koala/machine_learning/feature_selection/feature_selection.py 
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
class feature_selection(object):
    """
        Base class for feature selection.

        This class provides an abstract base for implementing feature selection algorithms. It supports
        incremental feature selection with stopping criteria based on thresholds.

        Attributes
        ----------
        name : str
            The name of the feature selection method.
        n_feature : int
            The number of features to select.
        incremental : bool
            Whether to perform incremental feature selection.
        incremental_stop_threshold : float
            The threshold for stopping incremental feature selection.
        t_threshold : int
            A time or iteration threshold for incremental selection.

        Methods
        -------
        get_n_feature()
            Get the number of features to select.
        set_n_feature(n_feature)
            Set the number of features to select.
        forward(X: Union[np.ndarray, torch.Tensor], device: str = 'cpu', *args, **kwargs)
            Apply feature selection and return the reduced features.
        fit_transform(X: Union[np.ndarray, torch.Tensor], device: str = 'cpu', *args, **kwargs)
            Fit the feature selection model and return the reduced features.
        fit(X: Union[np.ndarray, torch.Tensor], device: str = 'cpu', *args, **kwargs)
            Abstract method to fit the feature selection model to the input data.
        transform(X: Union[np.ndarray, torch.Tensor], device: str = 'cpu', *args, **kwargs)
            Abstract method to transform the input data using the fitted model.
    """

    def __init__(self, name: str = 'feature_selection', n_feature: int = None,  incremental: bool = True, incremental_stop_threshold: float = 0.01, t_threshold: int = 100, *args, **kwargs):
        """
            Initialize the feature selection class.

            Parameters
            ----------
            name : str, optional
                The name of the feature selection method. Default is 'feature_selection'.
            n_feature : int, optional
                The number of features to select. Default is None.
            incremental : bool, optional
                Whether to perform incremental feature selection. Default is True.
            incremental_stop_threshold : float, optional
                The threshold for stopping incremental feature selection. Default is 0.01.
            t_threshold : int, optional
                A time or iteration threshold for incremental selection. Default is 100.
            *args, **kwargs
                Additional arguments for customization.
        """

        self.name = name
        self.n_feature = n_feature
        self.incremental = incremental
        self.incremental_stop_threshold = incremental_stop_threshold
        self.t_threshold = t_threshold

    def get_n_feature(self):
        """
            Get the number of features to select.

            Returns
            -------
            int
                The number of features to select.
        """
        return self.n_feature

    def set_n_feature(self, n_feature):
        """
            Get the number of features to select.

            Returns
            -------
            int
                The number of features to select.
        """
        self.n_feature = n_feature

    def __call__(self, X: Union[np.ndarray, torch.Tensor], device: str = 'cup', *args, **kwargs):
        """
            Apply the feature selection model.

            Parameters
            ----------
            X : Union[np.ndarray, torch.Tensor]
                The input data for feature selection.
            device : str, optional
                The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.
            *args, **kwargs
                Additional arguments for the feature selection process.

            Returns
            -------
            Union[np.ndarray, torch.Tensor]
                The reduced features after selection.
        """
        return self.forward(X=X, device=device, *args, **kwargs)

    def forward(self, X: Union[np.ndarray, torch.Tensor], device: str = 'cup', *args, **kwargs):
        """
            Apply feature selection and return the reduced features.

            Parameters
            ----------
            X : Union[np.ndarray, torch.Tensor]
                The input data for feature selection.
            device : str, optional
                The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.
            *args, **kwargs
                Additional arguments for the feature selection process.

            Returns
            -------
            Union[np.ndarray, torch.Tensor]
                The reduced features after selection.
        """
        if isinstance(X, torch.Tensor):
            input_X = X.detach().cpu().numpy()
        else:
            input_X = X

        X_reduced = self.fit_transform(X=input_X, device=device, *args, **kwargs)

        return torch.tensor(X_reduced) if isinstance(X, torch.Tensor) and not isinstance(X_reduced, torch.Tensor) else X_reduced

    def fit_transform(self, X: Union[np.ndarray, torch.Tensor], device: str = 'cup', *args, **kwargs):
        """
            Fit the feature selection model and return the reduced features.

            Parameters
            ----------
            X : Union[np.ndarray, torch.Tensor]
                The input data for fitting and transformation.
            device : str, optional
                The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.
            *args, **kwargs
                Additional arguments for the feature selection process.

            Returns
            -------
            Union[np.ndarray, torch.Tensor]
                The reduced features after fitting and transformation.
        """
        if isinstance(X, torch.Tensor):
            input_X = X.detach().cpu().numpy()
        else:
            input_X = X

        self.fit(X=input_X, device=device, *args, **kwargs)
        X_reduced = self.transform(X=input_X, device=device, *args, **kwargs)

        return torch.tensor(X_reduced) if isinstance(X, torch.Tensor) and not isinstance(X_reduced, torch.Tensor) else X_reduced

    @abstractmethod
    def fit(self, X: Union[np.ndarray, torch.Tensor], device: str = 'cup', *args, **kwargs):
        """
            Fit the feature selection model to the input data.

            Parameters
            ----------
            X : Union[np.ndarray, torch.Tensor]
                The input data for fitting.
            device : str, optional
                The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.
            *args, **kwargs
                Additional arguments for fitting the feature selection model.
        """
        pass

    @abstractmethod
    def transform(self, X: Union[np.ndarray, torch.Tensor], device: str = 'cup', *args, **kwargs):
        """
            Transform the input data using the fitted feature selection model.

            Parameters
            ----------
            X : Union[np.ndarray, torch.Tensor]
                The input data to transform.
            device : str, optional
                The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.
            *args, **kwargs
                Additional arguments for the transformation process.
        """
        pass

__call__(X, device='cup', *args, **kwargs)

Apply the feature selection model.

Parameters:

Name Type Description Default
X Union[ndarray, Tensor]

The input data for feature selection.

 required
device str

The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.

 'cup'
*args

Additional arguments for the feature selection process.

 ()
**kwargs

Additional arguments for the feature selection process.

 ()

Returns:

Type Description
Union[ndarray, Tensor]

The reduced features after selection.
Source code in tinybig/koala/machine_learning/feature_selection/feature_selection.py 
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
def __call__(self, X: Union[np.ndarray, torch.Tensor], device: str = 'cup', *args, **kwargs):
    """
        Apply the feature selection model.

        Parameters
        ----------
        X : Union[np.ndarray, torch.Tensor]
            The input data for feature selection.
        device : str, optional
            The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.
        *args, **kwargs
            Additional arguments for the feature selection process.

        Returns
        -------
        Union[np.ndarray, torch.Tensor]
            The reduced features after selection.
    """
    return self.forward(X=X, device=device, *args, **kwargs)

__init__(name='feature_selection', n_feature=None, incremental=True, incremental_stop_threshold=0.01, t_threshold=100, *args, **kwargs)

Initialize the feature selection class.

Parameters:

Name Type Description Default
name str

The name of the feature selection method. Default is 'feature_selection'.

 'feature_selection'
n_feature int

The number of features to select. Default is None.

 None
incremental bool

Whether to perform incremental feature selection. Default is True.

 True
incremental_stop_threshold float

The threshold for stopping incremental feature selection. Default is 0.01.

 0.01
t_threshold int

A time or iteration threshold for incremental selection. Default is 100.

 100
*args

Additional arguments for customization.

 ()
**kwargs

Additional arguments for customization.

 ()
Source code in tinybig/koala/machine_learning/feature_selection/feature_selection.py 
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
def __init__(self, name: str = 'feature_selection', n_feature: int = None,  incremental: bool = True, incremental_stop_threshold: float = 0.01, t_threshold: int = 100, *args, **kwargs):
    """
        Initialize the feature selection class.

        Parameters
        ----------
        name : str, optional
            The name of the feature selection method. Default is 'feature_selection'.
        n_feature : int, optional
            The number of features to select. Default is None.
        incremental : bool, optional
            Whether to perform incremental feature selection. Default is True.
        incremental_stop_threshold : float, optional
            The threshold for stopping incremental feature selection. Default is 0.01.
        t_threshold : int, optional
            A time or iteration threshold for incremental selection. Default is 100.
        *args, **kwargs
            Additional arguments for customization.
    """

    self.name = name
    self.n_feature = n_feature
    self.incremental = incremental
    self.incremental_stop_threshold = incremental_stop_threshold
    self.t_threshold = t_threshold

fit(X, device='cup', *args, **kwargs) abstractmethod

Fit the feature selection model to the input data.

Parameters:

Name Type Description Default
X Union[ndarray, Tensor]

The input data for fitting.

 required
device str

The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.

 'cup'
*args

Additional arguments for fitting the feature selection model.

 ()
**kwargs

Additional arguments for fitting the feature selection model.

 ()
Source code in tinybig/koala/machine_learning/feature_selection/feature_selection.py 
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
@abstractmethod
def fit(self, X: Union[np.ndarray, torch.Tensor], device: str = 'cup', *args, **kwargs):
    """
        Fit the feature selection model to the input data.

        Parameters
        ----------
        X : Union[np.ndarray, torch.Tensor]
            The input data for fitting.
        device : str, optional
            The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.
        *args, **kwargs
            Additional arguments for fitting the feature selection model.
    """
    pass

fit_transform(X, device='cup', *args, **kwargs)

Fit the feature selection model and return the reduced features.

Parameters:

Name Type Description Default
X Union[ndarray, Tensor]

The input data for fitting and transformation.

 required
device str

The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.

 'cup'
*args

Additional arguments for the feature selection process.

 ()
**kwargs

Additional arguments for the feature selection process.

 ()

Returns:

Type Description
Union[ndarray, Tensor]

The reduced features after fitting and transformation.
Source code in tinybig/koala/machine_learning/feature_selection/feature_selection.py 
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
def fit_transform(self, X: Union[np.ndarray, torch.Tensor], device: str = 'cup', *args, **kwargs):
    """
        Fit the feature selection model and return the reduced features.

        Parameters
        ----------
        X : Union[np.ndarray, torch.Tensor]
            The input data for fitting and transformation.
        device : str, optional
            The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.
        *args, **kwargs
            Additional arguments for the feature selection process.

        Returns
        -------
        Union[np.ndarray, torch.Tensor]
            The reduced features after fitting and transformation.
    """
    if isinstance(X, torch.Tensor):
        input_X = X.detach().cpu().numpy()
    else:
        input_X = X

    self.fit(X=input_X, device=device, *args, **kwargs)
    X_reduced = self.transform(X=input_X, device=device, *args, **kwargs)

    return torch.tensor(X_reduced) if isinstance(X, torch.Tensor) and not isinstance(X_reduced, torch.Tensor) else X_reduced

forward(X, device='cup', *args, **kwargs)

Apply feature selection and return the reduced features.

Parameters:

Name Type Description Default
X Union[ndarray, Tensor]

The input data for feature selection.

 required
device str

The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.

 'cup'
*args

Additional arguments for the feature selection process.

 ()
**kwargs

Additional arguments for the feature selection process.

 ()

Returns:

Type Description
Union[ndarray, Tensor]

The reduced features after selection.
Source code in tinybig/koala/machine_learning/feature_selection/feature_selection.py 
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
def forward(self, X: Union[np.ndarray, torch.Tensor], device: str = 'cup', *args, **kwargs):
    """
        Apply feature selection and return the reduced features.

        Parameters
        ----------
        X : Union[np.ndarray, torch.Tensor]
            The input data for feature selection.
        device : str, optional
            The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.
        *args, **kwargs
            Additional arguments for the feature selection process.

        Returns
        -------
        Union[np.ndarray, torch.Tensor]
            The reduced features after selection.
    """
    if isinstance(X, torch.Tensor):
        input_X = X.detach().cpu().numpy()
    else:
        input_X = X

    X_reduced = self.fit_transform(X=input_X, device=device, *args, **kwargs)

    return torch.tensor(X_reduced) if isinstance(X, torch.Tensor) and not isinstance(X_reduced, torch.Tensor) else X_reduced

get_n_feature()

Get the number of features to select.

Returns:

Type Description
int

The number of features to select.
Source code in tinybig/koala/machine_learning/feature_selection/feature_selection.py 
80
81
82
83
84
85
86
87
88
89
def get_n_feature(self):
    """
        Get the number of features to select.

        Returns
        -------
        int
            The number of features to select.
    """
    return self.n_feature

set_n_feature(n_feature)

Get the number of features to select.

Returns:

Type Description
int

The number of features to select.
Source code in tinybig/koala/machine_learning/feature_selection/feature_selection.py 
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
def set_n_feature(self, n_feature):
    """
        Get the number of features to select.

        Returns
        -------
        int
            The number of features to select.
    """
    self.n_feature = n_feature

transform(X, device='cup', *args, **kwargs) abstractmethod

Transform the input data using the fitted feature selection model.

Parameters:

Name Type Description Default
X Union[ndarray, Tensor]

The input data to transform.

 required
device str

The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.

 'cup'
*args

Additional arguments for the transformation process.

 ()
**kwargs

Additional arguments for the transformation process.

 ()
Source code in tinybig/koala/machine_learning/feature_selection/feature_selection.py 
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
@abstractmethod
def transform(self, X: Union[np.ndarray, torch.Tensor], device: str = 'cup', *args, **kwargs):
    """
        Transform the input data using the fitted feature selection model.

        Parameters
        ----------
        X : Union[np.ndarray, torch.Tensor]
            The input data to transform.
        device : str, optional
            The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.
        *args, **kwargs
            Additional arguments for the transformation process.
    """
    pass