regression_rnn

Bases: rpn

A Recurrent Neural Network (RNN) model for regression tasks, implemented as the RPN model.

Parameters:

Name	Type	Description	Default
chain_length	int	The length of the chain structure used in the RNN layers.	required
dims	list[int] | tuple[int]	A list or tuple of integers representing the dimensions of each layer in the model. Must contain at least two dimensions.	required
name	str	The name of the RNN model. Default is 'rpn_rnn'.	'rpn_rnn'
channel_num	int	The number of channels in each layer. Default is 1.	1
width	int	The number of parallel heads in each layer. Default is 1.	1
bi_directional	bool	If True, enables bidirectional processing in the chain structure. Default is False.	False
with_multihop	bool	If True, enables multi-hop interdependence in the chain structure. Default is False.	False
h	int	The number of hops for multi-hop interdependence. Default is 1.	1
accumulative	bool	If True, accumulates multi-hop dependencies. Default is False.	False
with_inverse_approx	bool	If True, enables inverse approximation for chain interdependence. Default is False.	False
with_exponential_approx	bool	If True, enables exponential approximation for chain interdependence. Default is False.	False
self_dependence	bool	If True, enables self-dependence in the chain structure. Default is True.	True
self_scaling	float	The scaling factor for self-dependence. Default is 1.0.	1.0
with_bspline	bool	If True, enables B-spline expansion for data transformation. Default is False.	False
with_taylor	bool	If True, enables Taylor expansion for data transformation. Default is False.	False
d	int	The degree of Taylor or B-spline expansion. Default is 2.	2
with_hybrid_expansion	bool	If True, enables hybrid data expansion. Default is False.	False
with_dual_lphm	bool	If True, enables dual low-parametric hypermatrix reconciliation. Default is False.	False
with_lorr	bool	If True, enables low-rank parameterized reconciliation. Default is False.	False
r	int	The rank for low-rank parameterized reconciliation. Default is 3.	3
with_residual	bool	If True, enables residual connections in the layers. Default is False.	False
with_dual_lphm_interdependence	bool	If True, enables dual low-parametric hypermatrix interdependence. Default is False.	False
with_lorr_interdependence	bool	If True, enables low-rank interdependence. Default is False.	False
r_interdependence	int	The rank for low-rank interdependence. Default is 3.	3
enable_bias	bool	If True, enables bias in the layers. Default is False.	False
with_batch_norm	bool	If True, applies batch normalization to the layers. Default is False.	False
with_relu	bool	If True, applies ReLU activation to the layers. Default is True.	True
with_softmax	bool	If True, applies Softmax activation to the output layer. Default is True.	True
with_dropout	bool	If True, applies dropout to the layers. Default is False.	False
p	float	Dropout probability. Default is 0.25.	0.25
parameters_init_method	str	The method for parameter initialization. Default is 'xavier_normal'.	'xavier_normal'
device	str	The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.	'cpu'
*args	optional	Additional positional arguments for the rpn superclass.	()
**kwargs	optional	Additional keyword arguments for the rpn superclass.	{}

Raises:

Type	Description
ValueError	If dims has fewer than two dimensions.

Methods:

Name	Description
__init__	Initializes the RNN model.
forward	Performs a forward pass through the RNN model.

Source code in tinybig/model/rpn_regression_rnn.py

class regression_rnn(rpn):
    """
    A Recurrent Neural Network (RNN) model for regression tasks, implemented as the RPN model.

    Parameters
    ----------
    chain_length : int
        The length of the chain structure used in the RNN layers.
    dims : list[int] | tuple[int]
        A list or tuple of integers representing the dimensions of each layer in the model.
        Must contain at least two dimensions.
    name : str, optional
        The name of the RNN model. Default is 'rpn_rnn'.
    channel_num : int, optional
        The number of channels in each layer. Default is 1.
    width : int, optional
        The number of parallel heads in each layer. Default is 1.
    bi_directional : bool, optional
        If True, enables bidirectional processing in the chain structure. Default is False.
    with_multihop : bool, optional
        If True, enables multi-hop interdependence in the chain structure. Default is False.
    h : int, optional
        The number of hops for multi-hop interdependence. Default is 1.
    accumulative : bool, optional
        If True, accumulates multi-hop dependencies. Default is False.
    with_inverse_approx : bool, optional
        If True, enables inverse approximation for chain interdependence. Default is False.
    with_exponential_approx : bool, optional
        If True, enables exponential approximation for chain interdependence. Default is False.
    self_dependence : bool, optional
        If True, enables self-dependence in the chain structure. Default is True.
    self_scaling : float, optional
        The scaling factor for self-dependence. Default is 1.0.
    with_bspline : bool, optional
        If True, enables B-spline expansion for data transformation. Default is False.
    with_taylor : bool, optional
        If True, enables Taylor expansion for data transformation. Default is False.
    d : int, optional
        The degree of Taylor or B-spline expansion. Default is 2.
    with_hybrid_expansion : bool, optional
        If True, enables hybrid data expansion. Default is False.
    with_dual_lphm : bool, optional
        If True, enables dual low-parametric hypermatrix reconciliation. Default is False.
    with_lorr : bool, optional
        If True, enables low-rank parameterized reconciliation. Default is False.
    r : int, optional
        The rank for low-rank parameterized reconciliation. Default is 3.
    with_residual : bool, optional
        If True, enables residual connections in the layers. Default is False.
    with_dual_lphm_interdependence : bool, optional
        If True, enables dual low-parametric hypermatrix interdependence. Default is False.
    with_lorr_interdependence : bool, optional
        If True, enables low-rank interdependence. Default is False.
    r_interdependence : int, optional
        The rank for low-rank interdependence. Default is 3.
    enable_bias : bool, optional
        If True, enables bias in the layers. Default is False.
    with_batch_norm : bool, optional
        If True, applies batch normalization to the layers. Default is False.
    with_relu : bool, optional
        If True, applies ReLU activation to the layers. Default is True.
    with_softmax : bool, optional
        If True, applies Softmax activation to the output layer. Default is True.
    with_dropout : bool, optional
        If True, applies dropout to the layers. Default is False.
    p : float, optional
        Dropout probability. Default is 0.25.
    parameters_init_method : str, optional
        The method for parameter initialization. Default is 'xavier_normal'.
    device : str, optional
        The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.
    *args : optional
        Additional positional arguments for the `rpn` superclass.
    **kwargs : optional
        Additional keyword arguments for the `rpn` superclass.

    Raises
    ------
    ValueError
        If `dims` has fewer than two dimensions.

    Methods
    -------
    __init__(...)
        Initializes the RNN model.
    forward(x, device='cpu', *args, **kwargs)
        Performs a forward pass through the RNN model.

    """
    def __init__(
        self,
        chain_length: int,
        dims: list[int] | tuple[int],
        name: str = 'rpn_rnn',
        channel_num: int = 1,
        width: int = 1,
        # chain structure interdependence function parameters
        bi_directional: bool = False,
        with_multihop: bool = False, h: int = 1, accumulative: bool = False,
        with_inverse_approx: bool = False,
        with_exponential_approx: bool = False,
        self_dependence: bool = True,
        self_scaling: float = 1.0,
        # data expansion function
        with_bspline: bool = False,
        with_taylor: bool = False, d: int = 2,
        with_hybrid_expansion: bool = False,
        # parameter reconciliation function parameters
        with_dual_lphm: bool = False,
        with_lorr: bool = False, r: int = 3,
        with_residual: bool = False,
        # bilinear interdependence function parameters
        with_dual_lphm_interdependence: bool = False,
        with_lorr_interdependence: bool = False, r_interdependence: int = 3,
        # remainder function parameters
        enable_bias: bool = False,
        # output processing parameters
        with_batch_norm: bool = False,
        with_relu: bool = True,
        with_softmax: bool = True,
        with_dropout: bool = False, p: float = 0.25,
        # other parameters
        parameters_init_method: str = 'xavier_normal',
        device: str = 'cpu', *args, **kwargs
    ):
        """
        Initialize the RNN model as a RPN.

        Parameters
        ----------
        chain_length : int
            The length of the chain structure for the RNN layers.
        dims : list[int] | tuple[int]
            A list or tuple of integers representing the dimensions of each layer.
            Must contain at least two dimensions.
        name : str, optional
            The name of the RNN model. Default is 'rpn_rnn'.
        channel_num : int, optional
            The number of channels for each layer. Default is 1.
        width : int, optional
            The number of parallel heads in each layer. Default is 1.
        bi_directional : bool, optional
            If True, enables bidirectional processing in the chain structure. Default is False.
        with_multihop : bool, optional
            If True, enables multi-hop interdependence in the chain structure. Default is False.
        h : int, optional
            Number of hops for multi-hop interdependence. Default is 1.
        accumulative : bool, optional
            If True, accumulates multi-hop dependencies. Default is False.
        with_inverse_approx : bool, optional
            If True, enables inverse approximation for interdependence. Default is False.
        with_exponential_approx : bool, optional
            If True, enables exponential approximation for interdependence. Default is False.
        self_dependence : bool, optional
            If True, enables self-dependence in the chain structure. Default is True.
        self_scaling : float, optional
            Scaling factor for self-dependence. Default is 1.0.
        with_bspline : bool, optional
            If True, enables B-spline expansion for data transformation. Default is False.
        with_taylor : bool, optional
            If True, enables Taylor expansion for data transformation. Default is False.
        d : int, optional
            Degree of the expansion function (B-spline or Taylor). Default is 2.
        with_hybrid_expansion : bool, optional
            If True, enables hybrid data expansion. Default is False.
        with_dual_lphm : bool, optional
            If True, enables dual low-parametric hypermatrix reconciliation. Default is False.
        with_lorr : bool, optional
            If True, enables low-rank parameterized reconciliation. Default is False.
        r : int, optional
            Rank parameter for low-rank reconciliation. Default is 3.
        with_residual : bool, optional
            If True, adds residual connections to the layers. Default is False.
        with_dual_lphm_interdependence : bool, optional
            If True, enables dual low-parametric hypermatrix interdependence. Default is False.
        with_lorr_interdependence : bool, optional
            If True, enables low-rank interdependence. Default is False.
        r_interdependence : int, optional
            Rank for low-rank interdependence. Default is 3.
        enable_bias : bool, optional
            If True, enables bias in the layers. Default is False.
        with_batch_norm : bool, optional
            If True, applies batch normalization to the layers. Default is False.
        with_relu : bool, optional
            If True, applies ReLU activation to the layers. Default is True.
        with_softmax : bool, optional
            If True, applies Softmax activation to the output layer. Default is True.
        with_dropout : bool, optional
            If True, applies dropout to the layers. Default is False.
        p : float, optional
            Dropout probability. Default is 0.25.
        parameters_init_method : str, optional
            Initialization method for the parameters. Default is 'xavier_normal'.
        device : str, optional
            Device to perform computations ('cpu' or 'cuda'). Default is 'cpu'.
        *args : optional
            Additional positional arguments for the superclass.
        **kwargs : optional
            Additional keyword arguments for the superclass.

        Raises
        ------
        ValueError
            If `dims` is empty or contains fewer than two dimensions.
        """
        print('############# rpn-rnn model architecture ############')

        self.chain_length = chain_length

        if dims is None or len(dims) <= 1:
           raise ValueError('dims must not be empty and need to have at least two dimensions...')
        assert all(isinstance(d, int) and d > 0 for d in dims)

        # input embedding layer
        layers = []
        for m, n in zip(dims[0:-2], dims[1:-1]):
            print('m', m, 'n', n)
            #---------------- x to h -----------------
            layers.append(
                perceptron_layer(
                    m=m, n=n,
                    channel_num=channel_num,
                    width=width,
                    # -----------------------
                    with_bspline=with_bspline,
                    with_taylor=with_taylor, d=d,
                    with_hybrid_expansion=with_hybrid_expansion,
                    # -----------------------
                    with_dual_lphm=with_dual_lphm,
                    with_lorr=with_lorr, r=r,
                    enable_bias=enable_bias,
                    with_residual=with_residual,
                    # -----------------------
                    with_batch_norm=False,
                    with_relu=True,
                    with_softmax=False,
                    with_dropout=False, p=p,
                    # -----------------------
                    parameters_init_method=parameters_init_method,
                    device=device,
                )
            )
            # ---------------- h to h -----------------
            layers.append(
                chain_interdependence_layer(
                    m=n, n=n,
                    chain_length=chain_length,
                    channel_num=channel_num,
                    width=width,
                    # -----------------------
                    bi_directional=bi_directional,
                    with_multihop=with_multihop, h=h, accumulative=accumulative,
                    with_inverse_approx=with_inverse_approx,
                    with_exponential_approx=with_exponential_approx,
                    self_dependence=self_dependence,
                    self_scaling=self_scaling,
                    # -----------------------
                    with_dual_lphm=with_dual_lphm,
                    with_lorr=with_lorr, r=r,
                    with_residual=with_residual,
                    # -----------------------
                    with_dual_lphm_interdependence=with_dual_lphm_interdependence,
                    with_lorr_interdependence=with_lorr_interdependence,
                    r_interdependence=r_interdependence,
                    # -----------------------
                    enable_bias=enable_bias,
                    # -----------------------
                    with_batch_norm=with_batch_norm,
                    with_relu=with_relu,
                    with_softmax=False,
                    with_dropout=with_dropout, p=p,
                    # -----------------------
                    parameters_init_method=parameters_init_method,
                    device=device,
                )
            )
        #--------------- output layer: h to y ------------------
        layers.append(
            perceptron_layer(
                name='output_layer',
                m=dims[-2]*chain_length, n=dims[-1],
                channel_num=channel_num,
                width=width,
                # -----------------------
                with_bspline=with_bspline,
                with_taylor=with_taylor, d=d,
                with_hybrid_expansion=with_hybrid_expansion,
                # -----------------------
                with_dual_lphm=with_dual_lphm,
                with_lorr=with_lorr, r=r,
                enable_bias=enable_bias,
                with_residual=with_residual,
                # -----------------------
                with_batch_norm=False,
                with_relu=False,
                with_softmax=with_softmax,
                with_dropout=False, p=p,
                # -----------------------
                parameters_init_method=parameters_init_method,
                device=device,
            )
        )
        super().__init__(name=name, layers=layers, device=device, *args, **kwargs)

    def forward(self, x: torch.Tensor, device='cpu', *args, **kwargs):
        """
        Performs a forward pass through the RNN model.

        Parameters
        ----------
        x : torch.Tensor
            The input tensor of shape (batch_size, input_dim).
        device : str, optional
            The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.
        *args : optional
            Additional positional arguments.
        **kwargs : optional
            Additional keyword arguments.

        Returns
        -------
        torch.Tensor
            The output tensor after processing through the RNN model.
        """
        for layer in self.layers:
            if isinstance(layer, tinybig.layer.perceptron_layer):
                if layer.name is not None and layer.name == 'output_layer':
                    x = x.view(x.size(0), -1)
                    x = layer(x, device=device)
                else:
                    b, m = x.shape
                    x = x.view(b * self.chain_length, -1)
                    x = layer(x, device=device)
                    x = x.view(b, -1)
            else:
                x = layer(x, device=device)
        return x

__init__(chain_length, dims, name='rpn_rnn', channel_num=1, width=1, bi_directional=False, with_multihop=False, h=1, accumulative=False, with_inverse_approx=False, with_exponential_approx=False, self_dependence=True, self_scaling=1.0, with_bspline=False, with_taylor=False, d=2, with_hybrid_expansion=False, with_dual_lphm=False, with_lorr=False, r=3, with_residual=False, with_dual_lphm_interdependence=False, with_lorr_interdependence=False, r_interdependence=3, enable_bias=False, with_batch_norm=False, with_relu=True, with_softmax=True, with_dropout=False, p=0.25, parameters_init_method='xavier_normal', device='cpu', *args, **kwargs)

Initialize the RNN model as a RPN.

Parameters:

Name	Type	Description	Default
chain_length	int	The length of the chain structure for the RNN layers.	required
dims	list[int] | tuple[int]	A list or tuple of integers representing the dimensions of each layer. Must contain at least two dimensions.	required
name	str	The name of the RNN model. Default is 'rpn_rnn'.	'rpn_rnn'
channel_num	int	The number of channels for each layer. Default is 1.	1
width	int	The number of parallel heads in each layer. Default is 1.	1
bi_directional	bool	If True, enables bidirectional processing in the chain structure. Default is False.	False
with_multihop	bool	If True, enables multi-hop interdependence in the chain structure. Default is False.	False
h	int	Number of hops for multi-hop interdependence. Default is 1.	1
accumulative	bool	If True, accumulates multi-hop dependencies. Default is False.	False
with_inverse_approx	bool	If True, enables inverse approximation for interdependence. Default is False.	False
with_exponential_approx	bool	If True, enables exponential approximation for interdependence. Default is False.	False
self_dependence	bool	If True, enables self-dependence in the chain structure. Default is True.	True
self_scaling	float	Scaling factor for self-dependence. Default is 1.0.	1.0
with_bspline	bool	If True, enables B-spline expansion for data transformation. Default is False.	False
with_taylor	bool	If True, enables Taylor expansion for data transformation. Default is False.	False
d	int	Degree of the expansion function (B-spline or Taylor). Default is 2.	2
with_hybrid_expansion	bool	If True, enables hybrid data expansion. Default is False.	False
with_dual_lphm	bool	If True, enables dual low-parametric hypermatrix reconciliation. Default is False.	False
with_lorr	bool	If True, enables low-rank parameterized reconciliation. Default is False.	False
r	int	Rank parameter for low-rank reconciliation. Default is 3.	3
with_residual	bool	If True, adds residual connections to the layers. Default is False.	False
with_dual_lphm_interdependence	bool	If True, enables dual low-parametric hypermatrix interdependence. Default is False.	False
with_lorr_interdependence	bool	If True, enables low-rank interdependence. Default is False.	False
r_interdependence	int	Rank for low-rank interdependence. Default is 3.	3
enable_bias	bool	If True, enables bias in the layers. Default is False.	False
with_batch_norm	bool	If True, applies batch normalization to the layers. Default is False.	False
with_relu	bool	If True, applies ReLU activation to the layers. Default is True.	True
with_softmax	bool	If True, applies Softmax activation to the output layer. Default is True.	True
with_dropout	bool	If True, applies dropout to the layers. Default is False.	False
p	float	Dropout probability. Default is 0.25.	0.25
parameters_init_method	str	Initialization method for the parameters. Default is 'xavier_normal'.	'xavier_normal'
device	str	Device to perform computations ('cpu' or 'cuda'). Default is 'cpu'.	'cpu'
*args	optional	Additional positional arguments for the superclass.	()
**kwargs	optional	Additional keyword arguments for the superclass.	{}

Raises:

Type	Description
ValueError	If dims is empty or contains fewer than two dimensions.

Source code in tinybig/model/rpn_regression_rnn.py

def __init__(
    self,
    chain_length: int,
    dims: list[int] | tuple[int],
    name: str = 'rpn_rnn',
    channel_num: int = 1,
    width: int = 1,
    # chain structure interdependence function parameters
    bi_directional: bool = False,
    with_multihop: bool = False, h: int = 1, accumulative: bool = False,
    with_inverse_approx: bool = False,
    with_exponential_approx: bool = False,
    self_dependence: bool = True,
    self_scaling: float = 1.0,
    # data expansion function
    with_bspline: bool = False,
    with_taylor: bool = False, d: int = 2,
    with_hybrid_expansion: bool = False,
    # parameter reconciliation function parameters
    with_dual_lphm: bool = False,
    with_lorr: bool = False, r: int = 3,
    with_residual: bool = False,
    # bilinear interdependence function parameters
    with_dual_lphm_interdependence: bool = False,
    with_lorr_interdependence: bool = False, r_interdependence: int = 3,
    # remainder function parameters
    enable_bias: bool = False,
    # output processing parameters
    with_batch_norm: bool = False,
    with_relu: bool = True,
    with_softmax: bool = True,
    with_dropout: bool = False, p: float = 0.25,
    # other parameters
    parameters_init_method: str = 'xavier_normal',
    device: str = 'cpu', *args, **kwargs
):
    """
    Initialize the RNN model as a RPN.

    Parameters
    ----------
    chain_length : int
        The length of the chain structure for the RNN layers.
    dims : list[int] | tuple[int]
        A list or tuple of integers representing the dimensions of each layer.
        Must contain at least two dimensions.
    name : str, optional
        The name of the RNN model. Default is 'rpn_rnn'.
    channel_num : int, optional
        The number of channels for each layer. Default is 1.
    width : int, optional
        The number of parallel heads in each layer. Default is 1.
    bi_directional : bool, optional
        If True, enables bidirectional processing in the chain structure. Default is False.
    with_multihop : bool, optional
        If True, enables multi-hop interdependence in the chain structure. Default is False.
    h : int, optional
        Number of hops for multi-hop interdependence. Default is 1.
    accumulative : bool, optional
        If True, accumulates multi-hop dependencies. Default is False.
    with_inverse_approx : bool, optional
        If True, enables inverse approximation for interdependence. Default is False.
    with_exponential_approx : bool, optional
        If True, enables exponential approximation for interdependence. Default is False.
    self_dependence : bool, optional
        If True, enables self-dependence in the chain structure. Default is True.
    self_scaling : float, optional
        Scaling factor for self-dependence. Default is 1.0.
    with_bspline : bool, optional
        If True, enables B-spline expansion for data transformation. Default is False.
    with_taylor : bool, optional
        If True, enables Taylor expansion for data transformation. Default is False.
    d : int, optional
        Degree of the expansion function (B-spline or Taylor). Default is 2.
    with_hybrid_expansion : bool, optional
        If True, enables hybrid data expansion. Default is False.
    with_dual_lphm : bool, optional
        If True, enables dual low-parametric hypermatrix reconciliation. Default is False.
    with_lorr : bool, optional
        If True, enables low-rank parameterized reconciliation. Default is False.
    r : int, optional
        Rank parameter for low-rank reconciliation. Default is 3.
    with_residual : bool, optional
        If True, adds residual connections to the layers. Default is False.
    with_dual_lphm_interdependence : bool, optional
        If True, enables dual low-parametric hypermatrix interdependence. Default is False.
    with_lorr_interdependence : bool, optional
        If True, enables low-rank interdependence. Default is False.
    r_interdependence : int, optional
        Rank for low-rank interdependence. Default is 3.
    enable_bias : bool, optional
        If True, enables bias in the layers. Default is False.
    with_batch_norm : bool, optional
        If True, applies batch normalization to the layers. Default is False.
    with_relu : bool, optional
        If True, applies ReLU activation to the layers. Default is True.
    with_softmax : bool, optional
        If True, applies Softmax activation to the output layer. Default is True.
    with_dropout : bool, optional
        If True, applies dropout to the layers. Default is False.
    p : float, optional
        Dropout probability. Default is 0.25.
    parameters_init_method : str, optional
        Initialization method for the parameters. Default is 'xavier_normal'.
    device : str, optional
        Device to perform computations ('cpu' or 'cuda'). Default is 'cpu'.
    *args : optional
        Additional positional arguments for the superclass.
    **kwargs : optional
        Additional keyword arguments for the superclass.

    Raises
    ------
    ValueError
        If `dims` is empty or contains fewer than two dimensions.
    """
    print('############# rpn-rnn model architecture ############')

    self.chain_length = chain_length

    if dims is None or len(dims) <= 1:
       raise ValueError('dims must not be empty and need to have at least two dimensions...')
    assert all(isinstance(d, int) and d > 0 for d in dims)

    # input embedding layer
    layers = []
    for m, n in zip(dims[0:-2], dims[1:-1]):
        print('m', m, 'n', n)
        #---------------- x to h -----------------
        layers.append(
            perceptron_layer(
                m=m, n=n,
                channel_num=channel_num,
                width=width,
                # -----------------------
                with_bspline=with_bspline,
                with_taylor=with_taylor, d=d,
                with_hybrid_expansion=with_hybrid_expansion,
                # -----------------------
                with_dual_lphm=with_dual_lphm,
                with_lorr=with_lorr, r=r,
                enable_bias=enable_bias,
                with_residual=with_residual,
                # -----------------------
                with_batch_norm=False,
                with_relu=True,
                with_softmax=False,
                with_dropout=False, p=p,
                # -----------------------
                parameters_init_method=parameters_init_method,
                device=device,
            )
        )
        # ---------------- h to h -----------------
        layers.append(
            chain_interdependence_layer(
                m=n, n=n,
                chain_length=chain_length,
                channel_num=channel_num,
                width=width,
                # -----------------------
                bi_directional=bi_directional,
                with_multihop=with_multihop, h=h, accumulative=accumulative,
                with_inverse_approx=with_inverse_approx,
                with_exponential_approx=with_exponential_approx,
                self_dependence=self_dependence,
                self_scaling=self_scaling,
                # -----------------------
                with_dual_lphm=with_dual_lphm,
                with_lorr=with_lorr, r=r,
                with_residual=with_residual,
                # -----------------------
                with_dual_lphm_interdependence=with_dual_lphm_interdependence,
                with_lorr_interdependence=with_lorr_interdependence,
                r_interdependence=r_interdependence,
                # -----------------------
                enable_bias=enable_bias,
                # -----------------------
                with_batch_norm=with_batch_norm,
                with_relu=with_relu,
                with_softmax=False,
                with_dropout=with_dropout, p=p,
                # -----------------------
                parameters_init_method=parameters_init_method,
                device=device,
            )
        )
    #--------------- output layer: h to y ------------------
    layers.append(
        perceptron_layer(
            name='output_layer',
            m=dims[-2]*chain_length, n=dims[-1],
            channel_num=channel_num,
            width=width,
            # -----------------------
            with_bspline=with_bspline,
            with_taylor=with_taylor, d=d,
            with_hybrid_expansion=with_hybrid_expansion,
            # -----------------------
            with_dual_lphm=with_dual_lphm,
            with_lorr=with_lorr, r=r,
            enable_bias=enable_bias,
            with_residual=with_residual,
            # -----------------------
            with_batch_norm=False,
            with_relu=False,
            with_softmax=with_softmax,
            with_dropout=False, p=p,
            # -----------------------
            parameters_init_method=parameters_init_method,
            device=device,
        )
    )
    super().__init__(name=name, layers=layers, device=device, *args, **kwargs)

forward(x, device='cpu', *args, **kwargs)

Performs a forward pass through the RNN model.

Parameters:

Name	Type	Description	Default
x	Tensor	The input tensor of shape (batch_size, input_dim).	required
device	str	The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.	'cpu'
*args	optional	Additional positional arguments.	()
**kwargs	optional	Additional keyword arguments.	{}

Returns:

Type	Description
Tensor	The output tensor after processing through the RNN model.

Source code in tinybig/model/rpn_regression_rnn.py

def forward(self, x: torch.Tensor, device='cpu', *args, **kwargs):
    """
    Performs a forward pass through the RNN model.

    Parameters
    ----------
    x : torch.Tensor
        The input tensor of shape (batch_size, input_dim).
    device : str, optional
        The device to use for computation ('cpu' or 'cuda'). Default is 'cpu'.
    *args : optional
        Additional positional arguments.
    **kwargs : optional
        Additional keyword arguments.

    Returns
    -------
    torch.Tensor
        The output tensor after processing through the RNN model.
    """
    for layer in self.layers:
        if isinstance(layer, tinybig.layer.perceptron_layer):
            if layer.name is not None and layer.name == 'output_layer':
                x = x.view(x.size(0), -1)
                x = layer(x, device=device)
            else:
                b, m = x.shape
                x = x.view(b * self.chain_length, -1)
                x = layer(x, device=device)
                x = x.view(b, -1)
        else:
            x = layer(x, device=device)
    return x