Skip to content

Feynman Function Approximation

Author: Jiawei Zhang
(Released: July 8, 2024; latest Revision: July 8, 2024.)
 Jupyter Logo Yaml Logo Python Logo

In this example, we will build a 3-layer RPN model with extended_expansion, lorr_reconciliation and linear_remainder functions to approximate the feynman_function dataset with formula ID I.9.18 (or with numerical equation_index 4).

This formula describes Newton's law of universal gravitation with \(9\) arguments inputs, it is illustrated as follows: $$ f(G, m_1, m_2, x_1, y_1, z_1, x_2, y_2, z_2) = \frac{G \cdot m_1 \cdot m_2}{(x_2 - x_1)^2+(y_2 - y_1)^2+(z_2 - z_1)^2}, $$ where \(G, m_1, m_2, x_2, y_2, z_2 \in [1, 2]\) and \(x_1, y_1, z_1 \in [3, 4]\).

Python Code and Model Configurations

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
from tinybig.config import rpn_config
from tinybig.util import set_random_seed

print('loading configs...')
config_file_name = 'feynman_configs'
config_obj = rpn_config()
config = config_obj.load_yaml(cache_dir='./configs', config_file=config_file_name + '.yaml')
print(config)

print('setting up environments...')
device = config['configurations'].pop('device')
random_seed = config['configurations'].pop('random_seed')
set_random_seed(random_seed)
print('device: ', device, '; random_seed: ', random_seed)

print('instantiating objects from config...')
object_dict = config_obj.instantiate_object_from_config(config['configurations'])
data_obj, model_obj, learner_obj, metric_obj, result_obj = [object_dict[name] for name in ['data', 'model', 'learner', 'metric', 'result']]
print('parameter num: ', sum([parameter.numel() for parameter in model_obj.parameters()]))

print('loading dataset...')
data_loader = data_obj.load()
print(data_loader['str_equation'])

print('training model...')
training_records = learner_obj.train(model=model_obj, data_loader=data_loader, device=device,
                                     metric=metric_obj, disable_tqdm=True, display_step=100)
model_obj.save_ckpt(cache_dir='./ckpt', checkpoint_file=f'{config_file_name}_checkpoint')

print('testing model...')
test_result = learner_obj.test(model=model_obj, test_loader=data_loader['test_loader'], device=device,
                               metric=metric_obj)

print('evaluating result...')
print(metric_obj.__class__.__name__, metric_obj.evaluate(y_true=test_result['y_true'], y_pred=test_result['y_pred'], y_score=test_result['y_score'], ))

print('saving result...')
result_obj.save(test_result, cache_dir='./result', output_file='{}_result'.format(config_file_name))
result_obj.save(training_records, cache_dir='./result', output_file='{}_record'.format(config_file_name))

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
# three-layer rpn for Newton's law of universal gravitation function approximation
configurations:
  device: cpu
  random_seed: 4567

  data_configs:
    data_class: tinybig.data.feynman_function
    data_parameters:
      name: elementary_function
      train_batch_size: 100
      test_batch_size: 100
      equation_index: 4

  learner_configs:
    learner_class: tinybig.learner.backward_learner
    learner_parameters:
      name: error_backward_propagation
      n_epochs: 2000
      optimizer_configs:
        optimizer_class: torch.optim.AdamW
        optimizer_parameters:
          lr: 3.0e-03
          weight_decay: 1.0e-04
      lr_scheduler_configs:
        lr_scheduler_class: torch.optim.lr_scheduler.ExponentialLR
        lr_scheduler_parameters:
          gamma: 0.999
      loss_configs:
        loss_class: torch.nn.MSELoss
        loss_parameters:
          reduction: mean

  model_configs:
    model_class: tinybig.model.rpn
    model_parameters:
      name: reconciled_polynomial_network
      depth: 3
      depth_alloc: [1, 1, 1]
      layer_configs:
        - layer_class: tinybig.module.rpn_layer
          layer_parameters:
            name: rpn_layer
            m: 9
            n: 2
            width: 1
            width_alloc: [ 1 ]
            fusion_strategy: average
            head_configs:
              - head_class: tinybig.module.rpn_head
                head_parameters:
                  l: null
                  channel_num: 1
                  data_transformation_configs:
                    data_transformation_class: tinybig.expansion.extended_expansion
                    data_transformation_parameters:
                      name: extended_expansion
                      expansion_function_configs:
                        - expansion_class: tinybig.expansion.bspline_expansion
                          expansion_parameters:
                            name: bspline_expansion
                            t: 10
                            d: 4
                        - expansion_class: tinybig.expansion.taylor_expansion
                          expansion_parameters:
                            name: taylor_expansion
                            d: 2
                  parameter_fabrication_configs:
                    parameter_fabrication_class: tinybig.reconciliation.lorr_reconciliation
                    parameter_fabrication_parameters:
                      name: lorr_reconciliation
                      r: 1
                  remainder_configs:
                    remainder_class: tinybig.remainder.linear_remainder
                    remainder_parameters:
                      name: linear_remainder
                      require_parameters: True
                      enable_bias: False
                      activation_functions: torch.nn.functional.silu

        - layer_class: tinybig.module.rpn_layer
          layer_parameters:
            name: rpn_layer
            m: 2
            n: 2
            width: 1
            width_alloc: [1]
            fusion_strategy: average
            head_configs:
              - head_class: tinybig.module.rpn_head
                head_parameters:
                  l: null
                  channel_num: 1
                  data_transformation_configs:
                    data_transformation_class: tinybig.expansion.extended_expansion
                    data_transformation_parameters:
                      name: extended_expansion
                      expansion_function_configs:
                        - expansion_class: tinybig.expansion.bspline_expansion
                          expansion_parameters:
                            name: bspline_expansion
                            t: 10
                            d: 4
                        - expansion_class: tinybig.expansion.taylor_expansion
                          expansion_parameters:
                            name: taylor_expansion
                            d: 2
                  parameter_fabrication_configs:
                    parameter_fabrication_class: tinybig.reconciliation.lorr_reconciliation
                    parameter_fabrication_parameters:
                      name: lorr_reconciliation
                      r: 1
                  remainder_configs:
                    remainder_class: tinybig.remainder.linear_remainder
                    remainder_parameters:
                      name: linear_remainder
                      require_parameters: True
                      enable_bias: False
                      activation_functions: torch.nn.functional.silu

        - layer_class: tinybig.module.rpn_layer
          layer_parameters:
            name: rpn_layer
            m: 2
            n: 1
            width: 1
            width_alloc: [ 1 ]
            fusion_strategy: average
            head_configs:
              - head_class: tinybig.module.rpn_head
                head_parameters:
                  l: null
                  channel_num: 1
                  data_transformation_configs:
                    data_transformation_class: tinybig.expansion.extended_expansion
                    data_transformation_parameters:
                      name: extended_expansion
                      expansion_function_configs:
                        - expansion_class: tinybig.expansion.bspline_expansion
                          expansion_parameters:
                            name: bspline_expansion
                            t: 10
                            d: 4
                        - expansion_class: tinybig.expansion.taylor_expansion
                          expansion_parameters:
                            name: taylor_expansion
                            d: 2
                  parameter_fabrication_configs:
                    parameter_fabrication_class: tinybig.reconciliation.lorr_reconciliation
                    parameter_fabrication_parameters:
                      name: lorr_reconciliation
                      r: 1
                  remainder_configs:
                    remainder_class: tinybig.remainder.linear_remainder
                    remainder_parameters:
                      name: linear_remainder
                      require_parameters: True
                      enable_bias: False
                      activation_functions: torch.nn.functional.silu

  metric_configs:
    metric_class: tinybig.metric.mse
    metric_parameters:
      name: mse

  result_configs:
    result_class: tinybig.output.rpn_output
    result_parameters:
      name: rpn_output
rpn with nested expansion testing and evaluation outputs 
evaluating result...
mse 5.7441863398028705e-05