nudles commented on pull request #697: URL: https://github.com/apache/singa/pull/697#issuecomment-633166301
> > > Now, users can define a new layer like this. (Just three functions) > > ``` > class MyLayer(layer.Layer): > def __init__(self, a, b, c): > super(MyLayer, self).__init__() > self.l1 = layer.Linear(a,b) Using the new API, Linear layer can accept only one argument, i.e., the output feature dim. > self.l2 = layer.Linear(b,c) > # just need to specify names of params and states > self.param_names = ['W', 'b'] there is no params explicitly defined by MyModel; therefore there should be no direct params of MyModel. All params are from the sublayers of MyModel, i.e., l1 and l2. > self.state_names = self.param_names + ['running_mean', 'running_var'] I suggest to let the layers with params and states to override the set_param/get_param, set/get_states functions. > > def initialize(self, x): > ## init params and states no need to enable the graph manually > ## the graph will be enabled automatically > pass > > def forward(self, x): > y = self.l1(x) > y = self.l2(y) > ### other statements > return y > ``` > > An example of the result of get_params.([MyModel](https://github.com/apache/singa/blob/5f485c0d21b107a119792f4a676009dcfad0d50f/proof_of_concept_to_remove.py)) > > ```shell > {'MyModel.l1.W': [[-0.38202214 -0.12201381] > [-0.304533 -0.9495602 ] > [ 0.16352855 -0.01871952] > [ 0.8286818 -0.35981584]], > 'MyModel.l1.b': [0. 0.], > 'MyModel.bn1.scale': [1. 1.], > 'MyModel.bn1.bias': [0. 0.], > 'MyModel.dl1.l1.W': [[ 0.11860882 -0.79534566 -0.22841209 0.17164443] > [-0.6173898 0.66481847 -0.2978943 0.30236784]], > 'MyModel.dl1.l1.b': [0. 0. 0. 0.], > 'MyModel.dl1.l2.W': [[ 0.4442746 -1.1191332 ] > [ 0.38644505 0.8065585 ] > [-0.06753294 -0.19837534] > [-0.20991991 0.18191515]], > 'MyModel.dl1.l2.b': [0. 0.]} > ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
