chrishkchris opened a new pull request #773: URL: https://github.com/apache/singa/pull/773
Added the 3 optimizers RMSProp, AdaGrad, and Adam into opt.py. However, I still need to convert the old example to use new opt.py. 1. RmsProp ``` root@8d566103edbf:~/dcsysh/singa/examples/cnn# python3 train_cnn.py cnn mnist -l 0.001 Starting Epoch 0: Training loss = 419.288391, training accuracy = 0.853555 Evaluation accuracy = 0.962841, Elapsed Time = 4.750765s Starting Epoch 1: Training loss = 140.065247, training accuracy = 0.953742 Evaluation accuracy = 0.969752, Elapsed Time = 4.735909s Starting Epoch 2: Training loss = 104.745422, training accuracy = 0.964448 Evaluation accuracy = 0.971955, Elapsed Time = 4.730492s Starting Epoch 3: Training loss = 90.510818, training accuracy = 0.970368 Evaluation accuracy = 0.978866, Elapsed Time = 4.741611s Starting Epoch 4: Training loss = 82.076942, training accuracy = 0.972535 Evaluation accuracy = 0.980369, Elapsed Time = 4.733157s Starting Epoch 5: Training loss = 76.509102, training accuracy = 0.974687 Evaluation accuracy = 0.971655, Elapsed Time = 4.734507s Starting Epoch 6: Training loss = 70.361732, training accuracy = 0.977154 Evaluation accuracy = 0.986679, Elapsed Time = 4.733339s Starting Epoch 7: Training loss = 69.605270, training accuracy = 0.977771 Evaluation accuracy = 0.985477, Elapsed Time = 4.732546s Starting Epoch 8: Training loss = 69.644501, training accuracy = 0.978072 Evaluation accuracy = 0.983273, Elapsed Time = 4.726449s Starting Epoch 9: Training loss = 67.109238, training accuracy = 0.978338 Evaluation accuracy = 0.986478, Elapsed Time = 4.731823s ``` 2. AdaGrad ``` root@8d566103edbf:~/dcsysh/singa/examples/cnn# python3 train_cnn.py cnn mnist -l 0.02 Starting Epoch 0: Training loss = 488.669769, training accuracy = 0.849420 Evaluation accuracy = 0.958834, Elapsed Time = 4.417540s Starting Epoch 1: Training loss = 160.616241, training accuracy = 0.945471 Evaluation accuracy = 0.965645, Elapsed Time = 4.408587s Starting Epoch 2: Training loss = 125.146477, training accuracy = 0.958278 Evaluation accuracy = 0.968149, Elapsed Time = 4.420360s Starting Epoch 3: Training loss = 108.487717, training accuracy = 0.964114 Evaluation accuracy = 0.975761, Elapsed Time = 4.418548s Starting Epoch 4: Training loss = 94.765808, training accuracy = 0.968366 Evaluation accuracy = 0.977163, Elapsed Time = 4.414004s Starting Epoch 5: Training loss = 88.474594, training accuracy = 0.970134 Evaluation accuracy = 0.974059, Elapsed Time = 4.409218s Starting Epoch 6: Training loss = 83.103210, training accuracy = 0.971718 Evaluation accuracy = 0.981971, Elapsed Time = 4.411892s Starting Epoch 7: Training loss = 77.175125, training accuracy = 0.973903 Evaluation accuracy = 0.979467, Elapsed Time = 4.407397s Starting Epoch 8: Training loss = 72.816505, training accuracy = 0.975270 Evaluation accuracy = 0.981170, Elapsed Time = 4.407173s Starting Epoch 9: Training loss = 70.385757, training accuracy = 0.975720 Evaluation accuracy = 0.981871, Elapsed Time = 4.407057s ``` 3. Adam ``` root@8d566103edbf:~/dcsysh/singa/examples/cnn# python3 train_cnn_1.py cnn mnist -l 0.001 Starting Epoch 0: Training loss = 429.283813, training accuracy = 0.851237 Evaluation accuracy = 0.952524, Elapsed Time = 5.627855s Starting Epoch 1: Training loss = 200.920776, training accuracy = 0.934482 Evaluation accuracy = 0.959335, Elapsed Time = 5.624342s Starting Epoch 2: Training loss = 153.100571, training accuracy = 0.949190 Evaluation accuracy = 0.968650, Elapsed Time = 5.618489s Starting Epoch 3: Training loss = 130.312637, training accuracy = 0.956077 Evaluation accuracy = 0.970353, Elapsed Time = 5.638798s Starting Epoch 4: Training loss = 115.165848, training accuracy = 0.961429 Evaluation accuracy = 0.973257, Elapsed Time = 5.619699s Starting Epoch 5: Training loss = 102.825790, training accuracy = 0.965415 Evaluation accuracy = 0.975160, Elapsed Time = 5.621256s Starting Epoch 6: Training loss = 93.691055, training accuracy = 0.969334 Evaluation accuracy = 0.981871, Elapsed Time = 5.618617s Starting Epoch 7: Training loss = 87.255371, training accuracy = 0.970985 Evaluation accuracy = 0.976963, Elapsed Time = 5.618419s Starting Epoch 8: Training loss = 81.303070, training accuracy = 0.972902 Evaluation accuracy = 0.978566, Elapsed Time = 5.633997s Starting Epoch 9: Training loss = 77.283844, training accuracy = 0.974320 Evaluation accuracy = 0.979267, Elapsed Time = 5.621700s ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
