indhub opened a new issue #8299: test_cifar10 fails in CI master build URL: https://github.com/apache/incubator-mxnet/issues/8299 ## Description test_cifar10 fails in CI master build. Failed build: https://builds.apache.org/blue/organizations/jenkins/incubator-mxnet/detail/master/532/pipeline/ ## Environment info (Required) CI build Python 2 CPU MXNet commit hash: 1c1c788916d672ee3cafdc4c91d7002a94a59d13 ## Error Message: ``` ====================================================================== FAIL: test_dtype.test_cifar10 ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/workspace/tests/python/train/test_dtype.py", line 192, in test_cifar10 run_cifar10(train, val, use_module=True) File "/workspace/tests/python/train/test_dtype.py", line 136, in run_cifar10 assert (ret[0][1] > 0.08) AssertionError: -------------------- >> begin captured logging << -------------------- root: INFO: Start training with [cpu(0)] root: INFO: Epoch[0] Batch [50] Speed: 7880.07 samples/sec accuracy=0.109531 root: INFO: Epoch[0] Batch [100] Speed: 10121.26 samples/sec accuracy=0.092969 root: INFO: Epoch[0] Batch [150] Speed: 9358.31 samples/sec accuracy=0.094375 root: INFO: Epoch[0] Batch [200] Speed: 9408.26 samples/sec accuracy=0.100312 root: INFO: Epoch[0] Batch [250] Speed: 10316.01 samples/sec accuracy=0.099062 root: INFO: Epoch[0] Batch [300] Speed: 9913.46 samples/sec accuracy=0.093906 root: INFO: Epoch[0] Batch [350] Speed: 10337.40 samples/sec accuracy=0.103750 root: INFO: Epoch[0] Resetting Data Iterator root: INFO: Epoch[0] Time cost=5.339 root: INFO: Epoch[0] Validation-accuracy=0.099881 root: INFO: final accuracy = 0.099881 root: INFO: Epoch[0] Batch [50] Speed: 7869.73 samples/sec accuracy=0.123775 root: INFO: Epoch[0] Batch [100] Speed: 9542.55 samples/sec accuracy=0.099375 root: INFO: Epoch[0] Batch [150] Speed: 9407.05 samples/sec accuracy=0.094219 root: INFO: Epoch[0] Batch [200] Speed: 8906.41 samples/sec accuracy=0.100625 root: INFO: Epoch[0] Batch [250] Speed: 9453.06 samples/sec accuracy=0.098125 root: INFO: Epoch[0] Batch [300] Speed: 8874.91 samples/sec accuracy=0.093281 root: INFO: Epoch[0] Batch [350] Speed: 6026.26 samples/sec accuracy=0.104063 root: INFO: Epoch[0] Train-accuracy=0.097656 root: INFO: Epoch[0] Time cost=6.069 root: INFO: Epoch[0] Validation-accuracy=0.099881 root: INFO: final accuracy = 0.099960 root: INFO: Start training with [cpu(0)] root: INFO: Epoch[0] Batch [50] Speed: 4614.01 samples/sec accuracy=0.120938 root: INFO: Epoch[0] Batch [100] Speed: 6189.65 samples/sec accuracy=0.096562 root: INFO: Epoch[0] Batch [150] Speed: 6062.91 samples/sec accuracy=0.094844 root: INFO: Epoch[0] Batch [200] Speed: 6710.55 samples/sec accuracy=0.100312 root: INFO: Epoch[0] Batch [250] Speed: 5982.40 samples/sec accuracy=0.099062 root: INFO: Epoch[0] Batch [300] Speed: 7129.64 samples/sec accuracy=0.093906 root: INFO: Epoch[0] Batch [350] Speed: 6555.40 samples/sec accuracy=0.103750 root: INFO: Epoch[0] Resetting Data Iterator root: INFO: Epoch[0] Time cost=8.313 root: INFO: Epoch[0] Validation-accuracy=0.099881 root: INFO: final accuracy = 0.099881 root: INFO: Epoch[0] Batch [50] Speed: 4543.26 samples/sec accuracy=0.119638 root: INFO: Epoch[0] Batch [100] Speed: 5423.21 samples/sec accuracy=0.103750 root: INFO: Epoch[0] Batch [150] Speed: 5033.73 samples/sec accuracy=0.092813 root: INFO: Epoch[0] Batch [200] Speed: 9233.24 samples/sec accuracy=0.100469 root: INFO: Epoch[0] Batch [250] Speed: 9242.30 samples/sec accuracy=0.099375 root: INFO: Epoch[0] Batch [300] Speed: 10217.41 samples/sec accuracy=0.093125 root: INFO: Epoch[0] Batch [350] Speed: 8666.85 samples/sec accuracy=0.104063 root: INFO: Epoch[0] Train-accuracy=0.092188 root: INFO: Epoch[0] Time cost=7.249 root: INFO: Epoch[0] Validation-accuracy=0.099881 root: INFO: final accuracy = 0.099960 root: INFO: Start training with [cpu(0)] root: INFO: Epoch[0] Batch [50] Speed: 8083.13 samples/sec accuracy=0.102656 root: INFO: Epoch[0] Batch [100] Speed: 9373.58 samples/sec accuracy=0.101250 root: INFO: Epoch[0] Batch [150] Speed: 8239.63 samples/sec accuracy=0.092969 root: INFO: Epoch[0] Batch [200] Speed: 8116.80 samples/sec accuracy=0.101094 root: INFO: Epoch[0] Batch [250] Speed: 8595.15 samples/sec accuracy=0.099531 root: INFO: Epoch[0] Batch [300] Speed: 9090.94 samples/sec accuracy=0.093906 root: INFO: Epoch[0] Batch [350] Speed: 9014.57 samples/sec accuracy=0.103750 root: INFO: Epoch[0] Resetting Data Iterator root: INFO: Epoch[0] Time cost=5.879 root: INFO: Epoch[0] Validation-accuracy=0.099881 root: INFO: final accuracy = 0.099881 root: INFO: Epoch[0] Batch [50] Speed: 9343.27 samples/sec accuracy=0.102635 root: INFO: Epoch[0] Batch [100] Speed: 8417.44 samples/sec accuracy=0.095312 root: INFO: Epoch[0] Batch [150] Speed: 9007.86 samples/sec accuracy=0.101094 root: INFO: Epoch[0] Batch [200] Speed: 9894.47 samples/sec accuracy=0.103125 root: INFO: Epoch[0] Batch [250] Speed: 9899.44 samples/sec accuracy=0.100312 root: INFO: Epoch[0] Batch [300] Speed: 9939.62 samples/sec accuracy=0.095781 root: INFO: Epoch[0] Batch [350] Speed: 10021.53 samples/sec accuracy=0.101250 root: INFO: Epoch[0] Train-accuracy=0.097656 root: INFO: Epoch[0] Time cost=5.316 root: INFO: Epoch[0] Validation-accuracy=0.078521 root: INFO: final accuracy = 0.078425 --------------------- >> end captured logging << --------------------- [success] 29.22% test_autograd.test_autograd: 68.1240s [fail] 27.32% test_dtype.test_cifar10: 63.6942s [success] 21.17% test_bucketing.test_bucket_module: 49.3386s [success] 14.59% test_mlp.test_mlp: 33.9994s [success] 7.70% test_conv.test_mnist: 17.9464s ---------------------------------------------------------------------- Ran 5 tests in 237.792s FAILED (failures=1) ``` ## Steps to reproduce Build and run unit test ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
With regards, Apache Git Services
