joddiy commented on pull request #738: URL: https://github.com/apache/singa/pull/738#issuecomment-646663571
> Hi, > > The test_onnx_backend.py did not gave any error for Floor or Abs added. However following error with maxpool2d was generated > > ``` > .s.s.s.s.s.s.s.s.s.s.s.s.s.s.sssssssssssssssssssssssssssssssss.s.s.s.s.s.s.s.sssss.sss.sss.s.s.sFsFs.sssssssssss.sssssssssssssssss.s.s.s.s.s.sssss.s.s.s.s.s.s.s.s.s.sssssssss.s.s.s.s.s.s.s.s.s.s.s.sssss.s.sssssssssssssssssssssssssss.s.s.s.sssssssssssssssssssssssssssss.s.s.s.s.sssssssssssssss.s.s.sFsFsss.s.s.s.sssssss.s.s.s.s.s.s.s.s.s.s.s.s.sssssss.sssss.s.s.s.s.sss.s.s.s.s.s.s.sssss.s.sssssssssssssssssssss.s.s.s.sssssssssssss.s.s.s.s.s.s.sssssssssssssssssssssssss.s.s.sss.s.s.sssss.sss.s.s.s.sFsFs.sssssssssss.s.s.s.s.s.sssssssssssssssssssssssssss.s.s.sssss.s.sssssssssssssssssss.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.sss.s.s.s.sssssssssssssssss.s.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.s.s.s.s.s.s.s.sssssssssssssssssssssssssssssssss.s.s.s.s.s.s.s.sssssssssssssssssss.s.s.s.s.s.s.s.s.s.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.s.s.s.s.sssss.s.s.sssss.s.s.s.sssss.s.s.s.s.s.s.sss.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.sssssssssssss.s.s.s.s.s.s.s.s.s.sssssssssssssssssssss.s.sssssssssssssssssssssssssssssss.s.s.s.s.s.s.s.s.sssss.s.s.s.s.s.s.s.s > ====================================================================== > FAIL: test_averagepool_2d_same_lower_cpu (__main__.OnnxBackendNodeModelTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func > return test_func(*args, device=device, **kwargs) > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 313, in run > atol=model_test.atol) > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 178, in assert_similar_outputs > atol=atol) > File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 1533, in assert_allclose > verbose=verbose, header=header, equal_nan=equal_nan) > File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 846, in assert_array_compare > raise AssertionError(msg) > AssertionError: > Not equal to tolerance rtol=0.001, atol=1e-07 > > Mismatched elements: 189 / 3072 (6.15%) > Max absolute difference: 1.3230393 > Max relative difference: 3. > x: array([[[[ 1.764052e+00, 1.082105e+00, 6.894476e-01, ..., > 1.501069e+00, 8.121531e-01, 2.665550e-01], > [ 4.381333e-01, -1.760931e-01, -2.374533e-01, ...,... > y: array([[[[ 4.410131e-01, 5.410524e-01, 3.447238e-01, ..., > 7.505345e-01, 4.060766e-01, 1.332775e-01], > [ 2.190667e-01, -1.760931e-01, -2.374533e-01, ...,... > > ====================================================================== > FAIL: test_averagepool_2d_same_upper_cpu (__main__.OnnxBackendNodeModelTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func > return test_func(*args, device=device, **kwargs) > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 313, in run > atol=model_test.atol) > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 178, in assert_similar_outputs > atol=atol) > File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 1533, in assert_allclose > verbose=verbose, header=header, equal_nan=equal_nan) > File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 846, in assert_array_compare > raise AssertionError(msg) > AssertionError: > Not equal to tolerance rtol=0.001, atol=1e-07 > > Mismatched elements: 189 / 3072 (6.15%) > Max absolute difference: 0.9547153 > Max relative difference: 3. > x: array([[[[-0.176093, -0.237453, 0.757017, ..., 0.112902, -0.50158 , > -0.67406 ], > [-0.773234, -1.090172, -0.339745, ..., 0.040076, -0.369122,... > y: array([[[[-0.176093, -0.237453, 0.757017, ..., 0.112902, -0.50158 , > -0.33703 ], > [-0.773234, -1.090172, -0.339745, ..., 0.040076, -0.369122,... > > ====================================================================== > FAIL: test_equal_bcast_cpu (__main__.OnnxBackendNodeModelTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func > return test_func(*args, device=device, **kwargs) > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 313, in run > atol=model_test.atol) > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 178, in assert_similar_outputs > atol=atol) > File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 1533, in assert_allclose > verbose=verbose, header=header, equal_nan=equal_nan) > File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 846, in assert_array_compare > raise AssertionError(msg) > AssertionError: > Not equal to tolerance rtol=0.001, atol=1e-07 > > Mismatched elements: 1 / 60 (1.67%) > x: array([[[False, False, False, False, False], > [False, False, False, False, False], > [False, False, False, False, False],... > y: array([[[False, False, False, False, False], > [False, False, False, False, False], > [False, False, False, False, False],... > > ====================================================================== > FAIL: test_equal_cpu (__main__.OnnxBackendNodeModelTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func > return test_func(*args, device=device, **kwargs) > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 313, in run > atol=model_test.atol) > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 178, in assert_similar_outputs > atol=atol) > File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 1533, in assert_allclose > verbose=verbose, header=header, equal_nan=equal_nan) > File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 846, in assert_array_compare > raise AssertionError(msg) > AssertionError: > Not equal to tolerance rtol=0.001, atol=1e-07 > > Mismatched elements: 1 / 60 (1.67%) > x: array([[[False, False, False, False, False], > [False, False, False, False, False], > [False, False, False, False, True],... > y: array([[[False, False, False, False, False], > [False, False, False, False, False], > [False, False, False, False, True],... > > ====================================================================== > FAIL: test_maxpool_2d_same_lower_cpu (__main__.OnnxBackendNodeModelTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func > return test_func(*args, device=device, **kwargs) > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 313, in run > atol=model_test.atol) > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 178, in assert_similar_outputs > atol=atol) > File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 1533, in assert_allclose > verbose=verbose, header=header, equal_nan=equal_nan) > File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 846, in assert_array_compare > raise AssertionError(msg) > AssertionError: > Not equal to tolerance rtol=0.001, atol=1e-07 > > Mismatched elements: 35 / 3072 (1.14%) > Max absolute difference: 1.6961312 > Max relative difference: 0. > x: array([[[[ 1.764052e+00, 1.764052e+00, 9.787380e-01, ..., > 1.532779e+00, 1.469359e+00, 3.781625e-01], > [ 1.764052e+00, 1.764052e+00, 9.787380e-01, ...,... > y: array([[[[ 1.764052e+00, 1.764052e+00, 9.787380e-01, ..., > 1.532779e+00, 1.469359e+00, 3.781625e-01], > [ 1.764052e+00, 1.764052e+00, 9.787380e-01, ...,... > > ====================================================================== > FAIL: test_maxpool_2d_same_upper_cpu (__main__.OnnxBackendNodeModelTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func > return test_func(*args, device=device, **kwargs) > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 313, in run > atol=model_test.atol) > File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 178, in assert_similar_outputs > atol=atol) > File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 1533, in assert_allclose > verbose=verbose, header=header, equal_nan=equal_nan) > File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 846, in assert_array_compare > raise AssertionError(msg) > AssertionError: > Not equal to tolerance rtol=0.001, atol=1e-07 > > Mismatched elements: 37 / 3072 (1.2%) > Max absolute difference: 1.2028884 > Max relative difference: 0. > x: array([[[[ 1.764052, 0.978738, 2.240893, ..., 1.469359, 0.378163, > 0.378163], > [ 0.177426, -0.347912, 0.462782, ..., 0.976639, 0.706573,... > y: array([[[[ 1.764052, 0.978738, 2.240893, ..., 1.469359, 0.378163, > 0.378163], > [ 0.177426, -0.347912, 0.462782, ..., 0.976639, 0.706573,... > > ---------------------------------------------------------------------- > Ran 1114 tests in 1.197s > > FAILED (failures=6, skipped=852) > ``` > > Thanks and Regards, > Shashank Hi, @Shashankwer , thanks. These errors mostly take place because of float precision. You can just ignore these. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
