leezu commented on issue #18255: URL: https://github.com/apache/incubator-mxnet/issues/18255#issuecomment-721480830
Thanks @akarbown for following up on this issue. I first noticed the failure reported here due to the CI failure of https://github.com/apache/incubator-mxnet/pull/17794 It would previously immediately abort. I rebased the PR and included an update to the latest MKL version. Now the CI does not immediately abort but there are still 20 failed (aborted) tests. Perhaps there has been a change in MKL / MXNet that works around the immediate issue? https://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/unix-cpu/branches/PR-17794/runs/14/nodes/284/steps/417/log/?start=0 ``` [2020-11-02T17:50:40.151Z] [2020-11-02T17:50:40.151Z] =================================== FAILURES =================================== [2020-11-02T17:50:40.151Z] ____________________ tests/python/unittest/test_autograd.py ____________________ [2020-11-02T17:50:40.151Z] [gw2] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.151Z] worker 'gw2' crashed while running 'tests/python/unittest/test_autograd.py::test_unary_func' [2020-11-02T17:50:40.151Z] ____________________ tests/python/unittest/test_autograd.py ____________________ [2020-11-02T17:50:40.151Z] [gw3] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.151Z] worker 'gw3' crashed while running 'tests/python/unittest/test_autograd.py::test_training' [2020-11-02T17:50:40.151Z] ___________ tests/python/unittest/test_contrib_gluon_data_vision.py ____________ [2020-11-02T17:50:40.151Z] [gw1] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.151Z] worker 'gw1' crashed while running 'tests/python/unittest/test_contrib_gluon_data_vision.py::TestImage::test_imageiter' [2020-11-02T17:50:40.151Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:50:40.151Z] [gw4] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.151Z] worker 'gw4' crashed while running 'tests/python/unittest/test_numpy_op.py::test_np_linalg_svd[False-float32-shape3]' [2020-11-02T17:50:40.151Z] ________________ tests/python/unittest/test_contrib_intgemm.py _________________ [2020-11-02T17:50:40.151Z] [gw0] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.151Z] worker 'gw0' crashed while running 'tests/python/unittest/test_contrib_intgemm.py::test_contrib_intgemm_multiply[api0-8-64-1]' [2020-11-02T17:50:40.151Z] ________________ tests/python/unittest/test_gluon_estimator.py _________________ [2020-11-02T17:50:40.151Z] [gw7] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.151Z] worker 'gw7' crashed while running 'tests/python/unittest/test_gluon_estimator.py::test_trainer' [2020-11-02T17:50:40.151Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:50:40.151Z] [gw8] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.151Z] worker 'gw8' crashed while running 'tests/python/unittest/test_numpy_op.py::test_np_matmul[float64-True-add-write-shape_a9-shape_b9]' [2020-11-02T17:50:40.151Z] ___________________ tests/python/unittest/test_optimizer.py ____________________ [2020-11-02T17:50:40.151Z] [gw6] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.151Z] worker 'gw6' crashed while running 'tests/python/unittest/test_optimizer.py::test_lamb' [2020-11-02T17:50:40.151Z] ________________ tests/python/unittest/test_contrib_intgemm.py _________________ [2020-11-02T17:50:40.151Z] [gw9] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.151Z] worker 'gw9' crashed while running 'tests/python/unittest/test_contrib_intgemm.py::test_contrib_intgemm_multiply[api0-8-128-1]' [2020-11-02T17:50:40.405Z] _______________ tests/python/unittest/test_higher_order_grad.py ________________ [2020-11-02T17:50:40.405Z] [gw5] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.405Z] worker 'gw5' crashed while running 'tests/python/unittest/test_higher_order_grad.py::test_arctan' [2020-11-02T17:50:40.405Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:50:40.405Z] [gw11] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.405Z] worker 'gw11' crashed while running 'tests/python/unittest/test_numpy_op.py::test_np_mixed_mxnp_op_funcs' [2020-11-02T17:50:40.405Z] ________________ tests/python/unittest/test_contrib_intgemm.py _________________ [2020-11-02T17:50:40.405Z] [gw12] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.405Z] worker 'gw12' crashed while running 'tests/python/unittest/test_contrib_intgemm.py::test_contrib_intgemm_multiply[api0-8-64-2]' [2020-11-02T17:50:40.405Z] ___________________ tests/python/unittest/test_optimizer.py ____________________ [2020-11-02T17:50:40.405Z] [gw13] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.405Z] worker 'gw13' crashed while running 'tests/python/unittest/test_optimizer.py::test_lans' [2020-11-02T17:50:40.405Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:50:40.405Z] [gw10] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.405Z] worker 'gw10' crashed while running 'tests/python/unittest/test_numpy_op.py::test_npx_batch_dot' [2020-11-02T17:50:40.405Z] ________________ tests/python/unittest/test_contrib_intgemm.py _________________ [2020-11-02T17:50:40.405Z] [gw15] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.405Z] worker 'gw15' crashed while running 'tests/python/unittest/test_contrib_intgemm.py::test_contrib_intgemm_multiply[api0-8-192-1]' [2020-11-02T17:50:40.405Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:50:40.405Z] [gw17] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.405Z] worker 'gw17' crashed while running 'tests/python/unittest/test_numpy_op.py::test_np_matmul[float32-True-add-null-shape_a6-shape_b6]' [2020-11-02T17:50:40.405Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:50:40.405Z] [gw18] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.405Z] worker 'gw18' crashed while running 'tests/python/unittest/test_numpy_op.py::test_np_matmul[float32-True-write-add-shape_a4-shape_b4]' [2020-11-02T17:50:40.405Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:50:40.405Z] [gw19] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.405Z] worker 'gw19' crashed while running 'tests/python/unittest/test_numpy_op.py::test_np_matmul[float64-False-write-write-shape_a3-shape_b3]' [2020-11-02T17:50:40.405Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:50:40.405Z] [gw14] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.405Z] worker 'gw14' crashed while running 'tests/python/unittest/test_numpy_op.py::test_np_linalg_svd[False-float32-shape4]' [2020-11-02T17:50:40.405Z] _____________________ tests/python/unittest/test_metric.py _____________________ [2020-11-02T17:50:40.405Z] [gw16] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:50:40.405Z] worker 'gw16' crashed while running 'tests/python/unittest/test_metric.py::test_ce' [2020-11-02T17:50:40.405Z] =============================== warnings summary =============================== ``` ``` [2020-11-02T17:51:15.713Z] =================================== FAILURES =================================== [2020-11-02T17:51:15.713Z] ____________________ tests/python/unittest/test_autograd.py ____________________ [2020-11-02T17:51:15.713Z] [gw3] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw3' crashed while running 'tests/python/unittest/test_autograd.py::test_unary_func' [2020-11-02T17:51:15.713Z] ____________________ tests/python/unittest/test_autograd.py ____________________ [2020-11-02T17:51:15.713Z] [gw2] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw2' crashed while running 'tests/python/unittest/test_autograd.py::test_training' [2020-11-02T17:51:15.713Z] ___________ tests/python/unittest/test_contrib_gluon_data_vision.py ____________ [2020-11-02T17:51:15.713Z] [gw1] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw1' crashed while running 'tests/python/unittest/test_contrib_gluon_data_vision.py::TestImage::test_imageiter' [2020-11-02T17:51:15.713Z] ____________________ tests/python/unittest/test_executor.py ____________________ [2020-11-02T17:51:15.713Z] [gw0] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw0' crashed while running 'tests/python/unittest/test_executor.py::test_dot' [2020-11-02T17:51:15.713Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:51:15.713Z] [gw5] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw5' crashed while running 'tests/python/unittest/test_numpy_op.py::test_np_mixed_mxnp_op_funcs' [2020-11-02T17:51:15.713Z] ________ tests/python/unittest/test_numpy_contrib_gluon_data_vision.py _________ [2020-11-02T17:51:15.713Z] [gw8] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw8' crashed while running 'tests/python/unittest/test_numpy_contrib_gluon_data_vision.py::TestImage::test_bbox_augmenters' [2020-11-02T17:51:15.713Z] _______________ tests/python/unittest/test_gluon_data_vision.py ________________ [2020-11-02T17:51:15.713Z] [gw9] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw9' crashed while running 'tests/python/unittest/test_gluon_data_vision.py::test_random_gray' [2020-11-02T17:51:15.713Z] ___________________ tests/python/unittest/test_optimizer.py ____________________ [2020-11-02T17:51:15.713Z] [gw7] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw7' crashed while running 'tests/python/unittest/test_optimizer.py::test_lamb' [2020-11-02T17:51:15.713Z] __________________ tests/python/unittest/test_subgraph_op.py ___________________ [2020-11-02T17:51:15.713Z] [gw6] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw6' crashed while running 'tests/python/unittest/test_subgraph_op.py::test_subgraph_exe4[sym1-op_names1-default]' [2020-11-02T17:51:15.713Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:51:15.713Z] [gw4] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw4' crashed while running 'tests/python/unittest/test_numpy_op.py::test_npx_batch_dot' [2020-11-02T17:51:15.713Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:51:15.713Z] [gw13] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw13' crashed while running 'tests/python/unittest/test_numpy_op.py::test_np_linalg_svd[True-float64-shape10]' [2020-11-02T17:51:15.713Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:51:15.713Z] [gw11] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw11' crashed while running 'tests/python/unittest/test_numpy_op.py::test_np_matmul[float64-True-add-add-shape_a6-shape_b6]' [2020-11-02T17:51:15.713Z] _____________________ tests/python/unittest/test_metric.py _____________________ [2020-11-02T17:51:15.713Z] [gw15] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw15' crashed while running 'tests/python/unittest/test_metric.py::test_ce' [2020-11-02T17:51:15.713Z] ___________________ tests/python/unittest/test_optimizer.py ____________________ [2020-11-02T17:51:15.713Z] [gw16] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw16' crashed while running 'tests/python/unittest/test_optimizer.py::test_lans' [2020-11-02T17:51:15.713Z] ___________________ tests/python/unittest/test_extensions.py ___________________ [2020-11-02T17:51:15.713Z] [gw10] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw10' crashed while running 'tests/python/unittest/test_extensions.py::test_subgraph' [2020-11-02T17:51:15.713Z] _________________ tests/python/unittest/test_contrib_krprod.py _________________ [2020-11-02T17:51:15.713Z] [gw14] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw14' crashed while running 'tests/python/unittest/test_contrib_krprod.py::test_krprod_one_input' [2020-11-02T17:51:15.713Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:51:15.713Z] [gw19] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw19' crashed while running 'tests/python/unittest/test_numpy_op.py::test_np_linalg_slogdet[a_shape6-False-float64-add]' [2020-11-02T17:51:15.713Z] _________________ tests/python/unittest/test_contrib_krprod.py _________________ [2020-11-02T17:51:15.713Z] [gw17] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw17' crashed while running 'tests/python/unittest/test_contrib_krprod.py::test_krprod_two_inputs' [2020-11-02T17:51:15.713Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:51:15.713Z] [gw18] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw18' crashed while running 'tests/python/unittest/test_numpy_op.py::test_np_linalg_qr' [2020-11-02T17:51:15.713Z] ____________________ tests/python/unittest/test_numpy_op.py ____________________ [2020-11-02T17:51:15.713Z] [gw12] linux -- Python 3.6.9 /usr/bin/python3 [2020-11-02T17:51:15.713Z] worker 'gw12' crashed while running 'tests/python/unittest/test_numpy_op.py::test_npx_special_unary_func' [2020-11-02T17:51:15.713Z] =============================== warnings summary =============================== ``` I haven't looked into these failures in more detail. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
