[GitHub] [incubator-mxnet] leezu edited a comment on pull request #18445: updating ubuntu_cpu base image to 20.04 to observe failing tests regarding Python 3.8
leezu edited a comment on pull request #18445: URL: https://github.com/apache/incubator-mxnet/pull/18445#issuecomment-656233768 @ma-hei looking at the output of unix-cpu, I also see that the dataloader related tests are hanging in Tests / Python3: MKLDNN-MKL-CPU. The recordio test you point out (which also hangs) is also part of the data api. > Note: comparing this with the same job running on python 3.6, I see that the same test is running much later (at 99% completion compared to 89% in this job): There may be a change in the order which the tests are executed, but it shouldn't affect the hanging? cc @zhreshold there may be incompatibility of the data code with Python 3.8 ``` [2020-07-02T19:41:12.626Z] tests/python/unittest/test_gluon_data.py::test_recordimage_dataset_handle PASSED [ 3%] [2020-07-02T19:41:14.506Z] tests/python/unittest/test_gluon_data.py::test_recordimage_dataset_with_data_loader_multiworker PASSED [ 3%] [2020-07-02T20:01:21.465Z] Timeout (0:20:00)! [2020-07-02T20:01:21.465Z] Thread 0x7fb1f264c700 (most recent call first): [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 379 in _recv [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 414 in _recv_bytes [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 250 in recv [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/pool.py", line 576 in _handle_results [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 870 in run [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap [2020-07-02T20:01:21.465Z] [2020-07-02T20:01:21.465Z] Thread 0x7fb1f6e4d700 (most recent call first): [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/pool.py", line 528 in _handle_tasks [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 870 in run [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap [2020-07-02T20:01:21.465Z] [2020-07-02T20:01:21.465Z] Thread 0x7fb1f964e700 (most recent call first): [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/selectors.py", line 415 in select [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 930 in wait [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/pool.py", line 499 in _wait_for_updates [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/pool.py", line 519 in _handle_workers [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 870 in run [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap [2020-07-02T20:01:21.465Z] [2020-07-02T20:01:21.465Z] Thread 0x7fb1f9e4f700 (most recent call first): [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 379 in _recv [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 414 in _recv_bytes [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 250 in recv [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/pool.py", line 576 in _handle_results [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 870 in run [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap [2020-07-02T20:01:21.465Z] [2020-07-02T20:01:21.465Z] Thread 0x7fb1fc650700 (most recent call first): [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/pool.py", line 528 in _handle_tasks [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 870 in run [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap [2020-07-02T20:01:21.465Z] [2020-07-02T20:01:21.465Z] Thread 0x7fb200e51700 (most recent call first): [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/selectors.py", line 415 in select [2020-07-02T20:01:21.465Z] File
[GitHub] [incubator-mxnet] leezu edited a comment on pull request #18445: updating ubuntu_cpu base image to 20.04 to observe failing tests regarding Python 3.8
leezu edited a comment on pull request #18445: URL: https://github.com/apache/incubator-mxnet/pull/18445#issuecomment-656233768 @ma-hei looking at the output of unix-cpu, I also see that the dataloader related tests are hanging in Tests / Python3: MKLDNN-MKL-CPU. The recordio test you point out (which also hangs) is also part of the data api. cc @zhreshold there may be incompatibility of the data code with Python 3.8 ``` [2020-07-02T19:41:12.626Z] tests/python/unittest/test_gluon_data.py::test_recordimage_dataset_handle PASSED [ 3%] [2020-07-02T19:41:14.506Z] tests/python/unittest/test_gluon_data.py::test_recordimage_dataset_with_data_loader_multiworker PASSED [ 3%] [2020-07-02T20:01:21.465Z] Timeout (0:20:00)! [2020-07-02T20:01:21.465Z] Thread 0x7fb1f264c700 (most recent call first): [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 379 in _recv [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 414 in _recv_bytes [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 250 in recv [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/pool.py", line 576 in _handle_results [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 870 in run [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap [2020-07-02T20:01:21.465Z] [2020-07-02T20:01:21.465Z] Thread 0x7fb1f6e4d700 (most recent call first): [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/pool.py", line 528 in _handle_tasks [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 870 in run [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap [2020-07-02T20:01:21.465Z] [2020-07-02T20:01:21.465Z] Thread 0x7fb1f964e700 (most recent call first): [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/selectors.py", line 415 in select [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 930 in wait [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/pool.py", line 499 in _wait_for_updates [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/pool.py", line 519 in _handle_workers [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 870 in run [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap [2020-07-02T20:01:21.465Z] [2020-07-02T20:01:21.465Z] Thread 0x7fb1f9e4f700 (most recent call first): [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 379 in _recv [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 414 in _recv_bytes [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 250 in recv [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/pool.py", line 576 in _handle_results [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 870 in run [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap [2020-07-02T20:01:21.465Z] [2020-07-02T20:01:21.465Z] Thread 0x7fb1fc650700 (most recent call first): [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/pool.py", line 528 in _handle_tasks [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 870 in run [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap [2020-07-02T20:01:21.465Z] [2020-07-02T20:01:21.465Z] Thread 0x7fb200e51700 (most recent call first): [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/selectors.py", line 415 in select [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/connection.py", line 930 in wait [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/pool.py", line 499 in _wait_for_updates [2020-07-02T20:01:21.465Z] File "/usr/lib/python3.8/multiprocessing/pool.py", line 519 in _handle_workers
[GitHub] [incubator-mxnet] leezu edited a comment on pull request #18445: updating ubuntu_cpu base image to 20.04 to observe failing tests regarding Python 3.8
leezu edited a comment on pull request #18445: URL: https://github.com/apache/incubator-mxnet/pull/18445#issuecomment-636267475 When updating the base image, the Dockerfile would require some changes. You can test that the containers can still build, locally via: `cd mxnet/ci; DOCKER_CACHE_REGISTRY=mxnetci docker-compose -f docker/docker-compose.yml pull; DOCKER_CACHE_REGISTRY=mxnetci docker-compose -f docker/docker-compose.yml build` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
