Bug#995021: osmnx: autopkgtest regression on armhf: iteration over a 0-d array
Hi Jerome, On 03-11-2021 19:11, Jerome BENOIT wrote: > On Mon, 1 Nov 2021 20:49:58 +0100 Paul Gevers wrote: >> Is this run on all cores? Our armhf worker has 160 cores, so you may be >> running into limits you didn't expect. > > The upstream maintainer would like to know the number of cpus > from a Python shell. > import multiprocessing as mp; print(mp.cpu_count()); > > Can you get this information ? > In fact, I am curious to know it as well. >>> import multiprocessing as mp; print(mp.cpu_count()) 160 Paul OpenPGP_signature Description: OpenPGP digital signature
Bug#995021: osmnx: autopkgtest regression on armhf: iteration over a 0-d array
On Mon, 1 Nov 2021 20:49:58 +0100 Paul Gevers wrote: Hello Paul, Is this run on all cores? Our armhf worker has 160 cores, so you may be running into limits you didn't expect. The upstream maintainer would like to know the number of cpus from a Python shell. import multiprocessing as mp; print(mp.cpu_count()); Can you get this information ? In fact, I am curious to know it as well. Cheers, Jerome -- Jerome BENOIT | calculus+at-rezozer^dot*net https://qa.debian.org/developer.php?login=calcu...@rezozer.net AE28 AE15 710D FF1D 87E5 A762 3F92 19A6 7F36 C68B OpenPGP_signature Description: OpenPGP digital signature
Bug#995021: osmnx: autopkgtest regression on armhf: iteration over a 0-d array
On Mon, 1 Nov 2021 20:49:58 +0100 Paul Gevers wrote: Hello Paul, Is this run on all cores? Our armhf worker has 160 cores, so you may be running into limits you didn't expect. I can reproduce the issue on my amd64 box by substituting mp.cpu_count() by 160 in the called function. We are progressing. Cheers, Jerome -- Jerome BENOIT | calculus+at-rezozer^dot*net https://qa.debian.org/developer.php?login=calcu...@rezozer.net AE28 AE15 710D FF1D 87E5 A762 3F92 19A6 7F36 C68B OpenPGP_signature Description: OpenPGP digital signature
Bug#995021: osmnx: autopkgtest regression on armhf: iteration over a 0-d array
On Mon, 1 Nov 2021 20:49:58 +0100 Paul Gevers wrote: Hello Paul, Is this run on all cores? Our armhf worker has 160 cores, so you may be running into limits you didn't expect. This would be a surprising bug, indeed. I will have a closer look. But I think I have ultimately to contact the upstream author. Otherwise, is there a way to limit the number of cores to use through an environment variable (or something along this way) ? Cheers, Jerom -- Jerome BENOIT | calculus+at-rezozer^dot*net https://qa.debian.org/developer.php?login=calcu...@rezozer.net AE28 AE15 710D FF1D 87E5 A762 3F92 19A6 7F36 C68B OpenPGP_signature Description: OpenPGP digital signature
Bug#995021: osmnx: autopkgtest regression on armhf: iteration over a 0-d array
Hi Jerome, On 24-09-2021 22:23, Paul Gevers wrote: > The above exception was the direct cause of the following exception: > > def test_elevation(): > > G = ox.graph_from_address(address=address, dist=500, > dist_type="bbox", network_type="bike") > rasters = list(Path("tests/input_data").glob("elevation*.tif")) > > # add node elevations from a single raster file (some nodes will > be null) > G = ox.elevation.add_node_elevations_raster(G, rasters[0], cpus=1) ^^^ Suddenly strikes me, compared to the next failing line... > # add node elevations from multiple raster files >> G = ox.elevation.add_node_elevations_raster(G, rasters) Is this run on all cores? Our armhf worker has 160 cores, so you may be running into limits you didn't expect. Paul OpenPGP_signature Description: OpenPGP digital signature
Bug#995021: osmnx: autopkgtest regression on armhf: iteration over a 0-d array
Hello Paul, thanks for pointing out the issue. I will try to have a look this week-end. Cheers, Jerome On 24/09/2021 22:23, Paul Gevers wrote: Source: osmnx Version: 1.1.1+ds-2 X-Debbugs-CC: debian...@lists.debian.org Severity: serious User: debian...@lists.debian.org Usertags: regression Dear maintainer(s), With a recent upload of osmnx the autopkgtest of osmnx fails in testing on armhf when that autopkgtest is run with the binary packages of osmnx from unstable. It passes when run with only packages from testing. In tabular form: passfail osmnx from testing1.1.1+ds-2 all others from testingfrom testing I copied some of the output at the bottom of this report. On top of the failure, you test seems to be sending requests to the internet, please use the needs-internet restriction [0] to mark is as such. Currently this regression is blocking the migration to testing [1]. Can you please investigate the situation and fix it? More information about this bug and the reason for filing it can be found on https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation Paul [0] https://salsa.debian.org/ci-team/autopkgtest/-/blob/master/doc/README.package-tests.rst [1] https://qa.debian.org/excuses.php?package=osmnx https://ci.debian.net/data/autopkgtest/testing/armhf/o/osmnx/15474259/log.gz === FAILURES === test_elevation multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/usr/lib/python3.9/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "/usr/lib/python3.9/multiprocessing/pool.py", line 51, in starmapstar return list(itertools.starmap(args[0], args[1])) File "/tmp/autopkgtest-lxc.2hzrzn35/downtmp/build.elV/src/osmnx/elevation.py", line 49, in _query_raster return zip(nodes.index, values) TypeError: iteration over a 0-d array """ The above exception was the direct cause of the following exception: def test_elevation(): G = ox.graph_from_address(address=address, dist=500, dist_type="bbox", network_type="bike") rasters = list(Path("tests/input_data").glob("elevation*.tif")) # add node elevations from a single raster file (some nodes will be null) G = ox.elevation.add_node_elevations_raster(G, rasters[0], cpus=1) # add node elevations from multiple raster files G = ox.elevation.add_node_elevations_raster(G, rasters) /usr/share/doc/python-osmnx-doc/examples/tests/test_osmnx.py:201: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ osmnx/elevation.py:98: in add_node_elevations_raster results = sma.get() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , timeout = None def get(self, timeout=None): self.wait(timeout) if not self.ready(): raise TimeoutError if self._success: return self._value else: raise self._value E TypeError: iteration over a 0-d array /usr/lib/python3.9/multiprocessing/pool.py:771: TypeError -- Jerome BENOIT | calculus+at-rezozer^dot*net https://qa.debian.org/developer.php?login=calcu...@rezozer.net AE28 AE15 710D FF1D 87E5 A762 3F92 19A6 7F36 C68B OpenPGP_signature Description: OpenPGP digital signature
Bug#995021: osmnx: autopkgtest regression on armhf: iteration over a 0-d array
Source: osmnx Version: 1.1.1+ds-2 X-Debbugs-CC: debian...@lists.debian.org Severity: serious User: debian...@lists.debian.org Usertags: regression Dear maintainer(s), With a recent upload of osmnx the autopkgtest of osmnx fails in testing on armhf when that autopkgtest is run with the binary packages of osmnx from unstable. It passes when run with only packages from testing. In tabular form: passfail osmnx from testing1.1.1+ds-2 all others from testingfrom testing I copied some of the output at the bottom of this report. On top of the failure, you test seems to be sending requests to the internet, please use the needs-internet restriction [0] to mark is as such. Currently this regression is blocking the migration to testing [1]. Can you please investigate the situation and fix it? More information about this bug and the reason for filing it can be found on https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation Paul [0] https://salsa.debian.org/ci-team/autopkgtest/-/blob/master/doc/README.package-tests.rst [1] https://qa.debian.org/excuses.php?package=osmnx https://ci.debian.net/data/autopkgtest/testing/armhf/o/osmnx/15474259/log.gz === FAILURES === test_elevation multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/usr/lib/python3.9/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "/usr/lib/python3.9/multiprocessing/pool.py", line 51, in starmapstar return list(itertools.starmap(args[0], args[1])) File "/tmp/autopkgtest-lxc.2hzrzn35/downtmp/build.elV/src/osmnx/elevation.py", line 49, in _query_raster return zip(nodes.index, values) TypeError: iteration over a 0-d array """ The above exception was the direct cause of the following exception: def test_elevation(): G = ox.graph_from_address(address=address, dist=500, dist_type="bbox", network_type="bike") rasters = list(Path("tests/input_data").glob("elevation*.tif")) # add node elevations from a single raster file (some nodes will be null) G = ox.elevation.add_node_elevations_raster(G, rasters[0], cpus=1) # add node elevations from multiple raster files > G = ox.elevation.add_node_elevations_raster(G, rasters) /usr/share/doc/python-osmnx-doc/examples/tests/test_osmnx.py:201: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ osmnx/elevation.py:98: in add_node_elevations_raster results = sma.get() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = , timeout = None def get(self, timeout=None): self.wait(timeout) if not self.ready(): raise TimeoutError if self._success: return self._value else: > raise self._value E TypeError: iteration over a 0-d array /usr/lib/python3.9/multiprocessing/pool.py:771: TypeError OpenPGP_signature Description: OpenPGP digital signature