[Python-modules-team] Bug#942235:
Hi,
Running autopkgtest I have this error:
autopkgtest [17:27:31]: test command1: set -e ; for py in
$(py3versions -r 2>/dev/null) ; do cd "$AUTOPKGTEST_TMP" ; echo
"Testing with $py:" ; http_proxy= $py -m pytest -v --pyargs dask ;
done
autopkgtest [17:27:31]: test command1: [---
Testing with python3.8:
= test session starts ==
platform linux -- Python 3.8.0, pytest-4.6.6, py-1.8.0, pluggy-0.13.0
-- /usr/bin/python3.8
cachedir: .pytest_cache
rootdir: /tmp/autopkgtest.ZwzvJ1/autopkgtest_tmp
collecting ... collected 2144 items / 39 errors / 8 skipped / 2097 selected
ERRORS
___ ERROR collecting array/tests/test_array_core.py
ImportError while importing test module
'/usr/lib/python3/dist-packages/dask/array/tests/test_array_core.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3/dist-packages/pandas/__init__.py:30: in
from pandas._libs import hashtable as _hashtable, lib as _lib,
tslib as _tslib
/usr/lib/python3/dist-packages/pandas/_libs/__init__.py:3: in
from .tslibs import (
/usr/lib/python3/dist-packages/pandas/_libs/tslibs/__init__.py:3: in
from .conversion import localize_pydatetime, normalize_date
E ModuleNotFoundError: No module named 'pandas._libs.tslibs.conversion'
During handling of the above exception, another exception occurred:
/usr/lib/python3/dist-packages/dask/dataframe/__init__.py:2: in
from .core import (
/usr/lib/python3/dist-packages/dask/dataframe/core.py:10: in
import pandas as pd
/usr/lib/python3/dist-packages/pandas/__init__.py:34: in
raise ImportError(
E ImportError: C extension: No module named
'pandas._libs.tslibs.conversion' not built. If you want to import
pandas from the source directory, you may need to run 'python setup.py
build_ext --inplace --force' to build the C extensions first.
During handling of the above exception, another exception occurred:
/usr/lib/python3/dist-packages/dask/array/tests/test_array_core.py:21:
in
import dask.dataframe
/usr/lib/python3/dist-packages/dask/dataframe/__init__.py:55: in
raise ImportError(str(e) + "\n\n" + msg)
E ImportError: C extension: No module named
'pandas._libs.tslibs.conversion' not built. If you want to import
pandas from the source directory, you may need to run 'python setup.py
build_ext --inplace --force' to build the C extensions first.
E
E Dask dataframe requirements are not installed.
E
E Please either conda or pip install as follows:
E
E conda install dask # either conda install
E pip install dask[dataframe] --upgrade # or pip install
__ ERROR collecting array/tests/test_image.py __
/usr/lib/python3/dist-packages/skimage/__init__.py:162: in
from ._shared import geometry
E ImportError: cannot import name 'geometry' from 'skimage._shared'
(/usr/lib/python3/dist-packages/skimage/_shared/__init__.py)
During handling of the above exception, another exception occurred:
/usr/lib/python3/dist-packages/dask/array/tests/test_image.py:6: in
pytest.importorskip("skimage")
/usr/lib/python3/dist-packages/skimage/__init__.py:165: in
_raise_build_error(e)
/usr/lib/python3/dist-packages/skimage/__init__.py:138: in _raise_build_error
local_dir = osp.split(__file__)[0]
E NameError: name 'osp' is not defined
___ ERROR collecting bag/tests/test_avro.py
ImportError while importing test module
'/usr/lib/python3/dist-packages/dask/bag/tests/test_avro.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3/dist-packages/dask/bag/__init__.py:2: in
from .core import (
/usr/lib/python3/dist-packages/dask/bag/core.py:67: in
from ..bytes import open_files
/usr/lib/python3/dist-packages/dask/bytes/__init__.py:9: in
raise ImportError(
E ImportError: fsspec is required to use any file-system
functionality. Please install using
E conda install -c conda-forge 'fsspec>=0.3.3'
E or
E pip install 'fsspec>=0.3.3'
I will work on fsspec now and try to solve the pandas issues
Cheers,
Arias Emmanuel
@eamanu
http://eamanu.com
___
Python-modules-team mailing list
Python-modules-team@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: Adding Python 3.8 as a supported Python3 version
On 18/11/2019 07:07, Matthias Klose wrote: On 18.11.19 07:52, Alastair McKinstry wrote: So I've uploaded a python-xarray 0.14.0-2 which passes on python3.7. Its failing on python 3.8 but apparently due to pandas not ready on 3.8, so it should be ok to pass as the transition completes. I think that error is just pandas *not being built* for 3.8 yet: a Python version transition is ~500 binNMUs and that takes time. xarray really needs dask >= 2.0 but version 1 is in unstable; I've disabled some functionality (dask='parallelize') and marked some tests xfail until this is done. Alastair yes, people are aware of a needed dask update, see #942235. There's now also a dask 2.8 release available, I didn't look at that yet. dask is itself failing autopkgtests, and this is blocking pandas and pytest, so is a problem independently of xarray. (The breakage was originally caused by pytest 4, but pandas 0.25 Build-Depends: on that.) ___ Python-modules-team mailing list Python-modules-team@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: Adding Python 3.8 as a supported Python3 version
On 18.11.19 07:52, Alastair McKinstry wrote: > So I've uploaded a python-xarray 0.14.0-2 which passes on python3.7. > > Its failing on python 3.8 but apparently due to pandas not ready on 3.8, so it > should be ok to pass as the transition completes. > > xarray really needs dask >= 2.0 but version 1 is in unstable; I've disabled > some > functionality (dask='parallelize') and marked some tests xfail until this is > done. > > Alastair yes, people are aware of a needed dask update, see #942235. There's now also a dask 2.8 release available, I didn't look at that yet. > On 07/11/2019 14:08, Matthias Klose wrote: >> This weekend, I am planning to upload python3-defaults, adding python3.8 as a >> supported Python3 version. This may introduce some churn in unstable until >> the basic binNMUs are available as well. >> >> >> - python-xarray #944044. 1.4 is already broken with Python 3.7. For >> Ubuntu I fell back to 1.2.1. 1.2.3 might work as well, still seeing >> one or two test failures. >> >> Packages building on top of numpy/scipy/pandas, like the PyAstro stack, >> continue to work with these updates, despite some new test failures. >> >> Matthias >> >> [1] https://wiki.debian.org/Python/Python3.8. >> [2] >> https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.8;users=debian-pyt...@lists.debian.org >> >> ___ Python-modules-team mailing list Python-modules-team@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: dask: autopkgtest needs update for new version of pytest
Ubuntu have dask 2.6.0 and fsspec, but still have a few autopkgtest failures: https://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html dask itself - looks like trying to use a nonexistent temporary directory pyfftw - looks like a test treating a new (unexpected) warning as a failure satpy - looks like a missing dependency on fsspec ___ Python-modules-team mailing list Python-modules-team@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: Bug#942235: Bug#942235: dask: autopkgtest needs update for new version of pytest
On Mon, 2019-11-04 at 19:42 -0300, Emmanuel Arias wrote: > I can work on fsspec :-) if there aren't opposition Go for it. I should find time and figure out where I was at for bokeh javascript dependencies. I wonder if the js team would review my first couple of packages while I try to understand how js packaging works. Diane ___ Python-modules-team mailing list Python-modules-team@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: Bug#942235: Bug#942235: dask: autopkgtest needs update for new version of pytest
I can work on fsspec :-) if there aren't opposition
El lun., 4 de nov. de 2019 a la(s) 19:39, Scott Kitterman
(deb...@kitterman.com) escribió:
>
>
>
> On November 4, 2019 10:00:27 PM UTC, Diane Trout wrote:
> >On Tue, 2019-10-29 at 09:15 +0800, Drew Parsons wrote:
> >> On 2019-10-29 03:01, Rebecca N. Palmer wrote:
> >> > Assuming we're talking about
> >> >
> >> >
> >https://salsa.debian.org/python-team/modules/dask/blob/experimental/debian/patches/use-local-intersphinx.patch
> >> >
> >> > I think the actual problem is on the numpy line: it adds the local
> >> > inventory but doesn't remove the online one, so the tuple is too
> >> > long.
> >> > (I haven't actually tried this.)
> >>
> >> Thanks Rebecca, that makes sense. I can scrutinize that patch more
> >> closely.
> >>
> >
> >I pushed a fix for the use-local-intersphinx.patch to the experimental
> >branch.
> >
> >Intersphinx references are basically:
> >
> >"domain": ("canonical url", "somewhere else to look or None"),
> >
> >Having 3 values in the tuple confused it.
> >
> >Though to get a new version of dask it looks like we need something
> >called fsspec.
> >
> >
> >(There were many of these errors running autopkgtest)
> >During handling of the above exception, another exception occurred:
> >/usr/lib/python3/dist-packages/dask/dataframe/__init__.py:55: in
> >
> >raise ImportError(str(e) + "\n\n" + msg)
> >E ImportError: fsspec is required to use any file-system
> >functionality. Please install using
> >E conda install -c conda-forge 'fsspec>=0.3.3'
> >E or
> >E pip install 'fsspec>=0.3.3'
> >E
> >E Dask dataframe requirements are not installed.
> >E
>
> Looks like https://pypi.org/project/fsspec/
>
> Scott K
>
> ___
> Python-modules-team mailing list
> Python-modules-team@alioth-lists.debian.net
> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
___
Python-modules-team mailing list
Python-modules-team@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: Bug#942235: dask: autopkgtest needs update for new version of pytest
On November 4, 2019 10:00:27 PM UTC, Diane Trout wrote:
>On Tue, 2019-10-29 at 09:15 +0800, Drew Parsons wrote:
>> On 2019-10-29 03:01, Rebecca N. Palmer wrote:
>> > Assuming we're talking about
>> >
>> >
>https://salsa.debian.org/python-team/modules/dask/blob/experimental/debian/patches/use-local-intersphinx.patch
>> >
>> > I think the actual problem is on the numpy line: it adds the local
>> > inventory but doesn't remove the online one, so the tuple is too
>> > long.
>> > (I haven't actually tried this.)
>>
>> Thanks Rebecca, that makes sense. I can scrutinize that patch more
>> closely.
>>
>
>I pushed a fix for the use-local-intersphinx.patch to the experimental
>branch.
>
>Intersphinx references are basically:
>
>"domain": ("canonical url", "somewhere else to look or None"),
>
>Having 3 values in the tuple confused it.
>
>Though to get a new version of dask it looks like we need something
>called fsspec.
>
>
>(There were many of these errors running autopkgtest)
>During handling of the above exception, another exception occurred:
>/usr/lib/python3/dist-packages/dask/dataframe/__init__.py:55: in
>
>raise ImportError(str(e) + "\n\n" + msg)
>E ImportError: fsspec is required to use any file-system
>functionality. Please install using
>E conda install -c conda-forge 'fsspec>=0.3.3'
>E or
>E pip install 'fsspec>=0.3.3'
>E
>E Dask dataframe requirements are not installed.
>E
Looks like https://pypi.org/project/fsspec/
Scott K
___
Python-modules-team mailing list
Python-modules-team@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: dask: autopkgtest needs update for new version of pytest
On Tue, 2019-10-29 at 09:15 +0800, Drew Parsons wrote:
> On 2019-10-29 03:01, Rebecca N. Palmer wrote:
> > Assuming we're talking about
> >
> > https://salsa.debian.org/python-team/modules/dask/blob/experimental/debian/patches/use-local-intersphinx.patch
> >
> > I think the actual problem is on the numpy line: it adds the local
> > inventory but doesn't remove the online one, so the tuple is too
> > long.
> > (I haven't actually tried this.)
>
> Thanks Rebecca, that makes sense. I can scrutinize that patch more
> closely.
>
I pushed a fix for the use-local-intersphinx.patch to the experimental
branch.
Intersphinx references are basically:
"domain": ("canonical url", "somewhere else to look or None"),
Having 3 values in the tuple confused it.
Though to get a new version of dask it looks like we need something
called fsspec.
(There were many of these errors running autopkgtest)
During handling of the above exception, another exception occurred:
/usr/lib/python3/dist-packages/dask/dataframe/__init__.py:55: in
raise ImportError(str(e) + "\n\n" + msg)
E ImportError: fsspec is required to use any file-system
functionality. Please install using
E conda install -c conda-forge 'fsspec>=0.3.3'
E or
E pip install 'fsspec>=0.3.3'
E
E Dask dataframe requirements are not installed.
E
___
Python-modules-team mailing list
Python-modules-team@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: Bug#942235: python-xarray needed new dask as well?
Oops!!! Sorry, I reverted the change Cheers, Arias Emmanuel @eamanu http://eamanu.com El dom., 3 de nov. de 2019 a la(s) 19:49, Drew Parsons (dpars...@debian.org) escribió: > > On 2019-11-04 01:15, Emmanuel Arias wrote: > > Hi, > > > > I've just prepare the new upstream release (for some reason > > the upstream branch was not merge to master). [1] > > > > Emmanuel, the work-in-progress is in the experimental branch. Push to > experimental first before pulling into master. > > Drew ___ Python-modules-team mailing list Python-modules-team@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: Bug#942235: python-xarray needed new dask as well?
On 2019-11-04 01:15, Emmanuel Arias wrote: Hi, I've just prepare the new upstream release (for some reason the upstream branch was not merge to master). [1] Emmanuel, the work-in-progress is in the experimental branch. Push to experimental first before pulling into master. Drew ___ Python-modules-team mailing list Python-modules-team@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: Bug#942235: python-xarray needed new dask as well?
Hi, I've just prepare the new upstream release (for some reason the upstream branch was not merge to master). [1] But is necessary some work with patches, are failing. I can work on it tomorrow. [1] https://salsa.debian.org/python-team/modules/dask Cheers, Arias Emmanuel @eamanu http://eamanu.com El dom., 3 de nov. de 2019 a la(s) 11:24, Matthias Klose (d...@debian.org) escribió: > > it looks to me that python-xarray 0.14 needs a newer dask version as well. > > ___ > Python-modules-team mailing list > Python-modules-team@alioth-lists.debian.net > https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team ___ Python-modules-team mailing list Python-modules-team@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: python-xarray needed new dask as well?
it looks to me that python-xarray 0.14 needs a newer dask version as well. ___ Python-modules-team mailing list Python-modules-team@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: dask: autopkgtest needs update for new version of pytest
On 2019-10-29 03:01, Rebecca N. Palmer wrote: Assuming we're talking about https://salsa.debian.org/python-team/modules/dask/blob/experimental/debian/patches/use-local-intersphinx.patch I think the actual problem is on the numpy line: it adds the local inventory but doesn't remove the online one, so the tuple is too long. (I haven't actually tried this.) Thanks Rebecca, that makes sense. I can scrutinize that patch more closely. Drew ___ Python-modules-team mailing list Python-modules-team@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: dask: autopkgtest needs update for new version of pytest
Assuming we're talking about https://salsa.debian.org/python-team/modules/dask/blob/experimental/debian/patches/use-local-intersphinx.patch I think the actual problem is on the numpy line: it adds the local inventory but doesn't remove the online one, so the tuple is too long. (I haven't actually tried this.) ___ Python-modules-team mailing list Python-modules-team@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: dask: autopkgtest needs update for new version of pytest
Source: dask Followup-For: Bug #942235 Fixed upstream, latest version is pushed to the experimental branch. Needs sphinx-click, which is in the NEW queue. Drew -- System Information: Debian Release: bullseye/sid APT prefers unstable APT policy: (500, 'unstable'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 5.3.0-1-amd64 (SMP w/4 CPU cores) Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=en_AU.UTF-8, LC_CTYPE=en_AU.UTF-8 (charmap=UTF-8), LANGUAGE=en_AU:en (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled ___ Python-modules-team mailing list Python-modules-team@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
[Python-modules-team] Bug#942235: dask: autopkgtest needs update for new version of pytest
Source: dask
Version: 1.0.0+dfsg-2
Severity: serious
X-Debbugs-CC: debian...@lists.debian.org, pyt...@packages.debian.org
Tags: sid bullseye
User: debian...@lists.debian.org
Usertags: needs-update
Control: affects -1 src:pytest
Dear maintainers,
With a not so recent upload of pytest the autopkgtest of dask fails in
testing when that autopkgtest is run with the binary packages of pytest
from unstable. It passes when run with only packages from testing. In
tabular form:
passfail
pytest from testing4.6.5-2
dask from testing1.0.0+dfsg-2
versioned deps [0] from testingfrom unstable
all others from testingfrom testing
I copied some of the output at the bottom of this report.
Currently this regression is blocking the migration of pytest to testing
[1]. Of course, pytest shouldn't just break your autopkgtest (or even
worse, your package), but it seems to me that the change in pytest was
intended and your package needs to update to the new situation. The
package had two months to adapt before this bug was even filed.
If this is a real problem in your package (and not only in your
autopkgtest), the right binary package(s) from pytest should really add
a versioned Breaks on the unfixed version of (one of your) package(s).
Note: the Breaks is nice even if the issue is only in the autopkgtest as
it helps the migration software to figure out the right versions to
combine in the tests.
More information about this bug and the reason for filing it can be found on
https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation
Paul
[0] You can see what packages were added from the second line of the log
file quoted below. The migration software adds source package from
unstable to the list if they are needed to install packages from
pytest/4.6.5-2. I.e. due to versioned dependencies or breaks/conflicts.
[1] https://qa.debian.org/excuses.php?package=pytest
https://ci.debian.net/data/autopkgtest/testing/amd64/d/dask/3146640/log.gz
autopkgtest [19:13:10]: test command1: [---
Testing with python3.7:
= test session starts
==
platform linux -- Python 3.7.5rc1, pytest-4.6.5, py-1.8.0, pluggy-0.12.0
-- /usr/bin/python3.7
cachedir: .pytest_cache
rootdir: /tmp/autopkgtest-lxc.n16ld1px/downtmp/autopkgtest_tmp
collecting ... collected 5850 items / 2 errors / 7 skipped / 5841 selected
ERRORS
_ ERROR collecting dataframe/io/tests/test_parquet.py
__
/usr/lib/python3/dist-packages/pluggy/hooks.py:289: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
/usr/lib/python3/dist-packages/pluggy/manager.py:87: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
/usr/lib/python3/dist-packages/pluggy/manager.py:81: in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
/usr/lib/python3/dist-packages/_pytest/python.py:234: in
pytest_pycollect_makeitem
res = list(collector._genfunctions(name, obj))
/usr/lib/python3/dist-packages/_pytest/python.py:410: in _genfunctions
self.ihook.pytest_generate_tests(metafunc=metafunc)
/usr/lib/python3/dist-packages/pluggy/hooks.py:289: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
/usr/lib/python3/dist-packages/pluggy/manager.py:87: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
/usr/lib/python3/dist-packages/pluggy/manager.py:81: in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
/usr/lib/python3/dist-packages/_pytest/python.py:137: in
pytest_generate_tests
metafunc.parametrize(*marker.args, **marker.kwargs)
/usr/lib/python3/dist-packages/_pytest/python.py:1004: in parametrize
function_definition=self.definition,
/usr/lib/python3/dist-packages/_pytest/mark/structures.py:130: in
_for_parametrize
if len(param.values) != len(argnames):
E TypeError: object of type 'MarkDecorator' has no len()
__ ERROR collecting tests/test_dot.py
__
/usr/lib/python3/dist-packages/pluggy/hooks.py:289: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
/usr/lib/python3/dist-packages/pluggy/manager.py:87: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
/usr/lib/python3/dist-packages/pluggy/manager.py:81: in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
/usr/lib/python3/dist-packages/_pytest/python.py:234: in
pytest_pycollect_makeitem
res = list(collector._genfunctions(name, obj))
/usr/lib/python3/dist-packages/_pytest/python.py:410: in _genfunctions
self.ihook.pytest_generate_tests(metafunc=metafunc)
/usr/lib/python3/dist-packages/pluggy/hooks.py:289: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
