Bug#942235: dask: autopkgtest needs update for new version of pytest
Ubuntu have dask 2.6.0 and fsspec, but still have a few autopkgtest failures: https://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html dask itself - looks like trying to use a nonexistent temporary directory pyfftw - looks like a test treating a new (unexpected) warning as a failure satpy - looks like a missing dependency on fsspec
Bug#942235: [Python-modules-team] Bug#942235: Bug#942235: dask: autopkgtest needs update for new version of pytest
On Mon, 2019-11-04 at 19:42 -0300, Emmanuel Arias wrote: > I can work on fsspec :-) if there aren't opposition Go for it. I should find time and figure out where I was at for bokeh javascript dependencies. I wonder if the js team would review my first couple of packages while I try to understand how js packaging works. Diane
Bug#942235: [Python-modules-team] Bug#942235: Bug#942235: dask: autopkgtest needs update for new version of pytest
I can work on fsspec :-) if there aren't opposition
El lun., 4 de nov. de 2019 a la(s) 19:39, Scott Kitterman
(deb...@kitterman.com) escribió:
>
>
>
> On November 4, 2019 10:00:27 PM UTC, Diane Trout wrote:
> >On Tue, 2019-10-29 at 09:15 +0800, Drew Parsons wrote:
> >> On 2019-10-29 03:01, Rebecca N. Palmer wrote:
> >> > Assuming we're talking about
> >> >
> >> >
> >https://salsa.debian.org/python-team/modules/dask/blob/experimental/debian/patches/use-local-intersphinx.patch
> >> >
> >> > I think the actual problem is on the numpy line: it adds the local
> >> > inventory but doesn't remove the online one, so the tuple is too
> >> > long.
> >> > (I haven't actually tried this.)
> >>
> >> Thanks Rebecca, that makes sense. I can scrutinize that patch more
> >> closely.
> >>
> >
> >I pushed a fix for the use-local-intersphinx.patch to the experimental
> >branch.
> >
> >Intersphinx references are basically:
> >
> >"domain": ("canonical url", "somewhere else to look or None"),
> >
> >Having 3 values in the tuple confused it.
> >
> >Though to get a new version of dask it looks like we need something
> >called fsspec.
> >
> >
> >(There were many of these errors running autopkgtest)
> >During handling of the above exception, another exception occurred:
> >/usr/lib/python3/dist-packages/dask/dataframe/__init__.py:55: in
> >
> >raise ImportError(str(e) + "\n\n" + msg)
> >E ImportError: fsspec is required to use any file-system
> >functionality. Please install using
> >E conda install -c conda-forge 'fsspec>=0.3.3'
> >E or
> >E pip install 'fsspec>=0.3.3'
> >E
> >E Dask dataframe requirements are not installed.
> >E
>
> Looks like https://pypi.org/project/fsspec/
>
> Scott K
>
> ___
> Python-modules-team mailing list
> python-modules-t...@alioth-lists.debian.net
> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team
Bug#942235: [Python-modules-team] Bug#942235: dask: autopkgtest needs update for new version of pytest
On November 4, 2019 10:00:27 PM UTC, Diane Trout wrote:
>On Tue, 2019-10-29 at 09:15 +0800, Drew Parsons wrote:
>> On 2019-10-29 03:01, Rebecca N. Palmer wrote:
>> > Assuming we're talking about
>> >
>> >
>https://salsa.debian.org/python-team/modules/dask/blob/experimental/debian/patches/use-local-intersphinx.patch
>> >
>> > I think the actual problem is on the numpy line: it adds the local
>> > inventory but doesn't remove the online one, so the tuple is too
>> > long.
>> > (I haven't actually tried this.)
>>
>> Thanks Rebecca, that makes sense. I can scrutinize that patch more
>> closely.
>>
>
>I pushed a fix for the use-local-intersphinx.patch to the experimental
>branch.
>
>Intersphinx references are basically:
>
>"domain": ("canonical url", "somewhere else to look or None"),
>
>Having 3 values in the tuple confused it.
>
>Though to get a new version of dask it looks like we need something
>called fsspec.
>
>
>(There were many of these errors running autopkgtest)
>During handling of the above exception, another exception occurred:
>/usr/lib/python3/dist-packages/dask/dataframe/__init__.py:55: in
>
>raise ImportError(str(e) + "\n\n" + msg)
>E ImportError: fsspec is required to use any file-system
>functionality. Please install using
>E conda install -c conda-forge 'fsspec>=0.3.3'
>E or
>E pip install 'fsspec>=0.3.3'
>E
>E Dask dataframe requirements are not installed.
>E
Looks like https://pypi.org/project/fsspec/
Scott K
Bug#942235: dask: autopkgtest needs update for new version of pytest
On Tue, 2019-10-29 at 09:15 +0800, Drew Parsons wrote:
> On 2019-10-29 03:01, Rebecca N. Palmer wrote:
> > Assuming we're talking about
> >
> > https://salsa.debian.org/python-team/modules/dask/blob/experimental/debian/patches/use-local-intersphinx.patch
> >
> > I think the actual problem is on the numpy line: it adds the local
> > inventory but doesn't remove the online one, so the tuple is too
> > long.
> > (I haven't actually tried this.)
>
> Thanks Rebecca, that makes sense. I can scrutinize that patch more
> closely.
>
I pushed a fix for the use-local-intersphinx.patch to the experimental
branch.
Intersphinx references are basically:
"domain": ("canonical url", "somewhere else to look or None"),
Having 3 values in the tuple confused it.
Though to get a new version of dask it looks like we need something
called fsspec.
(There were many of these errors running autopkgtest)
During handling of the above exception, another exception occurred:
/usr/lib/python3/dist-packages/dask/dataframe/__init__.py:55: in
raise ImportError(str(e) + "\n\n" + msg)
E ImportError: fsspec is required to use any file-system
functionality. Please install using
E conda install -c conda-forge 'fsspec>=0.3.3'
E or
E pip install 'fsspec>=0.3.3'
E
E Dask dataframe requirements are not installed.
E
Bug#942235: dask: autopkgtest needs update for new version of pytest
On 2019-10-29 03:01, Rebecca N. Palmer wrote: Assuming we're talking about https://salsa.debian.org/python-team/modules/dask/blob/experimental/debian/patches/use-local-intersphinx.patch I think the actual problem is on the numpy line: it adds the local inventory but doesn't remove the online one, so the tuple is too long. (I haven't actually tried this.) Thanks Rebecca, that makes sense. I can scrutinize that patch more closely. Drew
Bug#942235: dask: autopkgtest needs update for new version of pytest
Assuming we're talking about https://salsa.debian.org/python-team/modules/dask/blob/experimental/debian/patches/use-local-intersphinx.patch I think the actual problem is on the numpy line: it adds the local inventory but doesn't remove the online one, so the tuple is too long. (I haven't actually tried this.)
Bug#942235: dask: autopkgtest needs update for new version of pytest
Source: dask
Followup-For: Bug #942235
Control: tags -1 + help
On 2019-10-27 10:03, Drew Parsons wrote:
Fixed upstream, latest version is pushed to the experimental branch.
Needs sphinx-click, which is in the NEW queue.
sphinx-click is now available.
The dask 2.6.0 build from experimental proceeds, but runs into an error
when buildings docs with sphinx:
PYTHONPATH=/home/projects/python/build/dask /usr/bin/make -C docs html
make[2]: Entering directory '/home/projects/python/build/dask/docs'
sphinx-build -b html -d build/doctrees source build/html
Running Sphinx v1.8.5
making output directory...
loading intersphinx inventory from
/usr/share/doc/python-pandas-doc/html/objects.inv...
Exception occurred:
File "/usr/lib/python3/dist-packages/sphinx/ext/intersphinx.py",
line 224, in load_mappings
name, (uri, inv) = key, value
ValueError: too many values to unpack (expected 2)
The accompanying error log does not give me more illumination:
# Sphinx version: 1.8.5
# Python version: 3.7.5 (CPython)
# Docutils version: 0.15.2 release
# Jinja2 version: 2.10.1
# Last messages:
# Loaded extensions:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/sphinx/cmd/build.py", line 303,
in build_main
args.tags, args.verbosity, args.jobs, args.keep_going)
File "/usr/lib/python3/dist-packages/sphinx/application.py", line 263,
in __init__
self._init_builder()
File "/usr/lib/python3/dist-packages/sphinx/application.py", line 325,
in _init_builder
self.emit('builder-inited')
File "/usr/lib/python3/dist-packages/sphinx/application.py", line 510,
in emit
return self.events.emit(event, self, *args)
File "/usr/lib/python3/dist-packages/sphinx/events.py", line 80, in
emit
results.append(callback(*args))
File "/usr/lib/python3/dist-packages/sphinx/ext/intersphinx.py", line
224, in load_mappings
name, (uri, inv) = key, value
ValueError: too many values to unpack (expected 2)
I can't tell if it's an internal error in sphinx, or triggered by dask,
so need help with the bug (cc:ing Diane, the dask uploader).
The last output in the build log refers to
python-pandas-doc/html/objects.inv. I'm not sure if it means pandas-doc
passed successfully, or if it indicates pandas is triggering the
ValueError. cc:ing Rebecca (pandas uploader) to ask if she's seen this
problem with pandas elsewhere.
Drew
Bug#942235: dask: autopkgtest needs update for new version of pytest
Source: dask Followup-For: Bug #942235 Fixed upstream, latest version is pushed to the experimental branch. Needs sphinx-click, which is in the NEW queue. Drew -- System Information: Debian Release: bullseye/sid APT prefers unstable APT policy: (500, 'unstable'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 5.3.0-1-amd64 (SMP w/4 CPU cores) Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=en_AU.UTF-8, LC_CTYPE=en_AU.UTF-8 (charmap=UTF-8), LANGUAGE=en_AU:en (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled
Bug#942235: dask: autopkgtest needs update for new version of pytest
Source: dask
Version: 1.0.0+dfsg-2
Severity: serious
X-Debbugs-CC: debian...@lists.debian.org, pyt...@packages.debian.org
Tags: sid bullseye
User: debian...@lists.debian.org
Usertags: needs-update
Control: affects -1 src:pytest
Dear maintainers,
With a not so recent upload of pytest the autopkgtest of dask fails in
testing when that autopkgtest is run with the binary packages of pytest
from unstable. It passes when run with only packages from testing. In
tabular form:
passfail
pytest from testing4.6.5-2
dask from testing1.0.0+dfsg-2
versioned deps [0] from testingfrom unstable
all others from testingfrom testing
I copied some of the output at the bottom of this report.
Currently this regression is blocking the migration of pytest to testing
[1]. Of course, pytest shouldn't just break your autopkgtest (or even
worse, your package), but it seems to me that the change in pytest was
intended and your package needs to update to the new situation. The
package had two months to adapt before this bug was even filed.
If this is a real problem in your package (and not only in your
autopkgtest), the right binary package(s) from pytest should really add
a versioned Breaks on the unfixed version of (one of your) package(s).
Note: the Breaks is nice even if the issue is only in the autopkgtest as
it helps the migration software to figure out the right versions to
combine in the tests.
More information about this bug and the reason for filing it can be found on
https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation
Paul
[0] You can see what packages were added from the second line of the log
file quoted below. The migration software adds source package from
unstable to the list if they are needed to install packages from
pytest/4.6.5-2. I.e. due to versioned dependencies or breaks/conflicts.
[1] https://qa.debian.org/excuses.php?package=pytest
https://ci.debian.net/data/autopkgtest/testing/amd64/d/dask/3146640/log.gz
autopkgtest [19:13:10]: test command1: [---
Testing with python3.7:
= test session starts
==
platform linux -- Python 3.7.5rc1, pytest-4.6.5, py-1.8.0, pluggy-0.12.0
-- /usr/bin/python3.7
cachedir: .pytest_cache
rootdir: /tmp/autopkgtest-lxc.n16ld1px/downtmp/autopkgtest_tmp
collecting ... collected 5850 items / 2 errors / 7 skipped / 5841 selected
ERRORS
_ ERROR collecting dataframe/io/tests/test_parquet.py
__
/usr/lib/python3/dist-packages/pluggy/hooks.py:289: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
/usr/lib/python3/dist-packages/pluggy/manager.py:87: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
/usr/lib/python3/dist-packages/pluggy/manager.py:81: in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
/usr/lib/python3/dist-packages/_pytest/python.py:234: in
pytest_pycollect_makeitem
res = list(collector._genfunctions(name, obj))
/usr/lib/python3/dist-packages/_pytest/python.py:410: in _genfunctions
self.ihook.pytest_generate_tests(metafunc=metafunc)
/usr/lib/python3/dist-packages/pluggy/hooks.py:289: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
/usr/lib/python3/dist-packages/pluggy/manager.py:87: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
/usr/lib/python3/dist-packages/pluggy/manager.py:81: in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
/usr/lib/python3/dist-packages/_pytest/python.py:137: in
pytest_generate_tests
metafunc.parametrize(*marker.args, **marker.kwargs)
/usr/lib/python3/dist-packages/_pytest/python.py:1004: in parametrize
function_definition=self.definition,
/usr/lib/python3/dist-packages/_pytest/mark/structures.py:130: in
_for_parametrize
if len(param.values) != len(argnames):
E TypeError: object of type 'MarkDecorator' has no len()
__ ERROR collecting tests/test_dot.py
__
/usr/lib/python3/dist-packages/pluggy/hooks.py:289: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
/usr/lib/python3/dist-packages/pluggy/manager.py:87: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
/usr/lib/python3/dist-packages/pluggy/manager.py:81: in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
/usr/lib/python3/dist-packages/_pytest/python.py:234: in
pytest_pycollect_makeitem
res = list(collector._genfunctions(name, obj))
/usr/lib/python3/dist-packages/_pytest/python.py:410: in _genfunctions
self.ihook.pytest_generate_tests(metafunc=metafunc)
/usr/lib/python3/dist-packages/pluggy/hooks.py:289: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
/usr/lib/python3/dist-packages/pluggy/manager.py
