Re: python3-mastodon fails CI due to bad magic?
tor. 28. jan. 2021 kl. 20:31 skrev Craig Small :
>
> I saw that too, and yet the logs have:
> import magic
> File "/usr/lib/python3/dist-packages/magic/__init__.py", line 361, in
>
>
> That's a python3-magic file, so it exists.
I guess you are right about that.
Have you noticed that the reprotest is the only place the test suite
is actually finding any tests? During build it doesn't find any. This
might not be related to reprotest at all.
>
> However, a bit of googling showed up this [1] where due to the fact that
> python3-magic uses ctype.util.find.library if the environment is "strange"
> (e.g. in a snap) the load fails.
> Possibly the reprotest environment qualifies as "strange" and the load line
> is failing in an odd way?
>
>return ctypes.cdll.LoadLibrary(find_library('magic'))
>
> Anyone else use python3-magic and reprotest? Does it work and why?
>
> - Craig
>
>
> 1: https://bugs.launchpad.net/snapcraft/+bug/1861026
>
> On Fri, 29 Jan 2021 at 03:08, Emmanuel Arias wrote:
>>
>> Seems that reprotest on salsa, is not installing the python3-magic.
>>
>> I don't think that can be a problem with cpython.
>>
>> On Thu, Jan 28, 2021 at 8:39 AM Håvard Flaget Aasen
>> wrote:
>>>
>>> tor. 28. jan. 2021 kl. 10:48 skrev Craig Small :
>>> >
>>> > Hi,
>>> > I'm trying to understand why python3-mastodon fails the unit tests.
>>>
>>> I'm not sure what you found out so far. When I grepped the log you
>>> linked, I couldn't find python3-magic or libmagic. I believe the
>>> packages isn't installed and the test therefore fails.
>>> I'm guessing now, but I think this has to do with reproducible build in
>>> salsa.
>>>
>>> >
>>> > An example is at [1] and it seems that importing magic is enough to
>>> > trigger it. The compat line is to handle both python3-magic and a
>>> > libmagic module both wanting magic.
>>> >
>>> > Is this a cpython thing? I'm not sure what I can do to fix this.
>>> >
>>> > File
>>> > "/tmp/reprotest.7a4W2G/build-experiment-1/build-experiment-1/.pybuild/cpython3_3.9_mastodon/build/mastodon/Mastodon.py",
>>> > line 55, in
>>> > import magic
>>> > File "/usr/lib/python3/dist-packages/magic/__init__.py", line 361, in
>>> >
>>> > add_compat(globals())
>>> > File "/usr/lib/python3/dist-packages/magic/__init__.py", line 325, in
>>> > add_compat
>>> > from magic import compat
>>> > File "/usr/lib/python3/dist-packages/magic/compat.py", line 61, in
>>> >
>>> > _open = _libraries['magic'].magic_open
>>> > File "/usr/lib/python3.9/ctypes/__init__.py", line 387, in __getattr__
>>> > func = self.__getitem__(name)
>>> > File "/usr/lib/python3.9/ctypes/__init__.py", line 392, in __getitem__
>>> > func = self._FuncPtr((name_or_ordinal, self))
>>> > AttributeError: python3.9: undefined symbol: magic_open
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > 1: https://salsa.debian.org/python-team/packages/mastodon/-/jobs/1312190
>>> >
>>>
Re: python3-mastodon fails CI due to bad magic?
tor. 28. jan. 2021 kl. 10:48 skrev Craig Small : > > Hi, > I'm trying to understand why python3-mastodon fails the unit tests. I'm not sure what you found out so far. When I grepped the log you linked, I couldn't find python3-magic or libmagic. I believe the packages isn't installed and the test therefore fails. I'm guessing now, but I think this has to do with reproducible build in salsa. > > An example is at [1] and it seems that importing magic is enough to trigger > it. The compat line is to handle both python3-magic and a libmagic module > both wanting magic. > > Is this a cpython thing? I'm not sure what I can do to fix this. > > File > "/tmp/reprotest.7a4W2G/build-experiment-1/build-experiment-1/.pybuild/cpython3_3.9_mastodon/build/mastodon/Mastodon.py", > line 55, in > import magic > File "/usr/lib/python3/dist-packages/magic/__init__.py", line 361, in > > add_compat(globals()) > File "/usr/lib/python3/dist-packages/magic/__init__.py", line 325, in > add_compat > from magic import compat > File "/usr/lib/python3/dist-packages/magic/compat.py", line 61, in > _open = _libraries['magic'].magic_open > File "/usr/lib/python3.9/ctypes/__init__.py", line 387, in __getattr__ > func = self.__getitem__(name) > File "/usr/lib/python3.9/ctypes/__init__.py", line 392, in __getitem__ > func = self._FuncPtr((name_or_ordinal, self)) > AttributeError: python3.9: undefined symbol: magic_open > > > > > > 1: https://salsa.debian.org/python-team/packages/mastodon/-/jobs/1312190 >
Re: python-mongoengine 0.21.0-1 review
Hi, Thanks for the review! tor. 24. des. 2020 kl. 05:31 skrev Louis-Philippe Véronneau : > > Hi, > > I've just finished reviewing your RFS for python-mongoengine 0.21.0-1 > and it all looks good! > > I can't help but notice you are using the tarballs from pypi though, > which means you don't have access to the upstream testsuite :( > > Do you think it would be possible to migrate to the github one and have > your package run the tests? I tested with the tarball from GitHub today, and almost all of the tests is testing against a MongoDB server, which we don't have. This of course means that almost all test fails. I'm not sure how much value it is to run the remaining tests during build. As for the autopkgtest, there might be possible to download, install, and run MongoDB during the test, that's what upstream is doing when they test the package, but that must be stretching what we can and should do when running autopkgtest in Debian. I'm open for suggestion here, but since we don't have the necessary package in the repository I believe it will be difficult to test this package properly. > > I would also suggest running them as an autopkgtests (here's an example > [1]). That would give you a non superficial one :) > > If you don't have time to do that right now, I'd be happy to sponsor > your package as-is if you commit to trying that out for the next release. > > Keep me posted! > > NB: I've removed your package from the sponsor queue on IRC so that > others don't inadvertently review it a second time :) > > [1]: > https://salsa.debian.org/python-team/packages/python-itemloaders/-/blob/debian/master/debian/tests/unittests > > -- > ⢀⣴⠾⠻⢶⣦⠀ > ⣾⠁⢠⠒⠀⣿⡁ Louis-Philippe Véronneau > ⢿⡄⠘⠷⠚⠋ po...@debian.org / veronneau.org > ⠈⠳⣄ >
Re: Python3.8 as default final status
On 27.03.2020 20:44, Scott Kitterman wrote: > On Friday, March 27, 2020 3:39:30 PM EDT Scott Kitterman wrote: >> On Friday, March 27, 2020 3:21:38 PM EDT Håvard Flaget Aasen wrote: >>> On 27.03.2020 20:09, Sergio Durigan Junior wrote: >>>> On Friday, March 27 2020, Scott Kitterman wrote: >>>>> The python3-defaults with python3.8 as the default python3 has migrated >>>>> to >>>>> Testing thanks to the release team hammering things around until it >>>>> went. >>>> >>>> Thanks for this. >>>> >>>>> Most of the outstanding autipkgtest failures with python3.8 were fixed >>>>> either in unstable or in git/BTS. Here are the remaining issues that >>>>> someone (who isn't me) should have a look at: >>>>> >>>>> celery/4.2.1-5: #952217 autorm 4/13 >>>> >>>> FWIW, I looked at this a little bit, but could not make much progress. >>>> I'm very interested in fixing this since it impacts pagure. I'll try to >>>> investigate more this weekend, but if someone else wants to take a look >>>> (and let me know), you're more than welcome! >>> >>> I believe I already fixed that package, it's waiting for someone to >>> review and upload it. Did you look at the repository in salsa? >>> >>> >>> Håvard >> >> I didn't. The ones I knew about having fixed in git or patches in the BTS, >> I didn't include. I didn't go through and check them all. >> >> Scott K > > I should read more carefully. I failed to note that this wasn't directed at > me. Nevermind. > > Scott K > Yet, I probably could have communicated better. I just placed it in the subject line on IRC, and assumed it would be found. Håvard
Re: Python3.8 as default final status
On 27.03.2020 20:09, Sergio Durigan Junior wrote: > On Friday, March 27 2020, Scott Kitterman wrote: > >> The python3-defaults with python3.8 as the default python3 has migrated to >> Testing thanks to the release team hammering things around until it went. > > Thanks for this. > >> Most of the outstanding autipkgtest failures with python3.8 were fixed >> either >> in unstable or in git/BTS. Here are the remaining issues that someone (who >> isn't me) should have a look at: >> >> celery/4.2.1-5: #952217 autorm 4/13 > > FWIW, I looked at this a little bit, but could not make much progress. > I'm very interested in fixing this since it impacts pagure. I'll try to > investigate more this weekend, but if someone else wants to take a look > (and let me know), you're more than welcome! > I believe I already fixed that package, it's waiting for someone to review and upload it. Did you look at the repository in salsa? Håvard
Re: python-babel
On 11.02.2020 16:55, Håvard Flaget Aasen wrote: On 11.02.2020 15:44, Thomas Goirand wrote: On 2/11/20 11:16 AM, Håvard Flaget Aasen wrote: I tried to build python-babel to sponsor the upload from what's in the Git in the DPMT, and it failed to build for me: # Generate the localedata folder data out of the xml files # downloaded in debian/repack scripts/import_cldr.py common Processing root.xml (Language = root; Territory = 001) root: Unsupported number system "adlm" in [... lots of lines like this one ...] I'm also getting these lines. I tested with the current 2.6.0 which also gives the same messages, the build log for 2.6.0 in Ubuntu prints the same messages. Now for the files actually being processed It is 719 in version 2.4.0 745 in version 2.6.0 and 764 in version 2.8.0 Not that it is supposed to be this way of course. Though I do believe the actual content in the package is correct. I can have a look at what we are actually importing in d/repack and see if anything has changed from version 2.4.0 to 2.8.0 root: Unsupported number system "vaii" in Traceback (most recent call last): File "scripts/import_cldr.py", line 968, in main() File "scripts/import_cldr.py", line 193, in main dump_json=bool(options.dump_json) File "scripts/import_cldr.py", line 207, in process_data _process_local_datas(sup, srcdir, destdir, force=force, dump_json=dump_json) File "scripts/import_cldr.py", line 435, in _process_local_datas write_datafile(data_filename, data, dump_json=dump_json) File "scripts/import_cldr.py", line 167, in write_datafile with open(path, 'wb') as outfile: IOError: [Errno 2] No such file or directory: '/<>/babel/locale-data/root.dat' I can't replicate this. I tried with cowbuilder/pbuilder and sbuild and the build completes fine on my systems. Håvard In such case, could you provide me with the source package, rather than just letting me try with Git? Maybe this is going to work then... Thats ok, how do you want it? the orig.tar.gz is just over 25 MB When i try I only use git, with these commands: $ gbp clone $ gbp buildpackage Håvard I'm sorry to have wasted your time yesterday. It is indeed my system that does something it shouldn't. I created a CI in salsa which duplicated your error. I believe the repository with the pristine-tar and upstream branch is correct I will fix the package today and readd it on IRC when it's done. I was able to reproduce the error on my own system with $ dpkg-buildpackage -us -ui Håvard
Re: python-babel
On 11.02.2020 15:44, Thomas Goirand wrote: On 2/11/20 11:16 AM, Håvard Flaget Aasen wrote: I tried to build python-babel to sponsor the upload from what's in the Git in the DPMT, and it failed to build for me: # Generate the localedata folder data out of the xml files # downloaded in debian/repack scripts/import_cldr.py common Processing root.xml (Language = root; Territory = 001) root: Unsupported number system "adlm" in [... lots of lines like this one ...] I'm also getting these lines. I tested with the current 2.6.0 which also gives the same messages, the build log for 2.6.0 in Ubuntu prints the same messages. Now for the files actually being processed It is 719 in version 2.4.0 745 in version 2.6.0 and 764 in version 2.8.0 Not that it is supposed to be this way of course. Though I do believe the actual content in the package is correct. I can have a look at what we are actually importing in d/repack and see if anything has changed from version 2.4.0 to 2.8.0 root: Unsupported number system "vaii" in Traceback (most recent call last): File "scripts/import_cldr.py", line 968, in main() File "scripts/import_cldr.py", line 193, in main dump_json=bool(options.dump_json) File "scripts/import_cldr.py", line 207, in process_data _process_local_datas(sup, srcdir, destdir, force=force, dump_json=dump_json) File "scripts/import_cldr.py", line 435, in _process_local_datas write_datafile(data_filename, data, dump_json=dump_json) File "scripts/import_cldr.py", line 167, in write_datafile with open(path, 'wb') as outfile: IOError: [Errno 2] No such file or directory: '/<>/babel/locale-data/root.dat' I can't replicate this. I tried with cowbuilder/pbuilder and sbuild and the build completes fine on my systems. Håvard In such case, could you provide me with the source package, rather than just letting me try with Git? Maybe this is going to work then... Thats ok, how do you want it? the orig.tar.gz is just over 25 MB When i try I only use git, with these commands: $ gbp clone $ gbp buildpackage Håvard
Re: python-babel
I tried to build python-babel to sponsor the upload from what's in the Git in the DPMT, and it failed to build for me: # Generate the localedata folder data out of the xml files # downloaded in debian/repack scripts/import_cldr.py common Processing root.xml (Language = root; Territory = 001) root: Unsupported number system "adlm" in [... lots of lines like this one ...] I'm also getting these lines. I tested with the current 2.6.0 which also gives the same messages, the build log for 2.6.0 in Ubuntu prints the same messages. Now for the files actually being processed It is 719 in version 2.4.0 745 in version 2.6.0 and 764 in version 2.8.0 Not that it is supposed to be this way of course. Though I do believe the actual content in the package is correct. I can have a look at what we are actually importing in d/repack and see if anything has changed from version 2.4.0 to 2.8.0 I believe this is the upstream change. [1] If it indeed are files were supposed to have in the package i guess it need some more work, if not we could either leave it as is, or comment out the log line. Håvard [1] https://github.com/python-babel/babel/commit/77dc9d4024b78c2339f7cf3bff1a2e8be8e2d0f7#diff-106585574531c31ab48c6bf45e566e40
Re: python-babel
I tried to build python-babel to sponsor the upload from what's in the Git in the DPMT, and it failed to build for me: # Generate the localedata folder data out of the xml files # downloaded in debian/repack scripts/import_cldr.py common Processing root.xml (Language = root; Territory = 001) root: Unsupported number system "adlm" in [... lots of lines like this one ...] I'm also getting these lines. I tested with the current 2.6.0 which also gives the same messages, the build log for 2.6.0 in Ubuntu prints the same messages. Now for the files actually being processed It is 719 in version 2.4.0 745 in version 2.6.0 and 764 in version 2.8.0 Not that it is supposed to be this way of course. Though I do believe the actual content in the package is correct. I can have a look at what we are actually importing in d/repack and see if anything has changed from version 2.4.0 to 2.8.0 root: Unsupported number system "vaii" in Traceback (most recent call last): File "scripts/import_cldr.py", line 968, in main() File "scripts/import_cldr.py", line 193, in main dump_json=bool(options.dump_json) File "scripts/import_cldr.py", line 207, in process_data _process_local_datas(sup, srcdir, destdir, force=force, dump_json=dump_json) File "scripts/import_cldr.py", line 435, in _process_local_datas write_datafile(data_filename, data, dump_json=dump_json) File "scripts/import_cldr.py", line 167, in write_datafile with open(path, 'wb') as outfile: IOError: [Errno 2] No such file or directory: '/<>/babel/locale-data/root.dat' I can't replicate this. I tried with cowbuilder/pbuilder and sbuild and the build completes fine on my systems. Håvard
Re: Updating pip
I'm not going to be able to deal with all that by myself, so this is a request for team members to step up and look at updating these packages. I have not checked which of these are DPMT maintained, so in some cases the action may be to work with the maintainer to update their package. I would suggest that people who decide to work on one of the above reply to this message so we don't end up with people doing duplicate works. I can begin with distro and pyparsing. H�vard
python-babel
Hello, The python-babel [1] package is currently broken, there isn't any bugs reported yet, but after the transit to python3.8 it failed the testsuite. New upstream version 2.7.0 and above fixes these issues. To update the package there is a 'repack' script which i believe is broken, I at least did most of the work manually. For that reason I haven't dared push it to the official repo yet, if anyone have time, it will be nice with a second opinion. It's still some work left, but it builds and tests fine. The version I updated is currently here [2] The script might only require a minor fix but i noticed that the files being downloaded already exist within Debian. The package unicode-cldr-core [3]. Is it possible to have this as a build-dependency, have a Files-Excluded in d/copyright file and drop the repack script altogether? If there still is a reason to change the files in the first place. If this makes it any easier/better of course. I'll appreciate all kinds of feedback. Håvard [1] https://tracker.debian.org/pkg/python-babel [2] https://salsa.debian.org/haava-guest/python-babel [3] https://tracker.debian.org/pkg/unicode-cldr-core
RFS [RC] py-postgresql and python-murmurhash
Hi, Can someone please do an upload of py-postgresql? It is marked with autoremoval tomorrow (16 December). This update fixes the issue. https://salsa.debian.org/python-team/modules/py-postgresql This repo doesn't have the pristine-tar branch, so it might be a bit more work to check the package. However it has CI which passed all tests. I can look into merging the old and new repo if it is of interest (the old has upstream and pristine-tar branch). I did see that the upstream source hasn't been updated for a couple of years and popcon is decreasing, so I am not sure how relevant this package is anymore. python-murmurhash is also marked with autoremoval, but not before 7 January. https://salsa.debian.org/python-team/modules/python-murmurhash I hope the changes is accepted. The tests i have run on the package have at least passed. Both packages is part of the ongoing testing transition python3.8 Thanks, Håvard
RFS [RC] jsonpickle, python-serverfiles
Hello, Can someone please do an upload of jsonpickle? It is marked for autoremoval on 19 January. This update fixes the issue, it also updates to the latest upstream release. https://salsa.debian.org/python-team/modules/jsonpickle I did the same on python-serverfiles, removed an rc bug and updated it to the latest upstream release. Though this has never gone all the way to testing, blocked for over a year because of a missing dependency. https://salsa.debian.org/python-team/modules/python-serverfiles Thanks, Håvard
advice on jsonpickle
I had a look at jsonpickle. Package: https://tracker.debian.org/pkg/jsonpickle The latest release is missing from the repository, it is also FTBFS in sid https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=946304 I forked the repo before i started, in case i broke something. https://salsa.debian.org/haava-guest/jsonpickle I imported the current .dsc file to get the repo up to date. Then i updated it to the newest upstream version and disabled the tests that failed (three of them). The package is now building properly. The tests that failed, do already have a issue ticket upstream, from someone else. Can someone take a quick look at my repo and see if it's ok to push it to the main one? I did think about a merge request, but do I need several requests to get all branches and tags updated? Should i let the bug ticket remain open until the tests has been fixed upstream, or is it enough that the package is being built again? Håvard
Re: Dealing with zope.interface unsatisfiable build-dependency.
If you still wish to disable tests for python 2, you might be looking for this export PYBUILD_DISABLE_python2=test That line in debian/rules should work. You have some more options here: https://wiki.debian.org/Python/Pybuild and perhaps the manpages. On 07.12.2019 09:12, peter green wrote: On 07/12/2019 07:47, peter green wrote: It would be preferable to only disable the testsuite for python2, but I have no idea how to do that, so my current debdiff disables the testsuite completely, I also ran into an issue with the package's clean target not cleaning up properly. Just realized I added moreutils to the build-depends, planning to use it in the clean target fix, but in the end I decided to just delete the file in question. So that build-dep should be dropped before upload.
Re: Prefered way to RFS
Thanks! Is this something that could be added here? https://wiki.debian.org/Python/GitPackaging#Sponsoring.2C_mentoring.2C_reviewing On 04.12.2019 18:42, Louis-Philippe V??ronneau wrote: If you are part of the DPMT or the PAPT, you can modify a package directly. If not, Merge Requests are accepted but since very few people have receive those notifications, sometimes it takes time to have them merged. For the sponsoring part, I've found that adding your requests to the topic of the #debian-python IRC channel on OFTC works really well. You can also send emails to this list. Cheers,
Prefered way to RFS
Hi, Is there any preferred way to RFS? Merge request, just push to the repo and mail this list or something else? I guess that uploading a package to mentors.debian isn't of much use since we are using salsa? Thanks, H??vard
Join DPMT
Hi, I wish to join the team. I don't bring any packages with me, but i intend to contribute to existing ones. I have read and accept the DPMT policy: https://salsa.debian.org/python-team/tools/python-modules/blob/master/policy.rst My salsa account is haava-guest Regards Håvard
