mkautodoc adopted

[Antonio Terceiro]

On Thu, Mar 14, 2024 at 06:20:11AM +, Julian Gilbey wrote:
> #1065143 O: mkautodoc -- AutoDoc for MarkDown

I picked this one up as I have at least one projects at $work that uses
it.


signature.asc
Description: PGP signature

Re: Proposed MBF: packages still using nose

[Antonio Terceiro]

On Sun, Aug 21, 2022 at 04:04:36PM +0300, Dmitry Shachnev wrote:
> Antonio Terceiro 
>python-ofxclient (U)
>rows (U)

can't fix those right away, but I think you should just go ahead and do
the MBF.


signature.asc
Description: PGP signature

Re: Are YOU interested in a potential remote sprint sometime in October/November? (yes YOU!)

[Antonio Terceiro]

On Wed, Aug 17, 2022 at 08:22:28PM -0400, Louis-Philippe Véronneau wrote:
> Hello folks,
> 
> At DC22, a few people seemed interested in a potential Python Team remote
> sprint, sometime between October and early December.
> 
> I'm thus writing to the list to see if there is indeed interest for this. If
> only 2 people reply, I won't push this further :)
> 
> Here are a few potential ideas of things people could work on, in no
> particular order:
> 
> - working on testing and merging the pybuild-autodep8 feature [1]

I'm very much invested in getting this done. I plan to be participate to
help make it happen.

> - fixing the ~50 packages that are still using 'python3 setup.py' [2]
> - reviewing and merging unnoticed salsa MRs on the Team's packages [3]
> - fixing policy violations [4]
> - upstreaming CPython patches [5]
> - trying to remove all remaining Python 2 packages [6]
> - working on PEP 668 [7] [8]
> - working on lintian tags for the team [9]
> - patching tracker.debian.org (Django) to show pending MRs [10] [11]
> 
> People are of course welcome to work on whatever other things they see fit
> for the betterment of the Team :)
> 
> I've done a remote sprint for the Clojure Team back in May and it went
> great.
> 
> Each people registered and we asked for a food budget, based on the "Meals
> and Incidentals Rate" the US government publishes for most cities in the
> world [12] [13]. I encourage you to look up your city, but I know I ended up
> eating pretty well :)
> 
> I would envision a 3 day sprint (Friday, Saturday, Sunday) being long enough
> yet not too long for most of us to make progress on key issues without being
> over-tiring.
> 
> Happy to hear back from y'all (please do if you're interested). If I see
> people are interested, I'll send a poll to find the most suitable dates.

I would be willing to join, at least all day on Friday and a few hours
during both days in the weekend.


signature.asc
Description: PGP signature

Python in the Debian infrastructure

[Antonio Terceiro]

On Fri, Aug 19, 2022 at 09:29:40AM -0300, Emmanuel Arias wrote:
> Hi,
> 
> I'm very interested :-)
> 
> 
> On Wed, Aug 17, 2022 at 9:23 PM Louis-Philippe Véronneau 
> wrote:
[...]
> > - patching tracker.debian.org (Django) to show pending MRs [10] [11]
> >
>  I'm just curious, we know or it's easy to know  if there're more parts of
> the Debian
> infrastructure where Python is used and we can help?

Lots of it. I made a talk about contributing with the Debian
infrastructure in 2020 online minidebconf, and here is the contents of
my summary slide:


## Summary

Project  | Ruby | OCaml | Python | Shell | Perl | JS
-|::|:-:|:--:|:-:|::|:---
[ci.debian.net][]|  x   |   ||   x   |  |
[dose3][]|  |   x   ||   |  |
[botch][]|  |   x   ||   |  |
[devscripts][]   |  |   |   x|   x   |  x   |
[licensecheck][] |  |   ||   |  x   |
[Debian Keyring][]   |  x   |   |   x|   x   |  |
[sources.debian.org][]   |  |   |   x|   |  |
[contributors.debian.org][]  |  |   |   x|   |  |
[Security Team][]|  |   |   x|   |  |
[debtags.debian.org][]   |  |   |   x|   |  |  x
[moinmoin][] |  |   |   x|   |  |
[nm.debian.org][]|  |   |   x|   |  |
[sso.debian.org][]   |  |   |   x|   |  |
[tracker.debian.org][]   |  |   |   x|   |  |


[ci.debian.net]: 

[dose3]: 

[botch]: 

[devscripts]: 

[licensecheck]: 


[Debian Keyring]: 

[sources.debian.org]: 

[contributors.debian.org]: 


[Security Team]: 

[debtags.debian.org]: 

[moinmoin]: 

[nm.debian.org]: 

[sso.debian.org]: 

[tracker.debian.org]: 


FWIW the recording is here (40 min):
https://peertube.debian.social/w/hLnps9yLxAwiz4FzNBaCuw


signature.asc
Description: PGP signature

Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

[Antonio Terceiro]

On Tue, Jul 26, 2022 at 03:37:50PM +, Scott Kitterman wrote:
> 
> 
> On July 26, 2022 2:50:19 PM UTC, Antonio Terceiro  wrote:
> >On Mon, Jul 25, 2022 at 09:04:27AM +0100, Julian Gilbey wrote:
> >> On Sat, Jul 23, 2022 at 07:52:19PM +0200, Louis-Philippe Véronneau wrote:
> >> > == pybuild improvements ==
> >> > 
> >> > getting the autopkgtest MR in would be great
> >> > 
> >> > https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/27
> >> > 
> >> > We need people to test this MR some more, although it seems fairly 
> >> > mature.
> >> > 
> >> > It might be a good idea to have a line in d/control to let us migrate 
> >> > from
> >> > the existing autopkgtests running unit tests to the new automated MR.
> >> 
> >> I've been looking at this a bit more.  I'm not sure what this last
> >> paragraph means: the new "automated" autopkgtest will only be run if
> >> the maintainer explicitly adds:
> >> 
> >> Testsuite: autopkgtest-pkg-pybuild
> >> 
> >> to debian/control (see the autodep8 MR at
> >> https://salsa.debian.org/ci-team/autodep8/-/merge_requests/27/diffs -
> >> it will never automatically detect a pybuild package).
> >> And a maintainer would presumably only add that if they are also
> >> removing their existing debian/tests/control (or want to run the
> >> pybuild tests in addition).
> >> 
> >> An alternative would be for the autodep8 patch to try to determine
> >> whether to run pybuild-autopkgtest.  One approach could be:
> >> 
> >> if the package would run autopkgtest-pkg-python:
> >>   if debian/control does not contain an override_dh_auto_test stanza:
> >> run pybuild-autopkgtest
> >> 
> >> Note, though, that if autodep8 is called, it will run all of the
> >> detected tests.  (At least that is what I believe happens from reading
> >> /usr/bin/autodep8; I haven't double-checked this.)  So, for example,
> >> if a package specifies
> >> 
> >> Testsuite: autopkgtest-pkg-python
> >> 
> >> it will also run the autopkgtest-pkg-pybuild suite as it will be
> >> detected as being a Python package, and vice versa.  That is a
> >> possible reason *not* to use the above suggestion, as it would
> >> potentially run pybuild-autopkgtest even if not desired.
> >
> >I think the notes did not capture the consensus correctly. The point was
> >that it should be possible to automate updating the `Testsuite:` field
> >to run tests with pybuild-autopkgtest, and that we should probably do
> >that across team packages with the help of some scripting.
> 
> I suppose this is fine as long as whoever does it fixes all the resulting RC 
> bugs.

Well the idea is to only actually commit the change and upload the
package with the new Testsuite value after ensuring that actually works,
i.e. that the autopkgtest passes.

We are not suggesting to do this blindly across the packages, sorry for
not making that clear.


signature.asc
Description: PGP signature

Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

[Antonio Terceiro]

On Mon, Jul 25, 2022 at 09:04:27AM +0100, Julian Gilbey wrote:
> On Sat, Jul 23, 2022 at 07:52:19PM +0200, Louis-Philippe Véronneau wrote:
> > == pybuild improvements ==
> > 
> > getting the autopkgtest MR in would be great
> > 
> > https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/27
> > 
> > We need people to test this MR some more, although it seems fairly mature.
> > 
> > It might be a good idea to have a line in d/control to let us migrate from
> > the existing autopkgtests running unit tests to the new automated MR.
> 
> I've been looking at this a bit more.  I'm not sure what this last
> paragraph means: the new "automated" autopkgtest will only be run if
> the maintainer explicitly adds:
> 
> Testsuite: autopkgtest-pkg-pybuild
> 
> to debian/control (see the autodep8 MR at
> https://salsa.debian.org/ci-team/autodep8/-/merge_requests/27/diffs -
> it will never automatically detect a pybuild package).
> And a maintainer would presumably only add that if they are also
> removing their existing debian/tests/control (or want to run the
> pybuild tests in addition).
> 
> An alternative would be for the autodep8 patch to try to determine
> whether to run pybuild-autopkgtest.  One approach could be:
> 
> if the package would run autopkgtest-pkg-python:
>   if debian/control does not contain an override_dh_auto_test stanza:
> run pybuild-autopkgtest
> 
> Note, though, that if autodep8 is called, it will run all of the
> detected tests.  (At least that is what I believe happens from reading
> /usr/bin/autodep8; I haven't double-checked this.)  So, for example,
> if a package specifies
> 
> Testsuite: autopkgtest-pkg-python
> 
> it will also run the autopkgtest-pkg-pybuild suite as it will be
> detected as being a Python package, and vice versa.  That is a
> possible reason *not* to use the above suggestion, as it would
> potentially run pybuild-autopkgtest even if not desired.

I think the notes did not capture the consensus correctly. The point was
that it should be possible to automate updating the `Testsuite:` field
to run tests with pybuild-autopkgtest, and that we should probably do
that across team packages with the help of some scripting.


signature.asc
Description: PGP signature

dh-python: add pybuild-autopkgtest, a test runner

[Antonio Terceiro]

Control: reassign -1 dh-python
Control: retitle -1 dh-python: add pybuild-autopkgtest, a test runner
Control: severity -1 wishlist
Control: tag -1 + patch
Control: forwarded -1 
https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/27

On Mon, Dec 20, 2021 at 04:03:17PM -0300, Antonio Terceiro wrote:
> On Wed, Dec 01, 2021 at 08:19:20PM -0300, Antonio Terceiro wrote:
> > Hi,
> > 
> > Adding debian-python@l.d.o
> > 
> > The context is #1000803 where Sandro asked about reducing duplication
> > when running upstream test suites both during the build and under
> > autopkgtest.
> > 
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1000803
> > 
> > On Tue, Nov 30, 2021 at 12:47:42AM -0500, Sandro Tosi wrote:
> > > > This is usually solved outside of autopkgtest itself.
> > > >
> > > > For example Ruby packages have a single place where tests are specified
> > > > (debian/ruby-tests.*{rb,rake,yaml}, and the appropriate
> > > > debian/tests/control file is autogenerated by autodep8. I would say 99%
> > > > of Ruby packages require no duplication, or even explicitly specifying
> > > > commands to run tests at all.
> > > >
> > > > In general, this is most efficiently solved by package type (e.g.
> > > > programming language), because the way the tests are run at build-time
> > > > usually is tied to the build system. You then usually need some test
> > > > runner, probably extracted from, or provided by, the build system, to
> > > > also provide the same funcionality in an autopkgtest-compatible way.
> > > >
> > > > All the package types that are well supported in autodep8 have their
> > > > specialized test runner tools that avoid this type of duplication.
> > > 
> > > I'm mostly interested in the python ecosystem, and the provided
> > > autodep8 test are extremely superficial (essentially only importing
> > > the main python module packaged), which is better than nothing but not
> > > the same level as what's in d/rules.
> > > 
> > > Are you aware of any effort to expand the Python autodep8 tests to
> > > uniform them into a comprehensive, and unique place to run tests at
> > > build-time and via autopkgtest?
> > 
> > I'm not.
> > 
> > > If no such effort currently exists, what would one have to do to start
> > > developing it? Not necessarily volunteering, just gathering
> > > information.
> > 
> > In general terms, I see this being implemented like this:
> > 
> > 0) modify pybuild so one can call e.g. `pybuild --autopkgtest`. That
> > should work almost the same as `pybuild --test`, but would copy the test
> > files under ${AUTOPKGTEST_TMP} and run the tests from there, instead of
> > assuming a previously built tree and trying to chdir into
> > .pybuild/*/build.
> > 
> > 1) add a way of being able to specify pybuild parameters outside of
> > debian/rules that can be used both during the build and under
> > autopkgtest
> > 
> >For Ruby, gem2deb-test-runner is driven by debian/ruby-tests.*, both
> >during the build, and under autopkgtest.
> > 
> >In pybuild, some aspects of the test run can already be done this
> >way, e.g. debian/pybuild.testfiles. Maybe we could have
> >debian/pybuild.env to read options in the same way as they are set in
> >debian/rules today (with environment variables).
> > 
> > 2) update autodep8 to generate the appropriate control file to call
> >`pybuild --autopkgtest`. This needs to be "smart" enough to not
> >automatically add this call to packages that are not ready for it. At
> >the moment, almost 1000 source packages have
> >`Testsuite: autopkgtest-pkg-python`. We would probably need a new
> >value, or (probably better) to additionally check for something else
> >in the source tree.
> > 
> > Each item has quite some details to be figured out, but this is the
> > general idea.
> 
> I have written a prototype implementation of this, and published it as
> https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/27

This is now ready to be seriously considered, please take a look.


signature.asc
Description: PGP signature

Re: Bug#1000803: autopkgtest: avoid duplication with autopkgtests and how unittests are run at build-time

[Antonio Terceiro]

On Wed, Dec 01, 2021 at 08:19:20PM -0300, Antonio Terceiro wrote:
> Hi,
> 
> Adding debian-python@l.d.o
> 
> The context is #1000803 where Sandro asked about reducing duplication
> when running upstream test suites both during the build and under
> autopkgtest.
> 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1000803
> 
> On Tue, Nov 30, 2021 at 12:47:42AM -0500, Sandro Tosi wrote:
> > > This is usually solved outside of autopkgtest itself.
> > >
> > > For example Ruby packages have a single place where tests are specified
> > > (debian/ruby-tests.*{rb,rake,yaml}, and the appropriate
> > > debian/tests/control file is autogenerated by autodep8. I would say 99%
> > > of Ruby packages require no duplication, or even explicitly specifying
> > > commands to run tests at all.
> > >
> > > In general, this is most efficiently solved by package type (e.g.
> > > programming language), because the way the tests are run at build-time
> > > usually is tied to the build system. You then usually need some test
> > > runner, probably extracted from, or provided by, the build system, to
> > > also provide the same funcionality in an autopkgtest-compatible way.
> > >
> > > All the package types that are well supported in autodep8 have their
> > > specialized test runner tools that avoid this type of duplication.
> > 
> > I'm mostly interested in the python ecosystem, and the provided
> > autodep8 test are extremely superficial (essentially only importing
> > the main python module packaged), which is better than nothing but not
> > the same level as what's in d/rules.
> > 
> > Are you aware of any effort to expand the Python autodep8 tests to
> > uniform them into a comprehensive, and unique place to run tests at
> > build-time and via autopkgtest?
> 
> I'm not.
> 
> > If no such effort currently exists, what would one have to do to start
> > developing it? Not necessarily volunteering, just gathering
> > information.
> 
> In general terms, I see this being implemented like this:
> 
> 0) modify pybuild so one can call e.g. `pybuild --autopkgtest`. That
> should work almost the same as `pybuild --test`, but would copy the test
> files under ${AUTOPKGTEST_TMP} and run the tests from there, instead of
> assuming a previously built tree and trying to chdir into
> .pybuild/*/build.
> 
> 1) add a way of being able to specify pybuild parameters outside of
> debian/rules that can be used both during the build and under
> autopkgtest
> 
>For Ruby, gem2deb-test-runner is driven by debian/ruby-tests.*, both
>during the build, and under autopkgtest.
> 
>In pybuild, some aspects of the test run can already be done this
>way, e.g. debian/pybuild.testfiles. Maybe we could have
>debian/pybuild.env to read options in the same way as they are set in
>debian/rules today (with environment variables).
> 
> 2) update autodep8 to generate the appropriate control file to call
>`pybuild --autopkgtest`. This needs to be "smart" enough to not
>automatically add this call to packages that are not ready for it. At
>the moment, almost 1000 source packages have
>`Testsuite: autopkgtest-pkg-python`. We would probably need a new
>value, or (probably better) to additionally check for something else
>in the source tree.
> 
> Each item has quite some details to be figured out, but this is the
> general idea.

I have written a prototype implementation of this, and published it as
https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/27


signature.asc
Description: PGP signature

Re: Bug#1000803: autopkgtest: avoid duplication with autopkgtests and how unittests are run at build-time

[Antonio Terceiro]

Hi,

Adding debian-python@l.d.o

The context is #1000803 where Sandro asked about reducing duplication
when running upstream test suites both during the build and under
autopkgtest.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1000803

On Tue, Nov 30, 2021 at 12:47:42AM -0500, Sandro Tosi wrote:
> > This is usually solved outside of autopkgtest itself.
> >
> > For example Ruby packages have a single place where tests are specified
> > (debian/ruby-tests.*{rb,rake,yaml}, and the appropriate
> > debian/tests/control file is autogenerated by autodep8. I would say 99%
> > of Ruby packages require no duplication, or even explicitly specifying
> > commands to run tests at all.
> >
> > In general, this is most efficiently solved by package type (e.g.
> > programming language), because the way the tests are run at build-time
> > usually is tied to the build system. You then usually need some test
> > runner, probably extracted from, or provided by, the build system, to
> > also provide the same funcionality in an autopkgtest-compatible way.
> >
> > All the package types that are well supported in autodep8 have their
> > specialized test runner tools that avoid this type of duplication.
> 
> I'm mostly interested in the python ecosystem, and the provided
> autodep8 test are extremely superficial (essentially only importing
> the main python module packaged), which is better than nothing but not
> the same level as what's in d/rules.
> 
> Are you aware of any effort to expand the Python autodep8 tests to
> uniform them into a comprehensive, and unique place to run tests at
> build-time and via autopkgtest?

I'm not.

> If no such effort currently exists, what would one have to do to start
> developing it? Not necessarily volunteering, just gathering
> information.

In general terms, I see this being implemented like this:

0) modify pybuild so one can call e.g. `pybuild --autopkgtest`. That
should work almost the same as `pybuild --test`, but would copy the test
files under ${AUTOPKGTEST_TMP} and run the tests from there, instead of
assuming a previously built tree and trying to chdir into
.pybuild/*/build.

1) add a way of being able to specify pybuild parameters outside of
debian/rules that can be used both during the build and under
autopkgtest

   For Ruby, gem2deb-test-runner is driven by debian/ruby-tests.*, both
   during the build, and under autopkgtest.

   In pybuild, some aspects of the test run can already be done this
   way, e.g. debian/pybuild.testfiles. Maybe we could have
   debian/pybuild.env to read options in the same way as they are set in
   debian/rules today (with environment variables).

2) update autodep8 to generate the appropriate control file to call
   `pybuild --autopkgtest`. This needs to be "smart" enough to not
   automatically add this call to packages that are not ready for it. At
   the moment, almost 1000 source packages have
   `Testsuite: autopkgtest-pkg-python`. We would probably need a new
   value, or (probably better) to additionally check for something else
   in the source tree.

Each item has quite some details to be figured out, but this is the
general idea.

If someone is willing to work on this I could help with guidance along
the way.


signature.asc
Description: PGP signature

Bug#996087: ITP: typeshed -- collection of library stubs for Python, with static types

[Antonio Terceiro]

Package: wnpp
Severity: wishlist
Owner: Antonio Terceiro 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-python@lists.debian.org

* Package name: typeshed
  Version : n/a
  Upstream Author : Several authors
* URL : https://github.com/python/typeshed
* License : Apache 2.0
  Programming Lang: Python
  Description : collection of library stubs for Python, with static types

 Typeshed contains external type annotations for the Python standard library
 and Python builtins, as well as third party packages as contributed by people
 external to those projects.

 This data can e.g. be used for static analysis, type checking or type
 inference.

 typeshed has been extracted from mypy, and is necessary in order to
 have type checking work for libraries that still don't provide their
 own type anotations. Users are encouraged to install individual types-*
 wheels, but this package will provide all the typing information in a
 single binary package.

 Preliminary packaging available at
 https://salsa.debian.org/terceiro/typeshed and will be moved into the
 Python team soon™.


signature.asc
Description: PGP signature

Re: mypy and typeshed

[Antonio Terceiro]

On Fri, Oct 01, 2021 at 08:14:20AM -0300, Antonio Terceiro wrote:
> On Thu, Sep 30, 2021 at 04:29:20PM -0300, Antonio Terceiro wrote:
> > Maybe I didn't dedicate enough time for this, but I couldn't figure out
> > how the pypi packages are produced from the git repository. Knowing this
> > would help creating such typeshed package by means of some scripting
> > (not necessarily volunteering, will be happy if someone beats me to it).
> 
> It turns out that this is done by this repository:
> 
> https://github.com/typeshed-internal/stub_uploader

I gave this a try, and came up with
https://salsa.debian.org/terceiro/typeshed

This seems to work, but there are still issues to solve, e.g. the
licensing status of the stub_uploader repository¹, and generating a
meaningful Provides: field.

¹ https://github.com/typeshed-internal/stub_uploader/issues/31


signature.asc
Description: PGP signature

Re: mypy and typeshed

[Antonio Terceiro]

On Thu, Sep 30, 2021 at 04:29:20PM -0300, Antonio Terceiro wrote:
> Maybe I didn't dedicate enough time for this, but I couldn't figure out
> how the pypi packages are produced from the git repository. Knowing this
> would help creating such typeshed package by means of some scripting
> (not necessarily volunteering, will be happy if someone beats me to it).

It turns out that this is done by this repository:

https://github.com/typeshed-internal/stub_uploader


signature.asc
Description: PGP signature

Re: mypy and typeshed

[Antonio Terceiro]

On Thu, Sep 23, 2021 at 09:25:26AM +0200, Michael R. Crusoe wrote:
> Do we package mypy to [A] support other packages, or are we trying to [B]
> deliver a "complete" offline developer experience? (even if that means
> shipping type hints for python packages not in Debian)
> 
> If the goal is [A], then we should stick to the add-hoc approach:
> "python-types-*" packages will only be made as needed for building other
> Debian source packages in our archives. In this instance, developers using
> the packaged version of mypy can use `mypy --install-types' to pull down
> the packages they need from PyPI to their local system/virtualenv.
> 
> http://mypy-lang.blogspot.com/2021/06/mypy-0900-released.html
> 
> If the goal is [B], then we need to be comprehensive.

Particularly, I try to develop all my Python work against Debian, and
having a mypy that is broken by default without installing stuff that is
not in the Debian archive is a major incovenience.

> Either way we choose, updates to these type hints may need to be made (and
> patches applied) to match the versions of the Python packages in the
> archive (if they have fallen behind or ahead of what is in the typeshed
> repository).
> 
> 
> > Does it make sense for Debian to package snapshots of all of typeshed in
> > one
> > binary package to save a proliferation of small python-types-foo packages?
> >
> 
> Hmm..  a joint source package makes sense (and might save the FTP team from
> reviewing 103 additional submissions), but as each "types-*" package in
> PyPI has its own public version it would be weird to not match their
> versions at all.
>
> Though I guess the monolithic package could have a very long "Provides"
> line mentioning all the components and their versions.
>
> The Copyright file will be interesting! For the two "python-types-*"
> packages I created by hand, I had to dig up the author information from the
> git history at the urging of the FTP gatekeepers :-)
> 
> Personally I'm more of an [A] person (out of laziness), but I won't block
> the [B] approach if there is sufficient interest that it doesn't primarily
> fall on my shoulders.
> 
> Feedback from the wider Debian Python community is welcome!

Maybe I didn't dedicate enough time for this, but I couldn't figure out
how the pypi packages are produced from the git repository. Knowing this
would help creating such typeshed package by means of some scripting
(not necessarily volunteering, will be happy if someone beats me to it).


signature.asc
Description: PGP signature

Re: python-django-js-asset_1.2.2-3_source.changes REJECTED

[Antonio Terceiro]

On Mon, Sep 20, 2021 at 11:14:44AM -0400, Sandro Tosi wrote:
> > That's because gbp does not use pristine-tar by default, and
> > debian/gbp.conf was missing `pristine-tar=True`. Just pushed a commit to
> > fix that.
> 
> I dont think this is the right approach: the default options to work
> on DPT packages should be in gbp default config file (or in another,
> global, config file), and not live in each and every package
> debian/gbp.conf file; it is already inconsistently maintained with
> several packages having uncommon settings that will take precedence
> over the default ones.

I agree with you in theory; my global gbp.cons enables pristine-tar.

However, having it duplicated in every package means we as a team work
consistently regardless of people's global configuration, and that's one
less detail people need to get just right to be able to contribute
effectively.

Also, one's global configuration might not apply to all the packages
they contribute to; it's easier for everyone if gbp just does the
right thing based on per-package configuration than expecting people to
remember to switch their defaults, or to pass options explicitly.


signature.asc
Description: PGP signature

Re: python-django-js-asset_1.2.2-3_source.changes REJECTED

[Antonio Terceiro]

On Sun, Sep 19, 2021 at 10:56:33PM +0200, Carsten Schoenert wrote:
> Hi Antonio,
> 
> thanks for your quick feedback!
> 
> Am 19.09.21 um 21:24 schrieb Antonio Terceiro:
> 
> > Looking from my side, the tarball from the archive (apt source
> > python-django-js-asset) and the one generated by pristine-tar are
> > identical:
> > 
> > 4b6a2c8625b8e96bbc4ff1588a27238d7d418b03  
> > /tmp/python-django-js-asset_1.2.2.orig.tar.gz
> > 4b6a2c8625b8e96bbc4ff1588a27238d7d418b03  
> > /tmp/archive/python-django-js-asset_1.2.2.orig.tar.gz
> > 
> >  From reading the REJECT email, I think it implies that the .dsc/.changes
> > you uploaded refer to an orig tarball with 6360 bytes. Do you still have
> > the exact artifacts that you uploaded?
> 
> No, not completely.
> But I played around a bit with gbp and pristine-tar too and I was able to
> recreate a tarball which has the same size and the same hashes as the
> *.tar.gz in the archive and the one you've posted by using pristine-tar
> manually.
> 
> If I clean out all completely and build the package from scratch by gbp I
> get again a wrong size and of course also different hashes.
> 
> Currently I've no real clue why gbp is creating here different results, will
> look again at this once Guido is around, I'm sure he can blame me that I'm
> doing something wrong. :P

That's because gbp does not use pristine-tar by default, and
debian/gbp.conf was missing `pristine-tar=True`. Just pushed a commit to
fix that.


signature.asc
Description: PGP signature

Re: python-django-js-asset_1.2.2-3_source.changes REJECTED

[Antonio Terceiro]

Hi,

On Sun, Sep 19, 2021 at 10:27:21AM +0200, Carsten Schoenert wrote:
> Hello Antonio,
> 
> as you initially did the first upload for this package I'm now heading
> over to you.
> 
> Am 19.09.21 um 09:48 schrieb Debian FTP Masters:
> > 
> > python-django-js-asset_1.2.2-3.dsc: Invalid size hash for 
> > python-django-js-asset_1.2.2.orig.tar.gz:
> > According to the control file the size hash should be 6360,
> > but python-django-js-asset_1.2.2.orig.tar.gz has 6367.
> > 
> > If you did not include python-django-js-asset_1.2.2.orig.tar.gz in your 
> > upload, a different version
> > might already be known to the archive software.
> 
> I've no idea where the size of 6360 for the orig.tar.gz file is coming
> from which DAK refers.
> 
> If I look at the source package sites or on snapshot.d.o I always see a
> size of 6367.
> 
> https://packages.debian.org/source/unstable/python-django-js-asset
> http://snapshot.debian.org/package/python-django-js-asset/1.2.2-1/
> 
> And this size was recreated by gbp locally.
> But the question is now how to proceed now. A quick & dirty solution
> would be to simply use the old tar.gz file (if available).
> Another option would be more dramatically and let remove the package
> completely from the archive. Do you have opinions on that?

Looking from my side, the tarball from the archive (apt source
python-django-js-asset) and the one generated by pristine-tar are
identical:

4b6a2c8625b8e96bbc4ff1588a27238d7d418b03  
/tmp/python-django-js-asset_1.2.2.orig.tar.gz
4b6a2c8625b8e96bbc4ff1588a27238d7d418b03  
/tmp/archive/python-django-js-asset_1.2.2.orig.tar.gz

From reading the REJECT email, I think it implies that the .dsc/.changes
you uploaded refer to an orig tarball with 6360 bytes. Do you still have
the exact artifacts that you uploaded?


signature.asc
Description: PGP signature

Re: Build an initial Debian source package with py2dsp from a GitHub project

[Antonio Terceiro]

On Sun, Apr 25, 2021 at 06:19:18PM -0400, Sandro Tosi wrote:
> > It would be useful if it could also be run against a tarball
> 
> this is already supported (but in general by py2dsp and in the context
> of --github), f.e.:
> 
> $ ./py2dsp --profile dpt --distribution unstable --revision 1 --gh
> https://github.com/indygreg/python-zstandard
> ./zstandard_0.14.1.orig.tar.gz
> 
> uses the locally available zstandard_0.14.1.orig.tar.gz tarball (which
> is not the latest available on gh) to create the source pkg with the
> github customizations
> 
> > or git repository (a directory) that one has already downloaded.
> 
> i dont see how starting from a git repo is useful, can you expand?

instead of generating a .dsc first and then importing it into a git
repository, it's more logical to me to import an upstream tarball into a
git repository first (gbp import-orig), and then generate the debian
packaging on top of that.


signature.asc
Description: PGP signature

Re: Build an initial Debian source package with py2dsp from a GitHub project

[Antonio Terceiro]

On Sun, Apr 25, 2021 at 12:22:19AM -0400, Sandro Tosi wrote:
> Let me know if you find this useful and if there are
> issues/enhancement you'd like to see added/fixed.

That's very useful, thanks!

It would be useful if it could also be run against a tarball or git
repository (a directory) that one has already downloaded.


signature.asc
Description: PGP signature

gotchas when running tests via pybuild?

[Antonio Terceiro]

Hi,

I'm working on python-eventlet, to fix a FTBFS bug and an
incompatibility with python3.9. I got both fixes ready, but I'm not
fighting with the fact that the test suite fails randomly. In my
experiments, the test suite fails ~40% of the time, but *only when run
during the build* (!!).

- I added a testsuite run to autopkgtest, and it consistently passes,
  100% of the time.
- If I run the same command as pybuild runs manually (python3 -m nose)
  on a fully patched tree, it passes 100% of the time.
- Only during the build -- when executed automatically by pybuild -- the
  test fails ~40% of the time.

The failures are the typical failures you get when testing concurrency
code: timeouts, race conditions, etc, and it happens on different tests
every time. What I don't quite understand yet is why this _only_ happens
during the Debian package build.

I already checked that the tests don't use anything else from the source
tree beyond the tests themselves and the python modules. For example the
autopkgtest copies tests/, and only it, to a temp directory and runs the
tests from there; and works every time. So in principle I wouldn't need
to have an explicit testfiles file.

Does anybody have an insight on cases like this? Are there any details
that I'm missing?

If anyone wants to try it, the git repository is up to date.


signature.asc
Description: PGP signature

Bug#834866: ITP: python-whitenoise -- static file serving for WSGI applications

[Antonio Terceiro]

Package: wnpp
Severity: wishlist
Owner: Antonio Terceiro <terce...@debian.org>

* Package name: python-whitenoise
  Version : 3.2.1
  Upstream Author : David Evans
* URL : http://whitenoise.evans.io
* License : MIT/Expat
  Programming Lang: Python
  Description : static file serving for WSGI applications

With a couple of lines of config, WhiteNoise allows your web app to serve its
own static files, making it a self-contained unit that can be deployed
anywhere without relying on nginx, Amazon S3 or any other external service.

This is specially useful if you want to deploy a WSGI application in an
application container (e.g. docker).

I plan to import this package into the Debian Python Modules Team
repositories, even though I don't plan to get myself deeply involved
with the team. I will subscribe to this specific package through the
package tracker.


signature.asc
Description: PGP signature

Re: Bug#798066: Multiarch-renamed python extensions not found during autopkgtest testing

[Antonio Terceiro]

On Thu, Dec 10, 2015 at 10:52:05PM -0800, Afif Elghraoui wrote:
> Hi,
> 
> على الأربعاء  9 كانون الأول 2015 ‫05:25، كتب Antonio Terceiro:
> > autopkgtest does not do anything special wrt dependencies, it will
> > install exactly what you told it to in debian/tests/control
> [...]
> > 
> > I am therefore closing this bug.
> 
> The problem is not that something is not installed. The problem is that
> the multiarch configuration for python is not right in autopkgtest (or
> schroot). During the package build, dh-python renames the compiled
> extension to contain the multiarch triplet. For some reason, only in
> autopkgtest, python is not properly configured to find the extension
> after it has been renamed, so autopkgtest runs requiring the compiled
> extensions fail.

ci.debian.net does not use schroot anymore, since a few weeks ago. and
autopkgtest does not do anything special to packages, it just installs
them on an otherwise regular testbed, be it a schroot chroot, an lxc
container (what ci.debian.net currently uses), and so on.

> So should we continue to have hacks like this to get autopkgtest to find
> multiarch-renamed extensions?
> 
> cd /usr/lib/python2.7/dist-packages/pysam
> gnutype=`dpkg-architecture -qDEB_TARGET_GNU_TYPE`
> for so in *.${gnutype}.so ; do sudo ln -sf $so `basename $so
> .${gnutype}.so`.so ; done
> 
> http://anonscm.debian.org/cgit/debian-med/python-pysam.git/tree/debian/tests/run-nose-tests

Again, there is nothing special in the autopkgtest test beds. I also
don't see other python packages that contain compiled extensions needing
to do this sort of thing.

I tried python-pysam here, and after some trial and error, I can also reproduce
the same issue outside of autopkgtest. The issue is that Python load path is
being confused by the fact that you are on root of the source package:

# pwd
/tmp/python-pysam-0.8.4+ds
# ls
AUTHORS  INSTALL MANIFEST.in  THANKS debian  install-CGAT-tools.sh  
pysam.py  samtools  setup.cfg  tests
COPYING  KNOWN_BUGS  README.rst   benchmark  doc pysam  
requirements.txt  save  setup.py   win32
# python
Python 2.7.11 (default, Dec  9 2015, 00:29:25) 
[GCC 5.3.1 20151205] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysam
Traceback (most recent call last):
  File "", line 1, in 
  File "pysam/__init__.py", line 1, in 
from pysam.libchtslib import *
ImportError: No module named libchtslib
>>> 
# cd /
# python
Python 2.7.11 (default, Dec  9 2015, 00:29:25) 
[GCC 5.3.1 20151205] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysam 
>>> 
# 

So your problem has nothing to do autopkgtest, other than the fact that
autopkgtest always starts the tests from the root of the source package.

-- 
Antonio Terceiro <terce...@debian.org>


signature.asc
Description: PGP signature

Re: Bug#798066: Multiarch-renamed python extensions not found during autopkgtest testing

[Antonio Terceiro]

Control:

On Tue, Dec 08, 2015 at 11:25:47PM -0800, Afif Elghraoui wrote:
> Hi, debian-python,
> 
> For Debian Med's python packages that have compiled extensions, I
> noticed that test suites run via autopkgtest fail because the package
> cannot find those extensions, as they've been renamed with the multiarch
> triplet.
> 
> It seems to only be a problem with autopkgtest because I have in one
> case manually installed such a package and successfully ran its test
> suite. I therefore reported #798066 [1] against autopkgtest, but quite
> some time has elapsed since then and I'd like to get this resolved.
> 
> Does anyone know why this might be the case? Autopkgtest for these
> packages is currently not very helpful because there are many spurious
> failures due to this issue. There are more details in my original bug
> report [1].
> 
> Many thanks and regards
> Afif
> 
> 1. http://bugs.debian.org/798066

autopkgtest does not do anything special wrt dependencies, it will
install exactly what you told it to in debian/tests/control (see the
spec¹ for reference), so I doubt this would be a problem with
autopkgtest.

¹ 
http://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/tree/doc/README.package-tests.rst

I am therefore closing this bug.

-- 
Antonio Terceiro <terce...@debian.org>


signature.asc
Description: PGP signature