Re: GnuPG signatures on PyPI: why so few?
Brian May writes: > Maybe there is some way of turning signatures on by default, so I don't > have to remember for every upload, if so, I haven't been able to work it > out yet however. I don't know how it should be done using the currently-recommended Twine tool. For Distutils, the ‘upload’ command has a ‘--sign’ option https://docs.python.org/3.1/distutils/uploading.html> to specify the distribution should be GnuPG signed, and the ‘--identity’ option specifies which GnuPG identity from your keyring should sign the distribution. To set an option default for Setuptools commands, you put it in the ‘setup.cfg’ file. So, in the code base's ‘setup.cfg’: [upload] # Sign distributions, and upload the signing public key? sign = true # Which GnuPG identity to use for signing? identity = christ...@example.org -- \ “Probably the earliest flyswatters were nothing more than some | `\sort of striking surface attached to the end of a long stick.” | _o__) —Jack Handey | Ben Finney
Re: GnuPG signatures on PyPI: why so few?
> On Mar 11, 2017, at 9:23 PM, Brian May wrote: > > Ben Finney writes: > >> However, this only works if upstream releases are actually accompanied >> by a valid GnuPG signature each time. The PyPI infrastructure supports >> this; why isn't it more widely encouraged? > > One reason I have found for myself: I can forget to sign the package > when uploading to PyPI, and PyPI doesn't let you make any changes after > the package is uploaded without changing the version (including adding > signature file). There is a long running bug report on this, it is not > going to get fixed (TLDR it is not a bug, it is a design feature to > allow for caching). I think there is no reason we *couldn’t* allow uploads of signatures after the fact, although it sort of makes an already tenuous security claim even more tenuous I think. Since distutils/setuptools don’t have reproducible “builds” of sdists if you wanted to after the fact upload a signature your choices are: * Hope you still have the original files you uploaded still laying around, sign *them* and then upload just the signature. * Download the files, open them up and inspect all of the files to ensure they match the correct state. * Download the files, blindly sign them. Of those options, the first one is no more or less “secure” than uploading alongside the original file, but the problem is that it’s impossible to differentiate that from the second two, and that could genuinely be worse (for example, compromised PyPI gives author a bad file, they sign it and upload the signature thus making everyone feel it is a “safe” file). Both the second options are kind of bad because it is incredibly easy to smuggle something bad into a tarball that a human being would miss. The larger reason why I personally haven’t implemented it is because I believe the GPG signatures on PyPI are largely useless from a practical standpoint and I’m not really interested in doing anything to improve them and I would personally much rather just disable them completely. > > Maybe there is some way of turning signatures on by default, so I don't > have to remember for every upload, if so, I haven't been able to work it > out yet however. Um, I think you can set something in setup.cfg for ``setup.py upload``. I don’t think there is anything like that for Twine. I can’t speak for Ian but I don’t personally see anything inherently wrong with adding a environment variable or config option to twine that allows it to always sign by default. Ian may feel differently though! — Donald Stufft
Re: PyPI source or github source?
On Sun, Mar 12, 2017 at 10:19 AM, Brian May wrote: > Sure, you could argue that PyPI source packages should contain > everything the github package does. In fact there is a PyPI tool to help > get the MANIFEST.in correct for such purposes - > https://pypi.python.org/pypi/check-manifest Anyone interested in packaging this? > Unfortunately, github releases cannot (AFAIK) easily be signed, unless > you retrieve signed git tags directly from git (which is not supported > by uscan AFAIK). Would be good if gbp buildpackage supported signing git > tags, I don't think it does either. uscan does support git but doesn't check OpenPGP signatures on tags. It would probably be easy to add that, please file a bug about it. > * Do we consider signed git tags / commits secure, considering they are > based on SHA1? Better than having unsigned tags/commits. > * Is there any point having signed PyPI releases when (very likely) the > underlying upstream git repository has no signatures? Yes, presumably the PyPI releases are built from the author's copy of the git repository, rather than directly from the online repository, hopefully they have verified all commits they pulled into it. > * Is there any point having signed PyPI releases when (very likely) the > public key is stored in an insecure DPMT respository on > git.debian.org? Yes, it is also stored in immutable places like the archive and snapshot.d.o. -- bye, pabs https://wiki.debian.org/PaulWise
Re: GnuPG signatures on PyPI: why so few?
Ben Finney writes: > However, this only works if upstream releases are actually accompanied > by a valid GnuPG signature each time. The PyPI infrastructure supports > this; why isn't it more widely encouraged? One reason I have found for myself: I can forget to sign the package when uploading to PyPI, and PyPI doesn't let you make any changes after the package is uploaded without changing the version (including adding signature file). There is a long running bug report on this, it is not going to get fixed (TLDR it is not a bug, it is a design feature to allow for caching). Maybe there is some way of turning signatures on by default, so I don't have to remember for every upload, if so, I haven't been able to work it out yet however. -- Brian May
PyPI source or github source?
Ben Finney writes: > Debian's UScan has the ability to find, download, and verify the GnuPG > signature for a package source release. Lintian will remind the > maintainer if a Debian source package is not taking advantage of this. A related issue is: Should we be using PyPI as our source of packages? Or github? You mention a very good reason why PyPI should be the preferred form, packages *can* be signed by the author. However, often packages in PyPI are not the complete source package you would get from github. They may be sufficient for installations, but often not for Debian packaging - which really should have the complete source package. e.g. they can be missing tests, license files, documentation that doesn't get installed, etc. Sure, you could argue that PyPI source packages should contain everything the github package does. In fact there is a PyPI tool to help get the MANIFEST.in correct for such purposes - https://pypi.python.org/pypi/check-manifest; however a counter argument could also be made that upstrems generally don't care about such issues - probably won't notice any regressions - if it installs from PyPI that is all that really matters - and the only way we can be sure we are getting the complete source iis from github. Unfortunately, github releases cannot (AFAIK) easily be signed, unless you retrieve signed git tags directly from git (which is not supported by uscan AFAIK). Would be good if gbp buildpackage supported signing git tags, I don't think it does either. I started off thinking github was the best source, then slowly swayed towards PyPI, now I am thinking maybe github again. Comments? Some related issues: * Do we consider signed git tags / commits secure, considering they are based on SHA1? * Is there any point having signed PyPI releases when (very likely) the underlying upstream git repository has no signatures? * Is there any point having signed PyPI releases when (very likely) the public key is stored in an insecure DPMT respository on git.debian.org? -- Brian May
GnuPG signatures on PyPI: why so few?
Howdy all, What prospects are there for PyPI to have GnuPG-signed packages by default? Debian's UScan has the ability to find, download, and verify the GnuPG signature for a package source release. Lintian will remind the maintainer if a Debian source package is not taking advantage of this. However, this only works if upstream releases are actually accompanied by a valid GnuPG signature each time. The PyPI infrastructure supports this; why isn't it more widely encouraged? This thread from 2016 has a possible answer: while you can use GPG as is to verify that yes, "Donald Stufft" signed a particular package, you cannot use it to determine if "Donald Stufft" is *allowed* to sign for that package, a valid signature from me on the requests project should be just as invalid as an invalid signature from anyone on the requests project. The only namespacing provided by GPG itself is "trusted key" vs "not trusted key". […] I am aware of a single tool anywhere that actively supports verifying the signatures that people upload to PyPI, and that is Debian's uscan program. […] All in all, I think that there is not a whole lot of point to having this feature in PyPI, it is predicated a bunch of invalid assumptions (as detailed above) and I do not believe end users are actually even using the keys that are being uploaded. […] Thus, I would like to remove this feature from PyPI […]. https://mail.python.org/pipermail/distutils-sig/2016-May/028933.html> The thread has some discussion, and Barry Warsaw makes the case for Debian's use for signed releases. The last (?) post in the thread has a kind of interim conclusion: My main concern when implementing this is how to communicate it to users […]. [A move that gives the impression] "we're getting rid of this thing that only kinda works now in favor of something amazing that doesn't exist yet" is just not a popular move. https://mail.python.org/pipermail/distutils-sig/2016-May/028946.html> In response to polite requests for signed releases, some upstream maintainers are now pointing to that thread and closing bug reports as “won't fix”. What prospect is there in the Python community to get signed upstream releases become the obvious norm? -- \ “It is the fundamental duty of the citizen to resist and to | `\ restrain the violence of the state.” —Noam Chomsky, 1971 | _o__) | Ben Finney
Re: Moving a package from collab-maint to python-modules
On Sat, 2017-03-11 at 18:14 +, Scott Kitterman wrote: > > On March 11, 2017 6:52:59 AM EST, Ghislain Vaillant > wrote: > > On Sat, 2017-03-11 at 11:24 +, Christopher Hoskin wrote: > > > Hello, > > > > > > I'd like to package python-jsonpointer for Debian. The filer of the > > > > RFP (Bug #754296) Pietro Battiston, has created a repository at > > > > > > https://anonscm.debian.org/cgit/collab-maint/python-jsonpointer.git > > > > > > but has no intention of becoming the maintainer, and the package has > > > > not been uploaded. The existing repository does not use git-dpm or > > pristine-tar. > > > > > > I'd like to maintain this package within DPMT. Is there a way I can > > > > migrate the existing repository, or should I just start again? > > > > > > Thanks. > > > > > > Christopher Hoskin > > > > I know Pietro (I co-maintain src:bottleneck with him) and he has been > > keen on moving his packages to team-maintenance under the DPMT. > > However, since the package in question was never released, and is > > essentially RFP's now, you can probably start from scratch. > > > > I am wondering whether it makes sense to use git-dpm for new packages, > > now that the team is planning to transition from using git-dpm to gbp. > > I personally used gbp straight away for my recent packages (see pytest- > > qt, pytest-xvfb) for examples. > > I think we should be consistent like we were with svn/git. Stick with one > thing and then switch everything over. If we don't, we aren't acting as a > team. > > Scott K Fair enough. Christopher, please disregard my previous comment. Ghis
Re: Moving a package from collab-maint to python-modules
On March 11, 2017 6:52:59 AM EST, Ghislain Vaillant wrote: >On Sat, 2017-03-11 at 11:24 +, Christopher Hoskin wrote: >> Hello, >> >> I'd like to package python-jsonpointer for Debian. The filer of the >RFP (Bug #754296) Pietro Battiston, has created a repository at >> >> https://anonscm.debian.org/cgit/collab-maint/python-jsonpointer.git >> >> but has no intention of becoming the maintainer, and the package has >not been uploaded. The existing repository does not use git-dpm or >pristine-tar. >> >> I'd like to maintain this package within DPMT. Is there a way I can >migrate the existing repository, or should I just start again? >> >> Thanks. >> >> Christopher Hoskin > >I know Pietro (I co-maintain src:bottleneck with him) and he has been >keen on moving his packages to team-maintenance under the DPMT. >However, since the package in question was never released, and is >essentially RFP's now, you can probably start from scratch. > >I am wondering whether it makes sense to use git-dpm for new packages, >now that the team is planning to transition from using git-dpm to gbp. >I personally used gbp straight away for my recent packages (see pytest- >qt, pytest-xvfb) for examples. I think we should be consistent like we were with svn/git. Stick with one thing and then switch everything over. If we don't, we aren't acting as a team. Scott K
Re: Moving a package from collab-maint to python-modules
On Sat, 2017-03-11 at 11:24 +, Christopher Hoskin wrote: > Hello, > > I'd like to package python-jsonpointer for Debian. The filer of the RFP (Bug > #754296) Pietro Battiston, has created a repository at > > https://anonscm.debian.org/cgit/collab-maint/python-jsonpointer.git > > but has no intention of becoming the maintainer, and the package has not been > uploaded. The existing repository does not use git-dpm or pristine-tar. > > I'd like to maintain this package within DPMT. Is there a way I can migrate > the existing repository, or should I just start again? > > Thanks. > > Christopher Hoskin I know Pietro (I co-maintain src:bottleneck with him) and he has been keen on moving his packages to team-maintenance under the DPMT. However, since the package in question was never released, and is essentially RFP's now, you can probably start from scratch. I am wondering whether it makes sense to use git-dpm for new packages, now that the team is planning to transition from using git-dpm to gbp. I personally used gbp straight away for my recent packages (see pytest- qt, pytest-xvfb) for examples. Hope this helps, Ghis
Re: Moving a package from collab-maint to python-modules
On Sat, Mar 11, 2017 at 11:24:36AM +, Christopher Hoskin wrote: > I'd like to package python-jsonpointer for Debian. The filer of the RFP (Bug > #754296) Pietro Battiston, has created a repository at > > https://anonscm.debian.org/cgit/collab-maint/python-jsonpointer.git > > but has no intention of becoming the maintainer, and the package has not been > uploaded. The existing repository does not use git-dpm or pristine-tar. > > I'd like to maintain this package within DPMT. Is there a way I can migrate > the existing repository, or should I just start again? Once you switched the RFP to ITP, and own it, 1) add a pristine-tar branch (just create branch without parents, and run `pristine-tar commit` (+ appropriate args)) 2) convert to git-dpm (it should be something with `git-dpm init…` iirc) 3) push that repository you now got to DPMT 4) remove the old repository from collab-mainnt -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature
Moving a package from collab-maint to python-modules
Hello, I'd like to package python-jsonpointer for Debian. The filer of the RFP (Bug #754296) Pietro Battiston, has created a repository at https://anonscm.debian.org/cgit/collab-maint/python-jsonpointer.git but has no intention of becoming the maintainer, and the package has not been uploaded. The existing repository does not use git-dpm or pristine-tar. I'd like to maintain this package within DPMT. Is there a way I can migrate the existing repository, or should I just start again? Thanks. Christopher Hoskin signature.asc Description: PGP signature
Re: Join the team
[Sophie Brun, 2017-03-07] > I would like to join the Python-modules team to maintain the > python-pyric package > (https://github.com/wraith-wireless/PyRIC) and maybe other packages > needed by the pkg-security-team. > > My alioth login is sbrun-guest > > I have read the DPMT policy > (https://python-modules.alioth.debian.org/python-modules-policy.html) > and I accept it. welcome on board! :) -- GPG: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645
