On Saturday, November 5, 2016, Nick Coghlan <ncogh...@gmail.com> wrote:
> On 4 November 2016 at 06:07, Nathaniel Smith <n...@pobox.com <javascript:;>> > wrote: > > I think we're drifting pretty far off topic here... IIRC the original > > discussion was about whether the travis-ci infrastructure could be > suborned > > to provide an sdist->wheel autobuilding service for pypi. (Answer: maybe, > > though it would be pretty awkward, and no one seems to be jumping up to > make > > it happen.) > > The hard part of designing any such system isn't so much the building > process, it's the authentication, authorisation and trust management > for the release publication step. At the moment, that amounts to "Give > the service your PyPI password, so PyPI will 100% trust that they're > you" due to limitations on the PyPI side of things, and "Hello web > service developer, I grant you 100% authority to impersonate me to the > rest of the world however you like" is a questionable idea in any > circumstance, let alone when we're talking about publishing software. > > Since we don't currently provide end-to-end package signing, PyPI > initiated builds would solve the trust problem by having PyPI trust > *itself*, and only require user credentials for the initial source > upload. This has the major downside that "safely" running arbitrary > code from unknown publishers is a Hard Problem, which is one of the > big reasons that Linux distros put so many hurdles in the way of > becoming a package maintainer (i.e. package maintainers get to run > arbitrary code not just inside the distro build system but also on end > user machines, usually with elevated privileges, so you want to > establish a pretty high level of trust before letting people do it). > If I understand correctly, conda-forge works on the same basic > principle - reviewing the publishers before granting them publication > access, rather than defending against arbitrarily malicious code at > build time. - https://conda-forge.github.io - https://github.com/conda-forge - https://github.com/conda-forge/feedstocks - https://github.com/conda-forge/conda-smithy > > A more promising long term path is trust federation, which many folks > will already be familiar with through granting other services access > to their GitHub repositories, or using Twitter/Facebook/Google/et al > to sign into various systems. That's not going to be a quick fix > though, as it's contingent on sorting out the Warehouse migration > challenges, and those are already significant enough without piling > additional proposed changes on top of the already pending work. - [ ] Warehouse: ENH,SEC: A table, form, API for creating and revoking OAuth authz - (project, key, UPLOAD_RELEASE) - key renewal date There are a few existing OAuth Server libraries for pyramid (Warehouse): - https://github.com/sneridagh/osiris - https://github.com/tilgovi/pyramid-oauthlib - https://github.com/elliotpeele/pyramid_oauth2_provider - [ ] CI Release utility secrets: - VCS commit signature checking keyring - Package signing key (GPG ASC) - Package signature pubkey I just found these: - https://gist.github.com/audreyr/5990987 - https://github.com/michaeljoseph/changes - https://pypi.python.org/pypi/jarn.mkrelease - scripted GPG - https://caremad.io/posts/2013/07/packaging-signing-not-holy-grail/ - SHOULD have OOB keyring dist - https://github.com/pypa/twine/issues/157 - twine uploads *.asc if it exists - http://pythonhosted.org/distlib/tutorial.html#signing-a-distribution - http://pythonhosted.org/distlib/tutorial.html#verifying-signatures - SHOULD specify a limited keying-dir/ - https://packaging.python.org/distributing/ - [ ] howto create an .asc signature - https://docs.travis-ci.com/user/deployment/pypi/ - https://github.com/travis-ci/dpl/blob/master/lib/dpl/provider/pypi.rb - [x] https://github.com/travis-ci/dpl/issues/253 - [ ] oauth key instead of pass - https://docs.travis-ci.com/user/deployment/releases/ - upload release to GitHub - [ ] Is it possible to maintain a simple index of GitHub-hosted releases and .asc signatures w/ gh-pages? (for backups) - twine: Is uploading GitHub releases in scope? - https://pypi.org/search/?q=Github+release > However, something that could potentially be achieved in the near term > given folks interested enough in the idea to set about designing it > would be a default recommendation for a Travis CI + Appveyor + GitHub > Releases based setup that automated everything *except* the final > upload to PyPI, but then also offered a relatively simple way for > folks to pull their built artifacts from GitHub and push them to PyPI > (such that their login credentials never left their local system). > Folks that care enough about who hosts their source code to want to > avoid GitHub, or have complex enough build system needs that Travis CI > isn't sufficient, are likely to be technically sophisticated enough to > adapt service specific instructions to their particular circumstances, > and any such design work now would be a useful template for full > automation at some point in the future. https://www.google.com/search&q=Travis+Appveyor+GitHub+pypi ... also useful: - GitLab CI - .gitlab-ci.yml , config.toml - https://docs.gitlab.com/ce/ci/docker/using_docker_images.html - Jenkins - https://wiki.jenkins-ci.org/display/JENKINS/Docker+Plugin - https://wiki.jenkins-ci.org/display/JENKINS/ShiningPanda+Plugin - https://wiki.jenkins-ci.org/display/JENKINS/GitHub+Plugin - https://wiki.jenkins-ci.org/display/JENKINS/Release+Plugin - Buildbot - http://docs.buildbot.net/latest/manual/cfg-workers-docker.html - GitFlow and HubFlow specify a release/ branch with actual release tags all on master/ - https://datasift.github.io/gitflow/IntroducingGitFlow.html > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com <javascript:;> | Brisbane, > Australia > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG@python.org <javascript:;> > https://mail.python.org/mailman/listinfo/distutils-sig >
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
