For automated deployment / continuous deployment / "continuous delivery":
- pip maintains a local cache - devpi can be configured as a transparent proxy cache (in front of pypi.org ) - GitLab CI can show a checkmark for a deploy pipeline stage On Saturday, November 5, 2016, Wes Turner <wes.tur...@gmail.com> wrote: > > > On Saturday, November 5, 2016, Nick Coghlan <ncogh...@gmail.com > <javascript:_e(%7B%7D,'cvml','ncogh...@gmail.com');>> wrote: > >> On 4 November 2016 at 06:07, Nathaniel Smith <n...@pobox.com> wrote: >> > I think we're drifting pretty far off topic here... IIRC the original >> > discussion was about whether the travis-ci infrastructure could be >> suborned >> > to provide an sdist->wheel autobuilding service for pypi. (Answer: >> maybe, >> > though it would be pretty awkward, and no one seems to be jumping up to >> make >> > it happen.) >> >> The hard part of designing any such system isn't so much the building >> process, it's the authentication, authorisation and trust management >> for the release publication step. At the moment, that amounts to "Give >> the service your PyPI password, so PyPI will 100% trust that they're >> you" due to limitations on the PyPI side of things, and "Hello web >> service developer, I grant you 100% authority to impersonate me to the >> rest of the world however you like" is a questionable idea in any >> circumstance, let alone when we're talking about publishing software. >> >> Since we don't currently provide end-to-end package signing, PyPI >> initiated builds would solve the trust problem by having PyPI trust >> *itself*, and only require user credentials for the initial source >> upload. This has the major downside that "safely" running arbitrary >> code from unknown publishers is a Hard Problem, which is one of the >> big reasons that Linux distros put so many hurdles in the way of >> becoming a package maintainer (i.e. package maintainers get to run >> arbitrary code not just inside the distro build system but also on end >> user machines, usually with elevated privileges, so you want to >> establish a pretty high level of trust before letting people do it). >> If I understand correctly, conda-forge works on the same basic >> principle - reviewing the publishers before granting them publication >> access, rather than defending against arbitrarily malicious code at >> build time. > > > - https://conda-forge.github.io > - https://github.com/conda-forge > - https://github.com/conda-forge/feedstocks > - https://github.com/conda-forge/conda-smithy > > >> >> A more promising long term path is trust federation, which many folks >> will already be familiar with through granting other services access >> to their GitHub repositories, or using Twitter/Facebook/Google/et al >> to sign into various systems. That's not going to be a quick fix >> though, as it's contingent on sorting out the Warehouse migration >> challenges, and those are already significant enough without piling >> additional proposed changes on top of the already pending work. > > > - [ ] Warehouse: ENH,SEC: A table, form, API for creating and revoking > OAuth authz > - (project, key, UPLOAD_RELEASE) > - key renewal date > > There are a few existing OAuth Server libraries for pyramid (Warehouse): > > - https://github.com/sneridagh/osiris > - https://github.com/tilgovi/pyramid-oauthlib > - https://github.com/elliotpeele/pyramid_oauth2_provider > > > > - [ ] CI Release utility secrets: > - VCS commit signature checking keyring > - Package signing key (GPG ASC) > - Package signature pubkey > > I just found these: > > - https://gist.github.com/audreyr/5990987 > - https://github.com/michaeljoseph/changes > > - https://pypi.python.org/pypi/jarn.mkrelease > - scripted GPG > > - https://caremad.io/posts/2013/07/packaging-signing-not-holy-grail/ > - SHOULD have OOB keyring dist > > - https://github.com/pypa/twine/issues/157 > - twine uploads *.asc if it exists > > - http://pythonhosted.org/distlib/tutorial.html#signing-a-distribution > - http://pythonhosted.org/distlib/tutorial.html#verifying-signatures > - SHOULD specify a limited keying-dir/ > > - https://packaging.python.org/distributing/ > - [ ] howto create an .asc signature > > - https://docs.travis-ci.com/user/deployment/pypi/ > - https://github.com/travis-ci/dpl/blob/master/lib/dpl/provider/pypi.rb > - [x] https://github.com/travis-ci/dpl/issues/253 > - [ ] oauth key instead of pass > > - https://docs.travis-ci.com/user/deployment/releases/ > - upload release to GitHub > > - [ ] Is it possible to maintain a simple index of GitHub-hosted releases > and .asc signatures w/ gh-pages? (for backups) > - twine: Is uploading GitHub releases in scope? > - https://pypi.org/search/?q=Github+release > > >> However, something that could potentially be achieved in the near term >> given folks interested enough in the idea to set about designing it >> would be a default recommendation for a Travis CI + Appveyor + GitHub >> Releases based setup that automated everything *except* the final >> upload to PyPI, but then also offered a relatively simple way for >> folks to pull their built artifacts from GitHub and push them to PyPI >> (such that their login credentials never left their local system). >> Folks that care enough about who hosts their source code to want to >> avoid GitHub, or have complex enough build system needs that Travis CI >> isn't sufficient, are likely to be technically sophisticated enough to >> adapt service specific instructions to their particular circumstances, >> and any such design work now would be a useful template for full >> automation at some point in the future. > > > https://www.google.com/search&q=Travis+Appveyor+GitHub+pypi > > ... also useful: > > - GitLab CI > - .gitlab-ci.yml , config.toml > - https://docs.gitlab.com/ce/ci/docker/using_docker_images.html > > - Jenkins > - https://wiki.jenkins-ci.org/display/JENKINS/Docker+Plugin > - https://wiki.jenkins-ci.org/display/JENKINS/ShiningPanda+Plugin > - https://wiki.jenkins-ci.org/display/JENKINS/GitHub+Plugin > - https://wiki.jenkins-ci.org/display/JENKINS/Release+Plugin > > - Buildbot > - http://docs.buildbot.net/latest/manual/cfg-workers-docker.html > > - GitFlow and HubFlow specify a release/ branch with actual release tags > all on master/ > - https://datasift.github.io/gitflow/IntroducingGitFlow.html > > > >> Cheers, >> Nick. >> >> -- >> Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia >> _______________________________________________ >> Distutils-SIG maillist - Distutils-SIG@python.org >> https://mail.python.org/mailman/listinfo/distutils-sig >> >
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig