On Saturday, November 5, 2016, Wes Turner <wes.tur...@gmail.com> wrote:
> For automated deployment / continuous deployment / "continuous delivery": > > - pip maintains a local cache > - devpi can be configured as a transparent proxy cache (in front of > pypi.org) > http://doc.devpi.net/latest/quickstart-pypimirror.html > > - GitLab CI can show a checkmark for a deploy pipeline stage > > On Saturday, November 5, 2016, Wes Turner <wes.tur...@gmail.com > <javascript:_e(%7B%7D,'cvml','wes.tur...@gmail.com');>> wrote: > >> >> >> On Saturday, November 5, 2016, Nick Coghlan <ncogh...@gmail.com> wrote: >> >>> On 4 November 2016 at 06:07, Nathaniel Smith <n...@pobox.com> wrote: >>> > I think we're drifting pretty far off topic here... IIRC the original >>> > discussion was about whether the travis-ci infrastructure could be >>> suborned >>> > to provide an sdist->wheel autobuilding service for pypi. (Answer: >>> maybe, >>> > though it would be pretty awkward, and no one seems to be jumping up >>> to make >>> > it happen.) >>> >>> The hard part of designing any such system isn't so much the building >>> process, it's the authentication, authorisation and trust management >>> for the release publication step. At the moment, that amounts to "Give >>> the service your PyPI password, so PyPI will 100% trust that they're >>> you" due to limitations on the PyPI side of things, and "Hello web >>> service developer, I grant you 100% authority to impersonate me to the >>> rest of the world however you like" is a questionable idea in any >>> circumstance, let alone when we're talking about publishing software. >>> >>> Since we don't currently provide end-to-end package signing, PyPI >>> initiated builds would solve the trust problem by having PyPI trust >>> *itself*, and only require user credentials for the initial source >>> upload. This has the major downside that "safely" running arbitrary >>> code from unknown publishers is a Hard Problem, which is one of the >>> big reasons that Linux distros put so many hurdles in the way of >>> becoming a package maintainer (i.e. package maintainers get to run >>> arbitrary code not just inside the distro build system but also on end >>> user machines, usually with elevated privileges, so you want to >>> establish a pretty high level of trust before letting people do it). >>> If I understand correctly, conda-forge works on the same basic >>> principle - reviewing the publishers before granting them publication >>> access, rather than defending against arbitrarily malicious code at >>> build time. >> >> >> - https://conda-forge.github.io >> - https://github.com/conda-forge >> - https://github.com/conda-forge/feedstocks >> - https://github.com/conda-forge/conda-smithy >> >> >>> >>> A more promising long term path is trust federation, which many folks >>> will already be familiar with through granting other services access >>> to their GitHub repositories, or using Twitter/Facebook/Google/et al >>> to sign into various systems. That's not going to be a quick fix >>> though, as it's contingent on sorting out the Warehouse migration >>> challenges, and those are already significant enough without piling >>> additional proposed changes on top of the already pending work. >> >> >> - [ ] Warehouse: ENH,SEC: A table, form, API for creating and revoking >> OAuth authz >> - (project, key, UPLOAD_RELEASE) >> - key renewal date >> >> There are a few existing OAuth Server libraries for pyramid (Warehouse): >> >> - https://github.com/sneridagh/osiris >> - https://github.com/tilgovi/pyramid-oauthlib >> - https://github.com/elliotpeele/pyramid_oauth2_provider >> >> >> >> - [ ] CI Release utility secrets: >> - VCS commit signature checking keyring >> - Package signing key (GPG ASC) >> - Package signature pubkey >> >> I just found these: >> >> - https://gist.github.com/audreyr/5990987 >> - https://github.com/michaeljoseph/changes >> >> - https://pypi.python.org/pypi/jarn.mkrelease >> - scripted GPG >> >> - https://caremad.io/posts/2013/07/packaging-signing-not-holy-grail/ >> - SHOULD have OOB keyring dist >> >> - https://github.com/pypa/twine/issues/157 >> - twine uploads *.asc if it exists >> >> - http://pythonhosted.org/distlib/tutorial.html#signing-a-distribution >> - http://pythonhosted.org/distlib/tutorial.html#verifying-signatures >> - SHOULD specify a limited keying-dir/ >> >> - https://packaging.python.org/distributing/ >> - [ ] howto create an .asc signature >> >> - https://docs.travis-ci.com/user/deployment/pypi/ >> - https://github.com/travis-ci/dpl/blob/master/lib/dpl/provider/pypi.rb >> - [x] https://github.com/travis-ci/dpl/issues/253 >> - [ ] oauth key instead of pass >> >> - https://docs.travis-ci.com/user/deployment/releases/ >> - upload release to GitHub >> >> - [ ] Is it possible to maintain a simple index of GitHub-hosted releases >> and .asc signatures w/ gh-pages? (for backups) >> - twine: Is uploading GitHub releases in scope? >> - https://pypi.org/search/?q=Github+release >> >> >>> However, something that could potentially be achieved in the near term >>> given folks interested enough in the idea to set about designing it >>> would be a default recommendation for a Travis CI + Appveyor + GitHub >>> Releases based setup that automated everything *except* the final >>> upload to PyPI, but then also offered a relatively simple way for >>> folks to pull their built artifacts from GitHub and push them to PyPI >>> (such that their login credentials never left their local system). >>> Folks that care enough about who hosts their source code to want to >>> avoid GitHub, or have complex enough build system needs that Travis CI >>> isn't sufficient, are likely to be technically sophisticated enough to >>> adapt service specific instructions to their particular circumstances, >>> and any such design work now would be a useful template for full >>> automation at some point in the future. >> >> >> https://www.google.com/search&q=Travis+Appveyor+GitHub+pypi >> >> ... also useful: >> >> - GitLab CI >> - .gitlab-ci.yml , config.toml >> - https://docs.gitlab.com/ce/ci/docker/using_docker_images.html >> >> - Jenkins >> - https://wiki.jenkins-ci.org/display/JENKINS/Docker+Plugin >> - https://wiki.jenkins-ci.org/display/JENKINS/ShiningPanda+Plugin >> - https://wiki.jenkins-ci.org/display/JENKINS/GitHub+Plugin >> - https://wiki.jenkins-ci.org/display/JENKINS/Release+Plugin >> >> - Buildbot >> - http://docs.buildbot.net/latest/manual/cfg-workers-docker.html >> >> - GitFlow and HubFlow specify a release/ branch with actual release tags >> all on master/ >> - https://datasift.github.io/gitflow/IntroducingGitFlow.html >> >> >> >>> Cheers, >>> Nick. >>> >>> -- >>> Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia >>> _______________________________________________ >>> Distutils-SIG maillist - Distutils-SIG@python.org >>> https://mail.python.org/mailman/listinfo/distutils-sig >>> >>
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
