[openstack-dev] [all][release][neutron][horizon] Publishing server projects to PyPI
Hey everybody! tl;dr - We'd like to start publishing the server projects to PyPI Background == The move to Zuul v3 has highlighted an odd situation we're in with some of our projects, most notably neutron and horizon plugin projects. Namely, those plugins need to depend on neutron and horizon, but we do not publish neutron and horizon releases to PyPI. There are a couple of reasons why we haven't historically published server projects. We were concerned that doing so would 'encourage' installation of the services from pip. Also that, because it is pip and not dpkg/rpm/emerge there's no chance that 'pip install nova' will result in a functioning service. (It's worth noting that pip and PyPI in general were in a much different state 6 years ago than today - thanks dstufft for all the great work!) I think it's safe to say that the 'ship has sailed' on those two issues ... which is to say we're already well past the point of return on either issue. pip is used or not used by deployers as they see fit, and I don't think 'pip install nova' not producing a working nova isn't going to actually surprise anyone. Moving Forward == Before we can do anything, we need to update some of the release validation scripts to allow this (they currently double-check that we're doing the right things with the right type of project): https://review.openstack.org/#/c/521115/ Once that's done, rather than doing a big-bang transition, the plan is to move server projects from using the release-openstack-server template to using the publish-to-pypi template as they are ready. This should simplify a great many things for horizon and neutron - and allow us to get rid of the horizon and neutron specific jobs and templates. There are a few gotchas we'll need to work through - notably there is another project there already named "Keystone" - although it seems not very used. I've reached out to the author to see if he's willing to relinquish. If he's not, we'll have to get creative. Thanks! Monty __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][release][neutron][horizon] Publishing server projects to PyPI
On 2017-11-17 17:27, Monty Taylor wrote:
> Hey everybody!
>
> tl;dr - We'd like to start publishing the server projects to PyPI
>
> Background
> ==
>
> The move to Zuul v3 has highlighted an odd situation we're in with some
> of our projects, most notably neutron and horizon plugin projects.
> Namely, those plugins need to depend on neutron and horizon, but we do
> not publish neutron and horizon releases to PyPI.
>
> There are a couple of reasons why we haven't historically published
> server projects. We were concerned that doing so would 'encourage'
> installation of the services from pip. Also that, because it is pip and
> not dpkg/rpm/emerge there's no chance that 'pip install nova' will
> result in a functioning service. (It's worth noting that pip and PyPI in
> general were in a much different state 6 years ago than today - thanks
> dstufft for all the great work!)
>
> I think it's safe to say that the 'ship has sailed' on those two issues
> ... which is to say we're already well past the point of return on
> either issue. pip is used or not used by deployers as they see fit, and
> I don't think 'pip install nova' not producing a working nova isn't
> going to actually surprise anyone.
>
> Moving Forward
> ==
>
> Before we can do anything, we need to update some of the release
> validation scripts to allow this (they currently double-check that we're
> doing the right things with the right type of project):
> https://review.openstack.org/#/c/521115/
>
> Once that's done, rather than doing a big-bang transition, the plan is
> to move server projects from using the release-openstack-server template
> to using the publish-to-pypi template as they are ready.
>
> This should simplify a great many things for horizon and neutron - and
> allow us to get rid of the horizon and neutron specific jobs and
> templates. There are a few gotchas we'll need to work through - notably
> there is another project there already named "Keystone" - although it
> seems not very used. I've reached out to the author to see if he's
> willing to relinquish. If he's not, we'll have to get creative.
One question on this: right now the dashboard and neutron plugins test
against current git head. Wouldn't installing from pypi mean that they
test against an older stable version?
Andreas
--
Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton,
HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][release][neutron][horizon] Publishing server projects to PyPI
On 2017-11-17 17:51:36 +0100 (+0100), Andreas Jaeger wrote: [...] > One question on this: right now the dashboard and neutron plugins test > against current git head. Wouldn't installing from pypi mean that they > test against an older stable version? I brought this up in #openstack-release as well, and Monty reminded me that the tox-siblings role attempts to address that for any repository declared as a required-project for a given job. That said, it's only a solution in CI automation so until we have a good story around running job definitions under Ansible in local development environments, I don't think we can completely get rid of the associated pip wrapper scripts in those various projects (but we can at least find a way to stop running them in the CI system in the meantime). -- Jeremy Stanley signature.asc Description: Digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][release][neutron][horizon] Publishing server projects to PyPI
On 11/17/2017 10:51 AM, Andreas Jaeger wrote: On 2017-11-17 17:27, Monty Taylor wrote: Hey everybody! tl;dr - We'd like to start publishing the server projects to PyPI Background == The move to Zuul v3 has highlighted an odd situation we're in with some of our projects, most notably neutron and horizon plugin projects. Namely, those plugins need to depend on neutron and horizon, but we do not publish neutron and horizon releases to PyPI. There are a couple of reasons why we haven't historically published server projects. We were concerned that doing so would 'encourage' installation of the services from pip. Also that, because it is pip and not dpkg/rpm/emerge there's no chance that 'pip install nova' will result in a functioning service. (It's worth noting that pip and PyPI in general were in a much different state 6 years ago than today - thanks dstufft for all the great work!) I think it's safe to say that the 'ship has sailed' on those two issues ... which is to say we're already well past the point of return on either issue. pip is used or not used by deployers as they see fit, and I don't think 'pip install nova' not producing a working nova isn't going to actually surprise anyone. Moving Forward == Before we can do anything, we need to update some of the release validation scripts to allow this (they currently double-check that we're doing the right things with the right type of project): https://review.openstack.org/#/c/521115/ Once that's done, rather than doing a big-bang transition, the plan is to move server projects from using the release-openstack-server template to using the publish-to-pypi template as they are ready. This should simplify a great many things for horizon and neutron - and allow us to get rid of the horizon and neutron specific jobs and templates. There are a few gotchas we'll need to work through - notably there is another project there already named "Keystone" - although it seems not very used. I've reached out to the author to see if he's willing to relinquish. If he's not, we'll have to get creative. One question on this: right now the dashboard and neutron plugins test against current git head. Wouldn't installing from pypi mean that they test against an older stable version? Yes - by default - so we actually probably can't get rid of all of the variants. However, the tox-siblings support in the tox jobs would allow us to remove the zuul-cloner and other magic from the install_tox.sh scripts in the repos and just let things work normally. Incidentally, it would also allow for the plugin projects to decide if they wanted to test against latest stable or master - or both. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
