Re: [openstack-dev] [release][infra][zuul][zuulv3][horizon][neutron] project-specific release job templates
Excerpts from Doug Hellmann's message of 2017-10-24 09:42:25 -0400: > The neutron queens milestone release is held up right now because > some of the repositories are using a release job template that isn't > recognized by the validation code in the releases repository. I'm > trying to decide between adding the custom job template to the > validation code or changing the release jobs for those neutron > repositories to use the one that isn't custom. I think we'll have > the same problem with horizon plugins using the custom job template > set up for them. > > It looks like the publish-to-pypi-neutron template modifies > publish-to-pypi by adding openstack/neutron to the required-repositories > list for the release-openstack-python job. That repository was also at > some point added directly to the release-openstack-python job. So > technically the extension via the template is not needed. The same > applies to publish-to-pypi-horizon. > > I see a few issues with keeping job template variants: > > 1. Having multiple release job variants complicates the release >repo validation logic. That logic was put in place after we >discovered several projects without release jobs defined at all, >so we definitely want to keep some level of validation in place. > > 2. As we continue to make changes to the release jobs, we're going >to have to consider whether to make those changes in multiple >places, which seems error prone. > > 3. As we find other projects with more dependencies, we're going >to end up with more custom templates. > > Those issues may be mitigated if we move the release job definitions > into the releases repo as we have discussed, because it will be > more obvious that we have multiple related templates that are > variants of one another and we can make the relevant changes all > together in one patch. > > One alternative to keeping multiple variants, and defining more in > the future, is to add required-repositories to the release-openstack-python > job directly, as we discover they are needed. Of course that means > we will clone repositories for some jobs that don't actually use > them. I don't know how big of an issue that really is, but the issue > of not knowing which instances of a job actually need a particular > dependency seems like more of a justification for keeping separate > templates than any runtime savings we would have by skipping a > couple of extra calls to git clone. > > It feels like we have two related by not necessarily dependent > policy questions we need to answer before we decide how to proceed: > > (a) Do we want to move the release job definitions from project-config > and openstack-zuul-jobs to the releases repo? > > (b) Do we want to have multiple release job templates due to custom > dependencies (or any other reason)? > > Thoughts? > > Doug > Based on the conversation in this thread and on IRC, we decided to keep the job variants and update the releases repo so projects can explicitly indicate that they are using those variants instead of the default. See https://review.openstack.org/515119 for that change, and there are related changes in the series for deliverable files that needed to be updated. Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [release][infra][zuul][zuulv3][horizon][neutron] project-specific release job templates
On 2017-10-24 16:24:39 -0700 (-0700), Armando M. wrote: [...] > I hope I represented the conversation correctly! [...] Totally accurate, thanks! -- Jeremy Stanley signature.asc Description: Digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [release][infra][zuul][zuulv3][horizon][neutron] project-specific release job templates
On 24 October 2017 at 13:13, Jeremy Stanleywrote: > On 2017-10-24 11:31:56 -0700 (-0700), Armando M. wrote: > [...] > > the work on neutron-lib is slowly progressing but it's not close enough > to > > allow us to break the dependency that requires neutron sub-projects to > add > > neutron to the list of required-projects (which I believe the sort of the > > error in the release pipeline stems from). > [...] > > It's not entirely clear to me what about a Neutron plug-in would > actually require the neutron source installed just to be able to > generate a tarball (or wheel) or build sphinx docs. Couldn't that be > solved even if things like installation or unit testing still > continue to need neutron? > Jeremy and I talked about this in [1]. To sum it up: the necessity of neutron for plugins is primarily for testing and installation to the best of my knowledge. Other coupling tied to bash scripts that control gate jobs could be easily addressed if it gets in the way. Therefore, should we move away from tox for release specific tasks, it's entirely possible to drop the custom project definition. I hope I represented the conversation correctly! [1] http://eavesdrop.openstack.org/irclogs/%23openstack-infra/latest.log.html#t2017-10-24T23:06:35 > -- > Jeremy Stanley > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [release][infra][zuul][zuulv3][horizon][neutron] project-specific release job templates
On 2017-10-24 11:31:56 -0700 (-0700), Armando M. wrote: [...] > the work on neutron-lib is slowly progressing but it's not close enough to > allow us to break the dependency that requires neutron sub-projects to add > neutron to the list of required-projects (which I believe the sort of the > error in the release pipeline stems from). [...] It's not entirely clear to me what about a Neutron plug-in would actually require the neutron source installed just to be able to generate a tarball (or wheel) or build sphinx docs. Couldn't that be solved even if things like installation or unit testing still continue to need neutron? -- Jeremy Stanley signature.asc Description: Digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [release][infra][zuul][zuulv3][horizon][neutron] project-specific release job templates
On 24 October 2017 at 10:35, Doug Hellmannwrote: > Excerpts from Jeremy Stanley's message of 2017-10-24 15:05:34 +: > > On 2017-10-24 09:42:25 -0400 (-0400), Doug Hellmann wrote: > > [...] > > > It looks like the publish-to-pypi-neutron template modifies > > > publish-to-pypi by adding openstack/neutron to the > required-repositories > > > list for the release-openstack-python job. That repository was also at > > > some point added directly to the release-openstack-python job. So > > > technically the extension via the template is not needed. The same > > > applies to publish-to-pypi-horizon. > > [...] > > > > Both are stop-gap solutions and I think either is fine in the short > > term to get us through the bulk of the release request backlog. > > Longer term we want to have neither of those. Python projects should > > be able to build sdists/wheels[1], documentation[2] and release > > notes[3] without tox (a behavior change which significantly > > complicates preinstalling source code from other projects, so best > > if their release jobs simply don't rely on doing that at all). > > > > > As we find other projects with more dependencies, we're going to > > > end up with more custom templates. > > [...] > > > > If we do, this only accelerates the need for something like a > > community goal for fixing this in those respective Python > > plugin/extension projects. > > Right. I know the neutron team has been working on their library system > for quite a while. Maybe someone can report on the progress there? > the work on neutron-lib is slowly progressing but it's not close enough to allow us to break the dependency that requires neutron sub-projects to add neutron to the list of required-projects (which I believe the sort of the error in the release pipeline stems from). > > I don't know if the horizon team is working on that (I'm just uninformed > about what they're doing). So maybe someone from that team can comment? > > > > > > One alternative to keeping multiple variants, and defining more in > > > the future, is to add required-repositories to the > release-openstack-python > > > job directly, as we discover they are needed. Of course that means > > > we will clone repositories for some jobs that don't actually use > > > them. I don't know how big of an issue that really is, but the issue > > > of not knowing which instances of a job actually need a particular > > > dependency seems like more of a justification for keeping separate > > > templates than any runtime savings we would have by skipping a > > > couple of extra calls to git clone. > > [...] > > > > Well, in this case you're talking about Zuul needing to calculate > > merge states for the neutron and horizon repos and then push these > > into every build of the affected jobs, which is no small amount of > > overhead on its own given the size of those particular repos. Also, > > once the tox-siblings role[4] is back in action (likely later this > > week?) Zuul will go back to preinstalling any required-projects from > > source into tox virtualenvs for these jobs, which is quite a bit > > more overhead still at that point. > > Yes, we don't want that, so I think that means we want to retain the > multiple project templates for the short-term. > > > > [1] http://git.openstack.org/cgit/openstack/governance/commit/? > id=2678231 > > [2] http://git.openstack.org/cgit/openstack/governance/commit/? > id=2c0cdd2 > > [3] http://git.openstack.org/cgit/openstack/governance/commit/? > id=df438a7 > > [4] http://git.openstack.org/cgit/openstack-infra/zuul-jobs/ > tree/roles/tox-siblings > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [release][infra][zuul][zuulv3][horizon][neutron] project-specific release job templates
Excerpts from Jeremy Stanley's message of 2017-10-24 15:05:34 +: > On 2017-10-24 09:42:25 -0400 (-0400), Doug Hellmann wrote: > [...] > > It looks like the publish-to-pypi-neutron template modifies > > publish-to-pypi by adding openstack/neutron to the required-repositories > > list for the release-openstack-python job. That repository was also at > > some point added directly to the release-openstack-python job. So > > technically the extension via the template is not needed. The same > > applies to publish-to-pypi-horizon. > [...] > > Both are stop-gap solutions and I think either is fine in the short > term to get us through the bulk of the release request backlog. > Longer term we want to have neither of those. Python projects should > be able to build sdists/wheels[1], documentation[2] and release > notes[3] without tox (a behavior change which significantly > complicates preinstalling source code from other projects, so best > if their release jobs simply don't rely on doing that at all). > > > As we find other projects with more dependencies, we're going to > > end up with more custom templates. > [...] > > If we do, this only accelerates the need for something like a > community goal for fixing this in those respective Python > plugin/extension projects. Right. I know the neutron team has been working on their library system for quite a while. Maybe someone can report on the progress there? I don't know if the horizon team is working on that (I'm just uninformed about what they're doing). So maybe someone from that team can comment? > > > One alternative to keeping multiple variants, and defining more in > > the future, is to add required-repositories to the release-openstack-python > > job directly, as we discover they are needed. Of course that means > > we will clone repositories for some jobs that don't actually use > > them. I don't know how big of an issue that really is, but the issue > > of not knowing which instances of a job actually need a particular > > dependency seems like more of a justification for keeping separate > > templates than any runtime savings we would have by skipping a > > couple of extra calls to git clone. > [...] > > Well, in this case you're talking about Zuul needing to calculate > merge states for the neutron and horizon repos and then push these > into every build of the affected jobs, which is no small amount of > overhead on its own given the size of those particular repos. Also, > once the tox-siblings role[4] is back in action (likely later this > week?) Zuul will go back to preinstalling any required-projects from > source into tox virtualenvs for these jobs, which is quite a bit > more overhead still at that point. Yes, we don't want that, so I think that means we want to retain the multiple project templates for the short-term. > > [1] http://git.openstack.org/cgit/openstack/governance/commit/?id=2678231 > [2] http://git.openstack.org/cgit/openstack/governance/commit/?id=2c0cdd2 > [3] http://git.openstack.org/cgit/openstack/governance/commit/?id=df438a7 > [4] > http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/tox-siblings __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [release][infra][zuul][zuulv3][horizon][neutron] project-specific release job templates
On 2017-10-24 09:42:25 -0400 (-0400), Doug Hellmann wrote: [...] > It looks like the publish-to-pypi-neutron template modifies > publish-to-pypi by adding openstack/neutron to the required-repositories > list for the release-openstack-python job. That repository was also at > some point added directly to the release-openstack-python job. So > technically the extension via the template is not needed. The same > applies to publish-to-pypi-horizon. [...] Both are stop-gap solutions and I think either is fine in the short term to get us through the bulk of the release request backlog. Longer term we want to have neither of those. Python projects should be able to build sdists/wheels[1], documentation[2] and release notes[3] without tox (a behavior change which significantly complicates preinstalling source code from other projects, so best if their release jobs simply don't rely on doing that at all). > As we find other projects with more dependencies, we're going to > end up with more custom templates. [...] If we do, this only accelerates the need for something like a community goal for fixing this in those respective Python plugin/extension projects. > One alternative to keeping multiple variants, and defining more in > the future, is to add required-repositories to the release-openstack-python > job directly, as we discover they are needed. Of course that means > we will clone repositories for some jobs that don't actually use > them. I don't know how big of an issue that really is, but the issue > of not knowing which instances of a job actually need a particular > dependency seems like more of a justification for keeping separate > templates than any runtime savings we would have by skipping a > couple of extra calls to git clone. [...] Well, in this case you're talking about Zuul needing to calculate merge states for the neutron and horizon repos and then push these into every build of the affected jobs, which is no small amount of overhead on its own given the size of those particular repos. Also, once the tox-siblings role[4] is back in action (likely later this week?) Zuul will go back to preinstalling any required-projects from source into tox virtualenvs for these jobs, which is quite a bit more overhead still at that point. [1] http://git.openstack.org/cgit/openstack/governance/commit/?id=2678231 [2] http://git.openstack.org/cgit/openstack/governance/commit/?id=2c0cdd2 [3] http://git.openstack.org/cgit/openstack/governance/commit/?id=df438a7 [4] http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/tox-siblings -- Jeremy Stanley signature.asc Description: Digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [release][infra][zuul][zuulv3][horizon][neutron] project-specific release job templates
Please see below: On Tue, Oct 24, 2017 at 10:11 AM, Thierry Carrezwrote: > Doug Hellmann wrote: >> [...] >> It feels like we have two related by not necessarily dependent >> policy questions we need to answer before we decide how to proceed: > > I don't have strong opinions on any of those two, I think anything would > work. If I had to pick between yes and no, here is my gut feeling: > >> (a) Do we want to move the release job definitions from project-config >> and openstack-zuul-jobs to the releases repo? > > Leaning towards yes as it will (imho) simplify maintenance. We'll still > have a couple of privileged jobs that will live on the infra side though? Agree with Thierry. >> (b) Do we want to have multiple release job templates due to custom >> dependencies (or any other reason)? > > Leaning towards no if we can avoid it, again to simplify operations. > That said, it will be difficult to enforce... Same. Let's start without it and revisit if we need them later. > -- > Thierry Carrez (ttx) > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Davanum Srinivas :: https://twitter.com/dims __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [release][infra][zuul][zuulv3][horizon][neutron] project-specific release job templates
On Tue, Oct 24, 2017 at 09:42:25AM -0400, Doug Hellmann wrote: > The neutron queens milestone release is held up right now because > some of the repositories are using a release job template that isn't > recognized by the validation code in the releases repository. I'm > trying to decide between adding the custom job template to the > validation code or changing the release jobs for those neutron > repositories to use the one that isn't custom. I think we'll have > the same problem with horizon plugins using the custom job template > set up for them. > > It looks like the publish-to-pypi-neutron template modifies > publish-to-pypi by adding openstack/neutron to the required-repositories > list for the release-openstack-python job. That repository was also at > some point added directly to the release-openstack-python job. So > technically the extension via the template is not needed. The same > applies to publish-to-pypi-horizon. > > I see a few issues with keeping job template variants: > > 1. Having multiple release job variants complicates the release >repo validation logic. That logic was put in place after we >discovered several projects without release jobs defined at all, >so we definitely want to keep some level of validation in place. > > 2. As we continue to make changes to the release jobs, we're going >to have to consider whether to make those changes in multiple >places, which seems error prone. This is my big concern. Chances are very likely things will get missed. > > 3. As we find other projects with more dependencies, we're going >to end up with more custom templates. > > Those issues may be mitigated if we move the release job definitions > into the releases repo as we have discussed, because it will be > more obvious that we have multiple related templates that are > variants of one another and we can make the relevant changes all > together in one patch. > > One alternative to keeping multiple variants, and defining more in > the future, is to add required-repositories to the release-openstack-python > job directly, as we discover they are needed. Of course that means > we will clone repositories for some jobs that don't actually use > them. I don't know how big of an issue that really is, but the issue > of not knowing which instances of a job actually need a particular > dependency seems like more of a justification for keeping separate > templates than any runtime savings we would have by skipping a > couple of extra calls to git clone. > > It feels like we have two related by not necessarily dependent > policy questions we need to answer before we decide how to proceed: > > (a) Do we want to move the release job definitions from project-config > and openstack-zuul-jobs to the releases repo? > This seems like a good compromise to me. If we at least have them all in one place, it will make it a lot easier to make sure we are able to get everything updated as needed. > (b) Do we want to have multiple release job templates due to custom > dependencies (or any other reason)? > I'm guessing we will want this. We will most likely at some point have some need to have unique requirements for certain jobs. > Thoughts? > > Doug > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [release][infra][zuul][zuulv3][horizon][neutron] project-specific release job templates
Doug Hellmann wrote: > [...] > It feels like we have two related by not necessarily dependent > policy questions we need to answer before we decide how to proceed: I don't have strong opinions on any of those two, I think anything would work. If I had to pick between yes and no, here is my gut feeling: > (a) Do we want to move the release job definitions from project-config > and openstack-zuul-jobs to the releases repo? Leaning towards yes as it will (imho) simplify maintenance. We'll still have a couple of privileged jobs that will live on the infra side though? > (b) Do we want to have multiple release job templates due to custom > dependencies (or any other reason)? Leaning towards no if we can avoid it, again to simplify operations. That said, it will be difficult to enforce... -- Thierry Carrez (ttx) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [release][infra][zuul][zuulv3][horizon][neutron] project-specific release job templates
The neutron queens milestone release is held up right now because some of the repositories are using a release job template that isn't recognized by the validation code in the releases repository. I'm trying to decide between adding the custom job template to the validation code or changing the release jobs for those neutron repositories to use the one that isn't custom. I think we'll have the same problem with horizon plugins using the custom job template set up for them. It looks like the publish-to-pypi-neutron template modifies publish-to-pypi by adding openstack/neutron to the required-repositories list for the release-openstack-python job. That repository was also at some point added directly to the release-openstack-python job. So technically the extension via the template is not needed. The same applies to publish-to-pypi-horizon. I see a few issues with keeping job template variants: 1. Having multiple release job variants complicates the release repo validation logic. That logic was put in place after we discovered several projects without release jobs defined at all, so we definitely want to keep some level of validation in place. 2. As we continue to make changes to the release jobs, we're going to have to consider whether to make those changes in multiple places, which seems error prone. 3. As we find other projects with more dependencies, we're going to end up with more custom templates. Those issues may be mitigated if we move the release job definitions into the releases repo as we have discussed, because it will be more obvious that we have multiple related templates that are variants of one another and we can make the relevant changes all together in one patch. One alternative to keeping multiple variants, and defining more in the future, is to add required-repositories to the release-openstack-python job directly, as we discover they are needed. Of course that means we will clone repositories for some jobs that don't actually use them. I don't know how big of an issue that really is, but the issue of not knowing which instances of a job actually need a particular dependency seems like more of a justification for keeping separate templates than any runtime savings we would have by skipping a couple of extra calls to git clone. It feels like we have two related by not necessarily dependent policy questions we need to answer before we decide how to proceed: (a) Do we want to move the release job definitions from project-config and openstack-zuul-jobs to the releases repo? (b) Do we want to have multiple release job templates due to custom dependencies (or any other reason)? Thoughts? Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev