Re: [openstack-dev] [openstack-infra] [neutron] Third Party CI Voting
Edgar, in a nutshell my point is that if we want to remove voting rights from every CI I'm fine with it. However, I think what's being discussed in this thread is already captured very well by [1] and believe the policy it outlines is perfectly fine for Neutron purposes. Salvatore [1] http://git.openstack.org/cgit/openstack/neutron/tree/doc/source/policies/thirdparty-ci.rst On 25 June 2015 at 17:08, Edgar Magana wrote: > Thank for your response Salvatore. I am not sure what is your position > in this topic? Are you fine removing voting rights to all Cis? > > Edgar > > From: Salvatore Orlando > Reply-To: "OpenStack Development Mailing List (not for usage questions)" > Date: Thursday, June 25, 2015 at 7:59 AM > To: "OpenStack Development Mailing List (not for usage questions)" > Subject: Re: [openstack-dev] [openstack-infra] [neutron] Third Party CI > Voting > > > > On 25 June 2015 at 16:08, John Davidge (jodavidg) > wrote: > >> Hi all, >> >> Recent neutron third party CI issues have got me thinking again about a >> topic which we discussed in Vancouver: >> >> Should any Third Party CI have voting rights for neutron patches in >> gerrit? >> > > Why should this be a decision for Neutron only? > > >> >> I’d like to suggest that they shouldn’t. >> >> A -1 from a third party CI tool can often be an indication that the CI >> tool itself or the third party plugin is broken, rather than there being >> issues with the patch under review. I don’t think there are many cases >> where a third party CI tool has caught a genuine issue that Jenkins has >> missed. With the current voting rights these CI tools cause a lot of noise >> when they experience problems. >> > > As far as I am aware no 3rd party CI tool has a better coverage than the > upstream one. > some 3rd party CIs exercise different code paths and might uncover some > issue that the upstream CI did not cover. There will surely be people > claiming this has happened a lot of times, and even a single issue found is > invaluable; I would agree with that, but I also think that a 3rd party CI > does not have to vote to be useful. > >> >> I’m not suggesting that the results of these tests be removed from the >> page altogether - there are some cases where their results are useful to >> the patch author/reviewer - but removing voting rights (or at least -1 >> rights) would save a patch from a –1 that might not be particularly >> meaningful. >> > > Frankly I find the overwhelming number of CI messages - and email > notifications even more annoying that random -1s. Thankfully you can hide > the formers and filter out the latters. > From the perspective of 3rd party CI maintainer I could use myself as an > example; I maintain a CI which has now been broken for about 48 hours. I am > busy with other tasks and cannot look at it now. I might be a terrible > person for this, but that's my problem. If the CI was not voting at least I > would not have annoyed people. (fwiw, I've disabled my CI now). > > Also, I believe we already agreed that a working CI is not anymore a > requirement, as long as the plugin/driver maintainers can provide a > reasonable proof that their integration works? > > Salvatore > > >> >> Thoughts? >> >> John >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-infra] [neutron] Third Party CI Voting
Totally agreed! Edgar From: Salvatore Orlando Reply-To: "OpenStack Development Mailing List (not for usage questions)" Date: Thursday, June 25, 2015 at 3:44 PM To: "OpenStack Development Mailing List (not for usage questions)" Subject: Re: [openstack-dev] [openstack-infra] [neutron] Third Party CI Voting Edgar, in a nutshell my point is that if we want to remove voting rights from every CI I'm fine with it. However, I think what's being discussed in this thread is already captured very well by [1] and believe the policy it outlines is perfectly fine for Neutron purposes. Salvatore [1] http://git.openstack.org/cgit/openstack/neutron/tree/doc/source/policies/thirdparty-ci.rst On 25 June 2015 at 17:08, Edgar Magana mailto:edgar.mag...@workday.com>> wrote: Thank for your response Salvatore. I am not sure what is your position in this topic? Are you fine removing voting rights to all Cis? Edgar From: Salvatore Orlando Reply-To: "OpenStack Development Mailing List (not for usage questions)" Date: Thursday, June 25, 2015 at 7:59 AM To: "OpenStack Development Mailing List (not for usage questions)" Subject: Re: [openstack-dev] [openstack-infra] [neutron] Third Party CI Voting On 25 June 2015 at 16:08, John Davidge (jodavidg) mailto:jodav...@cisco.com>> wrote: Hi all, Recent neutron third party CI issues have got me thinking again about a topic which we discussed in Vancouver: Should any Third Party CI have voting rights for neutron patches in gerrit? Why should this be a decision for Neutron only? I’d like to suggest that they shouldn’t. A -1 from a third party CI tool can often be an indication that the CI tool itself or the third party plugin is broken, rather than there being issues with the patch under review. I don’t think there are many cases where a third party CI tool has caught a genuine issue that Jenkins has missed. With the current voting rights these CI tools cause a lot of noise when they experience problems. As far as I am aware no 3rd party CI tool has a better coverage than the upstream one. some 3rd party CIs exercise different code paths and might uncover some issue that the upstream CI did not cover. There will surely be people claiming this has happened a lot of times, and even a single issue found is invaluable; I would agree with that, but I also think that a 3rd party CI does not have to vote to be useful. I’m not suggesting that the results of these tests be removed from the page altogether - there are some cases where their results are useful to the patch author/reviewer - but removing voting rights (or at least -1 rights) would save a patch from a –1 that might not be particularly meaningful. Frankly I find the overwhelming number of CI messages - and email notifications even more annoying that random -1s. Thankfully you can hide the formers and filter out the latters. From the perspective of 3rd party CI maintainer I could use myself as an example; I maintain a CI which has now been broken for about 48 hours. I am busy with other tasks and cannot look at it now. I might be a terrible person for this, but that's my problem. If the CI was not voting at least I would not have annoyed people. (fwiw, I've disabled my CI now). Also, I believe we already agreed that a working CI is not anymore a requirement, as long as the plugin/driver maintainers can provide a reasonable proof that their integration works? Salvatore Thoughts? John __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-infra] [neutron] Third Party CI Voting
We seem to be agreeing that having third party CI tools not vote –1 is a good idea. Personally I think it would be more beneficial to make it a rule rather than a recommendation. John From: Edgar Magana mailto:edgar.mag...@workday.com>> Reply-To: "OpenStack Development Mailing List (not for usage questions)" mailto:openstack-dev@lists.openstack.org>> Date: Friday, 26 June 2015 19:04 To: "OpenStack Development Mailing List (not for usage questions)" mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [openstack-infra] [neutron] Third Party CI Voting Totally agreed! Edgar From: Salvatore Orlando Reply-To: "OpenStack Development Mailing List (not for usage questions)" Date: Thursday, June 25, 2015 at 3:44 PM To: "OpenStack Development Mailing List (not for usage questions)" Subject: Re: [openstack-dev] [openstack-infra] [neutron] Third Party CI Voting Edgar, in a nutshell my point is that if we want to remove voting rights from every CI I'm fine with it. However, I think what's being discussed in this thread is already captured very well by [1] and believe the policy it outlines is perfectly fine for Neutron purposes. Salvatore [1] http://git.openstack.org/cgit/openstack/neutron/tree/doc/source/policies/thirdparty-ci.rst On 25 June 2015 at 17:08, Edgar Magana mailto:edgar.mag...@workday.com>> wrote: Thank for your response Salvatore. I am not sure what is your position in this topic? Are you fine removing voting rights to all Cis? Edgar From: Salvatore Orlando Reply-To: "OpenStack Development Mailing List (not for usage questions)" Date: Thursday, June 25, 2015 at 7:59 AM To: "OpenStack Development Mailing List (not for usage questions)" Subject: Re: [openstack-dev] [openstack-infra] [neutron] Third Party CI Voting On 25 June 2015 at 16:08, John Davidge (jodavidg) mailto:jodav...@cisco.com>> wrote: Hi all, Recent neutron third party CI issues have got me thinking again about a topic which we discussed in Vancouver: Should any Third Party CI have voting rights for neutron patches in gerrit? Why should this be a decision for Neutron only? I’d like to suggest that they shouldn’t. A -1 from a third party CI tool can often be an indication that the CI tool itself or the third party plugin is broken, rather than there being issues with the patch under review. I don’t think there are many cases where a third party CI tool has caught a genuine issue that Jenkins has missed. With the current voting rights these CI tools cause a lot of noise when they experience problems. As far as I am aware no 3rd party CI tool has a better coverage than the upstream one. some 3rd party CIs exercise different code paths and might uncover some issue that the upstream CI did not cover. There will surely be people claiming this has happened a lot of times, and even a single issue found is invaluable; I would agree with that, but I also think that a 3rd party CI does not have to vote to be useful. I’m not suggesting that the results of these tests be removed from the page altogether - there are some cases where their results are useful to the patch author/reviewer - but removing voting rights (or at least -1 rights) would save a patch from a –1 that might not be particularly meaningful. Frankly I find the overwhelming number of CI messages - and email notifications even more annoying that random -1s. Thankfully you can hide the formers and filter out the latters. >From the perspective of 3rd party CI maintainer I could use myself as an >example; I maintain a CI which has now been broken for about 48 hours. I am >busy with other tasks and cannot look at it now. I might be a terrible person >for this, but that's my problem. If the CI was not voting at least I would not >have annoyed people. (fwiw, I've disabled my CI now). Also, I believe we already agreed that a working CI is not anymore a requirement, as long as the plugin/driver maintainers can provide a reasonable proof that their integration works? Salvatore Thoughts? John __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing Lis
Re: [openstack-dev] [openstack-infra] [neutron] Third Party CI Voting
On 25 June 2015 at 16:08, John Davidge (jodavidg) wrote: > Hi all, > > Recent neutron third party CI issues have got me thinking again about a > topic which we discussed in Vancouver: > > Should any Third Party CI have voting rights for neutron patches in > gerrit? > Why should this be a decision for Neutron only? > > I’d like to suggest that they shouldn’t. > > A -1 from a third party CI tool can often be an indication that the CI > tool itself or the third party plugin is broken, rather than there being > issues with the patch under review. I don’t think there are many cases > where a third party CI tool has caught a genuine issue that Jenkins has > missed. With the current voting rights these CI tools cause a lot of noise > when they experience problems. > As far as I am aware no 3rd party CI tool has a better coverage than the upstream one. some 3rd party CIs exercise different code paths and might uncover some issue that the upstream CI did not cover. There will surely be people claiming this has happened a lot of times, and even a single issue found is invaluable; I would agree with that, but I also think that a 3rd party CI does not have to vote to be useful. > > I’m not suggesting that the results of these tests be removed from the > page altogether - there are some cases where their results are useful to > the patch author/reviewer - but removing voting rights (or at least -1 > rights) would save a patch from a –1 that might not be particularly > meaningful. > Frankly I find the overwhelming number of CI messages - and email notifications even more annoying that random -1s. Thankfully you can hide the formers and filter out the latters. >From the perspective of 3rd party CI maintainer I could use myself as an example; I maintain a CI which has now been broken for about 48 hours. I am busy with other tasks and cannot look at it now. I might be a terrible person for this, but that's my problem. If the CI was not voting at least I would not have annoyed people. (fwiw, I've disabled my CI now). Also, I believe we already agreed that a working CI is not anymore a requirement, as long as the plugin/driver maintainers can provide a reasonable proof that their integration works? Salvatore > > Thoughts? > > John > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-infra] [neutron] Third Party CI Voting
Hi, I agree, and only vote +1 myself, but I don’t agree on mandating it. If someone has invested enough in their CI to be confident, we just supply rope. Thanks, doug > On Jun 25, 2015, at 8:08 AM, John Davidge (jodavidg) > wrote: > > Hi all, > > Recent neutron third party CI issues have got me thinking again about a topic > which we discussed in Vancouver: > > Should any Third Party CI have voting rights for neutron patches in gerrit? > > I’d like to suggest that they shouldn’t. > > A -1 from a third party CI tool can often be an indication that the CI tool > itself or the third party plugin is broken, rather than there being issues > with the patch under review. I don’t think there are many cases where a third > party CI tool has caught a genuine issue that Jenkins has missed. With the > current voting rights these CI tools cause a lot of noise when they > experience problems. > > I’m not suggesting that the results of these tests be removed from the page > altogether - there are some cases where their results are useful to the patch > author/reviewer - but removing voting rights (or at least -1 rights) would > save a patch from a –1 that might not be particularly meaningful. > > Thoughts? > > John > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-infra] [neutron] Third Party CI Voting
I think this make sense. I have lost any faith on third party CIs after the Vancouver decision of not police them (I was doing that for a while and I was fine doing it). So, why to keep them with voting rights? I will say, let’s remove voting rights for all of them. Edgar From: "John Davidge (jodavidg)" Reply-To: "OpenStack Development Mailing List (not for usage questions)" Date: Thursday, June 25, 2015 at 7:08 AM To: "OpenStack Development Mailing List (not for usage questions)" Subject: [openstack-dev] [openstack-infra] [neutron] Third Party CI Voting Hi all, Recent neutron third party CI issues have got me thinking again about a topic which we discussed in Vancouver: Should any Third Party CI have voting rights for neutron patches in gerrit? I’d like to suggest that they shouldn’t. A -1 from a third party CI tool can often be an indication that the CI tool itself or the third party plugin is broken, rather than there being issues with the patch under review. I don’t think there are many cases where a third party CI tool has caught a genuine issue that Jenkins has missed. With the current voting rights these CI tools cause a lot of noise when they experience problems. I’m not suggesting that the results of these tests be removed from the page altogether - there are some cases where their results are useful to the patch author/reviewer - but removing voting rights (or at least -1 rights) would save a patch from a –1 that might not be particularly meaningful. Thoughts? John __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-infra] [neutron] Third Party CI Voting
Thank for your response Salvatore. I am not sure what is your position in this topic? Are you fine removing voting rights to all Cis? Edgar From: Salvatore Orlando Reply-To: "OpenStack Development Mailing List (not for usage questions)" Date: Thursday, June 25, 2015 at 7:59 AM To: "OpenStack Development Mailing List (not for usage questions)" Subject: Re: [openstack-dev] [openstack-infra] [neutron] Third Party CI Voting On 25 June 2015 at 16:08, John Davidge (jodavidg) mailto:jodav...@cisco.com>> wrote: Hi all, Recent neutron third party CI issues have got me thinking again about a topic which we discussed in Vancouver: Should any Third Party CI have voting rights for neutron patches in gerrit? Why should this be a decision for Neutron only? I’d like to suggest that they shouldn’t. A -1 from a third party CI tool can often be an indication that the CI tool itself or the third party plugin is broken, rather than there being issues with the patch under review. I don’t think there are many cases where a third party CI tool has caught a genuine issue that Jenkins has missed. With the current voting rights these CI tools cause a lot of noise when they experience problems. As far as I am aware no 3rd party CI tool has a better coverage than the upstream one. some 3rd party CIs exercise different code paths and might uncover some issue that the upstream CI did not cover. There will surely be people claiming this has happened a lot of times, and even a single issue found is invaluable; I would agree with that, but I also think that a 3rd party CI does not have to vote to be useful. I’m not suggesting that the results of these tests be removed from the page altogether - there are some cases where their results are useful to the patch author/reviewer - but removing voting rights (or at least -1 rights) would save a patch from a –1 that might not be particularly meaningful. Frankly I find the overwhelming number of CI messages - and email notifications even more annoying that random -1s. Thankfully you can hide the formers and filter out the latters. From the perspective of 3rd party CI maintainer I could use myself as an example; I maintain a CI which has now been broken for about 48 hours. I am busy with other tasks and cannot look at it now. I might be a terrible person for this, but that's my problem. If the CI was not voting at least I would not have annoyed people. (fwiw, I've disabled my CI now). Also, I believe we already agreed that a working CI is not anymore a requirement, as long as the plugin/driver maintainers can provide a reasonable proof that their integration works? Salvatore Thoughts? John __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
