Re: [openstack-dev] [CI] Try to introduce RFC mechanism to CI.

[Tang Chen]

To all,

Ok, thanks for the advices.

Thanks. :)

On 10/09/2015 08:27 PM, Jeremy Stanley wrote:

On 2015-10-09 17:00:27 +0800 (+0800), Tang Chen wrote:
[...]

I'm not sure if it is a good idea. Please help to review the
following BP.

https://blueprints.launchpad.net/openstack-ci/+spec/ci-rfc-mechanism

The Infra team doesn't rely on blueprints, and instead uses
http://specs.openstack.org/openstack-infra/infra-specs/ to plan
complex implementation work. I only just discovered that you can
disable blueprints on a project in Launchpad, so I've now done that
for the "openstack-ci" project.

That said, I don't expect the proposed feature would get very far.
We're committed to providing test results for all patchsets when
possible; that doesn't mean you are required to pay attention to the
results or even wait for them to be posted on your change while it's
still in a state of initial flux.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] Try to introduce RFC mechanism to CI.

[Jordan Pittier]

Hi,
On Fri, Oct 9, 2015 at 11:00 AM, Tang Chen  wrote:

> Hi,
>
> CI systems will run tests for each patch once it is submitted or modified.
> But most CI systems occupy a lot of resource, and take a long time to
> run tests (1 or 2 hours for one patch).
>
> I think, not all the patches submitted need to be tested. Even those
> patches
> with an approved BP and spec may be reworked for 20+ versions. So I think
> CI should support a RFC (Require For Comments) mechanism for developers
> to submit and review the code detail and rework. When the patches are
> fully ready, I mean all reviewers have agreed on the implementation detail,
> then CI will test the patches.

So have the humans do the hard work to eventually find out that the patch
breaks the world ?


> For a 20+ version patch-set, maybe 3 or 4 rounds
> of tests are enough. Just test the last 3 or 4 versions.
>
 How do know, when a new patchset arrives, that it's part of the last 3 or
4 versions ?

>
> This can significantly reduce CI overload.
>
> This workflow appears in many other OSS communities, such as Linux kernel,
> qemu and libvirt. Testers won't test patches with a [RFC] tag in the
> commit message.
> So I want to enable CI to support a similar mechanism.
>
> I'm not sure if it is a good idea. Please help to review the following BP.
>
> https://blueprints.launchpad.net/openstack-ci/+spec/ci-rfc-mechanism
>
> Thanks.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


I am running a 3rd party for Cinder. The amount of time to setup, operate
and watch after the CI results cost way more than the 1 or 2 servers it
take to run the jobs. So, I don"t want to be a party pooper here, but in my
opinion I am not sure it's worth the effort.

Note: I don"t know about nova or neutron.

Jordan
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] Try to introduce RFC mechanism to CI.

[Tang Chen]

On 10/09/2015 05:48 PM, Jordan Pittier wrote:

Hi,
On Fri, Oct 9, 2015 at 11:00 AM, Tang Chen > wrote:


Hi,

CI systems will run tests for each patch once it is submitted or
modified.
But most CI systems occupy a lot of resource, and take a long time to
run tests (1 or 2 hours for one patch).

I think, not all the patches submitted need to be tested. Even
those patches
with an approved BP and spec may be reworked for 20+ versions. So
I think
CI should support a RFC (Require For Comments) mechanism for
developers
to submit and review the code detail and rework. When the patches are
fully ready, I mean all reviewers have agreed on the
implementation detail,
then CI will test the patches. 

So have the humans do the hard work to eventually find out that the 
patch breaks the world ?


No. Developers of course will run some tests themselves before they 
submit patches.
It is just a waste of resource if reviewers are discussing about where 
this function should be,
or what the function should be named. After all these details are agreed 
on, run the CI.



For a 20+ version patch-set, maybe 3 or 4 rounds
of tests are enough. Just test the last 3 or 4 versions.

 How do know, when a new patchset arrives, that it's part of the last 
3 or 4 versions ?


I think it could work like this:
1. At first, developer submits v1 patch-set with RFC tag. CIs don't run.
2. After several versions reworked, like v5, v6, most reviewers have 
agreed on the implementation

is OK. Then submit v7 without RFC tag. Then CIs run.
3. After 3, 4 rounds of tests, v10 patch-set could be merged.

Thanks.



This can significantly reduce CI overload.

This workflow appears in many other OSS communities, such as Linux
kernel,
qemu and libvirt. Testers won't test patches with a [RFC] tag in
the commit message.
So I want to enable CI to support a similar mechanism.

I'm not sure if it is a good idea. Please help to review the
following BP.

https://blueprints.launchpad.net/openstack-ci/+spec/ci-rfc-mechanism

Thanks.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


I am running a 3rd party for Cinder. The amount of time to setup, 
operate and watch after the CI results cost way more than the 1 or 2 
servers it take to run the jobs. So, I don"t want to be a party pooper 
here, but in my opinion I am not sure it's worth the effort.


Note: I don"t know about nova or neutron.

Jordan



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [CI] Try to introduce RFC mechanism to CI.

[Tang Chen]

Hi,

CI systems will run tests for each patch once it is submitted or modified.
But most CI systems occupy a lot of resource, and take a long time to
run tests (1 or 2 hours for one patch).

I think, not all the patches submitted need to be tested. Even those 
patches

with an approved BP and spec may be reworked for 20+ versions. So I think
CI should support a RFC (Require For Comments) mechanism for developers
to submit and review the code detail and rework. When the patches are
fully ready, I mean all reviewers have agreed on the implementation detail,
then CI will test the patches. For a 20+ version patch-set, maybe 3 or 4 
rounds

of tests are enough. Just test the last 3 or 4 versions.

This can significantly reduce CI overload.

This workflow appears in many other OSS communities, such as Linux kernel,
qemu and libvirt. Testers won't test patches with a [RFC] tag in the 
commit message.

So I want to enable CI to support a similar mechanism.

I'm not sure if it is a good idea. Please help to review the following BP.

https://blueprints.launchpad.net/openstack-ci/+spec/ci-rfc-mechanism

Thanks.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] Try to introduce RFC mechanism to CI.

[Jeremy Stanley]

On 2015-10-09 17:00:27 +0800 (+0800), Tang Chen wrote:
[...]
> I'm not sure if it is a good idea. Please help to review the
> following BP.
> 
> https://blueprints.launchpad.net/openstack-ci/+spec/ci-rfc-mechanism

The Infra team doesn't rely on blueprints, and instead uses
http://specs.openstack.org/openstack-infra/infra-specs/ to plan
complex implementation work. I only just discovered that you can
disable blueprints on a project in Launchpad, so I've now done that
for the "openstack-ci" project.

That said, I don't expect the proposed feature would get very far.
We're committed to providing test results for all patchsets when
possible; that doesn't mean you are required to pay attention to the
results or even wait for them to be posted on your change while it's
still in a state of initial flux.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [CI] Try to introduce RFC mechanism to CI.

[Dmitry Tantsur]

On 10/09/2015 12:06 PM, Tang Chen wrote:


On 10/09/2015 05:48 PM, Jordan Pittier wrote:

Hi,
On Fri, Oct 9, 2015 at 11:00 AM, Tang Chen > wrote:

Hi,

CI systems will run tests for each patch once it is submitted or
modified.
But most CI systems occupy a lot of resource, and take a long time to
run tests (1 or 2 hours for one patch).

I think, not all the patches submitted need to be tested. Even
those patches
with an approved BP and spec may be reworked for 20+ versions. So
I think
CI should support a RFC (Require For Comments) mechanism for
developers
to submit and review the code detail and rework. When the patches are
fully ready, I mean all reviewers have agreed on the
implementation detail,
then CI will test the patches.

So have the humans do the hard work to eventually find out that the
patch breaks the world ?


No. Developers of course will run some tests themselves before they
submit patches.


Tests, but not all possible CI's. E.g. in ironic we 6 devstack-based 
jobs, I don't really expect a submitter to go through them manually. 
Actually, it's an awesome feature of our CI system that I would not give 
away :)


Also as a reviewer, I'm not sure I would like to argue on function 
names, while I'm not even sure that this change does not break the world.



It is just a waste of resource if reviewers are discussing about where
this function should be,
or what the function should be named. After all these details are agreed
on, run the CI.


For a 20+ version patch-set, maybe 3 or 4 rounds
of tests are enough. Just test the last 3 or 4 versions.

 How do know, when a new patchset arrives, that it's part of the last
3 or 4 versions ?


I think it could work like this:
1. At first, developer submits v1 patch-set with RFC tag. CIs don't run.
2. After several versions reworked, like v5, v6, most reviewers have
agreed on the implementation
 is OK. Then submit v7 without RFC tag. Then CIs run.
3. After 3, 4 rounds of tests, v10 patch-set could be merged.

Thanks.



This can significantly reduce CI overload.

This workflow appears in many other OSS communities, such as Linux
kernel,
qemu and libvirt. Testers won't test patches with a [RFC] tag in
the commit message.
So I want to enable CI to support a similar mechanism.

I'm not sure if it is a good idea. Please help to review the
following BP.

https://blueprints.launchpad.net/openstack-ci/+spec/ci-rfc-mechanism

Thanks.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


I am running a 3rd party for Cinder. The amount of time to setup,
operate and watch after the CI results cost way more than the 1 or 2
servers it take to run the jobs. So, I don"t want to be a party pooper
here, but in my opinion I am not sure it's worth the effort.

Note: I don"t know about nova or neutron.

Jordan



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev