Re: [DISCUSS] If BVT breaks, revert the commits...
Alex, On 10-Jul-2013, at 9:32 AM, Alex Huang alex.hu...@citrix.com wrote: Sorry this took a little longer than expected with the holiday in US. Here's the first proposal [1] on the automated test system. Comments welcome. Specifically, I have one question on if we should have a staging branch for all release branches and master where all checkins go and the build system automatically cherry-pick over commits that passes the tests. +1 Staging branch is better option than to revert the commits. One useful way might be to trigger a jenkins job once a patch is uploaded to review board and run BVT. But as of now Jenkins plugin for review board does not have git support. https://wiki.jenkins-ci.org/display/JENKINS/Reviewboard+Plugin Regards, Girish I believe we have a lot of pieces in place. Prassana, Sudha, Rayees, Ram Ganesh, myself, and a few others will be working to get this system in place. --Alex [1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Automated+Tests+Rules+and+Guidelines -Original Message- From: David Nalley [mailto:da...@gnsa.us] Sent: Friday, June 28, 2013 8:44 PM To: dev@cloudstack.apache.org Subject: Re: [DISCUSS] If BVT breaks, revert the commits... On Fri, Jun 28, 2013 at 8:18 PM, Alex Huang alex.hu...@citrix.com wrote: After Dave's complain in the vmsync [MERGE] thread about BVT in horrible shape on master, I went around to figure out what exactly happened. The best I can figure is that after a certain merge (I will leave out which merge as that's not important), BVT no longer runs automatically. It was promised to be fixed and there are people who are actively fixing it but it's been in this way for about two weeks. People running BVTs are working around the problem but it's not automated anymore and so it's no longer running on master. I understand people are nice and tried to be accommodating to other people by working around the problem but sometimes we just have to be an arse. So let me be that arse... New Rule If BVT or automated regression tests break on master or any release branch, we revert all commits that broke it. It doesn't matter if they promise to fix it within the next hour. If it's broken, the release manager will revert the commits and developers must resubmit. It sounds mean but it's the only way this problem can be fixed. To avoid having a bunch of reverts and resubmits, the developers should be able to request that BVT run on their branch and don't merge until BVT on their branch is at 100%. We will work on figuring out how to do that. Comments? --Alex +100 - not only +100 but I will increment ASFBots $beverage counter a few in your favor for suggesting this. --David
Re: [DISCUSS] If BVT breaks, revert the commits...
On Mon, Jul 15, 2013 at 03:06:41PM +0530, Girish Shilamkar wrote: Alex, On 10-Jul-2013, at 9:32 AM, Alex Huang alex.hu...@citrix.com wrote: Sorry this took a little longer than expected with the holiday in US. Here's the first proposal [1] on the automated test system. Comments welcome. Specifically, I have one question on if we should have a staging branch for all release branches and master where all checkins go and the build system automatically cherry-pick over commits that passes the tests. +1 Staging branch is better option than to revert the commits. One useful way might be to trigger a jenkins job once a patch is uploaded to review board and run BVT. A staging area would indeed be useful for passing tests and gating the checkins. But as of now Jenkins plugin for review board does not have git support. https://wiki.jenkins-ci.org/display/JENKINS/Reviewboard+Plugin Yeah - this is a perforce only plugin I'm afraid. -- Prasanna., Powered by BigRock.com
RE: [DISCUSS] If BVT breaks, revert the commits...
Sorry this took a little longer than expected with the holiday in US. Here's the first proposal [1] on the automated test system. Comments welcome. Specifically, I have one question on if we should have a staging branch for all release branches and master where all checkins go and the build system automatically cherry-pick over commits that passes the tests. I believe we have a lot of pieces in place. Prassana, Sudha, Rayees, Ram Ganesh, myself, and a few others will be working to get this system in place. --Alex [1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Automated+Tests+Rules+and+Guidelines -Original Message- From: David Nalley [mailto:da...@gnsa.us] Sent: Friday, June 28, 2013 8:44 PM To: dev@cloudstack.apache.org Subject: Re: [DISCUSS] If BVT breaks, revert the commits... On Fri, Jun 28, 2013 at 8:18 PM, Alex Huang alex.hu...@citrix.com wrote: After Dave's complain in the vmsync [MERGE] thread about BVT in horrible shape on master, I went around to figure out what exactly happened. The best I can figure is that after a certain merge (I will leave out which merge as that's not important), BVT no longer runs automatically. It was promised to be fixed and there are people who are actively fixing it but it's been in this way for about two weeks. People running BVTs are working around the problem but it's not automated anymore and so it's no longer running on master. I understand people are nice and tried to be accommodating to other people by working around the problem but sometimes we just have to be an arse. So let me be that arse... New Rule If BVT or automated regression tests break on master or any release branch, we revert all commits that broke it. It doesn't matter if they promise to fix it within the next hour. If it's broken, the release manager will revert the commits and developers must resubmit. It sounds mean but it's the only way this problem can be fixed. To avoid having a bunch of reverts and resubmits, the developers should be able to request that BVT run on their branch and don't merge until BVT on their branch is at 100%. We will work on figuring out how to do that. Comments? --Alex +100 - not only +100 but I will increment ASFBots $beverage counter a few in your favor for suggesting this. --David
Re: [DISCUSS] If BVT breaks, revert the commits...
On Sat, Jun 29, 2013 at 12:45 AM, Prasanna Santhanam t...@apache.org wrote: On Sat, Jun 29, 2013 at 12:18:17AM +, Alex Huang wrote: After Dave's complain in the vmsync [MERGE] thread about BVT in horrible shape on master, I went around to figure out what exactly happened. The best I can figure is that after a certain merge (I will leave out which merge as that's not important), BVT no longer runs automatically. It was promised to be fixed and there are people who are actively fixing it but it's been in this way for about two weeks. People running BVTs are working around the problem but it's not automated anymore and so it's no longer running on master. I understand people are nice and tried to be accommodating to other people by working around the problem but sometimes we just have to be an arse. So let me be that arse... New Rule If BVT or automated regression tests break on master or any release branch, we revert all commits that broke it. It doesn't matter if they promise to fix it within the next hour. If it's broken, the release manager will revert the commits and developers must resubmit. It sounds mean but it's the only way this problem can be fixed. To avoid having a bunch of reverts and resubmits, the developers should be able to request that BVT run on their branch and don't merge until BVT on their branch is at 100%. We will work on figuring out how to do that. Comments? Quick questions: 1. How do we test contributor code (from github/reviewboard) because we have no staging area/branch? Take a look at the pre-commit testing that hadoop does, that might be usable for us. Olamy did a blog post about it recently. 2. Squashing into a big patch is not a good idea. I prefer that we keep the clean commit history rebased atop master in the topic branch and run the bvt on it. This can be requested and there is ability in the test jobs to switch branches already. Hit Build (test-matrix job) - choose topic-branch on the form. 3. BVTs have long runtime (3 HVs - 4hrs). For large merges it makes sense since we're going to be waiting 72h for the review/merge request. But rest of the time I propose we run it against master only. Is this the same as you're proposing? 4. Who will fix the tests which are broken? Well if we treat master as frozen until the tests are fixed, then I'd suppose that folks wanting to get code in will be incentivized to at least help fix them so that master can start getting code again :) --David
[DISCUSS] If BVT breaks, revert the commits...
After Dave's complain in the vmsync [MERGE] thread about BVT in horrible shape on master, I went around to figure out what exactly happened. The best I can figure is that after a certain merge (I will leave out which merge as that's not important), BVT no longer runs automatically. It was promised to be fixed and there are people who are actively fixing it but it's been in this way for about two weeks. People running BVTs are working around the problem but it's not automated anymore and so it's no longer running on master. I understand people are nice and tried to be accommodating to other people by working around the problem but sometimes we just have to be an arse. So let me be that arse... New Rule If BVT or automated regression tests break on master or any release branch, we revert all commits that broke it. It doesn't matter if they promise to fix it within the next hour. If it's broken, the release manager will revert the commits and developers must resubmit. It sounds mean but it's the only way this problem can be fixed. To avoid having a bunch of reverts and resubmits, the developers should be able to request that BVT run on their branch and don't merge until BVT on their branch is at 100%. We will work on figuring out how to do that. Comments? --Alex
Re: [DISCUSS] If BVT breaks, revert the commits...
Alex, +1 From the jenkins side we could make more jobs with parameters so anyone can kick-off a test run on his/her branch. I don't know if that would work for the BVT tests though. This only thing i'm a bit afraid of is the technical side of reverting a merge. It's one of those this that you need to be careful with in git. Cheers, Hugo On Jun 28, 2013, at 5:18 PM, Alex Huang alex.hu...@citrix.com wrote: After Dave's complain in the vmsync [MERGE] thread about BVT in horrible shape on master, I went around to figure out what exactly happened. The best I can figure is that after a certain merge (I will leave out which merge as that's not important), BVT no longer runs automatically. It was promised to be fixed and there are people who are actively fixing it but it's been in this way for about two weeks. People running BVTs are working around the problem but it's not automated anymore and so it's no longer running on master. I understand people are nice and tried to be accommodating to other people by working around the problem but sometimes we just have to be an arse. So let me be that arse... New Rule If BVT or automated regression tests break on master or any release branch, we revert all commits that broke it. It doesn't matter if they promise to fix it within the next hour. If it's broken, the release manager will revert the commits and developers must resubmit. It sounds mean but it's the only way this problem can be fixed. To avoid having a bunch of reverts and resubmits, the developers should be able to request that BVT run on their branch and don't merge until BVT on their branch is at 100%. We will work on figuring out how to do that. Comments? --Alex
RE: [DISCUSS] If BVT breaks, revert the commits...
This only thing i'm a bit afraid of is the technical side of reverting a merge. It's one of those this that you need to be careful with in git. I think this means all merges must come in as one big squashed patch. If they come in as multiple commits or rebase, then it gets reverted automatically. --Alex
Re: [DISCUSS] If BVT breaks, revert the commits...
On Sat, Jun 29, 2013 at 12:18:17AM +, Alex Huang wrote: After Dave's complain in the vmsync [MERGE] thread about BVT in horrible shape on master, I went around to figure out what exactly happened. The best I can figure is that after a certain merge (I will leave out which merge as that's not important), BVT no longer runs automatically. It was promised to be fixed and there are people who are actively fixing it but it's been in this way for about two weeks. People running BVTs are working around the problem but it's not automated anymore and so it's no longer running on master. I understand people are nice and tried to be accommodating to other people by working around the problem but sometimes we just have to be an arse. So let me be that arse... New Rule If BVT or automated regression tests break on master or any release branch, we revert all commits that broke it. It doesn't matter if they promise to fix it within the next hour. If it's broken, the release manager will revert the commits and developers must resubmit. It sounds mean but it's the only way this problem can be fixed. To avoid having a bunch of reverts and resubmits, the developers should be able to request that BVT run on their branch and don't merge until BVT on their branch is at 100%. We will work on figuring out how to do that. Comments? Quick questions: 1. How do we test contributor code (from github/reviewboard) because we have no staging area/branch? 2. Squashing into a big patch is not a good idea. I prefer that we keep the clean commit history rebased atop master in the topic branch and run the bvt on it. This can be requested and there is ability in the test jobs to switch branches already. Hit Build (test-matrix job) - choose topic-branch on the form. 3. BVTs have long runtime (3 HVs - 4hrs). For large merges it makes sense since we're going to be waiting 72h for the review/merge request. But rest of the time I propose we run it against master only. Is this the same as you're proposing? 4. Who will fix the tests which are broken? -- Prasanna., Powered by BigRock.com
