Re: Patch testing
On Wed, Jan 26, 2011 at 10:05 AM, Nigel Daley wrote: > raid (contrib) test hanging: TestBlockFixer > > I forced 2 thread dumps. Both hung in the same place. Filed > https://issues.apache.org/jira/browse/MAPREDUCE-2283 This is a blocker > for turning on MR precommit. > Since this is contrib, I'd like to suggest just disabling this test temporarily. We can re-enable it once it's fixed. Not having MR pre-commit working has been pretty painful. -Todd > On Jan 25, 2011, at 11:19 PM, Nigel Daley wrote: > > > Started another trial run of MR precommit testing: > > > https://hudson.apache.org/hudson/view/G-L/view/Hadoop/job/PreCommit-MAPREDUCE-Build/17/ > > > > Let's see if 17th time is a charm... > > > > Nige > > > > On Jan 7, 2011, at 5:14 PM, Todd Lipcon wrote: > > > >> On Fri, Jan 7, 2011 at 2:11 PM, Nigel Daley wrote: > >> > >>> Hrm, the MR precommit test I'm running has hung (been running for 14 > hours > >>> so far). FWIW, 2 HDFS precommit tests are hung too. I suspect it > could be > >>> the NFS mounts on the machines. I forced a thread dump which you can > see in > >>> the console: > >>> > https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/10/console > >>> > >>> > >> Strange, haven't seen a hang like that before in > handleConnectionFailure. It > >> should retry for 15 minutes max in that loop. > >> > >> > >>> Any other ideas why these might be hanging? > >>> > >>> > >> There is an HDFS bug right now that can cause hangs on some tests - > >> HDFS-1529 - would appreciate if someone can take a look. But I don't > think > >> this is responsible for the MR hang above. > >> > >> -Todd > >> > >> > >>> On Jan 5, 2011, at 5:42 PM, Todd Lipcon wrote: > >>> > On Wed, Jan 5, 2011 at 4:39 PM, Nigel Daley wrote: > > > Thanks for looking into it Todd. Let's first see if you think it can > be > > fixed quickly. Let me know. > > > > > No problem, it wasn't too bad after all. Patch up on HADOOP-7087 which > >>> fixes > this test timeout for me. > > -Todd > > > > On Jan 5, 2011, at 4:33 PM, Todd Lipcon wrote: > > > >> On Wed, Jan 5, 2011 at 4:19 PM, Nigel Daley wrote: > >> > >>> Todd, would love to get > >>> https://issues.apache.org/jira/browse/MAPREDUCE-2121 fixed first > >>> since > >>> this is failing every night on trunk. > >>> > >> > >> What if we disable that test, move that issue to 0.22 blocker, and > then > >> enable the test-patch? I'll also look into that one today, but if > it's > >> something that will take a while to fix, I don't think we should > hold > >>> off > >> the useful testing for all the other patches. > >> > >> -Todd > >> > >> On Jan 5, 2011, at 2:45 PM, Todd Lipcon wrote: > >>> > Hi Nigel, > > MAPREDUCE-2172 has been fixed for a while. Are there any other > > particular > JIRAs you think need to be fixed before the MR test-patch queue > gets > enabled? I have a lot of outstanding patches and doing all the > > test-patch > turnaround manually on 3 different boxes is a real headache. > > Thanks > -Todd > > On Tue, Dec 21, 2010 at 1:33 PM, Nigel Daley > wrote: > > > Ok, HDFS is now enabled. You'll see a stream of updates shortly > on > > the > >>> ~30 > > Patch Available HDFS issues. > > > > Nige > > > > On Dec 20, 2010, at 12:42 PM, Jakob Homan wrote: > > > >> I committed HDFS-1511 this morning. We should be good to go. I > >>> can > >> haz snooty robot butler? > >> > >> On Fri, Dec 17, 2010 at 8:31 PM, Konstantin Boudnik < > >>> c...@apache.org> > > wrote: > >>> Thanks Jacob. I am wasted already but I can do it on Sun, I > think, > >>> unless it is done earlier. > >>> -- > >>> Take care, > >>> Konstantin (Cos) Boudnik > >>> > >>> > >>> > >>> On Fri, Dec 17, 2010 at 19:41, Jakob Homan > > wrote: > Ok. I'll get a patch out for 1511 tomorrow, unless someone > wants > > to > whip one up tonight. > > > On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley > > wrote: > > I agree with Cos on fixing HDFS-1511 first. Once that is done > >>> I'll > > enable hdfs patch testing. > > > > Cheers, > > Nige > > > > Sent from my iPhone4 > > > > On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik < > c...@apache.org > > > wrote: > > > >> One more issue needs to be addressed before test-patch is > >>> turned > > on > > HDFS is > >> https://issues.apache.org/jira/browse/HDFS-1511 > >> -- > >> Take care, > >> K
Re: Patch testing
raid (contrib) test hanging: TestBlockFixer I forced 2 thread dumps. Both hung in the same place. Filed https://issues.apache.org/jira/browse/MAPREDUCE-2283 This is a blocker for turning on MR precommit. Cheers, Nige On Jan 25, 2011, at 11:19 PM, Nigel Daley wrote: > Started another trial run of MR precommit testing: > https://hudson.apache.org/hudson/view/G-L/view/Hadoop/job/PreCommit-MAPREDUCE-Build/17/ > > Let's see if 17th time is a charm... > > Nige > > On Jan 7, 2011, at 5:14 PM, Todd Lipcon wrote: > >> On Fri, Jan 7, 2011 at 2:11 PM, Nigel Daley wrote: >> >>> Hrm, the MR precommit test I'm running has hung (been running for 14 hours >>> so far). FWIW, 2 HDFS precommit tests are hung too. I suspect it could be >>> the NFS mounts on the machines. I forced a thread dump which you can see in >>> the console: >>> https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/10/console >>> >>> >> Strange, haven't seen a hang like that before in handleConnectionFailure. It >> should retry for 15 minutes max in that loop. >> >> >>> Any other ideas why these might be hanging? >>> >>> >> There is an HDFS bug right now that can cause hangs on some tests - >> HDFS-1529 - would appreciate if someone can take a look. But I don't think >> this is responsible for the MR hang above. >> >> -Todd >> >> >>> On Jan 5, 2011, at 5:42 PM, Todd Lipcon wrote: >>> On Wed, Jan 5, 2011 at 4:39 PM, Nigel Daley wrote: > Thanks for looking into it Todd. Let's first see if you think it can be > fixed quickly. Let me know. > > No problem, it wasn't too bad after all. Patch up on HADOOP-7087 which >>> fixes this test timeout for me. -Todd > On Jan 5, 2011, at 4:33 PM, Todd Lipcon wrote: > >> On Wed, Jan 5, 2011 at 4:19 PM, Nigel Daley wrote: >> >>> Todd, would love to get >>> https://issues.apache.org/jira/browse/MAPREDUCE-2121 fixed first >>> since >>> this is failing every night on trunk. >>> >> >> What if we disable that test, move that issue to 0.22 blocker, and then >> enable the test-patch? I'll also look into that one today, but if it's >> something that will take a while to fix, I don't think we should hold >>> off >> the useful testing for all the other patches. >> >> -Todd >> >> On Jan 5, 2011, at 2:45 PM, Todd Lipcon wrote: >>> Hi Nigel, MAPREDUCE-2172 has been fixed for a while. Are there any other > particular JIRAs you think need to be fixed before the MR test-patch queue gets enabled? I have a lot of outstanding patches and doing all the > test-patch turnaround manually on 3 different boxes is a real headache. Thanks -Todd On Tue, Dec 21, 2010 at 1:33 PM, Nigel Daley wrote: > Ok, HDFS is now enabled. You'll see a stream of updates shortly on > the >>> ~30 > Patch Available HDFS issues. > > Nige > > On Dec 20, 2010, at 12:42 PM, Jakob Homan wrote: > >> I committed HDFS-1511 this morning. We should be good to go. I >>> can >> haz snooty robot butler? >> >> On Fri, Dec 17, 2010 at 8:31 PM, Konstantin Boudnik < >>> c...@apache.org> > wrote: >>> Thanks Jacob. I am wasted already but I can do it on Sun, I think, >>> unless it is done earlier. >>> -- >>> Take care, >>> Konstantin (Cos) Boudnik >>> >>> >>> >>> On Fri, Dec 17, 2010 at 19:41, Jakob Homan > wrote: Ok. I'll get a patch out for 1511 tomorrow, unless someone wants > to whip one up tonight. On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley > wrote: > I agree with Cos on fixing HDFS-1511 first. Once that is done >>> I'll > enable hdfs patch testing. > > Cheers, > Nige > > Sent from my iPhone4 > > On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik >>> > wrote: > >> One more issue needs to be addressed before test-patch is >>> turned > on > HDFS is >> https://issues.apache.org/jira/browse/HDFS-1511 >> -- >> Take care, >> Konstantin (Cos) Boudnik >> >> >> >> On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik < > c...@apache.org> > wrote: >>> Considering that because of these 4 faulty cases every patch > will >>> be >>> -1'ed a patch author will still have to look at it and make a > comment >>> why this particular -1 isn't valid. Lesser work, perhaps, but > messier >>> IMO. I'm not blocking it - I j
Re: Patch testing
Started another trial run of MR precommit testing: https://hudson.apache.org/hudson/view/G-L/view/Hadoop/job/PreCommit-MAPREDUCE-Build/17/ Let's see if 17th time is a charm... Nige On Jan 7, 2011, at 5:14 PM, Todd Lipcon wrote: > On Fri, Jan 7, 2011 at 2:11 PM, Nigel Daley wrote: > >> Hrm, the MR precommit test I'm running has hung (been running for 14 hours >> so far). FWIW, 2 HDFS precommit tests are hung too. I suspect it could be >> the NFS mounts on the machines. I forced a thread dump which you can see in >> the console: >> https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/10/console >> >> > Strange, haven't seen a hang like that before in handleConnectionFailure. It > should retry for 15 minutes max in that loop. > > >> Any other ideas why these might be hanging? >> >> > There is an HDFS bug right now that can cause hangs on some tests - > HDFS-1529 - would appreciate if someone can take a look. But I don't think > this is responsible for the MR hang above. > > -Todd > > >> On Jan 5, 2011, at 5:42 PM, Todd Lipcon wrote: >> >>> On Wed, Jan 5, 2011 at 4:39 PM, Nigel Daley wrote: >>> Thanks for looking into it Todd. Let's first see if you think it can be fixed quickly. Let me know. >>> No problem, it wasn't too bad after all. Patch up on HADOOP-7087 which >> fixes >>> this test timeout for me. >>> >>> -Todd >>> >>> On Jan 5, 2011, at 4:33 PM, Todd Lipcon wrote: > On Wed, Jan 5, 2011 at 4:19 PM, Nigel Daley wrote: > >> Todd, would love to get >> https://issues.apache.org/jira/browse/MAPREDUCE-2121 fixed first >> since >> this is failing every night on trunk. >> > > What if we disable that test, move that issue to 0.22 blocker, and then > enable the test-patch? I'll also look into that one today, but if it's > something that will take a while to fix, I don't think we should hold >> off > the useful testing for all the other patches. > > -Todd > > On Jan 5, 2011, at 2:45 PM, Todd Lipcon wrote: >> >>> Hi Nigel, >>> >>> MAPREDUCE-2172 has been fixed for a while. Are there any other particular >>> JIRAs you think need to be fixed before the MR test-patch queue gets >>> enabled? I have a lot of outstanding patches and doing all the test-patch >>> turnaround manually on 3 different boxes is a real headache. >>> >>> Thanks >>> -Todd >>> >>> On Tue, Dec 21, 2010 at 1:33 PM, Nigel Daley wrote: >>> Ok, HDFS is now enabled. You'll see a stream of updates shortly on the >> ~30 Patch Available HDFS issues. Nige On Dec 20, 2010, at 12:42 PM, Jakob Homan wrote: > I committed HDFS-1511 this morning. We should be good to go. I >> can > haz snooty robot butler? > > On Fri, Dec 17, 2010 at 8:31 PM, Konstantin Boudnik < >> c...@apache.org> wrote: >> Thanks Jacob. I am wasted already but I can do it on Sun, I think, >> unless it is done earlier. >> -- >> Take care, >> Konstantin (Cos) Boudnik >> >> >> >> On Fri, Dec 17, 2010 at 19:41, Jakob Homan wrote: >>> Ok. I'll get a patch out for 1511 tomorrow, unless someone wants to >>> whip one up tonight. >>> >>> >>> On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley wrote: I agree with Cos on fixing HDFS-1511 first. Once that is done >> I'll enable hdfs patch testing. Cheers, Nige Sent from my iPhone4 On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik >> wrote: > One more issue needs to be addressed before test-patch is >> turned on HDFS is > https://issues.apache.org/jira/browse/HDFS-1511 > -- > Take care, > Konstantin (Cos) Boudnik > > > > On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik < c...@apache.org> wrote: >> Considering that because of these 4 faulty cases every patch will >> be >> -1'ed a patch author will still have to look at it and make a comment >> why this particular -1 isn't valid. Lesser work, perhaps, but messier >> IMO. I'm not blocking it - I just feel like there's a better way. >> >> -- >> Take care, >> Konstantin (Cos) Boudnik >> >> >> >> On Fri, Dec 17, 2010 at 15:55, Jakob Homan >> wrote: If HDFS is added to the test-patch queue right now we get nothing but dozens of -1'ed patches. >>> There aren't dozens o
Re: Patch testing
On Fri, Jan 7, 2011 at 2:11 PM, Nigel Daley wrote: > Hrm, the MR precommit test I'm running has hung (been running for 14 hours > so far). FWIW, 2 HDFS precommit tests are hung too. I suspect it could be > the NFS mounts on the machines. I forced a thread dump which you can see in > the console: > https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/10/console > > Strange, haven't seen a hang like that before in handleConnectionFailure. It should retry for 15 minutes max in that loop. > Any other ideas why these might be hanging? > > There is an HDFS bug right now that can cause hangs on some tests - HDFS-1529 - would appreciate if someone can take a look. But I don't think this is responsible for the MR hang above. -Todd > On Jan 5, 2011, at 5:42 PM, Todd Lipcon wrote: > > > On Wed, Jan 5, 2011 at 4:39 PM, Nigel Daley wrote: > > > >> Thanks for looking into it Todd. Let's first see if you think it can be > >> fixed quickly. Let me know. > >> > >> > > No problem, it wasn't too bad after all. Patch up on HADOOP-7087 which > fixes > > this test timeout for me. > > > > -Todd > > > > > >> On Jan 5, 2011, at 4:33 PM, Todd Lipcon wrote: > >> > >>> On Wed, Jan 5, 2011 at 4:19 PM, Nigel Daley wrote: > >>> > Todd, would love to get > https://issues.apache.org/jira/browse/MAPREDUCE-2121 fixed first > since > this is failing every night on trunk. > > >>> > >>> What if we disable that test, move that issue to 0.22 blocker, and then > >>> enable the test-patch? I'll also look into that one today, but if it's > >>> something that will take a while to fix, I don't think we should hold > off > >>> the useful testing for all the other patches. > >>> > >>> -Todd > >>> > >>> On Jan 5, 2011, at 2:45 PM, Todd Lipcon wrote: > > > Hi Nigel, > > > > MAPREDUCE-2172 has been fixed for a while. Are there any other > >> particular > > JIRAs you think need to be fixed before the MR test-patch queue gets > > enabled? I have a lot of outstanding patches and doing all the > >> test-patch > > turnaround manually on 3 different boxes is a real headache. > > > > Thanks > > -Todd > > > > On Tue, Dec 21, 2010 at 1:33 PM, Nigel Daley wrote: > > > >> Ok, HDFS is now enabled. You'll see a stream of updates shortly on > >> the > ~30 > >> Patch Available HDFS issues. > >> > >> Nige > >> > >> On Dec 20, 2010, at 12:42 PM, Jakob Homan wrote: > >> > >>> I committed HDFS-1511 this morning. We should be good to go. I > can > >>> haz snooty robot butler? > >>> > >>> On Fri, Dec 17, 2010 at 8:31 PM, Konstantin Boudnik < > c...@apache.org> > >> wrote: > Thanks Jacob. I am wasted already but I can do it on Sun, I think, > unless it is done earlier. > -- > Take care, > Konstantin (Cos) Boudnik > > > > On Fri, Dec 17, 2010 at 19:41, Jakob Homan > >> wrote: > > Ok. I'll get a patch out for 1511 tomorrow, unless someone wants > >> to > > whip one up tonight. > > > > > > On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley > >> wrote: > >> I agree with Cos on fixing HDFS-1511 first. Once that is done > I'll > >> enable hdfs patch testing. > >> > >> Cheers, > >> Nige > >> > >> Sent from my iPhone4 > >> > >> On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik > > >> wrote: > >> > >>> One more issue needs to be addressed before test-patch is > turned > >> on > >> HDFS is > >>> https://issues.apache.org/jira/browse/HDFS-1511 > >>> -- > >>> Take care, > >>> Konstantin (Cos) Boudnik > >>> > >>> > >>> > >>> On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik < > >> c...@apache.org> > >> wrote: > Considering that because of these 4 faulty cases every patch > >> will > be > -1'ed a patch author will still have to look at it and make a > >> comment > why this particular -1 isn't valid. Lesser work, perhaps, but > >> messier > IMO. I'm not blocking it - I just feel like there's a better > >> way. > > -- > Take care, > Konstantin (Cos) Boudnik > > > > On Fri, Dec 17, 2010 at 15:55, Jakob Homan > > >> wrote: > >> If HDFS is added to the test-patch queue right now we get > >> nothing but dozens of -1'ed patches. > > There aren't dozens of patches being submitted currently. > The > >> -1 > > isn't the important thing, it's the grunt work of actually > running > > (and waiting) for the tests, test-patch, etc. that Hudson > does > >> so > >> that > > the developer doesn't have to. > > > >>>
Re: Patch testing
Hrm, the MR precommit test I'm running has hung (been running for 14 hours so far). FWIW, 2 HDFS precommit tests are hung too. I suspect it could be the NFS mounts on the machines. I forced a thread dump which you can see in the console: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/10/console Any other ideas why these might be hanging? Thanks, Nige On Jan 5, 2011, at 5:42 PM, Todd Lipcon wrote: > On Wed, Jan 5, 2011 at 4:39 PM, Nigel Daley wrote: > >> Thanks for looking into it Todd. Let's first see if you think it can be >> fixed quickly. Let me know. >> >> > No problem, it wasn't too bad after all. Patch up on HADOOP-7087 which fixes > this test timeout for me. > > -Todd > > >> On Jan 5, 2011, at 4:33 PM, Todd Lipcon wrote: >> >>> On Wed, Jan 5, 2011 at 4:19 PM, Nigel Daley wrote: >>> Todd, would love to get https://issues.apache.org/jira/browse/MAPREDUCE-2121 fixed first since this is failing every night on trunk. >>> >>> What if we disable that test, move that issue to 0.22 blocker, and then >>> enable the test-patch? I'll also look into that one today, but if it's >>> something that will take a while to fix, I don't think we should hold off >>> the useful testing for all the other patches. >>> >>> -Todd >>> >>> On Jan 5, 2011, at 2:45 PM, Todd Lipcon wrote: > Hi Nigel, > > MAPREDUCE-2172 has been fixed for a while. Are there any other >> particular > JIRAs you think need to be fixed before the MR test-patch queue gets > enabled? I have a lot of outstanding patches and doing all the >> test-patch > turnaround manually on 3 different boxes is a real headache. > > Thanks > -Todd > > On Tue, Dec 21, 2010 at 1:33 PM, Nigel Daley wrote: > >> Ok, HDFS is now enabled. You'll see a stream of updates shortly on >> the ~30 >> Patch Available HDFS issues. >> >> Nige >> >> On Dec 20, 2010, at 12:42 PM, Jakob Homan wrote: >> >>> I committed HDFS-1511 this morning. We should be good to go. I can >>> haz snooty robot butler? >>> >>> On Fri, Dec 17, 2010 at 8:31 PM, Konstantin Boudnik >> wrote: Thanks Jacob. I am wasted already but I can do it on Sun, I think, unless it is done earlier. -- Take care, Konstantin (Cos) Boudnik On Fri, Dec 17, 2010 at 19:41, Jakob Homan >> wrote: > Ok. I'll get a patch out for 1511 tomorrow, unless someone wants >> to > whip one up tonight. > > > On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley >> wrote: >> I agree with Cos on fixing HDFS-1511 first. Once that is done I'll >> enable hdfs patch testing. >> >> Cheers, >> Nige >> >> Sent from my iPhone4 >> >> On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik >> wrote: >> >>> One more issue needs to be addressed before test-patch is turned >> on >> HDFS is >>> https://issues.apache.org/jira/browse/HDFS-1511 >>> -- >>> Take care, >>> Konstantin (Cos) Boudnik >>> >>> >>> >>> On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik < >> c...@apache.org> >> wrote: Considering that because of these 4 faulty cases every patch >> will be -1'ed a patch author will still have to look at it and make a >> comment why this particular -1 isn't valid. Lesser work, perhaps, but >> messier IMO. I'm not blocking it - I just feel like there's a better >> way. -- Take care, Konstantin (Cos) Boudnik On Fri, Dec 17, 2010 at 15:55, Jakob Homan >> wrote: >> If HDFS is added to the test-patch queue right now we get >> nothing but dozens of -1'ed patches. > There aren't dozens of patches being submitted currently. The >> -1 > isn't the important thing, it's the grunt work of actually running > (and waiting) for the tests, test-patch, etc. that Hudson does >> so >> that > the developer doesn't have to. > > On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur < >> dhr...@gmail.com> wrote: >> +1, thanks for doing this. >> >> On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan < >> jgho...@gmail.com > >> wrote: >> >>> So, with test-patch updated to show the failing tests, saving the >>> developers the need to go and verify that the failed tests >> are >> all >>> known, how do people feel about turning on test-patch again >> for >> HDFS >>> and mapred? I think it'll help prevent any more tests from >> entering
Re: Patch testing
On Wed, Jan 05, 2011 at 08:46PM, Nigel Daley wrote: > Lol. At least now we know how much it's valued. I could get a foot long from > subway. But hurry Nige - pretty soon it's gonna be worthless ;( > Cheers, > Nige > > On Jan 5, 2011, at 8:11 PM, Stack wrote: > > > I'll give you $5.00 Nigel if you add HBase to the list below. > > St.Ack > > > > On Wed, Oct 20, 2010 at 1:31 PM, Nigel Daley wrote: > >> I think ZK, PIG, etc will also be included in the update I'm working on. > >> > >> Cheers, > >> Nige > >> > >> On Oct 20, 2010, at 1:27 PM, Mahadev Konar wrote: > >> > >>> Great. > >>> > >>> Nigel, > >>> Can you please document in somewhere on how you fixed it? We¹d like to fix > >>> it for ZooKeeper as well. > >>> > >>> Thanks > >>> mahadev > >>> > >>> > >>> On 10/20/10 1:23 PM, "Owen O'Malley" wrote: > >>> > > > On Oct 20, 2010, at 12:47 PM, Nigel Daley wrote: > > > I'm working to get the pre-commit patch testing running again for > > HDFS, HADOOP, and MAPREDUCE patches. > > That would be great, Nigel. > > Thanks, > Owen > > >>> > >> > >> signature.asc Description: Digital signature
Re: Patch testing
Lol. At least now we know how much it's valued. I could get a foot long from subway. Cheers, Nige On Jan 5, 2011, at 8:11 PM, Stack wrote: > I'll give you $5.00 Nigel if you add HBase to the list below. > St.Ack > > On Wed, Oct 20, 2010 at 1:31 PM, Nigel Daley wrote: >> I think ZK, PIG, etc will also be included in the update I'm working on. >> >> Cheers, >> Nige >> >> On Oct 20, 2010, at 1:27 PM, Mahadev Konar wrote: >> >>> Great. >>> >>> Nigel, >>> Can you please document in somewhere on how you fixed it? We¹d like to fix >>> it for ZooKeeper as well. >>> >>> Thanks >>> mahadev >>> >>> >>> On 10/20/10 1:23 PM, "Owen O'Malley" wrote: >>> On Oct 20, 2010, at 12:47 PM, Nigel Daley wrote: > I'm working to get the pre-commit patch testing running again for > HDFS, HADOOP, and MAPREDUCE patches. That would be great, Nigel. Thanks, Owen >>> >> >>
Re: Patch testing
I'll give you $5.00 Nigel if you add HBase to the list below. St.Ack On Wed, Oct 20, 2010 at 1:31 PM, Nigel Daley wrote: > I think ZK, PIG, etc will also be included in the update I'm working on. > > Cheers, > Nige > > On Oct 20, 2010, at 1:27 PM, Mahadev Konar wrote: > >> Great. >> >> Nigel, >> Can you please document in somewhere on how you fixed it? We¹d like to fix >> it for ZooKeeper as well. >> >> Thanks >> mahadev >> >> >> On 10/20/10 1:23 PM, "Owen O'Malley" wrote: >> >>> >>> >>> On Oct 20, 2010, at 12:47 PM, Nigel Daley wrote: >>> I'm working to get the pre-commit patch testing running again for HDFS, HADOOP, and MAPREDUCE patches. >>> >>> That would be great, Nigel. >>> >>> Thanks, >>> Owen >>> >> > >
Re: Patch testing
On Wed, Jan 5, 2011 at 4:39 PM, Nigel Daley wrote: > Thanks for looking into it Todd. Let's first see if you think it can be > fixed quickly. Let me know. > > No problem, it wasn't too bad after all. Patch up on HADOOP-7087 which fixes this test timeout for me. -Todd > On Jan 5, 2011, at 4:33 PM, Todd Lipcon wrote: > > > On Wed, Jan 5, 2011 at 4:19 PM, Nigel Daley wrote: > > > >> Todd, would love to get > >> https://issues.apache.org/jira/browse/MAPREDUCE-2121 fixed first since > >> this is failing every night on trunk. > >> > > > > What if we disable that test, move that issue to 0.22 blocker, and then > > enable the test-patch? I'll also look into that one today, but if it's > > something that will take a while to fix, I don't think we should hold off > > the useful testing for all the other patches. > > > > -Todd > > > > On Jan 5, 2011, at 2:45 PM, Todd Lipcon wrote: > >> > >>> Hi Nigel, > >>> > >>> MAPREDUCE-2172 has been fixed for a while. Are there any other > particular > >>> JIRAs you think need to be fixed before the MR test-patch queue gets > >>> enabled? I have a lot of outstanding patches and doing all the > test-patch > >>> turnaround manually on 3 different boxes is a real headache. > >>> > >>> Thanks > >>> -Todd > >>> > >>> On Tue, Dec 21, 2010 at 1:33 PM, Nigel Daley wrote: > >>> > Ok, HDFS is now enabled. You'll see a stream of updates shortly on > the > >> ~30 > Patch Available HDFS issues. > > Nige > > On Dec 20, 2010, at 12:42 PM, Jakob Homan wrote: > > > I committed HDFS-1511 this morning. We should be good to go. I can > > haz snooty robot butler? > > > > On Fri, Dec 17, 2010 at 8:31 PM, Konstantin Boudnik > wrote: > >> Thanks Jacob. I am wasted already but I can do it on Sun, I think, > >> unless it is done earlier. > >> -- > >> Take care, > >> Konstantin (Cos) Boudnik > >> > >> > >> > >> On Fri, Dec 17, 2010 at 19:41, Jakob Homan > wrote: > >>> Ok. I'll get a patch out for 1511 tomorrow, unless someone wants > to > >>> whip one up tonight. > >>> > >>> > >>> On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley > wrote: > I agree with Cos on fixing HDFS-1511 first. Once that is done I'll > enable hdfs patch testing. > > Cheers, > Nige > > Sent from my iPhone4 > > On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik > wrote: > > > One more issue needs to be addressed before test-patch is turned > on > HDFS is > > https://issues.apache.org/jira/browse/HDFS-1511 > > -- > > Take care, > > Konstantin (Cos) Boudnik > > > > > > > > On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik < > c...@apache.org> > wrote: > >> Considering that because of these 4 faulty cases every patch > will > >> be > >> -1'ed a patch author will still have to look at it and make a > comment > >> why this particular -1 isn't valid. Lesser work, perhaps, but > messier > >> IMO. I'm not blocking it - I just feel like there's a better > way. > >> > >> -- > >> Take care, > >> Konstantin (Cos) Boudnik > >> > >> > >> > >> On Fri, Dec 17, 2010 at 15:55, Jakob Homan > wrote: > If HDFS is added to the test-patch queue right now we get > nothing but dozens of -1'ed patches. > >>> There aren't dozens of patches being submitted currently. The > -1 > >>> isn't the important thing, it's the grunt work of actually > >> running > >>> (and waiting) for the tests, test-patch, etc. that Hudson does > so > that > >>> the developer doesn't have to. > >>> > >>> On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur < > dhr...@gmail.com> wrote: > +1, thanks for doing this. > > On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan < > jgho...@gmail.com > >>> > wrote: > > > So, with test-patch updated to show the failing tests, saving > >> the > > developers the need to go and verify that the failed tests > are > all > > known, how do people feel about turning on test-patch again > for > HDFS > > and mapred? I think it'll help prevent any more tests from > entering > > the "yeah, we know" category. > > > > Thanks, > > jg > > > > > > On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan < > jho...@yahoo-inc.com> wrote: > >> True, each patch would get a -1 and the failing tests would > >> need > to be > >> verified as those known bad (BTW, it would be great if > Hudson > could list > >> which tests failed in the message it posts to JIRA).
Re: Patch testing
Thanks for looking into it Todd. Let's first see if you think it can be fixed quickly. Let me know. Thanks, Nige On Jan 5, 2011, at 4:33 PM, Todd Lipcon wrote: > On Wed, Jan 5, 2011 at 4:19 PM, Nigel Daley wrote: > >> Todd, would love to get >> https://issues.apache.org/jira/browse/MAPREDUCE-2121 fixed first since >> this is failing every night on trunk. >> > > What if we disable that test, move that issue to 0.22 blocker, and then > enable the test-patch? I'll also look into that one today, but if it's > something that will take a while to fix, I don't think we should hold off > the useful testing for all the other patches. > > -Todd > > On Jan 5, 2011, at 2:45 PM, Todd Lipcon wrote: >> >>> Hi Nigel, >>> >>> MAPREDUCE-2172 has been fixed for a while. Are there any other particular >>> JIRAs you think need to be fixed before the MR test-patch queue gets >>> enabled? I have a lot of outstanding patches and doing all the test-patch >>> turnaround manually on 3 different boxes is a real headache. >>> >>> Thanks >>> -Todd >>> >>> On Tue, Dec 21, 2010 at 1:33 PM, Nigel Daley wrote: >>> Ok, HDFS is now enabled. You'll see a stream of updates shortly on the >> ~30 Patch Available HDFS issues. Nige On Dec 20, 2010, at 12:42 PM, Jakob Homan wrote: > I committed HDFS-1511 this morning. We should be good to go. I can > haz snooty robot butler? > > On Fri, Dec 17, 2010 at 8:31 PM, Konstantin Boudnik wrote: >> Thanks Jacob. I am wasted already but I can do it on Sun, I think, >> unless it is done earlier. >> -- >> Take care, >> Konstantin (Cos) Boudnik >> >> >> >> On Fri, Dec 17, 2010 at 19:41, Jakob Homan wrote: >>> Ok. I'll get a patch out for 1511 tomorrow, unless someone wants to >>> whip one up tonight. >>> >>> >>> On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley wrote: I agree with Cos on fixing HDFS-1511 first. Once that is done I'll enable hdfs patch testing. Cheers, Nige Sent from my iPhone4 On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik wrote: > One more issue needs to be addressed before test-patch is turned on HDFS is > https://issues.apache.org/jira/browse/HDFS-1511 > -- > Take care, > Konstantin (Cos) Boudnik > > > > On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik wrote: >> Considering that because of these 4 faulty cases every patch will >> be >> -1'ed a patch author will still have to look at it and make a comment >> why this particular -1 isn't valid. Lesser work, perhaps, but messier >> IMO. I'm not blocking it - I just feel like there's a better way. >> >> -- >> Take care, >> Konstantin (Cos) Boudnik >> >> >> >> On Fri, Dec 17, 2010 at 15:55, Jakob Homan wrote: If HDFS is added to the test-patch queue right now we get nothing but dozens of -1'ed patches. >>> There aren't dozens of patches being submitted currently. The -1 >>> isn't the important thing, it's the grunt work of actually >> running >>> (and waiting) for the tests, test-patch, etc. that Hudson does so that >>> the developer doesn't have to. >>> >>> On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur < dhr...@gmail.com> wrote: +1, thanks for doing this. On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan >> wrote: > So, with test-patch updated to show the failing tests, saving >> the > developers the need to go and verify that the failed tests are all > known, how do people feel about turning on test-patch again for HDFS > and mapred? I think it'll help prevent any more tests from entering > the "yeah, we know" category. > > Thanks, > jg > > > On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan < jho...@yahoo-inc.com> wrote: >> True, each patch would get a -1 and the failing tests would >> need to be >> verified as those known bad (BTW, it would be great if Hudson could list >> which tests failed in the message it posts to JIRA). But >> that's still > quite >> a bit less error-prone work than if the developer runs the >> tests and >> test-patch themselves. Also, with 22 being cut, there are a >> lot of > patches >> up in the air and several developers are juggling multiple patches. The >> more automation we can have, even if it's not perfect, will decrease > errors
Re: Patch testing
On Wed, Jan 5, 2011 at 4:19 PM, Nigel Daley wrote: > Todd, would love to get > https://issues.apache.org/jira/browse/MAPREDUCE-2121 fixed first since > this is failing every night on trunk. > What if we disable that test, move that issue to 0.22 blocker, and then enable the test-patch? I'll also look into that one today, but if it's something that will take a while to fix, I don't think we should hold off the useful testing for all the other patches. -Todd On Jan 5, 2011, at 2:45 PM, Todd Lipcon wrote: > > > Hi Nigel, > > > > MAPREDUCE-2172 has been fixed for a while. Are there any other particular > > JIRAs you think need to be fixed before the MR test-patch queue gets > > enabled? I have a lot of outstanding patches and doing all the test-patch > > turnaround manually on 3 different boxes is a real headache. > > > > Thanks > > -Todd > > > > On Tue, Dec 21, 2010 at 1:33 PM, Nigel Daley wrote: > > > >> Ok, HDFS is now enabled. You'll see a stream of updates shortly on the > ~30 > >> Patch Available HDFS issues. > >> > >> Nige > >> > >> On Dec 20, 2010, at 12:42 PM, Jakob Homan wrote: > >> > >>> I committed HDFS-1511 this morning. We should be good to go. I can > >>> haz snooty robot butler? > >>> > >>> On Fri, Dec 17, 2010 at 8:31 PM, Konstantin Boudnik > >> wrote: > Thanks Jacob. I am wasted already but I can do it on Sun, I think, > unless it is done earlier. > -- > Take care, > Konstantin (Cos) Boudnik > > > > On Fri, Dec 17, 2010 at 19:41, Jakob Homan wrote: > > Ok. I'll get a patch out for 1511 tomorrow, unless someone wants to > > whip one up tonight. > > > > > > On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley wrote: > >> I agree with Cos on fixing HDFS-1511 first. Once that is done I'll > >> enable hdfs patch testing. > >> > >> Cheers, > >> Nige > >> > >> Sent from my iPhone4 > >> > >> On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik > >> wrote: > >> > >>> One more issue needs to be addressed before test-patch is turned on > >> HDFS is > >>> https://issues.apache.org/jira/browse/HDFS-1511 > >>> -- > >>> Take care, > >>> Konstantin (Cos) Boudnik > >>> > >>> > >>> > >>> On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik > >> wrote: > Considering that because of these 4 faulty cases every patch will > be > -1'ed a patch author will still have to look at it and make a > >> comment > why this particular -1 isn't valid. Lesser work, perhaps, but > >> messier > IMO. I'm not blocking it - I just feel like there's a better way. > > -- > Take care, > Konstantin (Cos) Boudnik > > > > On Fri, Dec 17, 2010 at 15:55, Jakob Homan > >> wrote: > >> If HDFS is added to the test-patch queue right now we get > >> nothing but dozens of -1'ed patches. > > There aren't dozens of patches being submitted currently. The -1 > > isn't the important thing, it's the grunt work of actually > running > > (and waiting) for the tests, test-patch, etc. that Hudson does so > >> that > > the developer doesn't have to. > > > > On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur < > >> dhr...@gmail.com> wrote: > >> +1, thanks for doing this. > >> > >> On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan > > >> wrote: > >> > >>> So, with test-patch updated to show the failing tests, saving > the > >>> developers the need to go and verify that the failed tests are > >> all > >>> known, how do people feel about turning on test-patch again for > >> HDFS > >>> and mapred? I think it'll help prevent any more tests from > >> entering > >>> the "yeah, we know" category. > >>> > >>> Thanks, > >>> jg > >>> > >>> > >>> On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan < > >> jho...@yahoo-inc.com> wrote: > True, each patch would get a -1 and the failing tests would > need > >> to be > verified as those known bad (BTW, it would be great if Hudson > >> could list > which tests failed in the message it posts to JIRA). But > that's > >> still > >>> quite > a bit less error-prone work than if the developer runs the > tests > >> and > test-patch themselves. Also, with 22 being cut, there are a > lot > >> of > >>> patches > up in the air and several developers are juggling multiple > >> patches. The > more automation we can have, even if it's not perfect, will > >> decrease > >>> errors > we may make. > -jg > > Nigel Daley wrote: > > > > On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: > > > >>> It's also ready to run o
Re: Patch testing
Todd, would love to get https://issues.apache.org/jira/browse/MAPREDUCE-2121 fixed first since this is failing every night on trunk. Nige On Jan 5, 2011, at 2:45 PM, Todd Lipcon wrote: > Hi Nigel, > > MAPREDUCE-2172 has been fixed for a while. Are there any other particular > JIRAs you think need to be fixed before the MR test-patch queue gets > enabled? I have a lot of outstanding patches and doing all the test-patch > turnaround manually on 3 different boxes is a real headache. > > Thanks > -Todd > > On Tue, Dec 21, 2010 at 1:33 PM, Nigel Daley wrote: > >> Ok, HDFS is now enabled. You'll see a stream of updates shortly on the ~30 >> Patch Available HDFS issues. >> >> Nige >> >> On Dec 20, 2010, at 12:42 PM, Jakob Homan wrote: >> >>> I committed HDFS-1511 this morning. We should be good to go. I can >>> haz snooty robot butler? >>> >>> On Fri, Dec 17, 2010 at 8:31 PM, Konstantin Boudnik >> wrote: Thanks Jacob. I am wasted already but I can do it on Sun, I think, unless it is done earlier. -- Take care, Konstantin (Cos) Boudnik On Fri, Dec 17, 2010 at 19:41, Jakob Homan wrote: > Ok. I'll get a patch out for 1511 tomorrow, unless someone wants to > whip one up tonight. > > > On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley wrote: >> I agree with Cos on fixing HDFS-1511 first. Once that is done I'll >> enable hdfs patch testing. >> >> Cheers, >> Nige >> >> Sent from my iPhone4 >> >> On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik >> wrote: >> >>> One more issue needs to be addressed before test-patch is turned on >> HDFS is >>> https://issues.apache.org/jira/browse/HDFS-1511 >>> -- >>> Take care, >>> Konstantin (Cos) Boudnik >>> >>> >>> >>> On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik >> wrote: Considering that because of these 4 faulty cases every patch will be -1'ed a patch author will still have to look at it and make a >> comment why this particular -1 isn't valid. Lesser work, perhaps, but >> messier IMO. I'm not blocking it - I just feel like there's a better way. -- Take care, Konstantin (Cos) Boudnik On Fri, Dec 17, 2010 at 15:55, Jakob Homan >> wrote: >> If HDFS is added to the test-patch queue right now we get >> nothing but dozens of -1'ed patches. > There aren't dozens of patches being submitted currently. The -1 > isn't the important thing, it's the grunt work of actually running > (and waiting) for the tests, test-patch, etc. that Hudson does so >> that > the developer doesn't have to. > > On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur < >> dhr...@gmail.com> wrote: >> +1, thanks for doing this. >> >> On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan >> wrote: >> >>> So, with test-patch updated to show the failing tests, saving the >>> developers the need to go and verify that the failed tests are >> all >>> known, how do people feel about turning on test-patch again for >> HDFS >>> and mapred? I think it'll help prevent any more tests from >> entering >>> the "yeah, we know" category. >>> >>> Thanks, >>> jg >>> >>> >>> On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan < >> jho...@yahoo-inc.com> wrote: True, each patch would get a -1 and the failing tests would need >> to be verified as those known bad (BTW, it would be great if Hudson >> could list which tests failed in the message it posts to JIRA). But that's >> still >>> quite a bit less error-prone work than if the developer runs the tests >> and test-patch themselves. Also, with 22 being cut, there are a lot >> of >>> patches up in the air and several developers are juggling multiple >> patches. The more automation we can have, even if it's not perfect, will >> decrease >>> errors we may make. -jg Nigel Daley wrote: > > On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: > >>> It's also ready to run on MapReduce and HDFS but we won't >> turn it on >>> until these projects build and test cleanly. Looks like both >> these >>> projects >>> currently have test failures. >> >> Assuming the projects are compiling and building, is there a >> reason to >> not turn it on despite the test failures? Hudson is invaluable >> to >>> developers >> who then don't have to run the tests and test-patch >> themselves. We >>> didn't >> turn Hudson off when it was working p
Re: Patch testing
Hi Nigel, MAPREDUCE-2172 has been fixed for a while. Are there any other particular JIRAs you think need to be fixed before the MR test-patch queue gets enabled? I have a lot of outstanding patches and doing all the test-patch turnaround manually on 3 different boxes is a real headache. Thanks -Todd On Tue, Dec 21, 2010 at 1:33 PM, Nigel Daley wrote: > Ok, HDFS is now enabled. You'll see a stream of updates shortly on the ~30 > Patch Available HDFS issues. > > Nige > > On Dec 20, 2010, at 12:42 PM, Jakob Homan wrote: > > > I committed HDFS-1511 this morning. We should be good to go. I can > > haz snooty robot butler? > > > > On Fri, Dec 17, 2010 at 8:31 PM, Konstantin Boudnik > wrote: > >> Thanks Jacob. I am wasted already but I can do it on Sun, I think, > >> unless it is done earlier. > >> -- > >> Take care, > >> Konstantin (Cos) Boudnik > >> > >> > >> > >> On Fri, Dec 17, 2010 at 19:41, Jakob Homan wrote: > >>> Ok. I'll get a patch out for 1511 tomorrow, unless someone wants to > >>> whip one up tonight. > >>> > >>> > >>> On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley wrote: > I agree with Cos on fixing HDFS-1511 first. Once that is done I'll > enable hdfs patch testing. > > Cheers, > Nige > > Sent from my iPhone4 > > On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik > wrote: > > > One more issue needs to be addressed before test-patch is turned on > HDFS is > > https://issues.apache.org/jira/browse/HDFS-1511 > > -- > > Take care, > > Konstantin (Cos) Boudnik > > > > > > > > On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik > wrote: > >> Considering that because of these 4 faulty cases every patch will be > >> -1'ed a patch author will still have to look at it and make a > comment > >> why this particular -1 isn't valid. Lesser work, perhaps, but > messier > >> IMO. I'm not blocking it - I just feel like there's a better way. > >> > >> -- > >> Take care, > >> Konstantin (Cos) Boudnik > >> > >> > >> > >> On Fri, Dec 17, 2010 at 15:55, Jakob Homan > wrote: > If HDFS is added to the test-patch queue right now we get > nothing but dozens of -1'ed patches. > >>> There aren't dozens of patches being submitted currently. The -1 > >>> isn't the important thing, it's the grunt work of actually running > >>> (and waiting) for the tests, test-patch, etc. that Hudson does so > that > >>> the developer doesn't have to. > >>> > >>> On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur < > dhr...@gmail.com> wrote: > +1, thanks for doing this. > > On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan > wrote: > > > So, with test-patch updated to show the failing tests, saving the > > developers the need to go and verify that the failed tests are > all > > known, how do people feel about turning on test-patch again for > HDFS > > and mapred? I think it'll help prevent any more tests from > entering > > the "yeah, we know" category. > > > > Thanks, > > jg > > > > > > On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan < > jho...@yahoo-inc.com> wrote: > >> True, each patch would get a -1 and the failing tests would need > to be > >> verified as those known bad (BTW, it would be great if Hudson > could list > >> which tests failed in the message it posts to JIRA). But that's > still > > quite > >> a bit less error-prone work than if the developer runs the tests > and > >> test-patch themselves. Also, with 22 being cut, there are a lot > of > > patches > >> up in the air and several developers are juggling multiple > patches. The > >> more automation we can have, even if it's not perfect, will > decrease > > errors > >> we may make. > >> -jg > >> > >> Nigel Daley wrote: > >>> > >>> On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: > >>> > > It's also ready to run on MapReduce and HDFS but we won't > turn it on > > until these projects build and test cleanly. Looks like both > these > > projects > > currently have test failures. > > Assuming the projects are compiling and building, is there a > reason to > not turn it on despite the test failures? Hudson is invaluable > to > > developers > who then don't have to run the tests and test-patch > themselves. We > > didn't > turn Hudson off when it was working previously and there were > known > failures. I think one of the reasons we have more failing > tests now is > > the > higher cost of doing Hudson's work (not a great excuse I > know). This > > is > particularly true n
Re: Patch testing
Ok, HDFS is now enabled. You'll see a stream of updates shortly on the ~30 Patch Available HDFS issues. Nige On Dec 20, 2010, at 12:42 PM, Jakob Homan wrote: > I committed HDFS-1511 this morning. We should be good to go. I can > haz snooty robot butler? > > On Fri, Dec 17, 2010 at 8:31 PM, Konstantin Boudnik wrote: >> Thanks Jacob. I am wasted already but I can do it on Sun, I think, >> unless it is done earlier. >> -- >> Take care, >> Konstantin (Cos) Boudnik >> >> >> >> On Fri, Dec 17, 2010 at 19:41, Jakob Homan wrote: >>> Ok. I'll get a patch out for 1511 tomorrow, unless someone wants to >>> whip one up tonight. >>> >>> >>> On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley wrote: I agree with Cos on fixing HDFS-1511 first. Once that is done I'll enable hdfs patch testing. Cheers, Nige Sent from my iPhone4 On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik wrote: > One more issue needs to be addressed before test-patch is turned on HDFS > is > https://issues.apache.org/jira/browse/HDFS-1511 > -- > Take care, > Konstantin (Cos) Boudnik > > > > On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik wrote: >> Considering that because of these 4 faulty cases every patch will be >> -1'ed a patch author will still have to look at it and make a comment >> why this particular -1 isn't valid. Lesser work, perhaps, but messier >> IMO. I'm not blocking it - I just feel like there's a better way. >> >> -- >> Take care, >> Konstantin (Cos) Boudnik >> >> >> >> On Fri, Dec 17, 2010 at 15:55, Jakob Homan wrote: If HDFS is added to the test-patch queue right now we get nothing but dozens of -1'ed patches. >>> There aren't dozens of patches being submitted currently. The -1 >>> isn't the important thing, it's the grunt work of actually running >>> (and waiting) for the tests, test-patch, etc. that Hudson does so that >>> the developer doesn't have to. >>> >>> On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur >>> wrote: +1, thanks for doing this. On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan wrote: > So, with test-patch updated to show the failing tests, saving the > developers the need to go and verify that the failed tests are all > known, how do people feel about turning on test-patch again for HDFS > and mapred? I think it'll help prevent any more tests from entering > the "yeah, we know" category. > > Thanks, > jg > > > On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan > wrote: >> True, each patch would get a -1 and the failing tests would need to >> be >> verified as those known bad (BTW, it would be great if Hudson could >> list >> which tests failed in the message it posts to JIRA). But that's >> still > quite >> a bit less error-prone work than if the developer runs the tests and >> test-patch themselves. Also, with 22 being cut, there are a lot of > patches >> up in the air and several developers are juggling multiple patches. >> The >> more automation we can have, even if it's not perfect, will decrease > errors >> we may make. >> -jg >> >> Nigel Daley wrote: >>> >>> On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: >>> > It's also ready to run on MapReduce and HDFS but we won't turn it > on > until these projects build and test cleanly. Looks like both > these > projects > currently have test failures. Assuming the projects are compiling and building, is there a reason to not turn it on despite the test failures? Hudson is invaluable to > developers who then don't have to run the tests and test-patch themselves. We > didn't turn Hudson off when it was working previously and there were known failures. I think one of the reasons we have more failing tests now is > the higher cost of doing Hudson's work (not a great excuse I know). This > is particularly true now because several of the failing tests involve > tests timing out, making the whole testing regime even longer. >>> >>> Every single patch would get a -1 and need investigation. >>> Currently, > that >>> would be about 83 investigations between MR and HDFS issues that >>> are in >>> patch available state. Shouldn't we focus on getting these tests >>> fixed > or >>> removed/? Also, I nee
Re: Patch testing
I committed HDFS-1511 this morning. We should be good to go. I can haz snooty robot butler? On Fri, Dec 17, 2010 at 8:31 PM, Konstantin Boudnik wrote: > Thanks Jacob. I am wasted already but I can do it on Sun, I think, > unless it is done earlier. > -- > Take care, > Konstantin (Cos) Boudnik > > > > On Fri, Dec 17, 2010 at 19:41, Jakob Homan wrote: >> Ok. I'll get a patch out for 1511 tomorrow, unless someone wants to >> whip one up tonight. >> >> >> On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley wrote: >>> I agree with Cos on fixing HDFS-1511 first. Once that is done I'll enable >>> hdfs patch testing. >>> >>> Cheers, >>> Nige >>> >>> Sent from my iPhone4 >>> >>> On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik wrote: >>> One more issue needs to be addressed before test-patch is turned on HDFS is https://issues.apache.org/jira/browse/HDFS-1511 -- Take care, Konstantin (Cos) Boudnik On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik wrote: > Considering that because of these 4 faulty cases every patch will be > -1'ed a patch author will still have to look at it and make a comment > why this particular -1 isn't valid. Lesser work, perhaps, but messier > IMO. I'm not blocking it - I just feel like there's a better way. > > -- > Take care, > Konstantin (Cos) Boudnik > > > > On Fri, Dec 17, 2010 at 15:55, Jakob Homan wrote: >>> If HDFS is added to the test-patch queue right now we get >>> nothing but dozens of -1'ed patches. >> There aren't dozens of patches being submitted currently. The -1 >> isn't the important thing, it's the grunt work of actually running >> (and waiting) for the tests, test-patch, etc. that Hudson does so that >> the developer doesn't have to. >> >> On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur >> wrote: >>> +1, thanks for doing this. >>> >>> On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan wrote: >>> So, with test-patch updated to show the failing tests, saving the developers the need to go and verify that the failed tests are all known, how do people feel about turning on test-patch again for HDFS and mapred? I think it'll help prevent any more tests from entering the "yeah, we know" category. Thanks, jg On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan wrote: > True, each patch would get a -1 and the failing tests would need to be > verified as those known bad (BTW, it would be great if Hudson could > list > which tests failed in the message it posts to JIRA). But that's still quite > a bit less error-prone work than if the developer runs the tests and > test-patch themselves. Also, with 22 being cut, there are a lot of patches > up in the air and several developers are juggling multiple patches. > The > more automation we can have, even if it's not perfect, will decrease errors > we may make. > -jg > > Nigel Daley wrote: >> >> On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: >> It's also ready to run on MapReduce and HDFS but we won't turn it on until these projects build and test cleanly. Looks like both these projects currently have test failures. >>> >>> Assuming the projects are compiling and building, is there a reason >>> to >>> not turn it on despite the test failures? Hudson is invaluable to developers >>> who then don't have to run the tests and test-patch themselves. We didn't >>> turn Hudson off when it was working previously and there were known >>> failures. I think one of the reasons we have more failing tests >>> now is the >>> higher cost of doing Hudson's work (not a great excuse I know). >>> This is >>> particularly true now because several of the failing tests involve tests >>> timing out, making the whole testing regime even longer. >> >> Every single patch would get a -1 and need investigation. Currently, that >> would be about 83 investigations between MR and HDFS issues that are >> in >> patch available state. Shouldn't we focus on getting these tests >> fixed or >> removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS >> as >> well) before I turn this on. >> >> Cheers, >> Nige > > >>> >>> >>> >>> -- >>> Connect to me at http://www.facebook.com/dhruba >>> >> > >>> >> >
Re: Patch testing
Thanks Jacob. I am wasted already but I can do it on Sun, I think, unless it is done earlier. -- Take care, Konstantin (Cos) Boudnik On Fri, Dec 17, 2010 at 19:41, Jakob Homan wrote: > Ok. I'll get a patch out for 1511 tomorrow, unless someone wants to > whip one up tonight. > > > On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley wrote: >> I agree with Cos on fixing HDFS-1511 first. Once that is done I'll enable >> hdfs patch testing. >> >> Cheers, >> Nige >> >> Sent from my iPhone4 >> >> On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik wrote: >> >>> One more issue needs to be addressed before test-patch is turned on HDFS is >>> https://issues.apache.org/jira/browse/HDFS-1511 >>> -- >>> Take care, >>> Konstantin (Cos) Boudnik >>> >>> >>> >>> On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik wrote: Considering that because of these 4 faulty cases every patch will be -1'ed a patch author will still have to look at it and make a comment why this particular -1 isn't valid. Lesser work, perhaps, but messier IMO. I'm not blocking it - I just feel like there's a better way. -- Take care, Konstantin (Cos) Boudnik On Fri, Dec 17, 2010 at 15:55, Jakob Homan wrote: >> If HDFS is added to the test-patch queue right now we get >> nothing but dozens of -1'ed patches. > There aren't dozens of patches being submitted currently. The -1 > isn't the important thing, it's the grunt work of actually running > (and waiting) for the tests, test-patch, etc. that Hudson does so that > the developer doesn't have to. > > On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur > wrote: >> +1, thanks for doing this. >> >> On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan wrote: >> >>> So, with test-patch updated to show the failing tests, saving the >>> developers the need to go and verify that the failed tests are all >>> known, how do people feel about turning on test-patch again for HDFS >>> and mapred? I think it'll help prevent any more tests from entering >>> the "yeah, we know" category. >>> >>> Thanks, >>> jg >>> >>> >>> On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan >>> wrote: True, each patch would get a -1 and the failing tests would need to be verified as those known bad (BTW, it would be great if Hudson could list which tests failed in the message it posts to JIRA). But that's still >>> quite a bit less error-prone work than if the developer runs the tests and test-patch themselves. Also, with 22 being cut, there are a lot of >>> patches up in the air and several developers are juggling multiple patches. The more automation we can have, even if it's not perfect, will decrease >>> errors we may make. -jg Nigel Daley wrote: > > On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: > >>> It's also ready to run on MapReduce and HDFS but we won't turn it on >>> until these projects build and test cleanly. Looks like both these >>> projects >>> currently have test failures. >> >> Assuming the projects are compiling and building, is there a reason >> to >> not turn it on despite the test failures? Hudson is invaluable to >>> developers >> who then don't have to run the tests and test-patch themselves. We >>> didn't >> turn Hudson off when it was working previously and there were known >> failures. I think one of the reasons we have more failing tests now >> is >>> the >> higher cost of doing Hudson's work (not a great excuse I know). This >>> is >> particularly true now because several of the failing tests involve >>> tests >> timing out, making the whole testing regime even longer. > > Every single patch would get a -1 and need investigation. Currently, >>> that > would be about 83 investigations between MR and HDFS issues that are > in > patch available state. Shouldn't we focus on getting these tests > fixed >>> or > removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS > as > well) before I turn this on. > > Cheers, > Nige >>> >> >> >> >> -- >> Connect to me at http://www.facebook.com/dhruba >> > >> >
Re: Patch testing
Ok. I'll get a patch out for 1511 tomorrow, unless someone wants to whip one up tonight. On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley wrote: > I agree with Cos on fixing HDFS-1511 first. Once that is done I'll enable > hdfs patch testing. > > Cheers, > Nige > > Sent from my iPhone4 > > On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik wrote: > >> One more issue needs to be addressed before test-patch is turned on HDFS is >> https://issues.apache.org/jira/browse/HDFS-1511 >> -- >> Take care, >> Konstantin (Cos) Boudnik >> >> >> >> On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik wrote: >>> Considering that because of these 4 faulty cases every patch will be >>> -1'ed a patch author will still have to look at it and make a comment >>> why this particular -1 isn't valid. Lesser work, perhaps, but messier >>> IMO. I'm not blocking it - I just feel like there's a better way. >>> >>> -- >>> Take care, >>> Konstantin (Cos) Boudnik >>> >>> >>> >>> On Fri, Dec 17, 2010 at 15:55, Jakob Homan wrote: > If HDFS is added to the test-patch queue right now we get > nothing but dozens of -1'ed patches. There aren't dozens of patches being submitted currently. The -1 isn't the important thing, it's the grunt work of actually running (and waiting) for the tests, test-patch, etc. that Hudson does so that the developer doesn't have to. On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur wrote: > +1, thanks for doing this. > > On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan wrote: > >> So, with test-patch updated to show the failing tests, saving the >> developers the need to go and verify that the failed tests are all >> known, how do people feel about turning on test-patch again for HDFS >> and mapred? I think it'll help prevent any more tests from entering >> the "yeah, we know" category. >> >> Thanks, >> jg >> >> >> On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan >> wrote: >>> True, each patch would get a -1 and the failing tests would need to be >>> verified as those known bad (BTW, it would be great if Hudson could list >>> which tests failed in the message it posts to JIRA). But that's still >> quite >>> a bit less error-prone work than if the developer runs the tests and >>> test-patch themselves. Also, with 22 being cut, there are a lot of >> patches >>> up in the air and several developers are juggling multiple patches. The >>> more automation we can have, even if it's not perfect, will decrease >> errors >>> we may make. >>> -jg >>> >>> Nigel Daley wrote: On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: >> It's also ready to run on MapReduce and HDFS but we won't turn it on >> until these projects build and test cleanly. Looks like both these >> projects >> currently have test failures. > > Assuming the projects are compiling and building, is there a reason to > not turn it on despite the test failures? Hudson is invaluable to >> developers > who then don't have to run the tests and test-patch themselves. We >> didn't > turn Hudson off when it was working previously and there were known > failures. I think one of the reasons we have more failing tests now > is >> the > higher cost of doing Hudson's work (not a great excuse I know). This >> is > particularly true now because several of the failing tests involve >> tests > timing out, making the whole testing regime even longer. Every single patch would get a -1 and need investigation. Currently, >> that would be about 83 investigations between MR and HDFS issues that are in patch available state. Shouldn't we focus on getting these tests fixed >> or removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS as well) before I turn this on. Cheers, Nige >>> >>> >> > > > > -- > Connect to me at http://www.facebook.com/dhruba > >>> >
Re: Patch testing
I agree with Cos on fixing HDFS-1511 first. Once that is done I'll enable hdfs patch testing. Cheers, Nige Sent from my iPhone4 On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik wrote: > One more issue needs to be addressed before test-patch is turned on HDFS is > https://issues.apache.org/jira/browse/HDFS-1511 > -- > Take care, > Konstantin (Cos) Boudnik > > > > On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik wrote: >> Considering that because of these 4 faulty cases every patch will be >> -1'ed a patch author will still have to look at it and make a comment >> why this particular -1 isn't valid. Lesser work, perhaps, but messier >> IMO. I'm not blocking it - I just feel like there's a better way. >> >> -- >> Take care, >> Konstantin (Cos) Boudnik >> >> >> >> On Fri, Dec 17, 2010 at 15:55, Jakob Homan wrote: If HDFS is added to the test-patch queue right now we get nothing but dozens of -1'ed patches. >>> There aren't dozens of patches being submitted currently. The -1 >>> isn't the important thing, it's the grunt work of actually running >>> (and waiting) for the tests, test-patch, etc. that Hudson does so that >>> the developer doesn't have to. >>> >>> On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur wrote: +1, thanks for doing this. On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan wrote: > So, with test-patch updated to show the failing tests, saving the > developers the need to go and verify that the failed tests are all > known, how do people feel about turning on test-patch again for HDFS > and mapred? I think it'll help prevent any more tests from entering > the "yeah, we know" category. > > Thanks, > jg > > > On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan wrote: >> True, each patch would get a -1 and the failing tests would need to be >> verified as those known bad (BTW, it would be great if Hudson could list >> which tests failed in the message it posts to JIRA). But that's still > quite >> a bit less error-prone work than if the developer runs the tests and >> test-patch themselves. Also, with 22 being cut, there are a lot of > patches >> up in the air and several developers are juggling multiple patches. The >> more automation we can have, even if it's not perfect, will decrease > errors >> we may make. >> -jg >> >> Nigel Daley wrote: >>> >>> On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: >>> > It's also ready to run on MapReduce and HDFS but we won't turn it on > until these projects build and test cleanly. Looks like both these > projects > currently have test failures. Assuming the projects are compiling and building, is there a reason to not turn it on despite the test failures? Hudson is invaluable to > developers who then don't have to run the tests and test-patch themselves. We > didn't turn Hudson off when it was working previously and there were known failures. I think one of the reasons we have more failing tests now is > the higher cost of doing Hudson's work (not a great excuse I know). This > is particularly true now because several of the failing tests involve > tests timing out, making the whole testing regime even longer. >>> >>> Every single patch would get a -1 and need investigation. Currently, > that >>> would be about 83 investigations between MR and HDFS issues that are in >>> patch available state. Shouldn't we focus on getting these tests fixed > or >>> removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS as >>> well) before I turn this on. >>> >>> Cheers, >>> Nige >> >> > -- Connect to me at http://www.facebook.com/dhruba >>> >>
Re: Patch testing
On Fri, Dec 17, 2010 at 3:55 PM, Jakob Homan wrote: >> If HDFS is added to the test-patch queue right now we get >> nothing but dozens of -1'ed patches. > There aren't dozens of patches being submitted currently. The -1 > isn't the important thing, it's the grunt work of actually running > (and waiting) for the tests, test-patch, etc. that Hudson does so that > the developer doesn't have to. > I agree with Jakob. I've had to run and re-run the test-patch and unit tests probably 30 times over the last two weeks, and it takes a lot of effort, since my own infrastructure for doing this is a bit messy. I'd much rather just reply to the Hudson comments saying "these are known issues" than have to run the tests, check the results, copy and paste them and *then* say "these are known issues" anyway! > On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur wrote: >> +1, thanks for doing this. >> >> On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan wrote: >> >>> So, with test-patch updated to show the failing tests, saving the >>> developers the need to go and verify that the failed tests are all >>> known, how do people feel about turning on test-patch again for HDFS >>> and mapred? I think it'll help prevent any more tests from entering >>> the "yeah, we know" category. >>> >>> Thanks, >>> jg >>> >>> >>> On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan wrote: >>> > True, each patch would get a -1 and the failing tests would need to be >>> > verified as those known bad (BTW, it would be great if Hudson could list >>> > which tests failed in the message it posts to JIRA). But that's still >>> quite >>> > a bit less error-prone work than if the developer runs the tests and >>> > test-patch themselves. Also, with 22 being cut, there are a lot of >>> patches >>> > up in the air and several developers are juggling multiple patches. The >>> > more automation we can have, even if it's not perfect, will decrease >>> errors >>> > we may make. >>> > -jg >>> > >>> > Nigel Daley wrote: >>> >> >>> >> On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: >>> >> >>> It's also ready to run on MapReduce and HDFS but we won't turn it on >>> until these projects build and test cleanly. Looks like both these >>> projects >>> currently have test failures. >>> >>> >>> >>> Assuming the projects are compiling and building, is there a reason to >>> >>> not turn it on despite the test failures? Hudson is invaluable to >>> developers >>> >>> who then don't have to run the tests and test-patch themselves. We >>> didn't >>> >>> turn Hudson off when it was working previously and there were known >>> >>> failures. I think one of the reasons we have more failing tests now is >>> the >>> >>> higher cost of doing Hudson's work (not a great excuse I know). This >>> is >>> >>> particularly true now because several of the failing tests involve >>> tests >>> >>> timing out, making the whole testing regime even longer. >>> >> >>> >> Every single patch would get a -1 and need investigation. Currently, >>> that >>> >> would be about 83 investigations between MR and HDFS issues that are in >>> >> patch available state. Shouldn't we focus on getting these tests fixed >>> or >>> >> removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS as >>> >> well) before I turn this on. >>> >> >>> >> Cheers, >>> >> Nige >>> > >>> > >>> >> >> >> >> -- >> Connect to me at http://www.facebook.com/dhruba >> > -- Todd Lipcon Software Engineer, Cloudera
Re: Patch testing
One more issue needs to be addressed before test-patch is turned on HDFS is https://issues.apache.org/jira/browse/HDFS-1511 -- Take care, Konstantin (Cos) Boudnik On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik wrote: > Considering that because of these 4 faulty cases every patch will be > -1'ed a patch author will still have to look at it and make a comment > why this particular -1 isn't valid. Lesser work, perhaps, but messier > IMO. I'm not blocking it - I just feel like there's a better way. > > -- > Take care, > Konstantin (Cos) Boudnik > > > > On Fri, Dec 17, 2010 at 15:55, Jakob Homan wrote: >>> If HDFS is added to the test-patch queue right now we get >>> nothing but dozens of -1'ed patches. >> There aren't dozens of patches being submitted currently. The -1 >> isn't the important thing, it's the grunt work of actually running >> (and waiting) for the tests, test-patch, etc. that Hudson does so that >> the developer doesn't have to. >> >> On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur wrote: >>> +1, thanks for doing this. >>> >>> On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan wrote: >>> So, with test-patch updated to show the failing tests, saving the developers the need to go and verify that the failed tests are all known, how do people feel about turning on test-patch again for HDFS and mapred? I think it'll help prevent any more tests from entering the "yeah, we know" category. Thanks, jg On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan wrote: > True, each patch would get a -1 and the failing tests would need to be > verified as those known bad (BTW, it would be great if Hudson could list > which tests failed in the message it posts to JIRA). But that's still quite > a bit less error-prone work than if the developer runs the tests and > test-patch themselves. Also, with 22 being cut, there are a lot of patches > up in the air and several developers are juggling multiple patches. The > more automation we can have, even if it's not perfect, will decrease errors > we may make. > -jg > > Nigel Daley wrote: >> >> On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: >> It's also ready to run on MapReduce and HDFS but we won't turn it on until these projects build and test cleanly. Looks like both these projects currently have test failures. >>> >>> Assuming the projects are compiling and building, is there a reason to >>> not turn it on despite the test failures? Hudson is invaluable to developers >>> who then don't have to run the tests and test-patch themselves. We didn't >>> turn Hudson off when it was working previously and there were known >>> failures. I think one of the reasons we have more failing tests now is the >>> higher cost of doing Hudson's work (not a great excuse I know). This is >>> particularly true now because several of the failing tests involve tests >>> timing out, making the whole testing regime even longer. >> >> Every single patch would get a -1 and need investigation. Currently, that >> would be about 83 investigations between MR and HDFS issues that are in >> patch available state. Shouldn't we focus on getting these tests fixed or >> removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS as >> well) before I turn this on. >> >> Cheers, >> Nige > > >>> >>> >>> >>> -- >>> Connect to me at http://www.facebook.com/dhruba >>> >> >
Re: Patch testing
Considering that because of these 4 faulty cases every patch will be -1'ed a patch author will still have to look at it and make a comment why this particular -1 isn't valid. Lesser work, perhaps, but messier IMO. I'm not blocking it - I just feel like there's a better way. -- Take care, Konstantin (Cos) Boudnik On Fri, Dec 17, 2010 at 15:55, Jakob Homan wrote: >> If HDFS is added to the test-patch queue right now we get >> nothing but dozens of -1'ed patches. > There aren't dozens of patches being submitted currently. The -1 > isn't the important thing, it's the grunt work of actually running > (and waiting) for the tests, test-patch, etc. that Hudson does so that > the developer doesn't have to. > > On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur wrote: >> +1, thanks for doing this. >> >> On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan wrote: >> >>> So, with test-patch updated to show the failing tests, saving the >>> developers the need to go and verify that the failed tests are all >>> known, how do people feel about turning on test-patch again for HDFS >>> and mapred? I think it'll help prevent any more tests from entering >>> the "yeah, we know" category. >>> >>> Thanks, >>> jg >>> >>> >>> On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan wrote: >>> > True, each patch would get a -1 and the failing tests would need to be >>> > verified as those known bad (BTW, it would be great if Hudson could list >>> > which tests failed in the message it posts to JIRA). But that's still >>> quite >>> > a bit less error-prone work than if the developer runs the tests and >>> > test-patch themselves. Also, with 22 being cut, there are a lot of >>> patches >>> > up in the air and several developers are juggling multiple patches. The >>> > more automation we can have, even if it's not perfect, will decrease >>> errors >>> > we may make. >>> > -jg >>> > >>> > Nigel Daley wrote: >>> >> >>> >> On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: >>> >> >>> It's also ready to run on MapReduce and HDFS but we won't turn it on >>> until these projects build and test cleanly. Looks like both these >>> projects >>> currently have test failures. >>> >>> >>> >>> Assuming the projects are compiling and building, is there a reason to >>> >>> not turn it on despite the test failures? Hudson is invaluable to >>> developers >>> >>> who then don't have to run the tests and test-patch themselves. We >>> didn't >>> >>> turn Hudson off when it was working previously and there were known >>> >>> failures. I think one of the reasons we have more failing tests now is >>> the >>> >>> higher cost of doing Hudson's work (not a great excuse I know). This >>> is >>> >>> particularly true now because several of the failing tests involve >>> tests >>> >>> timing out, making the whole testing regime even longer. >>> >> >>> >> Every single patch would get a -1 and need investigation. Currently, >>> that >>> >> would be about 83 investigations between MR and HDFS issues that are in >>> >> patch available state. Shouldn't we focus on getting these tests fixed >>> or >>> >> removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS as >>> >> well) before I turn this on. >>> >> >>> >> Cheers, >>> >> Nige >>> > >>> > >>> >> >> >> >> -- >> Connect to me at http://www.facebook.com/dhruba >> >
Re: Patch testing
> If HDFS is added to the test-patch queue right now we get > nothing but dozens of -1'ed patches. There aren't dozens of patches being submitted currently. The -1 isn't the important thing, it's the grunt work of actually running (and waiting) for the tests, test-patch, etc. that Hudson does so that the developer doesn't have to. On Fri, Dec 17, 2010 at 3:48 PM, Dhruba Borthakur wrote: > +1, thanks for doing this. > > On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan wrote: > >> So, with test-patch updated to show the failing tests, saving the >> developers the need to go and verify that the failed tests are all >> known, how do people feel about turning on test-patch again for HDFS >> and mapred? I think it'll help prevent any more tests from entering >> the "yeah, we know" category. >> >> Thanks, >> jg >> >> >> On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan wrote: >> > True, each patch would get a -1 and the failing tests would need to be >> > verified as those known bad (BTW, it would be great if Hudson could list >> > which tests failed in the message it posts to JIRA). But that's still >> quite >> > a bit less error-prone work than if the developer runs the tests and >> > test-patch themselves. Also, with 22 being cut, there are a lot of >> patches >> > up in the air and several developers are juggling multiple patches. The >> > more automation we can have, even if it's not perfect, will decrease >> errors >> > we may make. >> > -jg >> > >> > Nigel Daley wrote: >> >> >> >> On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: >> >> >> It's also ready to run on MapReduce and HDFS but we won't turn it on >> until these projects build and test cleanly. Looks like both these >> projects >> currently have test failures. >> >>> >> >>> Assuming the projects are compiling and building, is there a reason to >> >>> not turn it on despite the test failures? Hudson is invaluable to >> developers >> >>> who then don't have to run the tests and test-patch themselves. We >> didn't >> >>> turn Hudson off when it was working previously and there were known >> >>> failures. I think one of the reasons we have more failing tests now is >> the >> >>> higher cost of doing Hudson's work (not a great excuse I know). This >> is >> >>> particularly true now because several of the failing tests involve >> tests >> >>> timing out, making the whole testing regime even longer. >> >> >> >> Every single patch would get a -1 and need investigation. Currently, >> that >> >> would be about 83 investigations between MR and HDFS issues that are in >> >> patch available state. Shouldn't we focus on getting these tests fixed >> or >> >> removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS as >> >> well) before I turn this on. >> >> >> >> Cheers, >> >> Nige >> > >> > >> > > > > -- > Connect to me at http://www.facebook.com/dhruba >
Re: Patch testing
+1, thanks for doing this. On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan wrote: > So, with test-patch updated to show the failing tests, saving the > developers the need to go and verify that the failed tests are all > known, how do people feel about turning on test-patch again for HDFS > and mapred? I think it'll help prevent any more tests from entering > the "yeah, we know" category. > > Thanks, > jg > > > On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan wrote: > > True, each patch would get a -1 and the failing tests would need to be > > verified as those known bad (BTW, it would be great if Hudson could list > > which tests failed in the message it posts to JIRA). But that's still > quite > > a bit less error-prone work than if the developer runs the tests and > > test-patch themselves. Also, with 22 being cut, there are a lot of > patches > > up in the air and several developers are juggling multiple patches. The > > more automation we can have, even if it's not perfect, will decrease > errors > > we may make. > > -jg > > > > Nigel Daley wrote: > >> > >> On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: > >> > It's also ready to run on MapReduce and HDFS but we won't turn it on > until these projects build and test cleanly. Looks like both these > projects > currently have test failures. > >>> > >>> Assuming the projects are compiling and building, is there a reason to > >>> not turn it on despite the test failures? Hudson is invaluable to > developers > >>> who then don't have to run the tests and test-patch themselves. We > didn't > >>> turn Hudson off when it was working previously and there were known > >>> failures. I think one of the reasons we have more failing tests now is > the > >>> higher cost of doing Hudson's work (not a great excuse I know). This > is > >>> particularly true now because several of the failing tests involve > tests > >>> timing out, making the whole testing regime even longer. > >> > >> Every single patch would get a -1 and need investigation. Currently, > that > >> would be about 83 investigations between MR and HDFS issues that are in > >> patch available state. Shouldn't we focus on getting these tests fixed > or > >> removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS as > >> well) before I turn this on. > >> > >> Cheers, > >> Nige > > > > > -- Connect to me at http://www.facebook.com/dhruba
Re: Patch testing
I do believe that it makes sense to wait a bit longer before doing this. If HDFS is added to the test-patch queue right now we get nothing but dozens of -1'ed patches. Number of failing tests has been reduced from 16 down to 4. Of those for 1 is a real bug (reviled by HDFS-903) and other three (all of the same nature) are needed to be investigated more. I'm for fixing the trunk first. On Fri, Dec 17, 2010 at 15:19, Jakob Homan wrote: > So, with test-patch updated to show the failing tests, saving the > developers the need to go and verify that the failed tests are all > known, how do people feel about turning on test-patch again for HDFS > and mapred? I think it'll help prevent any more tests from entering > the "yeah, we know" category. > > Thanks, > jg > > > On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan wrote: >> True, each patch would get a -1 and the failing tests would need to be >> verified as those known bad (BTW, it would be great if Hudson could list >> which tests failed in the message it posts to JIRA). But that's still quite >> a bit less error-prone work than if the developer runs the tests and >> test-patch themselves. Also, with 22 being cut, there are a lot of patches >> up in the air and several developers are juggling multiple patches. The >> more automation we can have, even if it's not perfect, will decrease errors >> we may make. >> -jg >> >> Nigel Daley wrote: >>> >>> On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: >>> > It's also ready to run on MapReduce and HDFS but we won't turn it on > until these projects build and test cleanly. Looks like both these > projects > currently have test failures. Assuming the projects are compiling and building, is there a reason to not turn it on despite the test failures? Hudson is invaluable to developers who then don't have to run the tests and test-patch themselves. We didn't turn Hudson off when it was working previously and there were known failures. I think one of the reasons we have more failing tests now is the higher cost of doing Hudson's work (not a great excuse I know). This is particularly true now because several of the failing tests involve tests timing out, making the whole testing regime even longer. >>> >>> Every single patch would get a -1 and need investigation. Currently, that >>> would be about 83 investigations between MR and HDFS issues that are in >>> patch available state. Shouldn't we focus on getting these tests fixed or >>> removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS as >>> well) before I turn this on. >>> >>> Cheers, >>> Nige >> >> >
Re: Patch testing
Doh. Forgot to include a shout-out to Nigel for adding the failing-tests-list to test-patch. Thanks! On Fri, Dec 17, 2010 at 3:23 PM, Eli Collins wrote: > +1 > > I think all the known failing tests should block the release as well. > > Thanks, > Eli > > On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan wrote: >> So, with test-patch updated to show the failing tests, saving the >> developers the need to go and verify that the failed tests are all >> known, how do people feel about turning on test-patch again for HDFS >> and mapred? I think it'll help prevent any more tests from entering >> the "yeah, we know" category. >> >> Thanks, >> jg >> >> >> On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan wrote: >>> True, each patch would get a -1 and the failing tests would need to be >>> verified as those known bad (BTW, it would be great if Hudson could list >>> which tests failed in the message it posts to JIRA). But that's still quite >>> a bit less error-prone work than if the developer runs the tests and >>> test-patch themselves. Also, with 22 being cut, there are a lot of patches >>> up in the air and several developers are juggling multiple patches. The >>> more automation we can have, even if it's not perfect, will decrease errors >>> we may make. >>> -jg >>> >>> Nigel Daley wrote: On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: >> It's also ready to run on MapReduce and HDFS but we won't turn it on >> until these projects build and test cleanly. Looks like both these >> projects >> currently have test failures. > > Assuming the projects are compiling and building, is there a reason to > not turn it on despite the test failures? Hudson is invaluable to > developers > who then don't have to run the tests and test-patch themselves. We didn't > turn Hudson off when it was working previously and there were known > failures. I think one of the reasons we have more failing tests now is > the > higher cost of doing Hudson's work (not a great excuse I know). This is > particularly true now because several of the failing tests involve tests > timing out, making the whole testing regime even longer. Every single patch would get a -1 and need investigation. Currently, that would be about 83 investigations between MR and HDFS issues that are in patch available state. Shouldn't we focus on getting these tests fixed or removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS as well) before I turn this on. Cheers, Nige >>> >>> >> >
Re: Patch testing
+1 I think all the known failing tests should block the release as well. Thanks, Eli On Fri, Dec 17, 2010 at 3:19 PM, Jakob Homan wrote: > So, with test-patch updated to show the failing tests, saving the > developers the need to go and verify that the failed tests are all > known, how do people feel about turning on test-patch again for HDFS > and mapred? I think it'll help prevent any more tests from entering > the "yeah, we know" category. > > Thanks, > jg > > > On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan wrote: >> True, each patch would get a -1 and the failing tests would need to be >> verified as those known bad (BTW, it would be great if Hudson could list >> which tests failed in the message it posts to JIRA). But that's still quite >> a bit less error-prone work than if the developer runs the tests and >> test-patch themselves. Also, with 22 being cut, there are a lot of patches >> up in the air and several developers are juggling multiple patches. The >> more automation we can have, even if it's not perfect, will decrease errors >> we may make. >> -jg >> >> Nigel Daley wrote: >>> >>> On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: >>> > It's also ready to run on MapReduce and HDFS but we won't turn it on > until these projects build and test cleanly. Looks like both these > projects > currently have test failures. Assuming the projects are compiling and building, is there a reason to not turn it on despite the test failures? Hudson is invaluable to developers who then don't have to run the tests and test-patch themselves. We didn't turn Hudson off when it was working previously and there were known failures. I think one of the reasons we have more failing tests now is the higher cost of doing Hudson's work (not a great excuse I know). This is particularly true now because several of the failing tests involve tests timing out, making the whole testing regime even longer. >>> >>> Every single patch would get a -1 and need investigation. Currently, that >>> would be about 83 investigations between MR and HDFS issues that are in >>> patch available state. Shouldn't we focus on getting these tests fixed or >>> removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS as >>> well) before I turn this on. >>> >>> Cheers, >>> Nige >> >> >
Re: Patch testing
So, with test-patch updated to show the failing tests, saving the developers the need to go and verify that the failed tests are all known, how do people feel about turning on test-patch again for HDFS and mapred? I think it'll help prevent any more tests from entering the "yeah, we know" category. Thanks, jg On Wed, Nov 17, 2010 at 5:08 PM, Jakob Homan wrote: > True, each patch would get a -1 and the failing tests would need to be > verified as those known bad (BTW, it would be great if Hudson could list > which tests failed in the message it posts to JIRA). But that's still quite > a bit less error-prone work than if the developer runs the tests and > test-patch themselves. Also, with 22 being cut, there are a lot of patches > up in the air and several developers are juggling multiple patches. The > more automation we can have, even if it's not perfect, will decrease errors > we may make. > -jg > > Nigel Daley wrote: >> >> On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: >> It's also ready to run on MapReduce and HDFS but we won't turn it on until these projects build and test cleanly. Looks like both these projects currently have test failures. >>> >>> Assuming the projects are compiling and building, is there a reason to >>> not turn it on despite the test failures? Hudson is invaluable to developers >>> who then don't have to run the tests and test-patch themselves. We didn't >>> turn Hudson off when it was working previously and there were known >>> failures. I think one of the reasons we have more failing tests now is the >>> higher cost of doing Hudson's work (not a great excuse I know). This is >>> particularly true now because several of the failing tests involve tests >>> timing out, making the whole testing regime even longer. >> >> Every single patch would get a -1 and need investigation. Currently, that >> would be about 83 investigations between MR and HDFS issues that are in >> patch available state. Shouldn't we focus on getting these tests fixed or >> removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS as >> well) before I turn this on. >> >> Cheers, >> Nige > >
Re: Patch testing
True, each patch would get a -1 and the failing tests would need to be verified as those known bad (BTW, it would be great if Hudson could list which tests failed in the message it posts to JIRA). But that's still quite a bit less error-prone work than if the developer runs the tests and test-patch themselves. Also, with 22 being cut, there are a lot of patches up in the air and several developers are juggling multiple patches. The more automation we can have, even if it's not perfect, will decrease errors we may make. -jg Nigel Daley wrote: On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: It's also ready to run on MapReduce and HDFS but we won't turn it on until these projects build and test cleanly. Looks like both these projects currently have test failures. Assuming the projects are compiling and building, is there a reason to not turn it on despite the test failures? Hudson is invaluable to developers who then don't have to run the tests and test-patch themselves. We didn't turn Hudson off when it was working previously and there were known failures. I think one of the reasons we have more failing tests now is the higher cost of doing Hudson's work (not a great excuse I know). This is particularly true now because several of the failing tests involve tests timing out, making the whole testing regime even longer. Every single patch would get a -1 and need investigation. Currently, that would be about 83 investigations between MR and HDFS issues that are in patch available state. Shouldn't we focus on getting these tests fixed or removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS as well) before I turn this on. Cheers, Nige
Re: Patch testing
On Nov 17, 2010, at 3:11 PM, Jakob Homan wrote: > > It's also ready to run on MapReduce and HDFS but we won't turn it on until > > these projects build and test cleanly. Looks like both these projects > > currently have test failures. > > Assuming the projects are compiling and building, is there a reason to not > turn it on despite the test failures? Hudson is invaluable to developers who > then don't have to run the tests and test-patch themselves. We didn't turn > Hudson off when it was working previously and there were known failures. I > think one of the reasons we have more failing tests now is the higher cost of > doing Hudson's work (not a great excuse I know). This is particularly true > now because several of the failing tests involve tests timing out, making the > whole testing regime even longer. Every single patch would get a -1 and need investigation. Currently, that would be about 83 investigations between MR and HDFS issues that are in patch available state. Shouldn't we focus on getting these tests fixed or removed/? Also, I need to get MAPREDUCE-2172 fixed (applies to HDFS as well) before I turn this on. Cheers, Nige
Re: Patch testing
> It's also ready to run on MapReduce and HDFS but we won't turn it on until these projects build and test cleanly. Looks like both these projects currently have test failures. Assuming the projects are compiling and building, is there a reason to not turn it on despite the test failures? Hudson is invaluable to developers who then don't have to run the tests and test-patch themselves. We didn't turn Hudson off when it was working previously and there were known failures. I think one of the reasons we have more failing tests now is the higher cost of doing Hudson's work (not a great excuse I know). This is particularly true now because several of the failing tests involve tests timing out, making the whole testing regime even longer. -Jakob
Re: Patch testing
Here are the jiras for hdfs test failures on trunk: HDFS-1306 TestFileAppend4 fails HDFS-1503 TestSaveNamespace fails when FI is enabled HDFS-1206 TestFiHFlush fails intermittently HDFS-1496 TestStorageRestore is failing after HDFS-903 fix HDFS-1502 TestBlockRecovery triggers NPE in assert HDFS-613 TestBalancer and TestBlockTokenWithDFS fail Balancer assert On Tue, Nov 16, 2010 at 10:46 PM, Nigel Daley wrote: > Just to follow up, the pre-commit patch testing is functioning again on > Zookeeper and Hadoop Common. It's also ready to run on MapReduce and HDFS > but we won't turn it on until these projects build and test cleanly. Looks > like both these projects currently have test failures. Are there Jira's to > fix these tests? > > New instructions for re-trigger testing of a patch are at > https://hudson.apache.org/hudson/job/PreCommit-Admin/ > > Cheers, > Nige > > On Oct 20, 2010, at 1:30 PM, Nigel Daley wrote: > >> Thanks Giri. >> >>> Longterm solution: >>> email patch process and queue creation is un-reliable. As a long term >>> solution we need to use jira-cli command line tool to read patch directly >>> from jira. >> >> Ya, that's what I've got mostly working on my machine. Any open Jira's for >> this work? If not, I'll open one. >> >>> And doing this would require introducing a new workflow in jira. Apart from >>> the current status called "patch-available" >> >> Not sure that's necessary. I think if we change the patch testing model a >> little, we can keep the current workflow. Also, given some recent advances >> in Hudson, I think we can reduce all the patch jobs down to 1 admin job >> covering *all* projects and 1 job per project -- and we should be able to >> drastically improve utilization of the slaves we have. Let's move this >> discussion to a ticket. LMK if I need to open one. >> >> Cheers, >> Nige >> >> On Oct 20, 2010, at 1:17 PM, Giridharan Kesavan wrote: >> >>> >>> Old times: >>> The hudson patch-testing admin job used to run on the master machine >>> hudson.zones.apache.org as this machine used to parse the emails from jira >>> and create the patch queue for testing. >>> >>> Current situation: >>> hudson.zones.apache.org machine is no more a master and we have a new >>> machine called aegis.apache.org as the master machine. >>> >>> This machine is configured to process the email and create the patch queue. >>> >>> Though we are able to get the email and create the patch queue on the new >>> aegis.apache.org machine, still we wont be able to trigger patch admin job >>> on the hudson master as this is not allowed in the new setup. >>> >>> Infra team doesn't want us to run any builds on the hudson master. Instead >>> they want all the builds to run on the slave machine. >>> >>> I got the password-less access for hudson user from hudson slave >>> h1.grid.sp2.yahoo.net to aegis.apache.org like 2 weeks back. >>> >>> The plan here is to read the patch queue on aegis from h1 and schedule >>> builds. >>> >>> Longterm solution: >>> email patch process and queue creation is un-reliable. As a long term >>> solution we need to use jira-cli command line tool to read patch directly >>> from jira. >>> And doing this would require introducing a new workflow in jira. Apart from >>> the current status called "patch-available" >>> >>> Thanks, >>> Giri >>> >>> On Oct 20, 2010, at 12:54 PM, Konstantin Boudnik wrote: >>> Thanks for getting to it, Nigel. This is absolutely useful and allows to keep weeds out of the trunk up to a certain degree. There were at least two slightly different sources of information about what's going on with test-patch process. Would you mind to post a brief about the situation so comments can be more substantial? Thanks, Cos On Wed, Oct 20, 2010 at 12:47PM, Nigel Daley wrote: > Folks, > > I'm working to get the pre-commit patch testing running again for HDFS, > HADOOP, and MAPREDUCE patches. Since I've been on a bit of a hiatus from > day-to-day involvement w/ the project over the last 6 months (new job), I > want to check in and make sure folks would still find this pre-commit > testing useful. Also, happy to hear any suggested improvement. Let me > know. > > Cheers, > Nige >>> >> > >
Re: Patch testing
On Tue, Nov 16, 2010 at 10:46PM, Nigel Daley wrote: > Just to follow up, the pre-commit patch testing is functioning again on > Zookeeper and Hadoop Common. It's also ready to run on MapReduce and HDFS > but we won't turn it on until these projects build and test cleanly. Looks > like both these projects currently have test failures. Are there Jira's to > fix these tests? In HDFS some of the tests are covered by JIRAs for sure. I haven't checked MR lately but it should be the same there. > New instructions for re-trigger testing of a patch are at > https://hudson.apache.org/hudson/job/PreCommit-Admin/ > > Cheers, > Nige > > On Oct 20, 2010, at 1:30 PM, Nigel Daley wrote: > > > Thanks Giri. > > > >> Longterm solution: > >> email patch process and queue creation is un-reliable. As a long term > >> solution we need to use jira-cli command line tool to read patch directly > >> from jira. > > > > Ya, that's what I've got mostly working on my machine. Any open Jira's for > > this work? If not, I'll open one. > > > >> And doing this would require introducing a new workflow in jira. Apart > >> from the current status called "patch-available" > > > > Not sure that's necessary. I think if we change the patch testing model a > > little, we can keep the current workflow. Also, given some recent advances > > in Hudson, I think we can reduce all the patch jobs down to 1 admin job > > covering *all* projects and 1 job per project -- and we should be able to > > drastically improve utilization of the slaves we have. Let's move this > > discussion to a ticket. LMK if I need to open one. > > > > Cheers, > > Nige > > > > On Oct 20, 2010, at 1:17 PM, Giridharan Kesavan wrote: > > > >> > >> Old times: > >> The hudson patch-testing admin job used to run on the master machine > >> hudson.zones.apache.org as this machine used to parse the emails from jira > >> and create the patch queue for testing. > >> > >> Current situation: > >> hudson.zones.apache.org machine is no more a master and we have a new > >> machine called aegis.apache.org as the master machine. > >> > >> This machine is configured to process the email and create the patch > >> queue. > >> Though we are able to get the email and create the patch queue on the new > >> aegis.apache.org machine, still we wont be able to trigger patch admin job > >> on the hudson master as this is not allowed in the new setup. > >> > >> Infra team doesn't want us to run any builds on the hudson master. Instead > >> they want all the builds to run on the slave machine. > >> > >> I got the password-less access for hudson user from hudson slave > >> h1.grid.sp2.yahoo.net to aegis.apache.org like 2 weeks back. > >> > >> The plan here is to read the patch queue on aegis from h1 and schedule > >> builds. > >> > >> Longterm solution: > >> email patch process and queue creation is un-reliable. As a long term > >> solution we need to use jira-cli command line tool to read patch directly > >> from jira. > >> And doing this would require introducing a new workflow in jira. Apart > >> from the current status called "patch-available" > >> > >> Thanks, > >> Giri > >> > >> On Oct 20, 2010, at 12:54 PM, Konstantin Boudnik wrote: > >> > >>> Thanks for getting to it, Nigel. This is absolutely useful and allows to > >>> keep > >>> weeds out of the trunk up to a certain degree. > >>> > >>> There were at least two slightly different sources of information about > >>> what's > >>> going on with test-patch process. Would you mind to post a brief about the > >>> situation so comments can be more substantial? > >>> > >>> Thanks, > >>> Cos > >>> > >>> On Wed, Oct 20, 2010 at 12:47PM, Nigel Daley wrote: > Folks, > > I'm working to get the pre-commit patch testing running again for HDFS, > HADOOP, and MAPREDUCE patches. Since I've been on a bit of a hiatus from > day-to-day involvement w/ the project over the last 6 months (new job), I > want to check in and make sure folks would still find this pre-commit > testing useful. Also, happy to hear any suggested improvement. Let me > know. > > Cheers, > Nige > >> > > > signature.asc Description: Digital signature
Re: Patch testing
Just to follow up, the pre-commit patch testing is functioning again on Zookeeper and Hadoop Common. It's also ready to run on MapReduce and HDFS but we won't turn it on until these projects build and test cleanly. Looks like both these projects currently have test failures. Are there Jira's to fix these tests? New instructions for re-trigger testing of a patch are at https://hudson.apache.org/hudson/job/PreCommit-Admin/ Cheers, Nige On Oct 20, 2010, at 1:30 PM, Nigel Daley wrote: > Thanks Giri. > >> Longterm solution: >> email patch process and queue creation is un-reliable. As a long term >> solution we need to use jira-cli command line tool to read patch directly >> from jira. > > Ya, that's what I've got mostly working on my machine. Any open Jira's for > this work? If not, I'll open one. > >> And doing this would require introducing a new workflow in jira. Apart from >> the current status called "patch-available" > > Not sure that's necessary. I think if we change the patch testing model a > little, we can keep the current workflow. Also, given some recent advances > in Hudson, I think we can reduce all the patch jobs down to 1 admin job > covering *all* projects and 1 job per project -- and we should be able to > drastically improve utilization of the slaves we have. Let's move this > discussion to a ticket. LMK if I need to open one. > > Cheers, > Nige > > On Oct 20, 2010, at 1:17 PM, Giridharan Kesavan wrote: > >> >> Old times: >> The hudson patch-testing admin job used to run on the master machine >> hudson.zones.apache.org as this machine used to parse the emails from jira >> and create the patch queue for testing. >> >> Current situation: >> hudson.zones.apache.org machine is no more a master and we have a new >> machine called aegis.apache.org as the master machine. >> >> This machine is configured to process the email and create the patch queue. >> >> Though we are able to get the email and create the patch queue on the new >> aegis.apache.org machine, still we wont be able to trigger patch admin job >> on the hudson master as this is not allowed in the new setup. >> >> Infra team doesn't want us to run any builds on the hudson master. Instead >> they want all the builds to run on the slave machine. >> >> I got the password-less access for hudson user from hudson slave >> h1.grid.sp2.yahoo.net to aegis.apache.org like 2 weeks back. >> >> The plan here is to read the patch queue on aegis from h1 and schedule >> builds. >> >> Longterm solution: >> email patch process and queue creation is un-reliable. As a long term >> solution we need to use jira-cli command line tool to read patch directly >> from jira. >> And doing this would require introducing a new workflow in jira. Apart from >> the current status called "patch-available" >> >> Thanks, >> Giri >> >> On Oct 20, 2010, at 12:54 PM, Konstantin Boudnik wrote: >> >>> Thanks for getting to it, Nigel. This is absolutely useful and allows to >>> keep >>> weeds out of the trunk up to a certain degree. >>> >>> There were at least two slightly different sources of information about >>> what's >>> going on with test-patch process. Would you mind to post a brief about the >>> situation so comments can be more substantial? >>> >>> Thanks, >>> Cos >>> >>> On Wed, Oct 20, 2010 at 12:47PM, Nigel Daley wrote: Folks, I'm working to get the pre-commit patch testing running again for HDFS, HADOOP, and MAPREDUCE patches. Since I've been on a bit of a hiatus from day-to-day involvement w/ the project over the last 6 months (new job), I want to check in and make sure folks would still find this pre-commit testing useful. Also, happy to hear any suggested improvement. Let me know. Cheers, Nige >> >
Re: Patch testing
Yes! Thanks Nigel. On Wed, Oct 20, 2010 at 12:47 PM, Nigel Daley wrote: > Folks, > > I'm working to get the pre-commit patch testing running again for HDFS, > HADOOP, and MAPREDUCE patches. Since I've been on a bit of a hiatus from > day-to-day involvement w/ the project over the last 6 months (new job), I > want to check in and make sure folks would still find this pre-commit testing > useful. Also, happy to hear any suggested improvement. Let me know. > > Cheers, > Nige >
Re: Patch testing
Awesome Nigel! That will be of real help. thanks, dhruba On Wed, Oct 20, 2010 at 12:47 PM, Nigel Daley wrote: > Folks, > > I'm working to get the pre-commit patch testing running again for HDFS, > HADOOP, and MAPREDUCE patches. Since I've been on a bit of a hiatus from > day-to-day involvement w/ the project over the last 6 months (new job), I > want to check in and make sure folks would still find this pre-commit > testing useful. Also, happy to hear any suggested improvement. Let me > know. > > Cheers, > Nige > -- Connect to me at http://www.facebook.com/dhruba
Re: Patch testing
Actually, there's a way to do it, but is isn't simple as you said and require certain JIRA management discipline like linking JIRAs properly, for instance. Another veritable here is to submit patches for all involved project at once or to be able to postpone a patch validation if patch-A isn't available when patch-B is submitted (project B depends on A, obviously). After that everything else is real easy: build dependencies, mvn-install artifacts, build the dependent one with locally installed artifacts, clean mvn cache after all. The tricky part though is to guarantee that these chained builds are executed on the same machine. Otherwise, local mvn repo synchronization will be one huge mess ;) So, in other words, let's restore the status quo first ;) IMO Cos On Wed, Oct 20, 2010 at 04:37PM, Aaron Myers wrote: > One feature request, which may be difficult to implement, and may not be > worth it: > > It would be nice to somehow support running tests for patches which require > changes that span core/MR/HDFS. As it stands, I believe an HDFS patch will > be built and run against a trunk jar, but if this HDFS patch requires some > not-yet-committed work that's being done in a HADOOP JIRA, the HDFS tests > will fail. > > Obviously, just getting hudson QA running again is a necessary pre-requisite > for this. :) > > Aaron signature.asc Description: Digital signature
Re: Patch testing
I ve just filed this jira for discussions about patch-testing. https://issues.apache.org/jira/browse/HADOOP-7003 Thanks, -Giri On Oct 20, 2010, at 1:30 PM, Nigel Daley wrote: > Thanks Giri. > >> Longterm solution: >> email patch process and queue creation is un-reliable. As a long term >> solution we need to use jira-cli command line tool to read patch directly >> from jira. > > Ya, that's what I've got mostly working on my machine. Any open Jira's for > this work? If not, I'll open one. > >> And doing this would require introducing a new workflow in jira. Apart from >> the current status called "patch-available" > > Not sure that's necessary. I think if we change the patch testing model a > little, we can keep the current workflow. Also, given some recent advances > in Hudson, I think we can reduce all the patch jobs down to 1 admin job > covering *all* projects and 1 job per project -- and we should be able to > drastically improve utilization of the slaves we have. Let's move this > discussion to a ticket. LMK if I need to open one. > > Cheers, > Nige > > On Oct 20, 2010, at 1:17 PM, Giridharan Kesavan wrote: > >> >> Old times: >> The hudson patch-testing admin job used to run on the master machine >> hudson.zones.apache.org as this machine used to parse the emails from jira >> and create the patch queue for testing. >> >> Current situation: >> hudson.zones.apache.org machine is no more a master and we have a new >> machine called aegis.apache.org as the master machine. >> >> This machine is configured to process the email and create the patch queue. >> >> Though we are able to get the email and create the patch queue on the new >> aegis.apache.org machine, still we wont be able to trigger patch admin job >> on the hudson master as this is not allowed in the new setup. >> >> Infra team doesn't want us to run any builds on the hudson master. Instead >> they want all the builds to run on the slave machine. >> >> I got the password-less access for hudson user from hudson slave >> h1.grid.sp2.yahoo.net to aegis.apache.org like 2 weeks back. >> >> The plan here is to read the patch queue on aegis from h1 and schedule >> builds. >> >> Longterm solution: >> email patch process and queue creation is un-reliable. As a long term >> solution we need to use jira-cli command line tool to read patch directly >> from jira. >> And doing this would require introducing a new workflow in jira. Apart from >> the current status called "patch-available" >> >> Thanks, >> Giri >> >> On Oct 20, 2010, at 12:54 PM, Konstantin Boudnik wrote: >> >>> Thanks for getting to it, Nigel. This is absolutely useful and allows to >>> keep >>> weeds out of the trunk up to a certain degree. >>> >>> There were at least two slightly different sources of information about >>> what's >>> going on with test-patch process. Would you mind to post a brief about the >>> situation so comments can be more substantial? >>> >>> Thanks, >>> Cos >>> >>> On Wed, Oct 20, 2010 at 12:47PM, Nigel Daley wrote: Folks, I'm working to get the pre-commit patch testing running again for HDFS, HADOOP, and MAPREDUCE patches. Since I've been on a bit of a hiatus from day-to-day involvement w/ the project over the last 6 months (new job), I want to check in and make sure folks would still find this pre-commit testing useful. Also, happy to hear any suggested improvement. Let me know. Cheers, Nige >> >
Re: Patch testing
One feature request, which may be difficult to implement, and may not be worth it: It would be nice to somehow support running tests for patches which require changes that span core/MR/HDFS. As it stands, I believe an HDFS patch will be built and run against a trunk jar, but if this HDFS patch requires some not-yet-committed work that's being done in a HADOOP JIRA, the HDFS tests will fail. Obviously, just getting hudson QA running again is a necessary pre-requisite for this. :) Aaron
Re: Patch testing
I think ZK, PIG, etc will also be included in the update I'm working on. Cheers, Nige On Oct 20, 2010, at 1:27 PM, Mahadev Konar wrote: > Great. > > Nigel, > Can you please document in somewhere on how you fixed it? We¹d like to fix > it for ZooKeeper as well. > > Thanks > mahadev > > > On 10/20/10 1:23 PM, "Owen O'Malley" wrote: > >> >> >> On Oct 20, 2010, at 12:47 PM, Nigel Daley wrote: >> >>> I'm working to get the pre-commit patch testing running again for >>> HDFS, HADOOP, and MAPREDUCE patches. >> >> That would be great, Nigel. >> >> Thanks, >>Owen >> >
Re: Patch testing
Thanks Giri! I'm totally agree that we need to move away from the email triggering. And in order to make changes to JIRA's workflow someone needs to work with INFRA folks or a person with admin access should suffice? I'm offering my help for either way, so lemme know. Cos On Wed, Oct 20, 2010 at 01:17PM, Giridharan Kesavan wrote: > > Old times: > The hudson patch-testing admin job used to run on the master machine > hudson.zones.apache.org as this machine used to parse the emails from jira > and create the patch queue for testing. > > Current situation: > hudson.zones.apache.org machine is no more a master and we have a new > machine called aegis.apache.org as the master machine. > > This machine is configured to process the email and create the patch queue. > > Though we are able to get the email and create the patch queue on the new > aegis.apache.org machine, still we wont be able to trigger patch admin job > on the hudson master as this is not allowed in the new setup. > > Infra team doesn't want us to run any builds on the hudson master. Instead > they want all the builds to run on the slave machine. > > I got the password-less access for hudson user from hudson slave > h1.grid.sp2.yahoo.net to aegis.apache.org like 2 weeks back. > > The plan here is to read the patch queue on aegis from h1 and schedule > builds. > > Longterm solution: > email patch process and queue creation is un-reliable. As a long term > solution we need to use jira-cli command line tool to read patch directly > from jira. And doing this would require introducing a new workflow in jira. > Apart from the current status called "patch-available" > > Thanks, > Giri > > On Oct 20, 2010, at 12:54 PM, Konstantin Boudnik wrote: > > > Thanks for getting to it, Nigel. This is absolutely useful and allows to > > keep > > weeds out of the trunk up to a certain degree. > > > > There were at least two slightly different sources of information about > > what's > > going on with test-patch process. Would you mind to post a brief about the > > situation so comments can be more substantial? > > > > Thanks, > > Cos > > > > On Wed, Oct 20, 2010 at 12:47PM, Nigel Daley wrote: > >> Folks, > >> > >> I'm working to get the pre-commit patch testing running again for HDFS, > >> HADOOP, and MAPREDUCE patches. Since I've been on a bit of a hiatus from > >> day-to-day involvement w/ the project over the last 6 months (new job), I > >> want to check in and make sure folks would still find this pre-commit > >> testing useful. Also, happy to hear any suggested improvement. Let me > >> know. > >> > >> Cheers, > >> Nige > signature.asc Description: Digital signature
Re: Patch testing
Thanks Giri. > Longterm solution: > email patch process and queue creation is un-reliable. As a long term > solution we need to use jira-cli command line tool to read patch directly > from jira. Ya, that's what I've got mostly working on my machine. Any open Jira's for this work? If not, I'll open one. > And doing this would require introducing a new workflow in jira. Apart from > the current status called "patch-available" Not sure that's necessary. I think if we change the patch testing model a little, we can keep the current workflow. Also, given some recent advances in Hudson, I think we can reduce all the patch jobs down to 1 admin job covering *all* projects and 1 job per project -- and we should be able to drastically improve utilization of the slaves we have. Let's move this discussion to a ticket. LMK if I need to open one. Cheers, Nige On Oct 20, 2010, at 1:17 PM, Giridharan Kesavan wrote: > > Old times: > The hudson patch-testing admin job used to run on the master machine > hudson.zones.apache.org as this machine used to parse the emails from jira > and create the patch queue for testing. > > Current situation: > hudson.zones.apache.org machine is no more a master and we have a new machine > called aegis.apache.org as the master machine. > > This machine is configured to process the email and create the patch queue. > > Though we are able to get the email and create the patch queue on the new > aegis.apache.org machine, still we wont be able to trigger patch admin job on > the hudson master as this is not allowed in the new setup. > > Infra team doesn't want us to run any builds on the hudson master. Instead > they want all the builds to run on the slave machine. > > I got the password-less access for hudson user from hudson slave > h1.grid.sp2.yahoo.net to aegis.apache.org like 2 weeks back. > > The plan here is to read the patch queue on aegis from h1 and schedule > builds. > > Longterm solution: > email patch process and queue creation is un-reliable. As a long term > solution we need to use jira-cli command line tool to read patch directly > from jira. > And doing this would require introducing a new workflow in jira. Apart from > the current status called "patch-available" > > Thanks, > Giri > > On Oct 20, 2010, at 12:54 PM, Konstantin Boudnik wrote: > >> Thanks for getting to it, Nigel. This is absolutely useful and allows to keep >> weeds out of the trunk up to a certain degree. >> >> There were at least two slightly different sources of information about >> what's >> going on with test-patch process. Would you mind to post a brief about the >> situation so comments can be more substantial? >> >> Thanks, >> Cos >> >> On Wed, Oct 20, 2010 at 12:47PM, Nigel Daley wrote: >>> Folks, >>> >>> I'm working to get the pre-commit patch testing running again for HDFS, >>> HADOOP, and MAPREDUCE patches. Since I've been on a bit of a hiatus from >>> day-to-day involvement w/ the project over the last 6 months (new job), I >>> want to check in and make sure folks would still find this pre-commit >>> testing useful. Also, happy to hear any suggested improvement. Let me >>> know. >>> >>> Cheers, >>> Nige >
Re: Patch testing
Great. Nigel, Can you please document in somewhere on how you fixed it? We¹d like to fix it for ZooKeeper as well. Thanks mahadev On 10/20/10 1:23 PM, "Owen O'Malley" wrote: > > > On Oct 20, 2010, at 12:47 PM, Nigel Daley wrote: > >> I'm working to get the pre-commit patch testing running again for >> HDFS, HADOOP, and MAPREDUCE patches. > > That would be great, Nigel. > > Thanks, > Owen >
Re: Patch testing
On Oct 20, 2010, at 12:47 PM, Nigel Daley wrote: I'm working to get the pre-commit patch testing running again for HDFS, HADOOP, and MAPREDUCE patches. That would be great, Nigel. Thanks, Owen
Re: Patch testing
Old times: The hudson patch-testing admin job used to run on the master machine hudson.zones.apache.org as this machine used to parse the emails from jira and create the patch queue for testing. Current situation: hudson.zones.apache.org machine is no more a master and we have a new machine called aegis.apache.org as the master machine. This machine is configured to process the email and create the patch queue. Though we are able to get the email and create the patch queue on the new aegis.apache.org machine, still we wont be able to trigger patch admin job on the hudson master as this is not allowed in the new setup. Infra team doesn't want us to run any builds on the hudson master. Instead they want all the builds to run on the slave machine. I got the password-less access for hudson user from hudson slave h1.grid.sp2.yahoo.net to aegis.apache.org like 2 weeks back. The plan here is to read the patch queue on aegis from h1 and schedule builds. Longterm solution: email patch process and queue creation is un-reliable. As a long term solution we need to use jira-cli command line tool to read patch directly from jira. And doing this would require introducing a new workflow in jira. Apart from the current status called "patch-available" Thanks, Giri On Oct 20, 2010, at 12:54 PM, Konstantin Boudnik wrote: > Thanks for getting to it, Nigel. This is absolutely useful and allows to keep > weeds out of the trunk up to a certain degree. > > There were at least two slightly different sources of information about what's > going on with test-patch process. Would you mind to post a brief about the > situation so comments can be more substantial? > > Thanks, > Cos > > On Wed, Oct 20, 2010 at 12:47PM, Nigel Daley wrote: >> Folks, >> >> I'm working to get the pre-commit patch testing running again for HDFS, >> HADOOP, and MAPREDUCE patches. Since I've been on a bit of a hiatus from >> day-to-day involvement w/ the project over the last 6 months (new job), I >> want to check in and make sure folks would still find this pre-commit >> testing useful. Also, happy to hear any suggested improvement. Let me >> know. >> >> Cheers, >> Nige
Re: Patch testing
Thanks for getting to it, Nigel. This is absolutely useful and allows to keep weeds out of the trunk up to a certain degree. There were at least two slightly different sources of information about what's going on with test-patch process. Would you mind to post a brief about the situation so comments can be more substantial? Thanks, Cos On Wed, Oct 20, 2010 at 12:47PM, Nigel Daley wrote: > Folks, > > I'm working to get the pre-commit patch testing running again for HDFS, > HADOOP, and MAPREDUCE patches. Since I've been on a bit of a hiatus from > day-to-day involvement w/ the project over the last 6 months (new job), I > want to check in and make sure folks would still find this pre-commit > testing useful. Also, happy to hear any suggested improvement. Let me > know. > > Cheers, > Nige signature.asc Description: Digital signature
Re: Patch testing
+1 There have been several tests failing on trunk for entirely too long, and I believe this will help prevent further regressions while we address those issues. Thanks a lot for volunteering to do this work, Nigel. Aaron On Wed, Oct 20, 2010 at 3:47 PM, Nigel Daley wrote: > Folks, > > I'm working to get the pre-commit patch testing running again for HDFS, > HADOOP, and MAPREDUCE patches. Since I've been on a bit of a hiatus from > day-to-day involvement w/ the project over the last 6 months (new job), I > want to check in and make sure folks would still find this pre-commit > testing useful. Also, happy to hear any suggested improvement. Let me > know. > > Cheers, > Nige >
