Assuming that applications built against 0.16.x should not have to be recompiled for 0.16.x+1, I have to assume that an intermittent bug has been introduced between hadoop 0.16.1 and 0.16.2 because I have run the HBase torture test 6 times against both 0.16.1 and 0.16.2. Hadoop 0.16.2 failed 3 out of 6 times. Hadoop 0.16.1 succeeded 6 out of 6 times.
If a recompile between 0.16.1 and 0.16.2 is required, then 0.16.2 gets a -1. If no recompile should be required, then 0.16.2 gets a -1 because it fails 50% of the time where 0.16.1 succeeded. --- Jim Kellerman, Senior Engineer; Powerset > -----Original Message----- > From: Jim Kellerman [mailto:[EMAIL PROTECTED] > Sent: Sunday, March 30, 2008 11:58 PM > To: [email protected] > Subject: RE: [VOTE] Release Hadoop 0.16.2 (candidate 0) > > Yes, it does appear to be intermittent. hbase-0.1.0 was built > against hadoop-0.16.0, so I assumed that there were no > incompatible changes. It just failed again with the exception: > > 2008-03-31 06:42:29,079 ERROR > [regionserver/0:0:0:0:0:0:0:0:8020.compactor] > hbase.HRegionServer$CompactSplitThread(267): Compaction > failed for region TestTable,0021017156,1206945717128 > java.lang.NullPointerException > at > org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClien > t.java:2262) > at > org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FS > DataOutputStream.java:51) > at > org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStre > am.java:67) > at > org.apache.hadoop.hbase.HStoreFile.writeSplitInfo(HStoreFile.java:267) > at > org.apache.hadoop.hbase.HStoreFile.writeReferenceFiles(HStoreF > ile.java:246) > at > org.apache.hadoop.hbase.HStoreFile.splitStoreFile(HStoreFile.java:239) > at > org.apache.hadoop.hbase.HRegion.splitRegion(HRegion.java:740) > at > org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.split > (HRegionServer.java:291) > at > org.apache.hadoop.hbase.HRegionServer$CompactSplitThread.run(H > RegionServer.java:255) > > and if you look at the source for DFSClient for 0.16.0, line 2262 is: > 2260: synchronized (dataQueue) { > 2261: if (response != null) { > 2262: response.close(); > 2263: } > > Obviously the code in DFSClient$DFSOutputStream has changed > considerably from 0.16.0 to 0.16.2. > > My assumption was that applications built against 0.16.x > should not have to be recompiled for 0.16.y. Is that not correct? > > --- > Jim Kellerman, Senior Engineer; Powerset > > > > -----Original Message----- > > From: Nigel Daley [mailto:[EMAIL PROTECTED] > > Sent: Sunday, March 30, 2008 11:20 PM > > To: [email protected] > > Subject: Re: [VOTE] Release Hadoop 0.16.2 (candidate 0) > > > > Perhaps it's an intermittent problem? Can you run a couple more > > times? > > > > On Mar 30, 2008, at 11:12 PM, Jim Kellerman wrote: > > > > > I did not recompile hbase-0.1.0 and it ran fine with hadoop-0.16. > > > [01]. I made two runs against 0.16.2 and they both failed > the same > > > way. > > > > > > However, I just re-ran against 0.16.2 and the test passed, > > so I must > > > have configured something incorrectly. > > > My bad!. > > > > > > +1 for 0.16.2 > > > > > > --- > > > Jim Kellerman, Senior Engineer; Powerset > > > > > > > > >> -----Original Message----- > > >> From: Nigel Daley [mailto:[EMAIL PROTECTED] > > >> Sent: Sunday, March 30, 2008 10:31 PM > > >> To: [email protected] > > >> Subject: Re: [VOTE] Release Hadoop 0.16.2 (candidate 0) > > >> > > >> Jim, did you recompile hbase with this new release? The > fact that > > >> the line numbers don't match means something is probably running > > >> using an old version. > > >> > > >> Nige > > >> > > >> On Mar 30, 2008, at 10:21 PM, Jim Kellerman wrote: > > >> > > >>> Yes, I am using candidate 0 which I downloaded from http:// > > >>> people.apache.org/~nigel/hadoop-0.16.2-candidate-0/ > > >>> > > >>> --- > > >>> Jim Kellerman, Senior Engineer; Powerset > > >>> > > >>> > > >>>> -----Original Message----- > > >>>> From: [EMAIL PROTECTED] > > >> [mailto:[EMAIL PROTECTED] > > >>>> Sent: Sunday, March 30, 2008 6:30 PM > > >>>> To: [email protected] > > >>>> Subject: Re: [VOTE] Release Hadoop 0.16.2 (candidate 0) > > >>>> > > >>>> Hi Jim, > > >>>> > > >>>> I checked the source codes. There is no way to get > > exceptions in > > >>>> these two line in DFSClient. Are you using candidate 0? > > >>>> 1506: private long bytesCurBlock = 0; // bytes writen in > > >>>> current block > > >>>> 2262: if (currentPacket != null) { > > >>>> > > >>>> Nicholas > > >>>> > > >>>> ----- Original Message ---- > > >>>> From: Jim Kellerman <[EMAIL PROTECTED]> > > >>>> To: "[email protected]" <[email protected]> > > >>>> Sent: Sunday, March 30, 2008 2:58:26 PM > > >>>> Subject: RE: [VOTE] Release Hadoop 0.16.2 (candidate 0) > > >>>> > > >>>> -1 > > >>>> > > >>>> HBase-0.1.0 has a torture test we run to qualify releases. > > >>>> This test ran fine on Hadoop-0.16.[01] but failed on > > >>>> Hadoop-0.16.2 release candidate with the following exceptions: > > >>>> > > >>>> 2008-03-30 17:32:58,993 ERROR [IPC Server handler 5 on 8020] > > >>>> hbase.HLog(380): Could not append to log. Opening new log. > > >> Exception: > > >>>> java.io.IOException: Stream closed. > > >>>> at > > >>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClie > > >>>> nt.java:1506) > > >>>> at > > >> org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:382) > > >>>> at > > >>>> org.apache.hadoop.dfs.DistributedFileSystem.create(Distributed > > >>>> FileSystem.java:123) > > >>>> at > > org.apache.hadoop.fs.FileSystem.create(FileSystem.java: > > >>>> 436) > > >>>> at > > >>>> > > org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java: > > >>>> 827) > > >>>> at > > >>>> > > >> > > > org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:379) > > >>>> at > > >>>> > > >> > > > org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:270) > > >>>> at > org.apache.hadoop.hbase.HLog.rollWriter(HLog.java:230) > > >>>> at > > >>>> org.apache.hadoop.hbase.HRegionServer$LogRoller.run(HRegionSer > > >>>> ver.java:539) > > >>>> 2008-03-30 17:32:58,993 ERROR > > >>>> [regionserver/0:0:0:0:0:0:0:0:8020.logRoller] > > >>>> hbase.HRegionServer$LogRoller(545): Log rolling failed > > >>>> java.lang.NullPointerException > > >>>> at > > >>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClien > > >>>> t.java:2262) > > >>>> at > > >>>> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FS > > >>>> DataOutputStream.java:51) > > >>>> at > > >>>> org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStre > > >>>> am.java:67) > > >>>> at > > >>>> > > >> > > > org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:932) > > >>>> at > org.apache.hadoop.hbase.HLog.rollWriter(HLog.java:217) > > >>>> at > > >>>> org.apache.hadoop.hbase.HRegionServer$LogRoller.run(HRegionSer > > >>>> ver.java:539) > > >>>> > > >>>> --- > > >>>> Jim Kellerman, Senior Engineer; Powerset > > >>>> > > >>>> > > >>>>> -----Original Message----- > > >>>>> From: Nigel Daley [mailto:[EMAIL PROTECTED] > > >>>>> Sent: Friday, March 28, 2008 10:19 PM > > >>>>> To: [email protected] > > >>>>> Subject: [VOTE] Release Hadoop 0.16.2 (candidate 0) > > >>>>> > > >>>>> I've created a candidate build for Hadoop 0.16.2. This fixes > > >>>>> 25 issues in 0.16.1. Now that HBase has it's own > > >> releases, it has > > >>>>> been removed from this release. > > >>>>> > > >>>>> Please download and try it out. > > >>>>> > > >>>>> http://people.apache.org/~nigel/hadoop-0.16.2-candidate-0/ > > >>>>> > > >>>>> Should we release this? Vote closes on Wednesday, April 2. > > >>>>> > > >>>>> Nigel > > >>>>> > > >>>>> > > >>>>> > > >>>>> No virus found in this incoming message. > > >>>>> Checked by AVG. > > >>>>> Version: 7.5.519 / Virus Database: 269.22.1/1348 - Release > > >>>>> Date: 3/28/2008 10:58 AM > > >>>>> > > >>>>> > > >>>> > > >>>> No virus found in this outgoing message. > > >>>> Checked by AVG. > > >>>> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release > > >>>> Date: 3/30/2008 12:32 PM > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> No virus found in this incoming message. > > >>>> Checked by AVG. > > >>>> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release > > >>>> Date: 3/30/2008 12:32 PM > > >>>> > > >>>> > > >>> > > >>> No virus found in this outgoing message. > > >>> Checked by AVG. > > >>> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release Date: > > >>> 3/30/2008 12:32 PM > > >>> > > >> > > >> > > >> No virus found in this incoming message. > > >> Checked by AVG. > > >> Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release > > >> Date: 3/30/2008 12:32 PM > > >> > > >> > > > > > > No virus found in this outgoing message. > > > Checked by AVG. > > > Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release Date: > > > 3/30/2008 12:32 PM > > > > > > > > > No virus found in this incoming message. > > Checked by AVG. > > Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release > > Date: 3/30/2008 12:32 PM > > > > > > No virus found in this outgoing message. > Checked by AVG. > Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release > Date: 3/30/2008 12:32 PM > > > No virus found in this incoming message. > Checked by AVG. > Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release > Date: 3/30/2008 12:32 PM > > No virus found in this outgoing message. Checked by AVG. Version: 7.5.519 / Virus Database: 269.22.1/1350 - Release Date: 3/30/2008 12:32 PM
