Re: Puzzling behaviour with HBase checksums

2013-07-07 Thread Varun Sharma
Created https://issues.apache.org/jira/browse/HDFS-4960. Thanks ! On Sat, Jul 6, 2013 at 3:55 PM, Stack wrote: > On Fri, Jul 5, 2013 at 5:21 PM, Varun Sharma wrote: > > > I just set this value in hbase-site.xml but still the 7 byte reads and > > lseek(s) persist. > > > > > c. dfs.client.read.

Re: Puzzling behaviour with HBase checksums

2013-07-06 Thread Stack
On Fri, Jul 5, 2013 at 5:21 PM, Varun Sharma wrote: > I just set this value in hbase-site.xml but still the 7 byte reads and > lseek(s) persist. > > c. dfs.client.read.shortcircuit.skip.checksum is the key to bypass checksum check at the client side. (from hdfs-2246 release notes). It is red he

Re: Puzzling behaviour with HBase checksums

2013-07-05 Thread Varun Sharma
Okay - I guess I now know what's going on here. Essentially there is a 7 byte header for each block which is read initially irrespective of whether this is a checksum/no checksum read. Some version checking is done here. >From what I can see, the FB branch (which I guess is more optimized for perfo

Re: Puzzling behaviour with HBase checksums

2013-07-05 Thread Varun Sharma
I just set this value in hbase-site.xml but still the 7 byte reads and lseek(s) persist. On Fri, Jul 5, 2013 at 4:22 PM, Ted Yu wrote: > What value did you set for dfs.client.read.shortcircuit.skip.checksum ? > > Cheers > > On Fri, Jul 5, 2013 at 2:55 PM, Varun Sharma wrote: > > > Hi, > > > >

Re: Puzzling behaviour with HBase checksums

2013-07-05 Thread Varun Sharma
Oh I never set that - what does it do, could that possibly be why this is causing problems ? THanks Varun On Fri, Jul 5, 2013 at 4:22 PM, Ted Yu wrote: > What value did you set for dfs.client.read.shortcircuit.skip.checksum ? > > Cheers > > On Fri, Jul 5, 2013 at 2:55 PM, Varun Sharma wrote:

Re: Puzzling behaviour with HBase checksums

2013-07-05 Thread Ted Yu
What value did you set for dfs.client.read.shortcircuit.skip.checksum ? Cheers On Fri, Jul 5, 2013 at 2:55 PM, Varun Sharma wrote: > Hi, > > We are running hbase with hbase.regionserver.checksum.verify set to true. > But we are seeing an equal # of seeks for .meta files on HDFS and data > block

Puzzling behaviour with HBase checksums

2013-07-05 Thread Varun Sharma
Hi, We are running hbase with hbase.regionserver.checksum.verify set to true. But we are seeing an equal # of seeks for .meta files on HDFS and data blocks. This is rather puzzling and I dont know if its broken. The hbase jar is compiled against 2.0.3-alpha and this behaviour occurs for both 0.94.