Re: Large discrepancy in hdfs hbase rootdir size after copytable operation.

Hi Colin, We also faced the scenario where after copying Table "A" from cluster to 1 to 2. Size of hdfs files between clusters was not equal. We also assumed that it should be equal. Hence we ran verifyRep job. I don't know whats the reason behind this discrepancy but i just wanted to share this s

Re: [VOTE] The 1st HBase 0.98.5 release candidate (RC0) is available, vote closing 8/11/2014

Running the IT IntegrationTestIngestWithVisibilityLabels fails. This is because we are not handling the Deletes in LoadTestDataGeneratorWithVisibilityLabels. As it is an issue only with IT test and no code level issues, you can take a call Andy.I have raised HBASE-11716 and attached a simple

Re: [VOTE] The 1st HBase 0.98.5 release candidate (RC0) is available, vote closing 8/11/2014

This is PE, so perhaps HBASE-11558 is operative. Or if you are running the test against a minicluster, perhaps HBASE-11621 ​Unfortunately I'm not set up yet for the small cluster YCSB tests I've run for previous 0.98 RCs. Next time.​ Any help providing insight as to perf changes in this release to

Re: Read operations hanged

bq. there was a compaction There was request for compaction. bq. if hbase hbck --repairHoles can fix this kind of thing? You can try the above command. As Qiang said, tracing back to the earlier failure would help determine root cause. Cheers On Sun, Aug 10, 2014 at 7:21 PM, Thomas Kwan wr

Re: Read operations hanged

Did you set "hbase.status.published" to true? if you enable it, master publish dead server list to clients every 10s by default, then client removes the cached regions on this server. so there must be sth wrong on dn29, please find the related first failure occurrence. you could also pastebin the

Re: Read operations hanged

Ted, >From the master log, there was a compaction around the time. 2014-08-09 22:50:51,176 DEBUG [827019302@qtp-63557232-287] client.HBaseAdmin: Trying to compact {ENCODED => 12c9a609765ad0bbd6468d93368f860a, NAME => 'm_data,2fd811c2b1d7476efb16499ccb2b823d,1406512331699.12c9a609765ad0bbd6468d93

Re: Read operations hanged

bq. it's host dn29.manage.com,60020,1407600154728 is dead but not processed yet Can you look back (from 22:50:51) in master log to see what happened to dn29 ? Thanks On Sun, Aug 10, 2014 at 2:51 PM, Thomas Kwan wrote: > Thanks for your help Ted. > > From the master's log, I see > > 2014-08-09

Re: Read operations hanged

Thanks for your help Ted. >From the master's log, I see 2014-08-09 22:50:51,176 DEBUG [827019302@qtp-63557232-287] client.HBaseAdmin: Trying to compact {ENCODED => 12c9a609765ad0bbd6468d93368f860a, NAME => 'm_data,2fd811c2b1d7476efb16499ccb2b823d,1406512331699.12c9a609765ad0bbd6468d93368f860a.',

Re: Aggresive compactions

Yes I did. Thanks for clarifying. On Aug 10, 2014 3:04 PM, "Ted Yu" wrote: bq. If I want aggressiveness from my cluster during these events I can place a larger value here You meant larger value for hbase.hstore.compaction.ratio, eight ? Sounds good. Cheers On Sun, Aug 10, 2014 at 11:48 AM,

Re: Large discrepancy in hdfs hbase rootdir size after copytable operation.

By the way I have copied the table across clusters, with the tables configured the same. the source cluster has an underlying ext2 filesystem, while the dest cluster had an underlying ext4 filesystem. The counts are the same for the tables. Will the filesystem account for the difference in director

Re: What is replacement of kv.getBuffer() in HBase0.98?

Thanks for clarification, Lars and Ram. As per Lars comment it seems like this change is meant for long term strategy where a Row is not backed by a single Byte[]. At present that's the not the case. Hence, i felt that it was less intuitive. In long term the new API would make more sense when calli

Re: Read operations hanged

Can you check master log to see why 'm_data,2fd811c2b1d7476efb16499ccb2b823d' went offline ? Thanks On Sun, Aug 10, 2014 at 12:13 PM, Thomas Kwan wrote: > Hi Ted, > > Hbase version is 0.96.0.2.0 > > Nothing interesting in the hbase log on dn29 and confirmed that region > server is running on d

Re: Read operations hanged

Hi Ted, Hbase version is 0.96.0.2.0 Nothing interesting in the hbase log on dn29 and confirmed that region server is running on dn29 When I do 'get', i see hbase(main):001:0> get 'm_data','2fd811c2b1d7476efb16499ccb2b823d' COLUMN CELL ERROR: org.apache.hadoop.hbase.N

Re: Aggresive compactions

bq. If I want aggressiveness from my cluster during these events I can place a larger value here You meant larger value for hbase.hstore.compaction.ratio, eight ? Sounds good. Cheers On Sun, Aug 10, 2014 at 11:48 AM, Geovanie Marquez < geovanie.marq...@gmail.com> wrote: > The default: 1.2F > h

Re: Aggresive compactions

The default: 1.2F hbase.hstore.compaction.ratio For minor compaction, this ratio is used to determine whether a given StoreFile which is larger than hbase.hstore.compaction.min.size is eligible for compaction. Its effect is to limit compaction of large StoreFiles. The value of hbase.hstore.compact

Re: Read operations hanged

bq. if I can just rmr stuff under /hbase-unsecure/splitWAL/... Please don't. Have you checked region server log on dn29.manage.com ? What hbase version are you using ? Cheers On Sun, Aug 10, 2014 at 10:27 AM, Thomas Kwan wrote: > And I have a program that do some read operations and it han

Read operations hanged

And I have a program that do some read operations and it hangs. And I am seeing 2014-08-10 12:22:05,359 DEBUG [main] client.HConnectionManager$HConnectionImplementation: Removed all cached region locations that map to dn29.manage.com,60020,1407600154728 2014-08-10 12:22:06,173 DEBUG [main] client.

Re: Aggresive compactions

What is the value for the config parameter 'hbase.hstore.compaction.ratio' ? Thanks On Sun, Aug 10, 2014 at 7:17 AM, Geovanie Marquez < geovanie.marq...@gmail.com> wrote: > I notice that when I have a regions with store file counts greater > than hbase.hstore.blockingStoreFiles, > on cluster st

Major compaction throttling

Hi, We are running CDH 4.5, 6 nodes hbase cluster (0.94.6). We configured major compaction to run once a week. During this time we are getting timeouts while writing to hbase. Is there any throttling parameter we can fine tune? I know that hbase doesn't operate the compaction on all 6 RS on the sa

Aggresive compactions

I notice that when I have a regions with store file counts greater than hbase.hstore.blockingStoreFiles, on cluster startup the number drops dramatically under this value to just under the blockingStoreFile parameter value in a relative short amount of time and then it stalls and doesn't fall more

Re: Large discrepancy in hdfs hbase rootdir size after copytable operation.

HBASE-11715 opened. 2014-08-10 7:12 GMT-04:00 Jean-Marc Spaggiari : > +1 too for a tool to produce a hash of a table. Like, one hash per region, > or as Lars said, one hash per range. You define the number of buckets you > want, run the MR job,

Re: Large discrepancy in hdfs hbase rootdir size after copytable operation.

+1 too for a tool to produce a hash of a table. Like, one hash per region, or as Lars said, one hash per range. You define the number of buckets you want, run the MR job, which produce a list of hash, and compare that from the 2 clusters. Might be pretty simple to do. The more buckets you define, t