Re: Region server crashes after GC pause time

2016-04-13 Thread Ding,Dongchao
Dump the jvm heap,analysis the the heap and find which query(s) cost so many memory? In my ever bad case,the RS crashed for Long GC pauses because of a big query on Batch Get operation. In addition,I think you can increase the men of JVM, 512m is so small for RS. 在 16/4/14 14:00, "karthi keya

Region server crashes after GC pause time

2016-04-13 Thread karthi keyan
Hi , i got this issue in HBase while at peak time handling more requests . can any one pls guide me to resolve the Long GC pauses in hbase . JDK-7 , JVM heap 512m HBase 0.98.13 INFO [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately

Re: HBase mapreduce job crawls on final 25% of maps

2016-04-13 Thread Colin Kincaid Williams
It appears that my issue was caused by the missing sections I mentioned in the second post. I ran a job with these settings, and my job finished in < 6 hours. Thanks for your suggestions because I have further ideas regarding issues moving forward. scan.setCaching(500);// 1 is the default

RE: Append Visibility Labels?

2016-04-13 Thread benedict.whittamsmith
Yes - it's a capability we would need to efficiently support permissioning. Good to know that we haven't killed off the old products! But I'm not sure the archaeological approach would scale. The generic facility you describe, caveats noted, certainly seems to fit our use case - especially if w

Re: Append Visibility Labels?

2016-04-13 Thread Andrew Purtell
I think Benedict was asking if it would be possible to add the capability. Actually the old product data doesn't have to die, Benedict. Set VERSIONS > 1 in your schema. The old cell version(s) carrying the old label set will still be there, accessible with a Scan that asks for N versions instea

Re: Append Visibility Labels?

2016-04-13 Thread Ted Yu
There is currently no API for appending Visibility Labels. checkAndPut() only allows you to compare value, not labels. On Wed, Apr 13, 2016 at 8:12 AM, wrote: > We sell data. A product can be defined as a permission to access data (at > a cell level). Visibility Labels look like a very good can

Append Visibility Labels?

2016-04-13 Thread benedict.whittamsmith
We sell data. A product can be defined as a permission to access data (at a cell level). Visibility Labels look like a very good candidate for implementing this model. The implementation works well until we create a new product over old data. We can set the visibility label for the new product

Re: HBase mapreduce job crawls on final 25% of maps

2016-04-13 Thread Colin Kincaid Williams
Hi Chien, 4. From 50-150k per * second * to 100-150k per * minute *, as stated above, so reads went *DOWN* significantly. I think you must have misread. I will take into account some of your other suggestions. Thanks, Colin On Tue, Apr 12, 2016 at 8:19 PM, Chien Le wrote: > Some things I wou