Re: HBase mapreduce job crawls on final 25% of maps

2016-04-13 Thread Colin Kincaid Williams
It appears that my issue was caused by the missing sections I mentioned in the second post. I ran a job with these settings, and my job finished in < 6 hours. Thanks for your suggestions because I have further ideas regarding issues moving forward. scan.setCaching(500);// 1 is the default

RE: Append Visibility Labels?

2016-04-13 Thread benedict.whittamsmith
Yes - it's a capability we would need to efficiently support permissioning. Good to know that we haven't killed off the old products! But I'm not sure the archaeological approach would scale. The generic facility you describe, caveats noted, certainly seems to fit our use case - especially if

Re: Append Visibility Labels?

2016-04-13 Thread Andrew Purtell
I think Benedict was asking if it would be possible to add the capability. Actually the old product data doesn't have to die, Benedict. Set VERSIONS > 1 in your schema. The old cell version(s) carrying the old label set will still be there, accessible with a Scan that asks for N versions

Re: Append Visibility Labels?

2016-04-13 Thread Ted Yu
There is currently no API for appending Visibility Labels. checkAndPut() only allows you to compare value, not labels. On Wed, Apr 13, 2016 at 8:12 AM, wrote: > We sell data. A product can be defined as a permission to access data (at > a cell level).

Append Visibility Labels?

2016-04-13 Thread benedict.whittamsmith
We sell data. A product can be defined as a permission to access data (at a cell level). Visibility Labels look like a very good candidate for implementing this model. The implementation works well until we create a new product over old data. We can set the visibility label for the new product

Re: HBase mapreduce job crawls on final 25% of maps

2016-04-13 Thread Colin Kincaid Williams
Hi Chien, 4. From 50-150k per * second * to 100-150k per * minute *, as stated above, so reads went *DOWN* significantly. I think you must have misread. I will take into account some of your other suggestions. Thanks, Colin On Tue, Apr 12, 2016 at 8:19 PM, Chien Le wrote: