Re: Custom preCompact RegionObserver crashes entire cluster on OOME: Heap Space

2013-02-12 Thread Mesika, Asaf
V I received from the >> InternalScanner before adding it the Result - i..e returning it from my own >> InternalScanner? > > You can change as per your need IMO > > -Anoop- > > ____ > From: Mesika, Asaf [asaf.mes...@gmail.com]

Re: Custom preCompact RegionObserver crashes entire cluster on OOME: Heap Space

2013-02-12 Thread Mesika, Asaf
is going wrong. Do you > have blooms being used? > > -Anoop- > > From: Mesika, Asaf [asaf.mes...@gmail.com] > Sent: Tuesday, February 12, 2013 11:16 AM > To: user@hbase.apache.org > Subject: Custom preCompact RegionObserver crashes entire c

Re: Custom preCompact RegionObserver crashes entire cluster on OOME: Heap Space

2013-02-12 Thread Mesika, Asaf
gt; > There is no queuing. Your assumption is correct only. It is written to the > writer as and when. (Just like how memstore flush doing the HFile write) As > Lars said a look at your code can tell if some thing is going wrong. Do you > have blooms being used? > > -Anoop- &g

Re: Custom preCompact RegionObserver crashes entire cluster on OOME: Heap Space

2013-02-11 Thread Mesika, Asaf
t; > Cheers > > On Mon, Feb 11, 2013 at 9:46 PM, Mesika, Asaf wrote: > >> Hi, >> >> I wrote a RegionObserver which does preCompact. >> I activated in pre-production, and then entire cluster dropped dead: One >> RegionServer after another crashed on

Custom preCompact RegionObserver crashes entire cluster on OOME: Heap Space

2013-02-11 Thread Mesika, Asaf
Hi, I wrote a RegionObserver which does preCompact. I activated in pre-production, and then entire cluster dropped dead: One RegionServer after another crashed on OutOfMemoryException: Heap Space. My preCompact method generates a KeyValue per each set of Column Qualifiers it sees. When I remove

Compressing data sent from HBase client

2013-02-08 Thread Mesika, Asaf
Hi, Was the possibility of compressing RPC data sent from HBase client to Region server? Some data pattern could benefit from this tremendously. Another thing - As a work-around, can be this be done by using a special coprocessor which will receive the List, decompress it and then call HRegion

Unique Put ends up twice in HBase?

2013-02-03 Thread Mesika, Asaf
Hi, Can the following scenario occur? Create a Put for (rk1, cf1, cq1, value=1L) run hTable.batch Inside implementation: Put is sent to server Server throws an exception, but Put still is saved (WAL, memstore) Flush kicks in Client attempts a retry and succeeds, thus same put is written for the s

Coprocessor HDFS jar hot deployment

2013-01-31 Thread Mesika, Asaf
Hi, If I deploy a Region Observer to a table using HDFS Jar, is it possible to deploy a new version, enable/disable the table to get that new jar hot deployed, without restarting all the region servers? Asaf Mesika Senior Developer / CSI Infrastructure Team Office: +972 (73) 285-8769 Fax: +

Re: increment-related bug Was: HBASE-7114 Increment does not extend Mutation but probably should

2013-01-30 Thread Mesika, Asaf
This may be an old one: https://issues.apache.org/jira/browse/HBASE-3787 > > > > On Wed, Jan 30, 2013 at 9:25 AM, Mesika, Asaf wrote: > >> Hi, >> >> We ran the QA test again, this time with INFO message on at the client >> side (HTable). >> We saw

Re: increment-related bug Was: HBASE-7114 Increment does not extend Mutation but probably should

2013-01-30 Thread Mesika, Asaf
Hi, We ran the QA test again, this time with INFO message on at the client side (HTable). We saw many retry attempts which failed on RPC timeouts (we use the default of 60 seconds). I guess when this error occurs, the increment shouldn't really happen, right? This may explain the diff we see f

Adding Custom Coprocessor metrics to HBase JMX Metrics

2013-01-29 Thread Mesika, Asaf
Hi, I wrote a RegionObserver. I'm collecting very important metrics in that observer and I would like to expose them in the JMX Server HBase is using. Is there a way to do that? Thanks, Asaf

WARN CleanerChore - Error while cleaning the logs

2013-01-28 Thread Mesika, Asaf
Hi, Recently after upgrading to 0.94.3, my unit test which is using HBase mini cluster keeps throwing this warning. Why does it want to delete a table folder? Can someone elaborate on this exception? My test it self is setting up two tables, in which only one is used. The one in the errors is

Re: HDFS disk space requirements

2013-01-11 Thread Mesika, Asaf
130 GB raw data will take in HBase since it adds the family name, qualifier and timestamp to each value, so it can even be 150GB. You can check it exactly, by loading only one row with one column and see how much it takes on the HDFS file system (run compaction first). Next, you 5 times that si

0.94.3 in Central (Maven)

2012-12-30 Thread Mesika, Asaf
Hi, I'm looking for a Maven repository that hosts latest HBase version. I see that central (http://mvnrepository.com/artifact/org.apache.hbase/hbase) only hosts 0.94.1 and not 0.94.4 Thanks, Asaf Mesika Senior Developer / CSI Infrastructure Team Office: +972 (73) 285-8769 Fax: +972 (9) 86

Re: HBase 0.94 security configurations

2012-12-18 Thread Mesika, Asaf
I suggest you use a Maven project. I can copy-paste a small pom.xml for you to get you started. It will solve all of those missing JARS you have. Just make sure to install M2Eclipse plugin (Eclipse Marketplace) On Dec 18, 2012, at 8:15 PM, Bob Futrelle wrote: > Nick's suggestions appear to have

Re: HBaseClient.call() hang

2012-12-18 Thread Mesika, Asaf
One thing I don't get: If the RS went down, then the RPC connection should have been reset, thus causing the client to interrupt, right? It shouldn't be a matter of timeout at all. On Dec 17, 2012, at 7:18 PM, Bryan Keller wrote: > It seems there was a cascading effect. The regionservers were b

Re: Checking if a coprocessor was loaded successfully from client

2012-12-10 Thread Mesika, Asaf
;> On Sun, Dec 9, 2012 at 2:25 PM, Ted Yu wrote: >> >>> On region server web UI, you should see the list of coprocessors loaded. >>> >>> But I guess you're looking for a programmatic way of detecting coprocessor >>> deployment. >>> >>&

Re: Checking if a coprocessor was loaded successfully from client

2012-12-10 Thread Mesika, Asaf
> >> On region server web UI, you should see the list of coprocessors loaded. >> >> But I guess you're looking for a programmatic way of detecting coprocessor >> deployment. >> >> Cheers >> >> On Sun, Dec 9, 2012 at 12:29 PM, Mesika, Asa

Re: Using HBaseAdmin.getAlterStatus

2012-12-09 Thread Mesika, Asaf
already disable-update-enable table and complete the alter transaction > before querying it (such that there isn't any pending operation to > report). If the querying is done in a parallel process/monitor thread, > then you may see in-flight results. > > On Mon, Dec 10, 2012 at 2

Using HBaseAdmin.getAlterStatus

2012-12-09 Thread Mesika, Asaf
Hi, I've tried using HBaseAdmin.getAlterStatus to check on a HBaseAdmin.modifyTable command I've issued, but it doesn't work. Sample Code: Modifying Table tableDescriptor = admin.getTableDescriptor(tableNameBytes); if (tableDescriptor.hasCoprocessor(observerClass

Re: Checking if a coprocessor was loaded successfully from client

2012-12-09 Thread Mesika, Asaf
0.94.0 On Dec 9, 2012, at 3:03 PM, yuzhih...@gmail.com wrote: > Which hbase version are you targeting ? > > Thanks > > > > On Dec 9, 2012, at 2:55 AM, "Mesika, Asaf" wrote: > >> Hi, >> >> I wrote a custom Region Observer. >> I

Checking if a coprocessor was loaded successfully from client

2012-12-09 Thread Mesika, Asaf
Hi, I wrote a custom Region Observer. I'm currently writing an Installer class for it. In this installer I'm adding the region observer by adding a coprocessor to the HTableDescriptor, and then calling modifyTable by HBaseAdmin. My question is: How can I check whether region observer was loaded

Re: How to configure in eclipse?

2012-11-13 Thread Mesika, Asaf
I got stuck on missing Maven dependencies. All our dependencies downloads goes through corporate gateway (Nexus). I'm not sure it's ok to request to add to follow personal test repo to Nexus: ghelmling.testing Gary Helmling test repo http://people.apache.org/~garyh/mvn/

Re: Region is not online: .META.,,1

2012-11-08 Thread Mesika, Asaf
nerCallable@39ed1b0b, org.apache.hadoop.hbase.NotServingRegionException: org.apache.hadoop.hbase.NotServingRegionException: Region is not online: .META.,,1 How can there be such a mismatch? On Nov 7, 2012, at 8:08 AM, Mesika, Asaf wrote: > After restart, running hbck results in: > 12/11/07 06:07:2

Re: Region is not online: .META.,,1

2012-11-08 Thread Mesika, Asaf
.html#d2067e11173 > > Thank you! > > Sincerely, > Leonid Fedotov > > > On Nov 6, 2012, at 10:08 PM, Mesika, Asaf wrote: > >> After restart, running hbck results in: >> 12/11/07 06:07:28 DEBUG client.MetaScanner:

Re: Region is not online: .META.,,1

2012-11-06 Thread Mesika, Asaf
6) at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:382) at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3120) HBase is version 0.94.0 On Nov 7, 2012, at 7:28 AM, Mesika, Asaf wrote: > Hi, > > I have a cluster with 10 RS and 1 Master. > From some reason, any call

Region is not online: .META.,,1

2012-11-06 Thread Mesika, Asaf
Hi, I have a cluster with 10 RS and 1 Master. From some reason, any call to the HBase fails. I ran hbck and got the same failure: Tue Nov 06 13:27:07 UTC 2012, org.apache.hadoop.hbase.client.ScannerCallable@7d8a8ce2, org.apache.hadoop.hbase.NotServingRegionException: org.apache.hadoop.hbase.N