Re: HBase random read performance

2013-09-30 Thread kiran
The thread ended abruptly..Can some suggest a solution for improved random read performance...Even we are facing a similar issue with multi-gets on a table with growing data... On Wed, Apr 17, 2013 at 6:03 PM, Michel Segel michael_se...@hotmail.comwrote: Wouldn't do that... Changing block size

Re: HBase memstore log entries

2013-09-30 Thread Jean-Marc Spaggiari
So, from the log to the source: , memsize= + StringUtils.humanReadableInt(flushed) + And this variable is assigned hre: flushed += this.memstore.heapSizeChange(kv, true); Where heapSizeChange is: /* * Calculate how the MemStore size has changed. Includes overhead of the * backing

export snapshot of cloned table fails

2013-09-30 Thread Siddharth Karandikar
Hi, I have reached a situation where good old ExportSnapshot fails Here is what I am doing 1) Create table MT2 2) Take a snapshot of MT2 - MT2SS1 3) Export MT2SS1 - works 4) Drop MT2, delete MT2SS1 from source system 5) Export back (import) MT2SS1 - works 6) Clone a table using MT2SS1 - CMT2

Re: export snapshot of cloned table fails

2013-09-30 Thread Matteo Bertozzi
The input name looks wrong, not an hfileLink. How that was created? could you dump the snapshots files with hbase org.apache.hadoop.hbase.SnapshotInfo -snapshot NAME -files Matteo On Mon, Sep 30, 2013 at 2:36 PM, Siddharth Karandikar siddharth.karandi...@gmail.com wrote: Hi, I have

filterRowKey returns true, but filterKeyValue still called

2013-09-30 Thread Bruno Dumon
Hi, I'm a bit confused by the behavior of Filter.filterRowKey(). The javadoc of Filter mentions this: [...] * li{@link #filterRowKey(byte[],int,int)} - true to drop this row, * if false, we will also call/li * li{@link #filterKeyValue(KeyValue)} - true to drop this key/value/li [...] So I

Re: Still not sure who to blame: Hbase, zookeeper, security?

2013-09-30 Thread Michael Segel
If that's the case... why not set up DNS? Assuming that you're on your own metal, you should be using DNS to manage your floor. (If not you, then your IS guys? ) If so, then they probably have DNS set up for the machines. You just need to get them to update the zones to include CNAMEs that

Coprocessor Errors

2013-09-30 Thread Young Kim
Hello, I've been running into this odd error when running a coprocessor on hbase-0.94.6: 13/09/30 14:53:35 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT 13/09/30 14:53:35 INFO zookeeper.ZooKeeper: Client

Re: Coprocessor Errors

2013-09-30 Thread Ted Yu
bq. unable to read call parameters: class com.google.common.collect.AbstractMapBasedMultimap overrides final method setMap.(Ljava/util/Map;)V Can you show us the classpath ? Thanks On Mon, Sep 30, 2013 at 3:01 PM, Young Kim y...@crunchyroll.com wrote: Hello, I've been running into this odd

Re: Coprocessor Errors

2013-09-30 Thread Young Kim
How would I go about displaying the classpath? On Monday, September 30, 2013 at 3:10 PM, Ted Yu wrote: bq. unable to read call parameters: class com.google.common.collect.AbstractMapBasedMultimap overrides final method setMap.(Ljava/util/Map;)V Can you show us the classpath ? Thanks

Re: Coprocessor Errors

2013-09-30 Thread Ted Yu
You can find example here: http://hbase.apache.org/book.html#driver On Mon, Sep 30, 2013 at 3:17 PM, Young Kim y...@crunchyroll.com wrote: How would I go about displaying the classpath? On Monday, September 30, 2013 at 3:10 PM, Ted Yu wrote: bq. unable to read call parameters: class

Re: Coprocessor Errors

2013-09-30 Thread Young Kim
Ah I see. Are you referring to the HBASE_CLASSPATH? If so, it doesn't seem like it is set in our hbase-env.sh. On Monday, September 30, 2013 at 3:30 PM, Ted Yu wrote: You can find example here: http://hbase.apache.org/book.html#driver On Mon, Sep 30, 2013 at 3:17 PM, Young Kim

RE: Coprocessor Errors

2013-09-30 Thread Vladimir Rodionov
Guava lib issue (more than one in a CLASSPATH)? http://code.google.com/p/guava-libraries/issues/detail?id=1318 Best regards, Vladimir Rodionov Principal Platform Engineer Carrier IQ, www.carrieriq.com e-mail: vrodio...@carrieriq.com From: Ted Yu

Re: Coprocessor Errors

2013-09-30 Thread Young Kim
Thanks for the feedback. That is indeed what it looks like. I just looked under /usr/lib/hbase/lib, and there were two instances of guava jars in our region and master servers. After removing the deprecated one, it seems like this error is still popping up. Does anyone know if a restart is

HTableUtil and equivalent

2013-09-30 Thread Graeme Wallace
Hi, I've got a scenario whereby i'm pulling a stream of messages off a Kafka topic, reformatting them and then i want to write them into HBase. I've seen various suggestions on how to improve performance - but wondered if there was a way whereby I could do something equivalent to HTableUtil but

RE: Coprocessor Errors

2013-09-30 Thread Vladimir Rodionov
Yes, you have to restart servers. FYI, the right version of guava - guava-11.0.2.jar Best regards, Vladimir Rodionov Principal Platform Engineer Carrier IQ, www.carrieriq.com e-mail: vrodio...@carrieriq.com From: Young Kim [y...@crunchyroll.com] Sent:

Re: Coprocessor Errors

2013-09-30 Thread Young Kim
Is there a reason why it has to be guava-11.0.2? We're trying to use guava-14.0.1. On Monday, September 30, 2013 at 5:59 PM, Vladimir Rodionov wrote: Yes, you have to restart servers. FYI, the right version of guava - guava-11.0.2.jar Best regards, Vladimir Rodionov Principal Platform

RE: Coprocessor Errors

2013-09-30 Thread Vladimir Rodionov
No backward API compatibility, for example. Best regards, Vladimir Rodionov Principal Platform Engineer Carrier IQ, www.carrieriq.com e-mail: vrodio...@carrieriq.com From: Young Kim [y...@crunchyroll.com] Sent: Monday, September 30, 2013 6:04 PM To:

Re: Coprocessor Errors

2013-09-30 Thread Young Kim
Is there something specific that uses guava-11.0.2? We're reliant on 14.0.1 due to some new data structures presented in guava. On Monday, September 30, 2013 at 6:22 PM, Vladimir Rodionov wrote: No backward API compatibility, for example. Best regards, Vladimir Rodionov Principal

Re: HBase memstore log entries

2013-09-30 Thread aiyoh79
Thanks for the clarification. :) aiyoh79 Jean-Marc Spaggiari wrote So, from the log to the source: , memsize= + StringUtils.humanReadableInt(flushed) + And this variable is assigned hre: flushed += this.memstore.heapSizeChange(kv, true); Where heapSizeChange is: /* *

RE: Coprocessor Errors

2013-09-30 Thread Vladimir Rodionov
HBase 0.94.x is not compatible with the latest guava: class com.google.common.collect.AbstractMapBasedMultimap overrides final method setMap.(Ljava/util/Map;)V AbstractMapBasedMultimap class hierarchy has been changed in 14 version, that is why you've got these errors. You have two choices

Re: How to understand the TS of each data version?

2013-09-30 Thread takeshi
Hi, yonghu I am not sure the following timestamp info. whether valuable for you, post it anyway. So, I wonder 1. If there are any predefined semantics of TS in HBase or the semantics of TS is application-specific? As I know, the timestamp is mainly used for 1. fetch order: from newest to

Re: export snapshot of cloned table fails

2013-09-30 Thread Siddharth Karandikar
Hi, Here is the output of SnapshotInfo hbase2-bigdata:~/siddharth/tools/hbase-0.94.10 # ./bin/hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo -Dfs.default.name=hdfs://hbase2-bigdata.vxindia.veritas.com:9000/ -snapshot CMT2SS1 -files Snapshot Info

Re: export snapshot of cloned table fails

2013-09-30 Thread Siddharth Karandikar
Btw, I am running on 0.94.10. On Tue, Oct 1, 2013 at 11:12 AM, Siddharth Karandikar siddharth.karandi...@gmail.com wrote: Hi, Here is the output of SnapshotInfo hbase2-bigdata:~/siddharth/tools/hbase-0.94.10 # ./bin/hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo