Re: timeouts with lots of coprocessor puts on single row

2013-08-27 Thread anil gupta
On Mon, Aug 26, 2013 at 10:56 PM, Olle Mårtensson olle.martens...@gmail.com wrote: Thank you for the link Anil it was a good explanation indeed. It's not recommended to do put/deletes across region servers like this. That was not my intention, I want to keep the region for the aggregates

HBase-Hive integration performance issues

2013-08-27 Thread Hao Ren
Hi, I am running Hive and HBase on Amazon EC2. By following the tutorial: https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration , I managed to create a HBase table from Hive and insert data into it. It works but with a low performance. To be specific, inserting 1.3 Gb (50 M

Data Deduplication in HBase

2013-08-27 Thread Anand Nalya
Hi, I have a use case in which I need to store segments of mp3 files in hbase. A song may come to the application in different ovelapping segments. For example, a 5 min song can have the following segments 0-1,0.5-2,2-4,3-5. As seen, some of the data is duplicate (3-4 is present in the last 2

Re: HBase-Hive integration performance issues

2013-08-27 Thread Matt Davies
Hao, A couple thoughts here. This could be related to many things. 1. Did you pre-split your regions? If not, you could be hot-spotting on a single server and then waiting for the region to split. If that is the case, you could actually only be using a single server for much of your load (if not

Re: Data Deduplication in HBase

2013-08-27 Thread Ted Yu
bq. Will hbase do some sort of deduplication? I don't think so. What is the granularity of segment overlap ? In the above example, it seems to be 0.5 Cheers On Tue, Aug 27, 2013 at 7:12 AM, Anand Nalya anand.na...@gmail.com wrote: Hi, I have a use case in which I need to store segments

Re: HBase-Hive integration performance issues

2013-08-27 Thread Hao Ren
Matt, Thank you for the lightning reply. I will try out what you have mentioned in these days, thus I could tell you some news on the issue in detail. Thank you again. Your suggestions show me the way. =) Hao Le 27/08/2013 16:13, Matt Davies a écrit : Hao, A couple thoughts here. This

[Question: replication] why only one regionserver is used during replication? 0.94.9

2013-08-27 Thread Demai Ni
hi, guys, I am using hbase 0.94.9. And setup replication from a 4-nodes master(3 regserver) to a 3-nodes slave(2 regserver). I can tell that all source regservers can successfully replicate data. However, it seems for each particular table, only one regserver will handle its replication at each

Re: [Question: replication] why only one regionserver is used during replication? 0.94.9

2013-08-27 Thread Jean-Daniel Cryans
Region servers replicate data written to them, so look at how your regions are distributed. J-D On Tue, Aug 27, 2013 at 11:29 AM, Demai Ni nid...@gmail.com wrote: hi, guys, I am using hbase 0.94.9. And setup replication from a 4-nodes master(3 regserver) to a 3-nodes slave(2 regserver).

Fwd: Hbase 0.94.6 stargate can't use multi get

2013-08-27 Thread Dmitriy Troyan
Hey all, I try to use multi get for receiving different versions of row but it give me only one always. For example I have table log, and column family data:get. I put a lot of versions of row/data:log. Now I try to get all versions of this key. As it said in manual

Re: [Question: replication] why only one regionserver is used during replication? 0.94.9

2013-08-27 Thread Demai Ni
J-D, thanks for the tip. On Tue, Aug 27, 2013 at 11:40 AM, Jean-Daniel Cryans jdcry...@apache.orgwrote: Region servers replicate data written to them, so look at how your regions are distributed. J-D On Tue, Aug 27, 2013 at 11:29 AM, Demai Ni nid...@gmail.com wrote: hi, guys, I am

Hbase 0.94.6 stargate can't use multi get

2013-08-27 Thread Dmitriy Troyan
Hey all, I try to use multi get for receiving different versions of row but it give me only one always. For example I have table log, and column family data:get. I put a lot of versions of row/data:log. Now I try to get all versions of this key. As it said in manual

Hbase 0.94.6 stargate can't use multi get

2013-08-27 Thread Dmitriy Troyan
Hey all, I try to use multi get for receiving different versions of row but it give me only one always. For example I have table log, and column family data:get. I put a lot of versions of row/data:log. Now I try to get all versions of this key. As it said in manual

Writing multiple tables from reducer

2013-08-27 Thread jamal sasha
Hi, I am new to hbase and am trying to achieve the following. I am reading data from hdfs in mapper and parsing it.. So, in reducer I want my output to write to hbase instead of hdfs But here is the thing. public static class MyTableReducer extends TableReducerText, Text,

Re: Writing multiple tables from reducer

2013-08-27 Thread Harsh J
You can use HBase's MultiTableOutputFormat: http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/MultiTableOutputFormat.html An example can be found in this blog post: http://www.wildnove.com/2011/07/19/tutorial-hadoop-and-hbase-multitableoutputformat/ On Wed, Aug 28, 2013 at 3:50

Region locality and core/thread availability

2013-08-27 Thread Kiru Pakkirisamy
I think my app wants to hit a particular region all the time. Since my table is only read-only (lookup). I have created more than one and randomly pick one to use. This way I can load the whole cluster. Is there a feature in Hbase which lets coprocessors run on another region server on

Re: [Question: replication] why only one regionserver is used during replication? 0.94.9

2013-08-27 Thread Toby Lazar
BEGIN:VCALENDAR VERSION:2.0 PRODID://RESEARCH IN MOTION//BIS 3.0 METHOD:REQUEST BEGIN:VEVENT X-RIM-REVISION:0 X-MICROSOFT-CDO-BUSYSTATUS:BUSY SUMMARY:Re: [Question: replication] why only one regionserver is used durin g replication? 0.94.9 CLASS:PUBLIC

how to export data from hbase to mysql?

2013-08-27 Thread ch huang
hi,all: any good idea? thanks

Re: how to export data from hbase to mysql?

2013-08-27 Thread Jean-Marc Spaggiari
Take a look at sqoop? Le 2013-08-27 23:08, ch huang justlo...@gmail.com a écrit : hi,all: any good idea? thanks

Re: how to export data from hbase to mysql?

2013-08-27 Thread James Taylor
Or if you'd like to be able to use SQL directly on it, take a look at Phoenix (https://github.com/forcedotcom/phoenix). James On Aug 27, 2013, at 8:14 PM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: Take a look at sqoop? Le 2013-08-27 23:08, ch huang justlo...@gmail.com a écrit :

Re: Hbase 0.94.6 stargate can't use multi get

2013-08-27 Thread Ravi Kiran
Hi , Can you please query for the schema of the table and show us here. Would like to know what is value for VERSIONS that you have set for the column family . I hope you have set it to 10. Ex:http://myhost.com:8080/log/schemahttp://myhost.com:8080/log/data:get/0,1377633354/?v=10 Regards

Re: how to export data from hbase to mysql?

2013-08-27 Thread ch huang
sqoop can not support export hbase data into mysql On Wed, Aug 28, 2013 at 11:13 AM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: Take a look at sqoop? Le 2013-08-27 23:08, ch huang justlo...@gmail.com a écrit : hi,all: any good idea? thanks