Questions related to HBase general use

2015-05-07 Thread java8964
Hi, I am kind of new to HBase. Currently our production run IBM BigInsight V3, comes with Hadoop 2.2 and HBase 0.96.0. We are mostly using HDFS and Hive/Pig for our BigData project, it works very good for our big datasets. Right now, we have a one dataset needs to be loaded from Mysql, about

Why my MR job running on HBase all are Rack-local map tasks

2014-02-24 Thread java8964
Hi, I have a 10 nodes cluster with 8 of them are datanode/tasknode/HbaseRegionNode. I have a HBase table with one column family and 1.5T data, spread across 55 regions on these 8 region servers. When I run a testing scan MR job, it will generate 55 mapper tasks, (Matching with 55 regions), but

Why the put without column in batch doesn't generate any error

2014-02-23 Thread java8964
Hi, I found some inconsistent behavior in the HBase, and wonder why. In the simple Put API call, if there is no content at all with this put, the local side of the client will throw IllegalArgumentException: No columns to insert to failed the put, as shown in the example of Lars George Book

RE: some data modeling questions related to Hbase

2014-02-14 Thread java8964
Phoenix ? See http://incubator.apache.org/projects/phoenix.html Cheers On Thu, Feb 13, 2014 at 6:49 PM, java8964 java8...@hotmail.com wrote: Hi, I am researching a project to load data from MySQL into HBase. The data is our user data. We want to analyze the user data with its

some data modeling questions related to Hbase

2014-02-13 Thread java8964
Hi, I am researching a project to load data from MySQL into HBase. The data is our user data. We want to analyze the user data with its activities and tracking. The user data is big. The record count reaches billion and whole data set size is about more than several Ts. The purpose of this data

Re: Hive + Hbase scanning performance

2014-02-10 Thread java8964
Hi, Ted: Our environment is using a distribution from a Vendor, so it is not easy just to patch it myself. But I can seek the option to see if the vendor is willing to patch it in next release. Before I do that, I just want to make sure patching the code is the ONLY solution. I read the source

RE: Hive + Hbase scanning performance

2014-02-10 Thread java8964
on the Configuration that the HTable sees that Hive uses to create the scanner it should work. -- Lars From: java8964 java8...@hotmail.com To: user@hbase.apache.org user@hbase.apache.org Sent: Monday, February 10, 2014 12:19 PM Subject: Re: Hive + Hbase