Re: integration issure about hive and hbase

2013-07-08 Thread Cheng Su
org.apache.hadoop.hive.ql.metadata.HiveException: Error in loading storage handler.ora.apache.hadoop.hive.hbase.HBaseStorageHandler FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask -- Regards, Cheng Su

Re: How does hive decide to launch how many map tasks?

2012-11-18 Thread Cheng Su
Hi, Bejoy. I find the mapred.min.split.size in mapred-default.xml, but there is no mapred.max.split.size property. I'm using hadoop 0.20.205.0. Maybe only newer versions support mapred.max.split.size? On Fri, Nov 16, 2012 at 8:08 PM, Cheng Su scarcer...@gmail.com wrote: Thank you so much

Why hadoop job pending for so long?

2012-11-16 Thread Cheng Su
org.apache.hadoop.mapred.IndexCache: Map ID attempt_201211151751_0016_m_02_0 not found in cache What's wrong with the job? -- Regards, Cheng Su

Re: Do I need any Pig knowledge to learn Hive?

2012-11-16 Thread Cheng Su
Hi, Majid. I think you don't need know about pig, but it will help if you have some SQL knowledge. On Fri, Nov 16, 2012 at 5:26 PM, Majid Azimi majid.merk...@gmail.com wrote: hi guys, Do I need any Pig knowledge to learn Hive? Or understanding Hadoop will suffice? -- Regards, Cheng Su

Re: How does hive decide to launch how many map tasks?

2012-11-16 Thread Cheng Su
mapper per file if your files are small. You can control the number of maps by controlling the split sizes. Mapred.min.split.size Mapred.max.split.size Regards Bejoy KS Sent from handheld, please excuse typos. -Original Message- From: Cheng Su scarcer...@gmail.com Date: Fri

Can I merge files after I loaded them into hive?

2012-11-15 Thread Cheng Su
that? -- Regards, Cheng Su

Re: Hive UDAF convert problem

2012-11-15 Thread Cheng Su
programming practice anyways. Mark On Mon, Nov 12, 2012 at 8:01 PM, Cheng Su scarcer...@gmail.com wrote: Hi all. I'm writing a hive UDAF to calculate page view per session. Java source is blow: public class CalculateAvgPVPerSession extends UDAF { /** * @author

Re: Can I merge files after I loaded them into hive?

2012-11-15 Thread Cheng Su
as well. Use combineFileInputFormat and start a map only job with Identity mapper with split size set to the required large file size. Regards Bejoy KS Sent from handheld, please excuse typos. -Original Message- From: Cheng Su scarcer...@gmail.com Date: Thu, 15 Nov 2012 16:03:44

How does hive decide to launch how many map tasks?

2012-11-15 Thread Cheng Su
files in it, the other has 2, when I execute select count(*) from table, only one map is launched. How can I increase the number of map tasks to improve the performance? Thanks. -- Regards, Cheng Su

Re: ClassNotFoundException when use hive java client of hive + hbase integration

2012-11-06 Thread Cheng Su
to the folder This way you need not add them manually everytime. On Mon, Nov 5, 2012 at 9:18 PM, Cheng Su scarcer...@gmail.com wrote: Mark, thank you so much for your suggestion. Although I've already add necessary jars to my hive aux path, thus I can execute my sql in hive CLI mode without

Re: Hbase connection closed when query multiple complicated hql with hive+hbase integration

2012-11-06 Thread Cheng Su
The exceptions seem to be another problem. They all happened on one node. And after the task attempts failed at that node, retried on other nodes and no exceptions. So that, the exception maybe have nothing to do with the performance issue. On Wed, Nov 7, 2012 at 11:07 AM, Cheng Su scarcer

ClassNotFoundException when use hive java client of hive + hbase integration

2012-11-05 Thread Cheng Su
as below: hive --service hiveserver -p 1 The server side error log says that HBaseSplit is not found. But why? How can I fix this? -- Regards, Cheng Su

Re: ClassNotFoundException when use hive java client of hive + hbase integration

2012-11-05 Thread Cheng Su
it in aux_lib. See this thread for reference: http://mail-archives.apache.org/mod_mbox/hive-user/201103.mbox/%3caanlktingqlgknqmizgoi+szfnexgcat8caqtovf8j...@mail.gmail.com%3E Mark On Mon, Nov 5, 2012 at 6:53 AM, Cheng Su scarcer...@gmail.com wrote: Hi, all. I have a hive+hbase integration cluster