RE: Joins Failing

2014-03-24 Thread Clay McDonald 
I believe I found my issue.

2014-03-24 15:49:38,775 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error 
running child : java.lang.OutOfMemoryError: Java heap space


Clay



From: Clay McDonald [mailto:stuart.mcdon...@bateswhite.com] 
Sent: Monday, March 24, 2014 2:07 PM
To: 'user@hive.apache.org'
Subject: Joins Failing

My join query is failing. Any suggestions on how I should troubleshoot this?

92651139753_0036_1_conf.xml to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp
2014-03-24 13:48:58,244 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done 
location: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp
2014-03-24 13:48:58,248 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to 
done: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036.summary_tmp
 to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036.summary
2014-03-24 13:48:58,249 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to 
done: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp
 to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml
2014-03-24 13:48:58,251 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to 
done: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036-1395683232406-root-SELECT+COUNT%28A.UPC_ID%29+AS+CNT%0D%0AFR...B.UPC_ID%28Stage-1395683335043-0-0-FAILED-default.jhist_tmp
 to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036-1395683232406-root-SELECT+COUNT%28A.UPC_ID%29+AS+CNT%0D%0AFR...B.UPC_ID%28Stage-1395683335043-0-0-FAILED-default.jhist
2014-03-24 13:48:58,251 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped 
JobHistoryEventHandler. super.stop()
2014-03-24 13:48:58,252 INFO [Thread-65] 
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job 
diagnostics to Task failed task_1392651139753_0036_m_11
Job failed as tasks failed. failedMaps:1 failedReduces:0


Thanks Clay

Re: Joins Failing

2014-03-24 Thread Divakar Reddy 
I hope, this property will fix your issue.

set mapred.reduce.child.java.opts=-Xmx4096m;





On Mon, Mar 24, 2014 at 3:59 PM, Clay McDonald 
stuart.mcdon...@bateswhite.com wrote:

 I believe I found my issue.

 2014-03-24 15:49:38,775 FATAL [main] org.apache.hadoop.mapred.YarnChild:
 Error running child : java.lang.OutOfMemoryError: Java heap space


 Clay



 From: Clay McDonald [mailto:stuart.mcdon...@bateswhite.com]
 Sent: Monday, March 24, 2014 2:07 PM
 To: 'user@hive.apache.org'
 Subject: Joins Failing

 My join query is failing. Any suggestions on how I should troubleshoot
 this?

 92651139753_0036_1_conf.xml to hdfs://
 dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp
 2014-03-24 13:48:58,244 INFO [Thread-65]
 org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to
 done location: hdfs://
 dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp
 2014-03-24 13:48:58,248 INFO [Thread-65]
 org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
 done: hdfs://
 dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036.summary_tmpto
  hdfs://
 dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036.summary
 2014-03-24 13:48:58,249 INFO [Thread-65]
 org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
 done: hdfs://
 dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmpto
  hdfs://
 dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml
 2014-03-24 13:48:58,251 INFO [Thread-65]
 org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
 done: hdfs://
 dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036-1395683232406-root-SELECT+COUNT%28A.UPC_ID%29+AS+CNT%0D%0AFR...B.UPC_ID%28Stage-1395683335043-0-0-FAILED-default.jhist_tmpto
  hdfs://
 dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036-1395683232406-root-SELECT+COUNT%28A.UPC_ID%29+AS+CNT%0D%0AFR...B.UPC_ID%28Stage-1395683335043-0-0-FAILED-default.jhist
 2014-03-24 13:48:58,251 INFO [Thread-65]
 org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped
 JobHistoryEventHandler. super.stop()
 2014-03-24 13:48:58,252 INFO [Thread-65]
 org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job
 diagnostics to Task failed task_1392651139753_0036_m_11
 Job failed as tasks failed. failedMaps:1 failedReduces:0


 Thanks Clay

RE: Joins Failing

2014-03-24 Thread java8964 
It looks like his job failed in OOM in mapper tasks:
Job failed as tasks failed. failedMaps:1 failedReduces:0
So what he need is to increase the mapper heap size request.
Yong

Date: Mon, 24 Mar 2014 16:16:50 -0400
Subject: Re: Joins Failing
From: divakarredd...@gmail.com
To: user@hive.apache.org
CC: jason.herba...@bateswhite.com

I hope, this property will fix your issue.

set mapred.reduce.child.java.opts=-Xmx4096m;






On Mon, Mar 24, 2014 at 3:59 PM, Clay McDonald stuart.mcdon...@bateswhite.com 
wrote:

I believe I found my issue.



2014-03-24 15:49:38,775 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error 
running child : java.lang.OutOfMemoryError: Java heap space





Clay







From: Clay McDonald [mailto:stuart.mcdon...@bateswhite.com]

Sent: Monday, March 24, 2014 2:07 PM

To: 'user@hive.apache.org'

Subject: Joins Failing



My join query is failing. Any suggestions on how I should troubleshoot this?



92651139753_0036_1_conf.xml to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp

2014-03-24 13:48:58,244 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done 
location: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp

2014-03-24 13:48:58,248 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to 
done: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036.summary_tmp
 to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036.summary

2014-03-24 13:48:58,249 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to 
done: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp
 to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml


2014-03-24 13:48:58,251 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to 
done: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036-1395683232406-root-SELECT+COUNT%28A.UPC_ID%29+AS+CNT%0D%0AFR...B.UPC_ID%28Stage-1395683335043-0-0-FAILED-default.jhist_tmp
 to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036-1395683232406-root-SELECT+COUNT%28A.UPC_ID%29+AS+CNT%0D%0AFR...B.UPC_ID%28Stage-1395683335043-0-0-FAILED-default.jhist


2014-03-24 13:48:58,251 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped 
JobHistoryEventHandler. super.stop()

2014-03-24 13:48:58,252 INFO [Thread-65] 
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job 
diagnostics to Task failed task_1392651139753_0036_m_11

Job failed as tasks failed. failedMaps:1 failedReduces:0





Thanks Clay