RE: Joins Failing

2014-03-24 Thread java8964 
It looks like his job failed in OOM in mapper tasks:
Job failed as tasks failed. failedMaps:1 failedReduces:0
So what he need is to increase the mapper heap size request.
Yong

Date: Mon, 24 Mar 2014 16:16:50 -0400
Subject: Re: Joins Failing
From: divakarredd...@gmail.com
To: user@hive.apache.org
CC: jason.herba...@bateswhite.com

I hope, this property will fix your issue.

set mapred.reduce.child.java.opts=-Xmx4096m;






On Mon, Mar 24, 2014 at 3:59 PM, Clay McDonald  
wrote:

I believe I found my issue.



2014-03-24 15:49:38,775 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error 
running child : java.lang.OutOfMemoryError: Java heap space





Clay







From: Clay McDonald [mailto:stuart.mcdon...@bateswhite.com]

Sent: Monday, March 24, 2014 2:07 PM

To: 'user@hive.apache.org'

Subject: Joins Failing



My join query is failing. Any suggestions on how I should troubleshoot this?



92651139753_0036_1_conf.xml to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp

2014-03-24 13:48:58,244 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done 
location: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp

2014-03-24 13:48:58,248 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to 
done: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036.summary_tmp
 to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036.summary

2014-03-24 13:48:58,249 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to 
done: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp
 to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml


2014-03-24 13:48:58,251 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to 
done: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036-1395683232406-root-SELECT+COUNT%28A.UPC_ID%29+AS+CNT%0D%0AFR...B.UPC_ID%28Stage-1395683335043-0-0-FAILED-default.jhist_tmp
 to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036-1395683232406-root-SELECT+COUNT%28A.UPC_ID%29+AS+CNT%0D%0AFR...B.UPC_ID%28Stage-1395683335043-0-0-FAILED-default.jhist


2014-03-24 13:48:58,251 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped 
JobHistoryEventHandler. super.stop()

2014-03-24 13:48:58,252 INFO [Thread-65] 
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job 
diagnostics to Task failed task_1392651139753_0036_m_11

Job failed as tasks failed. failedMaps:1 failedReduces:0





Thanks Clay

Re: Joins Failing

2014-03-24 Thread Divakar Reddy 
I hope, this property will fix your issue.

set mapred.reduce.child.java.opts=-Xmx4096m;





On Mon, Mar 24, 2014 at 3:59 PM, Clay McDonald <
stuart.mcdon...@bateswhite.com> wrote:

> I believe I found my issue.
>
> 2014-03-24 15:49:38,775 FATAL [main] org.apache.hadoop.mapred.YarnChild:
> Error running child : java.lang.OutOfMemoryError: Java heap space
>
>
> Clay
>
>
>
> From: Clay McDonald [mailto:stuart.mcdon...@bateswhite.com]
> Sent: Monday, March 24, 2014 2:07 PM
> To: 'user@hive.apache.org'
> Subject: Joins Failing
>
> My join query is failing. Any suggestions on how I should troubleshoot
> this?
>
> 92651139753_0036_1_conf.xml to hdfs://
> dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp
> 2014-03-24 13:48:58,244 INFO [Thread-65]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to
> done location: hdfs://
> dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp
> 2014-03-24 13:48:58,248 INFO [Thread-65]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
> done: hdfs://
> dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036.summary_tmpto
>  hdfs://
> dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036.summary
> 2014-03-24 13:48:58,249 INFO [Thread-65]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
> done: hdfs://
> dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmpto
>  hdfs://
> dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml
> 2014-03-24 13:48:58,251 INFO [Thread-65]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to
> done: hdfs://
> dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036-1395683232406-root-SELECT+COUNT%28A.UPC_ID%29+AS+CNT%0D%0AFR...B.UPC_ID%28Stage-1395683335043-0-0-FAILED-default.jhist_tmpto
>  hdfs://
> dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036-1395683232406-root-SELECT+COUNT%28A.UPC_ID%29+AS+CNT%0D%0AFR...B.UPC_ID%28Stage-1395683335043-0-0-FAILED-default.jhist
> 2014-03-24 13:48:58,251 INFO [Thread-65]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped
> JobHistoryEventHandler. super.stop()
> 2014-03-24 13:48:58,252 INFO [Thread-65]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job
> diagnostics to Task failed task_1392651139753_0036_m_11
> Job failed as tasks failed. failedMaps:1 failedReduces:0
>
>
> Thanks Clay
>

RE: Joins Failing

2014-03-24 Thread Clay McDonald 
I believe I found my issue.

2014-03-24 15:49:38,775 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error 
running child : java.lang.OutOfMemoryError: Java heap space


Clay



From: Clay McDonald [mailto:stuart.mcdon...@bateswhite.com] 
Sent: Monday, March 24, 2014 2:07 PM
To: 'user@hive.apache.org'
Subject: Joins Failing

My join query is failing. Any suggestions on how I should troubleshoot this?

92651139753_0036_1_conf.xml to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp
2014-03-24 13:48:58,244 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done 
location: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp
2014-03-24 13:48:58,248 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to 
done: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036.summary_tmp
 to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036.summary
2014-03-24 13:48:58,249 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to 
done: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml_tmp
 to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036_conf.xml
2014-03-24 13:48:58,251 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to 
done: 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036-1395683232406-root-SELECT+COUNT%28A.UPC_ID%29+AS+CNT%0D%0AFR...B.UPC_ID%28Stage-1395683335043-0-0-FAILED-default.jhist_tmp
 to 
hdfs://dc-bigdata5.bateswhite.com:8020/mr-history/tmp/root/job_1392651139753_0036-1395683232406-root-SELECT+COUNT%28A.UPC_ID%29+AS+CNT%0D%0AFR...B.UPC_ID%28Stage-1395683335043-0-0-FAILED-default.jhist
2014-03-24 13:48:58,251 INFO [Thread-65] 
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped 
JobHistoryEventHandler. super.stop()
2014-03-24 13:48:58,252 INFO [Thread-65] 
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job 
diagnostics to Task failed task_1392651139753_0036_m_11
Job failed as tasks failed. failedMaps:1 failedReduces:0


Thanks Clay