Hi,
I tried running same query from Simple JDBC Client instead of Spring Batch
framework its was working fine.
But now when I am running it again I am getting new error
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2015-02-26 11:41:50,233 Stage-1 map = 0%, reduce = 0%
2015-02-26 11:42:11,975 Stage-1 map = 47%, reduce = 0%, Cumulative CPU 15.13
sec
2015-02-26 11:42:24,593 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 27.55
sec
java.io.IOException: Could not find status of job:job_1424930506879_0003
at
org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:294)
at
org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:547)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:426)
at
org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:72)
Ended Job = job_1424930506879_0003 with exception 'java.io.IOException(Could
not find status of job:job_1424930506879_0003)'
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
Job HistoryServer is running properly.
Thanks
Shweta Jadhav
-----Vaibhav Gumashta <[email protected]> wrote: -----
To: "[email protected]" <[email protected]>
From: Vaibhav Gumashta <[email protected]>
Date: 02/26/2015 02:08AM
Subject: Re: ERROR OutOfMemoryError: Java heap space
Hi,
auth=nosasl should be only provided in JDBC url if
"hive.server2.authentication” is explicitly set to NOSASL in your
hive-site. Otherwise, the jdbc client tries to open a plain socket while the
the server is expecting a sasl negotiation to happen. Due to an issue with
thrift, this results in OOM on client side.
—Vaibhav
From: Srinivas Thunga <[email protected]>
Reply-To: "[email protected]" <[email protected]>
Date: Wednesday, February 25, 2015 at 3:40 AM
To: "[email protected]" <[email protected]>
Subject: Re: ERROR OutOfMemoryError: Java heap space
Is your problem solved?
Thanks & Regards,
Srinivas T
On Wed, Feb 25, 2015 at 4:17 PM, Srinivas Thunga <[email protected]>
wrote:
Hi,
You can set fetch size based certain datatype like blob, image, etc
statement.setFetchSize(1000);
Try
this.
Thanks & Regards,
Srinivas T
On Wed, Feb 25, 2015 at 4:10 PM, Jadhav Shweta <[email protected]> wrote:
Query is working fine if i have small data set in same table.
But its throwing error for large data set
Thanks
Shweta Jadhav
-----Srinivas Thunga <[email protected]> wrote: -----
To: "[email protected]" <[email protected]>
From: Srinivas Thunga <[email protected]>
Date: 02/25/2015 03:45PM
Subject: Re: ERROR OutOfMemoryError: Java heap space
not required. Make sure there will not be unnecessary looping in your code
Thanks & Regards,
Srinivas T
On Wed, Feb 25, 2015 at 2:58 PM, Jadhav Shweta <[email protected]> wrote:
Do I need to increase HADOOP_HEAPSIZE
I haven't set it.
I have cluster with 4 machines each having 8GB ram.
I read somewhere that we should mention auth=noSasl in hive jdbc url. Is it
necessary
Thanks
Shweta Jadhav
-----Jadhav Shweta <[email protected]> wrote: -----
To: [email protected]
From: Jadhav Shweta <[email protected]>
Date: 02/25/2015 02:54PM
Subject: Re: ERROR OutOfMemoryError: Java heap space
I am running the query using spring batch framework.
Thanks
Shweta Jadhav
-----Srinivas Thunga <[email protected]> wrote: -----
To: "[email protected]" <[email protected]>
From: Srinivas Thunga <[email protected]>
Date: 02/25/2015 02:43PM
Subject: Re: ERROR OutOfMemoryError: Java heap space
Hi,
Might be problem in your code.
Generally it happens when there are unnecessary looping.
Please check your java code.
Thanks & Regards,
Srinivas T
On Wed, Feb 25, 2015 at 2:38 PM, Jadhav Shweta <[email protected]> wrote:
Hi
I have installed apache hive 0.13.0.
I have configured metastore as postgres db.
Query is working fine in beeline command line interface but giving Heap space
error while executing using JDBC java client.
Thanks
Shweta Jadhav
-----Srinivas Thunga <[email protected]> wrote: -----
To: "[email protected]" <[email protected]>
From: Srinivas Thunga <[email protected]>
Date: 02/25/2015 02:32PM
Subject: Re: ERROR OutOfMemoryError: Java heap space
Hi,
Let me know how you configured Hive?
R u using cloudera?
If you are installed Hive separately then have you configured Metastore?
Thanks & Regards,
Srinivas T
On Wed, Feb 25, 2015 at 2:27 PM, Jadhav Shweta <[email protected]> wrote:
Hi
I am running simple select query
select * from table;
thanks
Shweta Jadhav
-----Srinivas Thunga <[email protected]> wrote: -----
To: "[email protected]" <[email protected]>
From: Srinivas Thunga <[email protected]>
Date: 02/25/2015 01:09PM
Subject: Re:
Hi,
Can you place the query as well which you are trying?
Thanks & Regards,
Srinivas T
On Wed, Feb 25, 2015 at 1:02 PM, Jadhav Shweta <[email protected]> wrote:
Hi
I am trying to run hive query
Its getting executed from beeline interface
but its throwing
java.lang.OutOfMemoryError: Java heap space
error when connecting using jdbc.
I am using hive 0.13.0 version and hiveserver2.
which parameters i need to configure for the same.
thanks
Shweta Jadhav
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
