RE: LLAP - Hive on Tez - ERROR on SQL query

[Aaron Grubb]

I experienced weird classpath issues when deploying LLAP on a from-scratch 
cluster. Things like, it wasn’t able to find hive-default.xml and had no 
default configuration provider, so I had to include a hive-default.xml with 
every possible setting inside the Yarnfile package to get it to run. This 
doesn’t necessarily apply to HDP but my suggestion would be to trace which 
classes/configurations are being loaded as much as possible and verify they 
line up with what you expected.

From: Fernando Antunes 
Sent: Friday, February 28, 2020 2:21 PM
To: user@hive.apache.org
Subject: LLAP - Hive on Tez - ERROR on SQL query

Hi.

I’m trying to make LLAP on HDP 3.1.4 with Hive 3.1.0 and Kerberos enabled works.

When I run a SQL query like

 select count(*) from database group by column;

I've got the following error:

Caused by: java.lang.IllegalStateException
at 
com.google.common.base.Preconditions.checkState(Preconditions.java:492)
at 
org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator.(LlapTaskCommunicator.java:165)

The error only happens when I use LLAP, using HiverServer2 works.



Anyone knows how to fix this ?




More details

0) I’ve searched the Internet and found others with the same error but without 
answer

http://mail-archives.apache.org/mod_mbox/tez-user/201910.mbox/%3cdb7p194mb0380535868265df6862b849e93...@db7p194mb0380.eurp194.prod.outlook.com%3e
https://community.cloudera.com/t5/Support-Questions/Hive-Tez-with-llap-query-fail/m-p/270597


1) Application log form YARN


2020-02-27 18:24:43,533 [ERROR] [main] |app.DAGAppMaster|: Error starting 
DAGAppMaster
org.apache.tez.dag.api.TezUncheckedException: 
java.lang.reflect.InvocationTargetException
at 
org.apache.tez.dag.app.TaskCommunicatorManager.createCustomTaskCommunicator(TaskCommunicatorManager.java:215)
at 
org.apache.tez.dag.app.TaskCommunicatorManager.createTaskCommunicator(TaskCommunicatorManager.java:184)
at 
org.apache.tez.dag.app.TaskCommunicatorManager.(TaskCommunicatorManager.java:152)
at 
org.apache.tez.dag.app.DAGAppMaster.createTaskCommunicatorManager(DAGAppMaster.java:1163)
at 
org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:536)
at 
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at 
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at 
org.apache.tez.dag.app.TaskCommunicatorManager.createCustomTaskCommunicator(TaskCommunicatorManager.java:213)
... 11 more
Caused by: java.lang.IllegalStateException
at 
com.google.common.base.Preconditions.checkState(Preconditions.java:492)
at 
org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator.(LlapTaskCommunicator.java:165)
... 16 more
2020-02-27 18:24:43,535 [INFO] [shutdown-hook-0] |app.DAGAppMaster|: 
DAGAppMasterShutdownHook invoked

2) From the  LlapTaskCommunicator.java file I found :


 public LlapTaskCommunicator(
  TaskCommunicatorContext taskCommunicatorContext) {
super(taskCommunicatorContext);
Credentials credentials = taskCommunicatorContext.getAMCredentials();
if (credentials != null) {
  @SuppressWarnings("unchecked")
  Token llapToken =
  
(Token)credentials.getToken(LlapTokenIdentifier.KIND_NAME);
  this.token = llapToken;
} else {
  this.token = null;
}
if (LOG.isInfoEnabled()) {
  LOG.info<http://LOG.info>("Task communicator with a token " + token);
}
Preconditions.checkState((token != null) == 
UserGroupInformation.isSecurityEnabled()); <=. Line 165

3) From my application log I could see that token variable is not null

Task communicator with a token Kind: LLAP_TOKEN, Service: , Ident: (LLAP_TOKEN; 
LLAP_TOKEN 
owner=hive/x.a@x.a.com<mailto:owner=hive/x.a@x.a.com>,
 renewer=hive, realUser=, issueDate=1582827878248, maxDate=1584037478248, 
sequenceNumber=59, masterKeyId=184, cluster hive_llap0, app ID , signing false)


3) Reading Preco

LLAP - Hive on Tez - ERROR on SQL query

[Fernando Antunes]

Hi. 

I’m trying to make LLAP on HDP 3.1.4 with Hive 3.1.0 and Kerberos enabled 
works. 

When I run a SQL query like 

 select count(*) from database group by column;

I've got the following error:

Caused by: java.lang.IllegalStateException
at 
com.google.common.base.Preconditions.checkState(Preconditions.java:492)
at 
org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator.(LlapTaskCommunicator.java:165)

The error only happens when I use LLAP, using HiverServer2 works.


Anyone knows how to fix this ?




More details

0) I’ve searched the Internet and found others with the same error but without 
answer

http://mail-archives.apache.org/mod_mbox/tez-user/201910.mbox/%3cdb7p194mb0380535868265df6862b849e93...@db7p194mb0380.eurp194.prod.outlook.com%3e
 

https://community.cloudera.com/t5/Support-Questions/Hive-Tez-with-llap-query-fail/m-p/270597
 



1) Application log form YARN


2020-02-27 18:24:43,533 [ERROR] [main] |app.DAGAppMaster|: Error starting 
DAGAppMaster
org.apache.tez.dag.api.TezUncheckedException: 
java.lang.reflect.InvocationTargetException
at 
org.apache.tez.dag.app.TaskCommunicatorManager.createCustomTaskCommunicator(TaskCommunicatorManager.java:215)
at 
org.apache.tez.dag.app.TaskCommunicatorManager.createTaskCommunicator(TaskCommunicatorManager.java:184)
at 
org.apache.tez.dag.app.TaskCommunicatorManager.(TaskCommunicatorManager.java:152)
at 
org.apache.tez.dag.app.DAGAppMaster.createTaskCommunicatorManager(DAGAppMaster.java:1163)
at 
org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:536)
at 
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at 
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at 
org.apache.tez.dag.app.TaskCommunicatorManager.createCustomTaskCommunicator(TaskCommunicatorManager.java:213)
... 11 more
Caused by: java.lang.IllegalStateException
at 
com.google.common.base.Preconditions.checkState(Preconditions.java:492)
at 
org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator.(LlapTaskCommunicator.java:165)
... 16 more
2020-02-27 18:24:43,535 [INFO] [shutdown-hook-0] |app.DAGAppMaster|: 
DAGAppMasterShutdownHook invoked

2) From the  LlapTaskCommunicator.java file I found :

 public LlapTaskCommunicator(
  TaskCommunicatorContext taskCommunicatorContext) {
super(taskCommunicatorContext);
Credentials credentials = taskCommunicatorContext.getAMCredentials();
if (credentials != null) {
  @SuppressWarnings("unchecked")
  Token llapToken =
  
(Token)credentials.getToken(LlapTokenIdentifier.KIND_NAME);
  this.token = llapToken;
} else {
  this.token = null;
}
if (LOG.isInfoEnabled()) {
  LOG.info("Task communicator with a token " + token);
}
Preconditions.checkState((token != null) == 
UserGroupInformation.isSecurityEnabled()); <=. Line 165

3) From my application log I could see that token variable is not null 

Task communicator with a token Kind: LLAP_TOKEN, Service: , Ident: (LLAP_TOKEN; 
LLAP_TOKEN owner=hive/x.a@x.a.com, renewer=hive, realUser=, 
issueDate=1582827878248, maxDate=1584037478248, sequenceNumber=59, 
masterKeyId=184, cluster hive_llap0, app ID , signing false)


3) Reading Preconditions.java the checkState method


   * Ensures the truth of an expression involving the state of the calling 
instance, but not
   * involving any parameters to the calling method.
   *
   * @param expression a boolean expression
   * @throws IllegalStateException if {@code expression} is false
   * @see Verify#verify Verify.verify()
   */
  public static void checkState(boolean expression) {
if (!expression) {
  throw new IllegalStateException();
}
  }

4) from UseGroupInformation.java : the isSecurityEnabled method.  
ensureInitialized does not change 

Hive on tez error

[Sateesh Karuturi]

I am trying to connect hive database(execution.engine value changed to tez)
using Java code... In case of select query its working But in the case
of INSERT getting an error:
The error looks like.
Error while processing statement: FAILED: Execution Error, return code 1
from org.apache.hadoop.hive.ql.exec.tez.TezTask
Please help me out

Re: Hive on tez error

[Jianfeng (Jeff) Zhang]

Please refer this wiki page  
https://cwiki.apache.org/confluence/display/TEZ/How+to+Diagnose+Tez+App


Best Regard,
Jeff Zhang


From: Sateesh Karuturi 
sateesh.karutu...@gmail.commailto:sateesh.karutu...@gmail.com
Reply-To: u...@tez.apache.orgmailto:u...@tez.apache.org 
u...@tez.apache.orgmailto:u...@tez.apache.org
Date: Friday, August 28, 2015 at 12:33 AM
To: user@hive.apache.orgmailto:user@hive.apache.org 
user@hive.apache.orgmailto:user@hive.apache.org, 
u...@tez.apache.orgmailto:u...@tez.apache.org 
u...@tez.apache.orgmailto:u...@tez.apache.org
Subject: Hive on tez error


I am trying to connect hive database(execution.engine value changed to tez) 
using Java code... In case of select query its working But in the case of 
INSERT getting an error:
The error looks like.
Error while processing statement: FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.tez.TezTask
Please help me out

Re: tez error

[Jianfeng (Jeff) Zhang]

+ hive mail list

Might be a hive bug.


Best Regard,
Jeff Zhang


From: hanked...@emar.commailto:hanked...@emar.com 
hanked...@emar.commailto:hanked...@emar.com
Reply-To: user u...@tez.apache.orgmailto:u...@tez.apache.org
Date: Thursday, August 27, 2015 at 5:51 PM
To: user u...@tez.apache.orgmailto:u...@tez.apache.org
Subject: tez error


Hi ~all

I build tez-0.7.0 with 2.6.0-cdh5.4.0,  deploy and run demo wordcount 
success.
but, hive on tez problems.
error infomation:



Status: Running (Executing on YARN cluster with App id 
application_1440664563598_0005)


VERTICES  STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED

Map 1 FAILED -1  00   -1   0   0
Reducer 2 KILLED  1  001   0   0

VERTICES: 00/02  [--] 0%ELAPSED TIME: 0.24 s

Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1440664563598_0005_1_00, 
diagnostics=[Vertex vertex_1440664563598_0005_1_00 [Map 1] killed/failed due 
to:ROOT_INPUT_INIT_FAILURE, Vertex Input: test initializer failed, 
vertex=vertex_1440664563598_0005_1_00 [Map 1], 
java.lang.IllegalArgumentException: Illegal Capacity: -12185
at java.util.ArrayList.init(ArrayList.java:156)
at 
org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:330)
at 
org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:306)
at 
org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:408)
at 
org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:129)
at 
org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:245)
at 
org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:239)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at 
org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:239)
at 
org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:226)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
]
Vertex killed, vertexName=Reducer 2, vertexId=vertex_1440664563598_0005_1_01, 
diagnostics=[Vertex received Kill in INITED state., Vertex 
vertex_1440664563598_0005_1_01 [Reducer 2] killed/failed due to:null]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1
FAILED: Execution Error, return code 2 from 
org.apache.hadoop.hive.ql.exec.tez.TezTask



hanked...@emar.commailto:hanked...@emar.com

Re: Hive - Tez error with big join - Container expired.

[Gopal Vijayaraghavan]

 I have a pretty big Hive Query. I¹m joining over 3 Hive-Tables which
have thousands of lines each. I¹m grouping this join by several columns.

Hive-on-Tez shouldn¹t have any issue even with billion of lines on a JOIN.

 0 failed, info=[Containercontainer_1434357133795_0008_01_39 finished
while trying to launch. Diagnostics: [Container failed. Container expired
since it was unused]], TaskAttempt 1 failed,

Looks like your node manager is actually not spinning up a container that
was allocated (i.e allocation succeeded, but the task spin up failed).

Which YARN scheduler are you running (fair/capacity?) and do you have any
idea on what the logs on the NodeManager logs say about trying to spin up
this container?

If I¹m not wrong, you need to also check if the YARN user has a ulimit set
for the total number of processes on the NM nodes.


 
Cheers,
Gopal

Re: Hive - Tez error with big join - Container expired.

[Jianfeng (Jeff) Zhang]

Tez will hold the idle containers for a while, but it would also expire the 
container if it reach some threshold.
Have you set property tez.am.container.idle.release-timeout-max.millis in 
tez-site.xml ? And can you attach the yarn app log ?



Best Regard,
Jeff Zhang


From: Daniel Klinger d...@web-computing.demailto:d...@web-computing.de
Reply-To: user@hive.apache.orgmailto:user@hive.apache.org 
user@hive.apache.orgmailto:user@hive.apache.org
Date: Thursday, June 18, 2015 at 5:35 AM
To: user@hive.apache.orgmailto:user@hive.apache.org 
user@hive.apache.orgmailto:user@hive.apache.org
Subject: Hive - Tez error with big join - Container expired.

Hi all,

I have a pretty big Hive Query. I'm joining over 3 Hive-Tables which have 
thousands of lines each. I'm grouping this join by several columns. In the 
Hive-Shell this query only reach about 80%. After about 1400 seconds its 
canceling with the following error:

Status: Failed
Vertex failed, vertexName=Map 2, vertexId=vertex_1434357133795_0008_1_01, 
diagnostics=[Task failed, taskId=task_1434357133795_0008_1_01_33, 
diagnostics=[TaskAttempt 0 failed, 
info=[Containercontainer_1434357133795_0008_01_39 finished while trying to 
launch. Diagnostics: [Container failed. Container expired since it was 
unused]], TaskAttempt 1 failed, 
info=[Containercontainer_1434357133795_0008_01_55 finished while trying to 
launch. Diagnostics: [Container failed. Container expired since it was 
unused]], TaskAttempt 2 failed, 
info=[Containercontainer_1434357133795_0008_01_72 finished while trying to 
launch. Diagnostics: [Container failed. Container expired since it was 
unused]], TaskAttempt 3 failed, 
info=[Containercontainer_1434357133795_0008_01_000101 finished while trying to 
launch. Diagnostics: [Container failed. Container expired since it was 
unused]]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex 
vertex_1434357133795_0008_1_01 [Map 2] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0
FAILED: Execution Error, return code 2 from 
org.apache.hadoop.hive.ql.exec.tez.TezTask

My yarn resource manager is at 100% during the whole execution (using all of 
the 300 GB memory). I tried to extend the live time of my containers with the 
following setting in the yarn-site.xml but no success:

yarn.resourcemanager.rm.container-allocation.expiry-interval-ms = 120

After this change my query stays at 0% over thousands of seconds. The query 
itself is working (tested with less data). How can I solve this problem.

Thanks for your help.

Greetz
DK

Hive - Tez error with big join - Container expired.

[Daniel Klinger]

Hi all,

 

I have a pretty big Hive Query. I'm joining over 3 Hive-Tables which have
thousands of lines each. I'm grouping this join by several columns. In the
Hive-Shell this query only reach about 80%. After about 1400 seconds its
canceling with the following error:  

 

Status: Failed

Vertex failed, vertexName=Map 2, vertexId=vertex_1434357133795_0008_1_01,
diagnostics=[Task failed, taskId=task_1434357133795_0008_1_01_33,
diagnostics=[TaskAttempt 0 failed,
info=[Containercontainer_1434357133795_0008_01_39 finished while trying
to launch. Diagnostics: [Container failed. Container expired since it was
unused]], TaskAttempt 1 failed,
info=[Containercontainer_1434357133795_0008_01_55 finished while trying
to launch. Diagnostics: [Container failed. Container expired since it was
unused]], TaskAttempt 2 failed,
info=[Containercontainer_1434357133795_0008_01_72 finished while trying
to launch. Diagnostics: [Container failed. Container expired since it was
unused]], TaskAttempt 3 failed,
info=[Containercontainer_1434357133795_0008_01_000101 finished while trying
to launch. Diagnostics: [Container failed. Container expired since it was
unused]]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex
vertex_1434357133795_0008_1_01 [Map 2] killed/failed due to:null]

DAG failed due to vertex failure. failedVertices:1 killedVertices:0

FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.tez.TezTask

 

My yarn resource manager is at 100% during the whole execution (using all of
the 300 GB memory). I tried to extend the live time of my containers with
the following setting in the yarn-site.xml but no success:

 

yarn.resourcemanager.rm.container-allocation.expiry-interval-ms = 120

 

After this change my query stays at 0% over thousands of seconds. The query
itself is working (tested with less data). How can I solve this problem.

 

Thanks for your help.

 

Greetz

DK

hive on tez error

[Sateesh Karuturi]

iam using hive 1.0.0 and tez 0.5.2.. whenever iam trying to open the hive
getting following error:
Exception in thread main java.lang.RuntimeException: java.io.IOException:
Previous writer likely failed to write
hdfs://localhost:9000/tmp/hive/hadoop/_tez_session_dir/002dad89-59b6-43c9-92f9-1c7b2232b1c4/tez.
Failing because I am unlikely to write too.
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:457)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:626)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:570)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: Previous writer likely failed to write
hdfs://localhost:9000/tmp/hive/hadoop/_tez_session_dir/002dad89-59b6-43c9-92f9-1c7b2232b1c4/tez.
Failing because I am unlikely to write too.
at
org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:979)
at
org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:860)
at
org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:803)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:228)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:154)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:122)
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:454)
... 8 more

Re: hive on tez error

[Sateesh Karuturi]

(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp0  0 127.0.0.1:9000  0.0.0.0:*
LISTEN  8030/java
tcp0  0 127.0.0.1:9000  127.0.0.1:40274
ESTABLISHED 8030/java
tcp0  0 127.0.0.1:40442 127.0.0.1:9000
 ESTABLISHED 8323/java
tcp0  0 127.0.0.1:9000  127.0.0.1:40442
ESTABLISHED 8030/java
tcp0  0 127.0.0.1:40441 127.0.0.1:9000
 TIME_WAIT   -
tcp0  0 127.0.0.1:40274 127.0.0.1:9000
 ESTABLISHED 8157/java

On Tue, Jun 16, 2015 at 1:36 AM, Steve Howard stevedhow...@gmail.com
wrote:

 What does netstat -anp | grep 9000 show?

 On Mon, Jun 15, 2015 at 3:47 PM, Sateesh Karuturi 
 sateesh.karutu...@gmail.com wrote:

 iam using hive 1.0.0 and tez 0.5.2.. whenever iam trying to open the hive
 getting following error:
 Exception in thread main java.lang.RuntimeException:
 java.io.IOException: Previous writer likely failed to write
 hdfs://localhost:9000/tmp/hive/hadoop/_tez_session_dir/002dad89-59b6-43c9-92f9-1c7b2232b1c4/tez.
 Failing because I am unlikely to write too.
 at
 org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:457)
 at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:626)
 at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:570)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:483)
 at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
 Caused by: java.io.IOException: Previous writer likely failed to write
 hdfs://localhost:9000/tmp/hive/hadoop/_tez_session_dir/002dad89-59b6-43c9-92f9-1c7b2232b1c4/tez.
 Failing because I am unlikely to write too.
 at
 org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:979)
 at
 org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:860)
 at
 org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:803)
 at
 org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:228)
 at
 org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:154)
 at
 org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:122)
 at
 org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:454)
 ... 8 more

Re: hive on tez error

[Steve Howard]

What does netstat -anp | grep 9000 show?

On Mon, Jun 15, 2015 at 3:47 PM, Sateesh Karuturi 
sateesh.karutu...@gmail.com wrote:

 iam using hive 1.0.0 and tez 0.5.2.. whenever iam trying to open the hive
 getting following error:
 Exception in thread main java.lang.RuntimeException:
 java.io.IOException: Previous writer likely failed to write
 hdfs://localhost:9000/tmp/hive/hadoop/_tez_session_dir/002dad89-59b6-43c9-92f9-1c7b2232b1c4/tez.
 Failing because I am unlikely to write too.
 at
 org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:457)
 at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:626)
 at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:570)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:483)
 at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
 Caused by: java.io.IOException: Previous writer likely failed to write
 hdfs://localhost:9000/tmp/hive/hadoop/_tez_session_dir/002dad89-59b6-43c9-92f9-1c7b2232b1c4/tez.
 Failing because I am unlikely to write too.
 at
 org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:979)
 at
 org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:860)
 at
 org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:803)
 at
 org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:228)
 at
 org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:154)
 at
 org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:122)
 at
 org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:454)
 ... 8 more

Re: hive tez error

[r7raul1...@163.com]

hive -hiveconf hive.root.logger=DEBUG,consoleTo See  some debug log



r7raul1...@163.com
 
From: Sateesh Karuturi
Date: 2015-06-08 16:15
To: user
Subject: hive tez error
getting FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.tez.TezTask
 error... when iam trying to perform insert operation on hive(set 
hive.execution.engine=tez).
whenever hive.execution.engine value is set to mr its works fine

please help me out 

hive tez error

[Sateesh Karuturi]

getting FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.tez.TezTask
 error... when iam trying to perform insert operation on hive(set
hive.execution.engine=tez).
whenever hive.execution.engine value is set to mr its works fine

please help me out

Re: hive tez error

[Jianfeng (Jeff) Zhang]

Sorry, to make it more clear. The applicationId is the placeholder of your 
applicationId of your hive session. You can look for it in the RM web UI



Best Regard,
Jeff Zhang


From: Sateesh Karuturi 
sateesh.karutu...@gmail.commailto:sateesh.karutu...@gmail.com
Reply-To: user@hive.apache.orgmailto:user@hive.apache.org 
user@hive.apache.orgmailto:user@hive.apache.org
Date: Monday, June 8, 2015 at 3:52 PM
To: user@hive.apache.orgmailto:user@hive.apache.org 
user@hive.apache.orgmailto:user@hive.apache.org
Subject: Re: hive tez error

when i try to check getting this

hadoop@localhost:~$ yarn logs -applicationId
options parsing failed: Missing argument for option: applicationId
Retrieve logs for completed YARN applications.
usage: yarn logs -applicationId application ID [OPTIONS]

general options are:
 -appOwner Application Owner   AppOwner (assumed to be current user if
 not specified)
 -containerId Container ID ContainerId (must be specified if node
 address is specified)
 -nodeAddress Node Address NodeAddress in the format nodename:port
 (must be specified if container id is
 specified)


On Mon, Jun 8, 2015 at 1:20 PM, Jianfeng (Jeff) Zhang 
jzh...@hortonworks.commailto:jzh...@hortonworks.com wrote:

Could you check the yarn app logs ?

By invoking command : yarn logs -applicationId

Best Regard,
Jeff Zhang


From: Sateesh Karuturi 
sateesh.karutu...@gmail.commailto:sateesh.karutu...@gmail.com
Reply-To: user@hive.apache.orgmailto:user@hive.apache.org 
user@hive.apache.orgmailto:user@hive.apache.org
Date: Monday, June 8, 2015 at 3:45 PM
To: user@hive.apache.orgmailto:user@hive.apache.org 
user@hive.apache.orgmailto:user@hive.apache.org
Subject: hive tez error

getting FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.tez.TezTask
 error... when iam trying to perform insert operation on hive(set 
hive.execution.engine=tez).
whenever hive.execution.engine value is set to mr its works fine

please help me out

Re: hive tez error

[Sateesh Karuturi]

when i try to check getting this

hadoop@localhost:~$ yarn logs -applicationId
options parsing failed: Missing argument for option: applicationId
Retrieve logs for completed YARN applications.
usage: yarn logs -applicationId application ID [OPTIONS]

general options are:
 -appOwner Application Owner   AppOwner (assumed to be current user if
 not specified)
 -containerId Container ID ContainerId (must be specified if node
 address is specified)
 -nodeAddress Node Address NodeAddress in the format nodename:port
 (must be specified if container id is
 specified)


On Mon, Jun 8, 2015 at 1:20 PM, Jianfeng (Jeff) Zhang 
jzh...@hortonworks.com wrote:


  Could you check the yarn app logs ?

  By invoking command : “yarn logs -applicationId

  Best Regard,
 Jeff Zhang


   From: Sateesh Karuturi sateesh.karutu...@gmail.com
 Reply-To: user@hive.apache.org user@hive.apache.org
 Date: Monday, June 8, 2015 at 3:45 PM
 To: user@hive.apache.org user@hive.apache.org
 Subject: hive tez error

   getting FAILED: Execution Error, return code 1 from
 org.apache.hadoop.hive.ql.exec.tez.TezTask
  error... when iam trying to perform insert operation on hive(set
 hive.execution.engine=tez).
 whenever hive.execution.engine value is set to mr its works fine

  please help me out

Hive on Tez Error

[peterm_second]

Hi Guys,
I am trying to enable tez on my hiveserver2 but everytime I get this error

Exception in thread main java.lang.RuntimeException: 
java.io.IOException: Previous writer likely failed to write 
hdfs://hadoop-nn.mo-data.com:9000/tmp/hive/root/_tez_session_dir/a0087fb2-1430-43fa-b3e1-06644ab4961d/*. 
Failing because I am unlikely to write too.
at 
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:457)

at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:672)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.io.IOException: Previous writer likely failed to write 
hdfs://hadoop-nn.mo-data.com:9000/tmp/hive/root/_tez_session_dir/a0087fb2-1430-43fa-b3e1-06644ab4961d/*. 
Failing because I am unlikely to write too.
at 
org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:974)
at 
org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:855)
at 
org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:798)
at 
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:228)
at 
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:154)
at 
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:122)
at 
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:454)

... 7 more

I am a noob when it comes to tez and hive, has anyone seen this problem 
? Any ideas what might be causing it.
As far as I understand tez doesn't use the distributed cashe so it tries 
to localize all the resources for each session. I have no idea what 
process is holding those files and causing it to fail with this error.

I am using Hive 0.14.0 and tez 0.6

Regards,
Peter