Re: hive 1.1.0 on tez0.53 error

Could you file a HIVE jira for this? The session scratch dir should not be deleted after a query completes. We can pull in folks from both the respective Hive and Tez communities to debug why this is happening. The Tez AM only tries to delete the staging data on AM shutdown which in this scena

Re: Re: hive 1.1.0 on tez0.53 error

Here is hive log: Status: Running (Executing on YARN cluster with App id application_1433219182593_180456) Map 1: -/- Reducer 2: 0/5 Reducer 3: 0/5 Map 1: 0(+1)/1 Reducer 2: 0/5 Reducer 3: 0/5 Map 1: 1/1 Reducer 2: 0(+1)/5 Reducer 3: 0/5 Map 1: 1/1 Reducer 2: 0(+5)/5 Reducer 3: 0/5 Map 1: 1/

Re: hive 1.1.0 on tez0.53 error

That particular log is a red herring and not really an issue that is causing the failure. The main problem based on the log is this: 2015-06-17 18:00:43,543 INFO [AsyncDispatcher event handler] history.HistoryEventHandler: [HISTORY][DAG:dag_1433219182593_180456_3][Event:DAG_FINISHED]: dagId

hive 1.1.0 on tez0.53 error

Hive return: Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask I check log found : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /tmp/hive/lujian/_tez_session_dir/86bc0010-4816-4251-95aa-bb37b8d029da/.tez/

RE: ClassNotFoundException with custom InputFormat.

If I understand this right, there is a jar with user code in it. The jar needs to be available during split creation but it is not available. Is split creation happening on the client or on the AM. If its happening on the AM, and the AM is not getting the jars then how are you specifying the jar

Re: ClassNotFoundException with custom InputFormat.

cross posting down to dev… should continue the discussion there I believe. as I understand it, all Cascading users familiar with packaging a Hadoop job jar with a lib folder, in which the packaged custom InputFormat is placed — pulled from maven etc, will have this issue. this also expands to p

ClassNotFoundException with custom InputFormat.

Hi, we are currently running into a problem when a user of Cascading uses a custom InputFormat with Tez. The ApplicationMaster is running into a ClassNotFoundException when calculating the splits, since we are unable to control the environment/classpath visibile to the ApplicationMaster. We have a

Re: 2nd port on Application Master

So created https://issues.apache.org/jira/browse/TEZ-2561 for it. As of the patch if have to see if i find time… thanks Johannes > On 17 Jun 2015, at 19:05, Hitesh Shah wrote: > > This may likely be TaskAttemptListenerImpTezDag used for communication > between the AM and the task containers. T

Re: 2nd port on Application Master

This may likely be TaskAttemptListenerImpTezDag used for communication between the AM and the task containers. There are no configuration knobs for it currently as it is always internal to a cluster. If you don’t mind, can you file a jira and contribute a patch to add a clarifying log message t

2nd port on Application Master

Hey guys, i’m trying to hunt down some port usage from the tez AM. Log looks like 2015-06-17 18:33:30,104 INFO [main] history.HistoryEventHandler: Initializing HistoryEventHandler 2015-06-17 18:33:30,110 INFO [main] impl.SimpleHistoryLoggingService: Log file location for SimpleHistoryLoggingSer