Manuel Godbert created TEZ-3461: ----------------------------------- Summary: Tez not working in local mode for M/R jobs Key: TEZ-3461 URL: https://issues.apache.org/jira/browse/TEZ-3461 Project: Apache Tez Issue Type: Bug Reporter: Manuel Godbert
I have map/reduce jobs that work as expected within YARN, and I want to see if Tez can help me improving their performance. Alas, I am experiencing issues and I want to understand what happens, to see if I can adapt my code or if I can suggest Tez enhancements. For this I need to be able to debug jobs from within eclipse, with breakpoints in Tez source code etc. I am working on a linux (ubuntu) platform I use the latest Tez version I found, i.e. 0.9.0-SNAPSHOT (also tried with 0.7.0) I have set up the hortonworks mini dev cluster https://github.com/hortonworks/mini-dev-cluster I am trying to run the basic WordCount2 code found here https://hadoop.apache.org/docs/r2.7.2/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html#Example:_WordCount_v2.0 I added the following code to have tez running locally: conf.set("mapreduce.framework.name", "yarn-tez"); conf.setBoolean("tez.local.mode", true); conf.set("fs.default.name", "file:///"); conf.setBoolean("tez.runtime.optimize.local.fetch", true); And I am getting the following error: 2016-09-27 18:32:34 Running Dag: dag_1474992804027_0003_1 2016-09-27 18:32:34 Running Dag: dag_1474992804027_0003_1 Exception in thread "main" java.lang.NullPointerException at org.apache.tez.client.LocalClient.getApplicationReport(LocalClient.java:153) at org.apache.tez.dag.api.client.rpc.DAGClientRPCImpl.getAppReport(DAGClientRPCImpl.java:231) at org.apache.tez.dag.api.client.rpc.DAGClientRPCImpl.createAMProxyIfNeeded(DAGClientRPCImpl.java:251) at org.apache.tez.dag.api.client.rpc.DAGClientRPCImpl.getDAGStatus(DAGClientRPCImpl.java:96) at org.apache.tez.dag.api.client.DAGClientImpl.getDAGStatusViaAM(DAGClientImpl.java:360) at org.apache.tez.dag.api.client.DAGClientImpl.getDAGStatusInternal(DAGClientImpl.java:220) at org.apache.tez.dag.api.client.DAGClientImpl.getDAGStatus(DAGClientImpl.java:268) at org.apache.tez.dag.api.client.MRDAGClient.getDAGStatus(MRDAGClient.java:58) at org.apache.tez.mapreduce.client.YARNRunner.getJobStatus(YARNRunner.java:710) at org.apache.tez.mapreduce.client.YARNRunner.submitJob(YARNRunner.java:650) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:240) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308) at WordCount2.main(WordCount2.java:136) -- This message was sent by Atlassian JIRA (v6.3.4#6332)