Hello we are trying to move to using YARN instead of legacy MapReduce
and we are seeing a lot of IOExeceptions in our pig jobs namely:
WARN org.apache.pig.tools.pigstats.PigStatsUtil - Failed to get running job
java.io.IOException
at
org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:317)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:385)
at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:487)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:184)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:624)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:622)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at
org.apache.hadoop.mapred.JobClient.getJobUsingCluster(JobClient.java:622)
at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:640)
at
org.apache.pig.tools.pigstats.PigStatsUtil.addSuccessJobStats(PigStatsUtil.java:345)
at
org.apache.pig.tools.pigstats.PigStatsUtil.accumulateStats(PigStatsUtil.java:257)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:324)
at org.apache.pig.PigServer.launchPlan(PigServer.java:1266)
at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1251)
at org.apache.pig.PigServer.execute(PigServer.java:1241)
at org.apache.pig.PigServer.executeBatch(PigServer.java:335)
at
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:137)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
at org.apache.pig.Main.run(Main.java:475)
at org.apache.pig.Main.main(Main.java:157)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
WARN org.apache.pig.tools.pigstats.PigStatsUtil - Failed to get
RunningJob for job job_1376720568599_6735
I was wondering if any one has seen this before and has any
suggestions on how to fix it. The jobs do seem to execute correctly
(there is output) but im guessing there is some configuration value I
don't have set correctly that would get rid of this error.
