[ https://issues.apache.org/jira/browse/PIG-4918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-4918: ---------------------------- Attachment: PIG-4918-1.patch > Pig on Tez cannot switch pig.temp.dir to another fs > --------------------------------------------------- > > Key: PIG-4918 > URL: https://issues.apache.org/jira/browse/PIG-4918 > Project: Pig > Issue Type: Bug > Reporter: Daniel Dai > Assignee: Daniel Dai > Fix For: 0.17.0, 0.16.1 > > Attachments: PIG-4918-1.patch > > > If pig.temp.dir points to another fs, Pig fails. One such case is the > defaultFS is set to s3, but use hdfs as temp dir. Error message: > {code} > org.apache.pig.backend.hadoop.executionengine.JobCreationException: ERROR > 2017: Internal error creating job configuration. > at > org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler.getJob(TezJobCompiler.java:141) > at > org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler.compile(TezJobCompiler.java:79) > at > org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher.launchPig(TezLauncher.java:194) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:304) > at org.apache.pig.PigServer.launchPlan(PigServer.java:1431) > at > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1416) > at org.apache.pig.PigServer.execute(PigServer.java:1405) > at org.apache.pig.PigServer.executeBatch(PigServer.java:456) > at org.apache.pig.PigServer.executeBatch(PigServer.java:439) > at > org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:171) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:234) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205) > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81) > at org.apache.pig.Main.run(Main.java:631) > at org.apache.pig.Main.main(Main.java:177) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.util.RunJar.run(RunJar.java:221) > at org.apache.hadoop.util.RunJar.main(RunJar.java:136) > Caused by: java.lang.IllegalArgumentException: Wrong FS: > hdfs://pig-aws-devenv-5.openstacklocal:8020/tmp/daijy/temp-265134702/automaton-1.11-8.jar, > expected: s3a://pig-aws-devenv > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:658) > at org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:478) > at > org.apache.pig.backend.hadoop.executionengine.tez.TezResourceManager.addTezResource(TezResourceManager.java:82) > at > org.apache.pig.backend.hadoop.executionengine.tez.TezResourceManager.addTezResources(TezResourceManager.java:106) > at > org.apache.pig.backend.hadoop.executionengine.tez.plan.TezPlanContainer.getLocalResources(TezPlanContainer.java:107) > at > org.apache.pig.backend.hadoop.executionengine.tez.TezJobCompiler.getJob(TezJobCompiler.java:95) > ... 20 more > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)