PigServer fails to run mapreduce against hadoop 0.23.
-----------------------------------------------------
Key: PIG-2635
URL: https://issues.apache.org/jira/browse/PIG-2635
Project: Pig
Issue Type: Bug
Affects Versions: 0.9.2
Reporter: Brian Tieman
Running Pig scripts through Grunt works without hiccup but I have a utility
program that runs commands through PigServer. When running that way, I run
into two problems.
The first, which I can hack around, is that I need to pass in to PigServer the
hbase configuration. When I do this with
ConfigurationUtil.toProperties(hbaseConfig), the new properties get passed
in-not the deprecated ones. Later on down the road, Pig looks for
fs.default.name which is the deprecated property for fs.defaultfs and throws
[1] The hack is to back map the fs.default.name property to the value of
fs.defaultfs after calling ConfigurationUtil.toProperties();
That brings me to the second problem which I haven't been able to hack around -
When PigServer runs, it throws the exception[2]. Poking through the code,
there is no longer an org.apache.hadoop.mapred.jobcontrol.JobControl.addJob()
method distributed with hadoop 0.23-only an
org.apache.hadoop.mapred.jobcontrol.JobControl.addJobs() method-which also
appears to be deprecated.
Again, Grunt works fine-just PigServer shows these problems. Has anyone
resolved this? Or should I log a bug?
Thanks!
[1]
java.lang.NullPointerException
at java.net.URI$Parser.parse(URI.java:3003)
at java.net.URI.<init>(URI.java:578)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.setHdfsServers(QueryParser.java:485)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.LoadClause(QueryParser.java:1618)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1366)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:1013)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:800)
at
org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1612)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1562)
at org.apache.pig.PigServer.registerQuery(PigServer.java:534)
at org.apache.pig.PigServer.registerQuery(PigServer.java:558)
at com.cerner.kepler.pig.console.KeyDump.dumpKeys(KeyDump.java:227)
[2]
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Lja
va/lang/String;
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:258)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:147)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:382)
at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1209)
at org.apache.pig.PigServer.storeEx(PigServer.java:885)
at org.apache.pig.PigServer.store(PigServer.java:827)
at com.cerner.kepler.pig.console.KeyDump.dumpKeys(KeyDump.java:231)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
