[
https://issues.apache.org/jira/browse/PIG-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohini Palaniswamy updated PIG-2791:
------------------------------------
Attachment: PIG-2791-5-trunk.patch
Daniel,
Realized what caused your test failure. Had removed conf_file.delete() in
MiniDFSCluster.java as it was causing tests to randomly fail without
hadoop-site.xml if two builds were running simultaneously in hudson (patch
builds and actual builds). If you switch between versions (23 and then 20),
then the hadoop-site.xml created in /<homedir>/pigtest/conf is not correct and
causes failure during MiniDfsCluster setup. Saw that in 0.10 it is created in
build/classes instead of home dir and that fixes the problem better. Updated
the trunk patch putting the conf_file.delete() back.
Should I create a separate jira for putting conf_file.delete back in pig 0.10
with build/classes or post the patch here itself as this jira is not closed
yet. Don't want developers to waste time debugging this as a issue.
> Pig does not work with Namenode Federation
> ------------------------------------------
>
> Key: PIG-2791
> URL: https://issues.apache.org/jira/browse/PIG-2791
> Project: Pig
> Issue Type: Bug
> Components: grunt
> Affects Versions: 0.10.0
> Environment: Pig QE
> Reporter: patrick white
> Assignee: Rohini Palaniswamy
> Priority: Blocker
> Attachments: PIG-2791-0.patch, PIG-2791-1.patch, PIG-2791-2.patch,
> PIG-2791-3-branch10.patch, PIG-2791-3-trunk.patch, PIG-2791-4-branch10.patch,
> PIG-2791-4-trunk.patch, PIG-2791-5-trunk.patch, asf_test_notes.txt
>
>
> The Yahoo Pig QE team ran into a blocking issue when trying to test
> Client-Side Mount Tables, on a Federated cluster with two NNs, this blocks
> Pig Testing on Federation.
> Federation relies strongly on the use of CSMT with viewFS, QE found that in
> this configuration it is not possible to enter grunt shell because Pig makes
> a call to getDefaultReplication() on the fs, which is ambiguous over viewFS
> and causes core to throw a
> org.apache.hadoop.fs.viewfs.NotInMountpointException: "getDefaultReplication
> on empty path is invalid".
> This in turn cause Pig to exit with an internal error as follows:
> 2012-07-06 22:20:25,657 [main] INFO org.apache.pig.Main - Apache Pig version
> 0.10.1.0.1206081058 (r1348169) compiled Jun 08 2012, 17:58:42
> 2012-07-06 22:20:26,074 [main] WARN org.apache.hadoop.conf.Configuration -
> mapred.used.genericoptionsparser is deprecated. Instead, use
> mapreduce.client.genericoptionsparser.used
> 2012-07-06 22:20:26,076 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
> to hadoop file system at: viewfs:///
> 2012-07-06 22:20:26,080 [main] WARN org.apache.hadoop.conf.Configuration -
> fs.default.name is deprecated. Instead, use fs.defaultFS
> 2012-07-06 22:20:26,522 [main] ERROR org.apache.pig.Main - ERROR 2999:
> Unexpected internal error. getDefaultReplication on empty path is invalid
> 2012-07-06 22:20:26,522 [main] WARN org.apache.pig.Main - There is no log
> file to write to.
> 2012-07-06 22:20:26,522 [main] ERROR org.apache.pig.Main -
> org.apache.hadoop.fs.viewfs.NotInMountpointException: getDefaultReplication
> on empty path is invalid
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem.getDefaultReplication(ViewFileSystem.java:482)
> at
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:77)
> at
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58)
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:205)
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:118)
> at org.apache.pig.impl.PigContext.connect(PigContext.java:208)
> at org.apache.pig.PigServer.<init>(PigServer.java:246)
> at org.apache.pig.PigServer.<init>(PigServer.java:231)
> at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:47)
> at org.apache.pig.Main.run(Main.java:487)
> at org.apache.pig.Main.main(Main.java:111)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
