[ https://issues.apache.org/jira/browse/PIG-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134304#comment-14134304 ]
Bhooshan Mogal commented on PIG-3441: ------------------------------------- Sorry for the delay. I debugged this issue further last week. I have created a project at https://github.com/bdmogal/pig-3441 to demonstrate this issue. I have a dummy {{MyFileSystem.java}} class that adds a resource {{myfs-site.xml}} to default resources and a pig script {{src/main/bin/myfs.pig}} that invokes this class. We tested this last week and found that properties from {{myfs-site.xml}} are not found by pig with or without applying this patch on trunk. If you then add {{myfs-site.xml}} using {{Configuration.addResource}} like [here|https://github.com/bdmogal/pig-3441/blob/master/src/main/java/pig/pig3441/MyFileSystem.java#L34], the properties become available. [~daijy] could you please try this and let me know? > Allow Pig to use default resources from Configuration objects > ------------------------------------------------------------- > > Key: PIG-3441 > URL: https://issues.apache.org/jira/browse/PIG-3441 > Project: Pig > Issue Type: Bug > Components: impl > Affects Versions: 0.11.1 > Reporter: Bhooshan Mogal > Assignee: Daniel Dai > Attachments: PIG-3441-2.patch, PIG-3441-3.patch, PIG-3441.patch, > PIG-3441_1.patch > > > Pig currently ignores parameters from configuration files added statically to > Configuration objects as Configuration.addDefaultResource(filename.xml). > Consider the following scenario - > In a hadoop FileSystem driver for a non-HDFS filesystem you load properties > specific to that FileSystem in a static initializer block in the class that > extends org.apache.hadoop.fs.Filesystem for your FileSystem like below - > {code} > class MyFileSystem extends FileSystem { > static { > Configuration.addDefaultResource("myfs-default.xml"); > Configuration.addDefaultResource("myfs-site.xml"); > } > } > {code} > Interfaces like the Hadoop CLI, Hive, Hadoop M/R can find configuration > parameters defined in these configuration files as long as they are on the > classpath. > However, Pig cannot find parameters from these files, because it ignores > configuration files added statically. > Pig should allow users to specify if they would like pig to read parameters > from resources loaded statically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)