[ https://issues.apache.org/jira/browse/PIG-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566853#comment-13566853 ]
Cheolsoo Park commented on PIG-3135: ------------------------------------ [~prkommireddi] Overall looks good to me. I have 3 questions. * Can we call the property "pig.use.overriden.hadoop.configs"? What we're overriding here are basically Hadoop confs, so I think that a more specific name is better. Do you agree? * On a related note, can you update the following comment in testJobConfGeneration? From: {code} // This should fail as pig expects classpath to be set {code} To something like: {code} // This should fail as pig expects Hadoop configs are present in classpath. {code} * Can you add this new property to conf/pig.properties with some explanation, so people can know about it? It would be nice if we could mention that this is a Mr-mode-specific property too. Please let me know if you have a better suggestion regarding how to document this. Thanks! > HExecutionEngine should look for resources in user passed Properties > -------------------------------------------------------------------- > > Key: PIG-3135 > URL: https://issues.apache.org/jira/browse/PIG-3135 > Project: Pig > Issue Type: Bug > Affects Versions: 0.10.0 > Reporter: Prashant Kommireddi > Assignee: Prashant Kommireddi > Attachments: PIG-3135.patch > > > Looking at this snippet: > {code} > private void init(Properties properties) throws ExecException { > . > . > . > // Check existence of hadoop-site.xml or core-site.xml > Configuration testConf = new Configuration(); > ClassLoader cl = testConf.getClassLoader(); > URL hadoop_site = cl.getResource( HADOOP_SITE ); > URL core_site = cl.getResource( CORE_SITE ); > > if( hadoop_site == null && core_site == null ) { > throw new ExecException("Cannot find hadoop configurations in > classpath (neither hadoop-site.xml nor core-site.xml was found in the > classpath)." + > " If you plan to use local mode, please put -x local > option in command line", > 4010); > } > {code} > This assumes the resources (*-site.xml) are set on the classpath, but this > will not always be the case when run with Pig's Java APIs. One could want to > programatically set the resources and the code here should additionally check > if they are available in there. > Example: When a Configuration object is created and resources are added > before passing it on to Pig. > {code} > Configuration conf = new Configuration(false); > conf.addResource("foo/core-site.xml"); > conf.addResource("bar/hadoop-site.xml"); > PigServer pServer = new PigServer(ExecType.MAPREDUCE, conf); > {code} > The above conf is not used right now to obtain resources. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira