[ https://issues.apache.org/jira/browse/PIG-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565042#comment-13565042 ]
Prashant Kommireddi commented on PIG-3145: ------------------------------------------ Agreed. I am guessing your changes would go into HExecutionEngine primarily. Would be good to get both changes in around the same time. > Parameters in core-site.xml and mapred-site.xml are not correctly substituted > ----------------------------------------------------------------------------- > > Key: PIG-3145 > URL: https://issues.apache.org/jira/browse/PIG-3145 > Project: Pig > Issue Type: Bug > Reporter: Cheolsoo Park > Assignee: Cheolsoo Park > > To reproduce the issue, please do the following: > # Parameterize the address of name node in core-site.xml. > {code} > <property> > <name>fs.default.name</name> > <value>hdfs://${foo}:8020</value> > </property> > {code} > # Set the value of "foo" via -D option. > {code} > export PIG_OPTS="-Dfoo=mr1-0.cheolsoo.com" > {code} > # Pig fails with the following error. > {code} > 2013-01-28 18:54:02,786 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting > to hadoop file system at: hdfs://${foo}:8020 > 2013-01-28 18:54:02,805 [main] ERROR org.apache.pig.Main - ERROR 2999: > Unexpected internal error. null > Details at logfile: /home/cheolsoo/pig-cdh/pig_1359428042522.log > {code} > Note that the parameter $\{foo\} in core-site.xml is not expanded. This is > because the addresses of name node and job tracker are read directly from > core-site.xml instead of reading via Configuration.get(). > {code:title=HExecutionEngine.java} > // properties is Java Properties > cluster = properties.getProperty(JOB_TRACKER_LOCATION); > nameNode = properties.getProperty(FILE_SYSTEM_LOCATION); > {code} > Replacing these lines with Configuration.get() fixes the issue. > {code:title=HExecutionEngine.java} > // jc is Hadoop Configuration > cluster = jc.get(JOB_TRACKER_LOCATION); > nameNode = jc.get(FILE_SYSTEM_LOCATION); > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira