[
https://issues.apache.org/jira/browse/PIG-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565038#comment-13565038
]
Cheolsoo Park commented on PIG-3145:
------------------------------------
Hi Prashant,
Thank you very much for pointing that out.
Indeed, PIg-3135 is in a similar category, but but I think that my problem is a
bit different from yours. In my case, *-site.xml files are present in
classpath, but system properties (-Dkey=value) that are passed to JVM are not
honored. They are different, aren't they?
> Parameters in core-site.xml and mapred-site.xml are not correctly substituted
> -----------------------------------------------------------------------------
>
> Key: PIG-3145
> URL: https://issues.apache.org/jira/browse/PIG-3145
> Project: Pig
> Issue Type: Bug
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
>
> To reproduce the issue, please do the following:
> # Parameterize the address of name node in core-site.xml.
> {code}
> <property>
> <name>fs.default.name</name>
> <value>hdfs://${foo}:8020</value>
> </property>
> {code}
> # Set the value of "foo" via -D option.
> {code}
> export PIG_OPTS="-Dfoo=mr1-0.cheolsoo.com"
> {code}
> # Pig fails with the following error.
> {code}
> 2013-01-28 18:54:02,786 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
> to hadoop file system at: hdfs://${foo}:8020
> 2013-01-28 18:54:02,805 [main] ERROR org.apache.pig.Main - ERROR 2999:
> Unexpected internal error. null
> Details at logfile: /home/cheolsoo/pig-cdh/pig_1359428042522.log
> {code}
> Note that the parameter $\{foo\} in core-site.xml is not expanded. This is
> because the addresses of name node and job tracker are read directly from
> core-site.xml instead of reading via Configuration.get().
> {code:title=HExecutionEngine.java}
> // properties is Java Properties
> cluster = properties.getProperty(JOB_TRACKER_LOCATION);
> nameNode = properties.getProperty(FILE_SYSTEM_LOCATION);
> {code}
> Replacing these lines with Configuration.get() fixes the issue.
> {code:title=HExecutionEngine.java}
> // jc is Hadoop Configuration
> cluster = jc.get(JOB_TRACKER_LOCATION);
> nameNode = jc.get(FILE_SYSTEM_LOCATION);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
