[
https://issues.apache.org/jira/browse/PIG-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13586480#comment-13586480
]
Santhosh Srinivasan commented on PIG-3145:
------------------------------------------
The patch looks good. Chatted offline with Cheolsoo about how this was missed
in the first place and he has explained that in his previous comment. The unit
tests were run for Hadoop 23 which excluded TestHBaseStorage. However, it
failed for Hadoop 20. This is now fixed.
+1
> Parameters in core-site.xml and mapred-site.xml are not correctly substituted
> -----------------------------------------------------------------------------
>
> Key: PIG-3145
> URL: https://issues.apache.org/jira/browse/PIG-3145
> Project: Pig
> Issue Type: Bug
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.12
>
> Attachments: PIG-3145-2.patch, PIG-3145.patch
>
>
> To reproduce the issue, please do the following:
> # Parameterize the address of name node in core-site.xml.
> {code}
> <property>
> <name>fs.default.name</name>
> <value>hdfs://${foo}:8020</value>
> </property>
> {code}
> # Set the value of "foo" via -D option.
> {code}
> export PIG_OPTS="-Dfoo=mr1-0.cheolsoo.com"
> {code}
> # Pig fails with the following error.
> {code}
> 2013-01-28 18:54:02,786 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
> to hadoop file system at: hdfs://${foo}:8020
> 2013-01-28 18:54:02,805 [main] ERROR org.apache.pig.Main - ERROR 2999:
> Unexpected internal error. null
> Details at logfile: /home/cheolsoo/pig-cdh/pig_1359428042522.log
> {code}
> Note that the parameter $\{foo\} in core-site.xml is not expanded. This is
> because the addresses of name node and job tracker are read directly from
> core-site.xml instead of reading via Configuration.get().
> {code:title=HExecutionEngine.java}
> // properties is Java Properties
> cluster = properties.getProperty(JOB_TRACKER_LOCATION);
> nameNode = properties.getProperty(FILE_SYSTEM_LOCATION);
> {code}
> Replacing these lines with Configuration.get() fixes the issue.
> {code:title=HExecutionEngine.java}
> // jc is Hadoop Configuration
> cluster = jc.get(JOB_TRACKER_LOCATION);
> nameNode = jc.get(FILE_SYSTEM_LOCATION);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
