[
https://issues.apache.org/jira/browse/HADOOP-7425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064494#comment-13064494
]
Sudharsan Sampath commented on HADOOP-7425:
-------------------------------------------
Hi Steven,
Yes. You are right. This issue is seen on the trunk too. But to me, configuring
twice should not be an issue but processing the configured property without
realizing that the same configuration has been processed already could be
avoided.
Will it be ok if instead of maintaining a List, we maintain a Set of keyspecs
in the KeyFieldHelper and provide an equals method that returns true if its
params match exactly as the passed in. The downside is that if some job wants
this kind of double hashing done on the same keyspec that would be difficult
asking for a custom partitioner. But I doubt if there would be such a scenario.
This would also ensure we get the correct partition and increase performance as
we avoid a hash invoke for every record.
> ReflectionUtils.setConf would configure anything Configurable twice
> -------------------------------------------------------------------
>
> Key: HADOOP-7425
> URL: https://issues.apache.org/jira/browse/HADOOP-7425
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
> Affects Versions: 0.21.0, 0.23.0
> Reporter: steven zhuang
> Attachments: test.tar
>
>
> In the setConf method of org.apache.hadoop.util.ReflectionUtils, any
> instance of Configurable would be configured twice.
> In 0.21.0, KeyFieldBasedPartitioner implements the Configurable interface.
> When configured twice, it get two KeyDescription and gives out wrong
> partition number.
> public static void setConf(Object theObject, Configuration conf) {
> if (conf != null) {
> if (theObject instanceof Configurable) {
> ((Configurable) theObject).setConf(conf);
> }
> setJobConf(theObject, conf);
> }
> }
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
