[
https://issues.apache.org/jira/browse/MAPREDUCE-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858629#action_12858629
]
Dick King commented on MAPREDUCE-1641:
--------------------------------------
The subject of this jira is that in one case a user specified that a particular
archive be both in {{mapred.cache.archives}} and {{mapred.cache.files}} . This
duplication was not forseen in the code, which happened to do something
unpleasant. While the particular instance that triggered the bug report was
accidental, it occurs to me that you could want that -- especially with a
{{.jar}} file.
I see the point that if we silently explode a {{.jar}} file as well as copying
it locally [and using it as a {{-classpath}}, presumably] we could be exposed
to pain -- not in terms of the code working, because the exploded copy wouldn't
ever be accessed, but in terms of cleanup performance.
For the time being, I'll just error out as described on 16/Apr/10 08:12 PM .
> Job submission should fail if same uri is added for mapred.cache.files and
> mapred.cache.archives
> ------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1641
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1641
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distributed-cache
> Reporter: Amareshwari Sriramadasu
> Assignee: Dick King
> Fix For: 0.22.0
>
>
> The behavior of mapred.cache.files and mapred.cache.archives is different
> during localization in the following way:
> If a jar file is added to mapred.cache.files, it will be localized under
> TaskTracker under a unique path.
> If a jar file is added to mapred.cache.archives, it will be localized under a
> unique path in a directory named the jar file name, and will be unarchived
> under the same directory.
> If same jar file is passed for both the configurations, the behavior
> undefined. Thus the job submission should fail.
> Currently, since distributed cache processes files before archives, the jar
> file will be just localized and not unarchived.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
