[
https://issues.apache.org/jira/browse/MAPREDUCE-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858374#action_12858374
]
Amareshwari Sriramadasu commented on MAPREDUCE-1641:
----------------------------------------------------
bq. Perhaps we should allow this, and both localize the file and unarchive it?
What do you think?
We should not make the file option to unarchive the file. We have seen many use
cases where users do not want their jars to be unjarred, for example HADOOP-5175
bq. We perform the check for conflicts between mapred.cache.files and
mapred.cache.archives when the user finally submits the offending JobConf .
+1
bq. In particular, I plan to make a new class DistributedCache.DuplicatedURI
extends InvalidJobConfException and throw that .
+1
> Job submission should fail if same uri is added for mapred.cache.files and
> mapred.cache.archives
> ------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1641
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1641
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distributed-cache
> Reporter: Amareshwari Sriramadasu
> Assignee: Dick King
> Fix For: 0.22.0
>
>
> The behavior of mapred.cache.files and mapred.cache.archives is different
> during localization in the following way:
> If a jar file is added to mapred.cache.files, it will be localized under
> TaskTracker under a unique path.
> If a jar file is added to mapred.cache.archives, it will be localized under a
> unique path in a directory named the jar file name, and will be unarchived
> under the same directory.
> If same jar file is passed for both the configurations, the behavior
> undefined. Thus the job submission should fail.
> Currently, since distributed cache processes files before archives, the jar
> file will be just localized and not unarchived.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
