[
https://issues.apache.org/jira/browse/MAPREDUCE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16193828#comment-16193828
]
Chris Trezzo commented on MAPREDUCE-5951:
-----------------------------------------
Hi [~sjlee0], [~kasha], [~mingma], [~jlowe], and [~vrushalic]!
I have made another push to get MapReduce support for the shared cache
committed. I have rebased the patch, added documentation and fixed all
warnings/issues that I see so far. At this point, I need a reviewer for the
final review. I know this patch is a big one, so if there is anything I can do
to help with the review process to make it easier to review, or if there is
someone else who might be interested in the review, please let me know. Some
good news:
# Much of the patch has already been reviewed by [~kasha] [~sjlee0] and
[~mingma] during previous iterations.
# I have ensured that the entire feature is behind a switch. As such, when
disabled (default) there are no effects for the user.
# I have functionally tested this patch on a pseudo distributed cluster.
# I have deployed this patch to a larger test cluster and ran jobs with the
patch.
# There is very similar code running in production that has been working for
years at this point.
My main goal is to commit this to trunk and branch-2 (2.9.0). If it can make it
into branch-3.0 for GA that would be great as well, but I understand that the
beta is already out ([~andrew.wang] please let me know what you think). Once I
get a +1 on the patch, I would be happy to do the work to commit.
Thanks in advance for the help and effort. I really do appreciate it!
> Add support for the YARN Shared Cache
> -------------------------------------
>
> Key: MAPREDUCE-5951
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5951
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Reporter: Chris Trezzo
> Assignee: Chris Trezzo
> Labels: BB2015-05-TBR
> Attachments: MAPREDUCE-5951-Overview.001.pdf,
> MAPREDUCE-5951-trunk.016.patch, MAPREDUCE-5951-trunk.017.patch,
> MAPREDUCE-5951-trunk.018.patch, MAPREDUCE-5951-trunk.019.patch,
> MAPREDUCE-5951-trunk-020.patch, MAPREDUCE-5951-trunk-021.patch,
> MAPREDUCE-5951-trunk-v10.patch, MAPREDUCE-5951-trunk-v11.patch,
> MAPREDUCE-5951-trunk-v12.patch, MAPREDUCE-5951-trunk-v13.patch,
> MAPREDUCE-5951-trunk-v14.patch, MAPREDUCE-5951-trunk-v15.patch,
> MAPREDUCE-5951-trunk-v1.patch, MAPREDUCE-5951-trunk-v2.patch,
> MAPREDUCE-5951-trunk-v3.patch, MAPREDUCE-5951-trunk-v4.patch,
> MAPREDUCE-5951-trunk-v5.patch, MAPREDUCE-5951-trunk-v6.patch,
> MAPREDUCE-5951-trunk-v7.patch, MAPREDUCE-5951-trunk-v8.patch,
> MAPREDUCE-5951-trunk-v9.patch
>
>
> Implement the necessary changes so that the MapReduce application can
> leverage the new YARN shared cache (i.e. YARN-1492).
> Specifically, allow per-job configuration so that MapReduce jobs can specify
> which set of resources they would like to cache (i.e. jobjar, libjars,
> archives, files).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
