[
https://issues.apache.org/jira/browse/TEZ-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374065#comment-15374065
]
Bikas Saha commented on TEZ-3334:
---------------------------------
YARN-4577 for classpath isolation of aux services.
Perhaps the first thing could be the POC.
Which is take existing MR shuffle and change its packaging to org.apache.tez.
Then add it as tez_shuffle in YARN alongside mapreduce_shuffle. And verify that
tez jobs use Tez shuffle and MR jobs use MR shuffle (both shuffle services
running the same code effectively).
After that we can create follow up jiras for new features and improvements to
tez shuffle.
Sounds like a plan?
> Tez Custom Shuffle Handler
> --------------------------
>
> Key: TEZ-3334
> URL: https://issues.apache.org/jira/browse/TEZ-3334
> Project: Apache Tez
> Issue Type: New Feature
> Reporter: Jonathan Eagles
>
> For conditions where auto-parallelism is reduced (e.g. TEZ-3222), a custom
> shuffle handler could help reduce the number of fetches and could more
> efficiently fetch data. In particular if a reducer is fetching 100 pieces
> serially from the same mapper it could do this in one fetch call.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
