[
https://issues.apache.org/jira/browse/TEZ-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983301#comment-13983301
]
Hitesh Shah commented on TEZ-802:
---------------------------------
bq. assignVertexManager
It requires some digging into to understand what happens if a vertex has both a
1:1 as well as a scatter-gather edge. As a follow-up jira, I think, we should
move all logic into a common building blocks to handle all known edge types as
well as inputs. This would allow someone to create a vertex manager for a
vertex that has an HDFS input as well as a shuffle and allow it to slow start
(but disable auto-reduce in such cases). A common vertex manager, that handles
inputs and all known basic known types, should also work but creating building
blocks that can be embedded into any manager would be useful in the long run.
For handling any vertex with custom edges, we can fall back to a simple
inputready/immediate start manager.
Maybe add a bit more code comments to mention things such as a vertex with both
1:1 and scatter-gather edges will use an input ready ( based on order of
if-else clauses).
bq. createDAGPlanWithMixedEdges
Test should be changed to have vertices with custom edges as well as inputs to
account for all cases.
> Determination of Task Placement for 1-1 Edges
> ---------------------------------------------
>
> Key: TEZ-802
> URL: https://issues.apache.org/jira/browse/TEZ-802
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-802.1.patch, TEZ-802.2.patch
>
>
> Currently, task placement is determined by TaskLocationHints that are
> specified by the user in the API or during runtime. Everything else runs as
> without placement constraints. While this works for broadcase and
> scatter-gather edges, it does not work for 1-1 edges.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
