[
https://issues.apache.org/jira/browse/TEZ-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291516#comment-14291516
]
Bikas Saha commented on TEZ-391:
--------------------------------
I am glad that you have come to the conclusion that VertexGroup can be
symmetrically used to created shared outputs in the same manner as it is
currently used to create shared inputs. I had thought about shared edge
implementation after this jira was opened and this seemed like the most natural
solution. I should have noted that down in a design note earlier but looks like
we are on the same page. Before going down the implementation path, it would be
great if you could leave a design note that outline the flow - from API spec to
how the logic flows through to the tasks. This will help clear out the design
and enabled others to understand it better.
> SharedEdge - Support for passing same output from a vertex as input to two
> different vertices
> ---------------------------------------------------------------------------------------------
>
> Key: TEZ-391
> URL: https://issues.apache.org/jira/browse/TEZ-391
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Rohini Palaniswamy
> Assignee: Jeff Zhang
> Attachments: TEZ-391-WIP-1.patch, TEZ-391-WIP-2.patch,
> TEZ-391-WIP-3.patch
>
>
> We need this for lot of usecases. For cases where multi-query is turned off
> and for optimizing unions. Currently those are BROADCAST or ONE-ONE edges and
> we write the output multiple times.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
