[ https://issues.apache.org/jira/browse/TEZ-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15643079#comment-15643079 ]
Ming Ma commented on TEZ-3465: ------------------------------ +1. BTW, do you expect that is how applications will do cartesian product as shown in the example, e.g. in processor#run the application needs to fetch data from all inputs except for one and cache in memory and then iterate through the remaining input to do the join? If the data size is more dynamic, wonder if there is a way for the application to find the largest input at runtime and use that as the last input for iteration. > Support broadcast edge into cartesian product vertex and forbid other edges > --------------------------------------------------------------------------- > > Key: TEZ-3465 > URL: https://issues.apache.org/jira/browse/TEZ-3465 > Project: Apache Tez > Issue Type: Sub-task > Reporter: Zhiyuan Yang > Assignee: Zhiyuan Yang > Attachments: TEZ-3465.1.patch, TEZ-3465.2.patch, TEZ-3465.3.patch, > TEZ-3465.4.patch > > > Cartesian product vertex manager should support other incoming edge type. > Currently only broadcast edge is necessary, although potentially more edge > types could also be. Custom edge need its own vertex manager which can't work > with Cartesian product VM, so it has to be forbade. -- This message was sent by Atlassian JIRA (v6.3.4#6332)