> Chris, > Are you referring to TEZ-2237, or is there some additional work that isn't > reported yet ? > 2237 isn't related to Broadcast edges (not just yet anyway); I don't think > memory is causing issues either - increasing it is an optimization. > > If you're running into additional issues outside of TEZ-2237, can you please > open another jira or post details.
I am referring to that. The stoppages seem to share the pattern of two broadcast edges (of the same output) from one vertex to two vertices. Cyrille has shared with me his full partitioned pipeline graph for the DAG. its huge, but it seems one of the receivers of the broadcast seems to get stuck. I’m reading between the lines with the little info I do have. Anyway, this is a pattern I am seeing, doesn’t mean I think its the issue, so I am asking if this is a known issue. That said, I just added a rule to prevent Cascading from creating this particular manifestation of them, as they are a side-effect of some other rules we didn’t account for. Hopefully he has time to test next week sometime. — Chris K Wensel [email protected]
