Hello, Consider the following graph: A ---- E ----> B With source vertex A connected to destination vertex B through edge E.
Edge E is defined through: https://github.com/apache/tez/blob/fe22f3276d6d97f6b5dfab24490ee2ca32bf73c3/tez-api/src/main/java/org/apache/tez/dag/api/EdgeProperty.java#L136-L156 with a null EdgeManagerPluginDescriptor, to have the edge manager setup at runtime. Assume that both vertex A and vertex B have a custom VertexManager, and need to be configured at runtime. Vertex B waits for vertex A to be configured, before attempting to configure itself. As part of its configuration, vertex B will use: https://github.com/apache/tez/blob/fe22f3276d6d97f6b5dfab24490ee2ca32bf73c3/tez-api/src/main/java/org/apache/tez/dag/api/VertexManagerPluginContext.java#L172-L208 to create the edge manager for E amongst other things. However, because of the following code: https://github.com/apache/tez/blob/fe22f3276d6d97f6b5dfab24490ee2ca32bf73c3/tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/VertexImpl.java#L2898-L2909 which adds edge E to the set of uninitialized edges, when vertex A has been reconfigured, then it's call to: https://github.com/apache/tez/blob/fe22f3276d6d97f6b5dfab24490ee2ca32bf73c3/tez-api/src/main/java/org/apache/tez/dag/api/VertexManagerPluginContext.java#L349-L353 will end up not sending any Vertexstate.CONFIGURE updates: https://github.com/apache/tez/blob/fe22f3276d6d97f6b5dfab24490ee2ca32bf73c3/tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/VertexImpl.java#L1932 because canInitVertex returns false, as vertex.uninitializedEdges is not empty: https://github.com/apache/tez/blob/fe22f3276d6d97f6b5dfab24490ee2ca32bf73c3/tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/VertexImpl.java#L3172 Now, vertex A has no way of creating an edge manager for edge E, so that edge will never be initialized - only vertex B does that initialization. This ends up in a deadlock, as vertex B never gets the notification that vertex A has been configured. What is then the right way of constructing this graph. Do uninitialized target edges need to be monitored in source vertices at all? i.e. is this code needed: https://github.com/apache/tez/blob/fe22f3276d6d97f6b5dfab24490ee2ca32bf73c3/tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/VertexImpl.java#L2898-L2909 Thank you for any insight. Adrian
