[ https://issues.apache.org/jira/browse/BEAM-8016?focusedWorklogId=347649&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-347649 ]
ASF GitHub Bot logged work on BEAM-8016: ---------------------------------------- Author: ASF GitHub Bot Created on: 21/Nov/19 19:43 Start Date: 21/Nov/19 19:43 Worklog Time Spent: 10m Work Description: KevinGG commented on issue #10132: [BEAM-8016] Pipeline Graph URL: https://github.com/apache/beam/pull/10132#issuecomment-557240292 > Does this mean that non-cached PCollections will also be displayed in the dot graph with a generated name? If any of the non-cached PCollections belongs to an output of a top level transform, then yes, they will be rendered as a circle with no name (we use `label=""` to override whatever name they might have) in it. Most of the non-cached PCollections exist in-between non-top-level transforms (such as sub transforms of a composite transform the user defines) as intermediate input-outputs within a composite transform. Since the DOT doesn't support expanding composite transforms, they will not be rendered. If in the future, we support expanding those transforms in graph, they will be rendered as circles without names too. And one more scenario: if the user re-executes some cells with `PCollection = AppliedPTransform` in notebook, since PCollections in those cells are re-evaluated, the original AppliedPTransforms will not automatically be replaced/pruned because there might still be valid downstream transforms in the pipeline, those previous refs to PCollections got re-evaluated will be rendered as circles without names. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 347649) Time Spent: 6h 50m (was: 6h 40m) > Render Beam Pipeline as DOT with Interactive Beam > --------------------------------------------------- > > Key: BEAM-8016 > URL: https://issues.apache.org/jira/browse/BEAM-8016 > Project: Beam > Issue Type: Improvement > Components: runner-py-interactive > Reporter: Ning Kang > Assignee: Ning Kang > Priority: Major > Time Spent: 6h 50m > Remaining Estimate: 0h > > With work in https://issues.apache.org/jira/browse/BEAM-7760, Beam pipeline > converted to DOT then rendered should mark user defined variables on edges. > With work in https://issues.apache.org/jira/browse/BEAM-7926, it might be > redundant or confusing to render arbitrary random sample PCollection data on > edges. > We'll also make sure edges in the graph corresponds to output -> input > relationship in the user defined pipeline. Each edge is one output. If > multiple down stream inputs take the same output, it should be rendered as > one edge diverging into two instead of two edges. > For advanced interactivity highlight where each execution highlights the part > of the pipeline really executed from the original pipeline, we'll also > provide the support in beta. -- This message was sent by Atlassian Jira (v8.3.4#803005)