Hey Joey, Dataflow and Beam playground are 2 options as you mentioned, locally many SDKs have local runner options with a visual component. For example, in Python you can use the interactive runner with the apache-beam-jupyterlab-sidepanel extension <https://cloud.google.com/dataflow/docs/guides/interactive-pipeline-development#visualize_the_data_through_the_interactive_beam_inspector> to view pipelines visually locally (this is similar to what the notebooks you reference are doing). You can also just call some of these pieces directly <https://stackoverflow.com/questions/72592971/way-to-visualize-beam-pipeline-run-with-directrunner> without an extension. Go has a dot runner <https://pkg.go.dev/github.com/apache/beam/sdks/v2@v2.50.0/go/pkg/beam/runners/dot> that produces a visual representation of a pipeline. Java has a similar dot renderer <https://mehmandarov.com/apache-beam-pipeline-graph/>.
Thanks, Danny On Thu, Aug 31, 2023 at 6:38 PM Joey Tran <joey.t...@schrodinger.com> wrote: > Hi all, > > What're all the current options for visualizing a pipeline? I'm guessing > Dataflow has a visualization. I saw that there are also Apache Beam > notebooks through GCP, and I'm aware of the Beam playground, but is there > an easy way to create and view the visualization locally? For example, I > might have a large codebase that's used to construct and run a pipeline, > and in this case I don't think any of those three solutions would be very > easy to use to visualize my pipeline (though I could be wrong) > > Best, > Joey > > -- > > Joey Tran | Senior Developer Il | AutoDesigner TL > > *he/him* > > [image: Schrödinger, Inc.] <https://schrodinger.com/> >
