Hi, When I build a data flow using the Beam SDK, can someone point me to the
code that represents the underlying representation of the beam model itself?
Is there an API that lets me retrieve the underlying protobuf-based graph for
the data flow? Perhaps some pointers to what code in the runne
The proto representation isn't (yet) part of the public API, and is still
under active development. However, if you're curious you can see it via
calling
pipeline.to_runner_api()
in Python or manually invoking classes under
https://github.com/apache/beam/tree/master/runners/core-constructio
Go currently prints out the model pipeline (as well as the Dataflow
representation) if you use the Dataflow runner. Pass --dry_run=true to not
actually submit a job, but just print out the representations. The graphx
package can also be used to generate a model pipeline manually.
On Tue, Mar 20,
: 3/21/18 12:18 AM (GMT+02:00) To: dev@beam.apache.org Subject: Re:
Common model for runners
The proto representation isn't (yet) part of the public API, and is still under
active development. However, if you're curious you can see it via calling
pipeline.to_runner_api()
in
AM (GMT+02:00)
> To: dev@beam.apache.org
> Subject: Re: Common model for runners
>
> The proto representation isn't (yet) part of the public API, and is still
> under active development. However, if you're curious you can see it via
> calling
>
> pipeline.t
