For contrast, the Go SDK provides an Impulse transform directly (analogous to PBegin, part of the model) and has a ParDo0 (which like PDone has no output Pcollections). The numeral suffixing the go ParDo functions indicate the number of Output Pcollections are expected from the passed in DoFm.
On Tue, Jul 14, 2020, 5:03 PM Robert Bradshaw <rober...@google.com> wrote: > Yes, PBegin and PDone are used in the SDKs, but are not part of the model. > > I would be supportive of making PBegin more public to denote that a > transform is a "root" of the pipeline. PDone was required for Java, > however I don't think there's any use for it in the Python SDK (a > transform can simply not return any value (which is equivalent to > returning None) if it has no outputs. > > On Mon, Jul 13, 2020 at 8:17 PM Udi Meiri <eh...@google.com> wrote: > > > > Details: > > One item of interest that came up during the implementation of > BEAM-10258 [1] is how to treat PTransforms that act like sources or sinks. > > These transforms have either no input or output PCollections, > respectively. > > > > Internally, we use PBegin and PDone to denote this. (ex: [2]) > > IIUC, PBegin and PDone aren't part of the Beam model, and in the > pipeline description they manifest as empty input and output lists. > > > > To support type hinting, I propose making these types public. > > They are not currently listed in [3] and the documentation implies > they're internal. > > Java SDK already supports these types and makes them public. [4] > > > > > > [1] https://github.com/apache/beam/pull/12009 > > [2] > https://github.com/apache/beam/blob/e73e1d1cce93930fa3d85046b9bbae7c724926bf/sdks/python/apache_beam/io/gcp/experimental/spannerio.py#L471-L506 > > [3] > https://github.com/apache/beam/blob/e73e1d1cce93930fa3d85046b9bbae7c724926bf/sdks/python/apache_beam/pvalue.py#L61 > > [4] > https://github.com/apache/beam/blob/e73e1d1cce93930fa3d85046b9bbae7c724926bf/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/PTransform.java#L53-L56 >
