Re: Confusing multiple output semantics in Python

2019-11-11 Thread Sam Rohde
I made https://github.com/apache/beam/pull/9954 that explores this. Thanks for the insight, Ning. Internally, we use a different representation. On Thu, Nov 7, 2019 at 2:27 PM Ning Kang wrote: > Hi Sam, > > Thanks for clarifying the accessor to output when building a pipeline. > > Internally, w

Re: Confusing multiple output semantics in Python

2019-11-07 Thread Ning Kang
Hi Sam, Thanks for clarifying the accessor to output when building a pipeline. Internally, we have AppliedPTransform, where the output is always a dictionary: https://github.com/apache/beam/blob/master/sdks/python/apache_beam/pipeline.py#L770 And it seems to me that with key 'None', the output wi

Confusing multiple output semantics in Python

2019-11-07 Thread Sam Rohde
Hi All, In the Python SDK there are three ways of representing the output of a PTransform with multiple PCollections: - dictionary: PCollection tag --> PCollection - tuple: index --> PCollection - DoOutputsTuple: tag, index, or field name --> PCollection I find this inconsistent way of