TheNeuralBit commented on PR #17384:
URL: https://github.com/apache/beam/pull/17384#issuecomment-1112774127
@robertwb I think this is largely ready. I added a new `WindowedBatch`
implementation, `HomogeneousWindowedBatch` which is effectively a wrapper
around `WindowedValue`, where we know the value is a batch. We now enforce that
batches are partitioned by `(timestamp, windows, pane_info)` and only create
`HomoegeneousWindowedBatch` instances.
The old implementation, `ConcreteWindowedBatch` isn't used, it's probably
worth just backing that out and saving it for a future PR.
Other notes: I've added support for {Timestamp, PaneInfo, Window}Param on
process_batch (and we validate only these are used at construction time). Other
DoFn params that were discussed in https://s.apache.org/batched-dofns are not
yet supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
