hjtran commented on PR #35283:
URL: https://github.com/apache/beam/pull/35283#issuecomment-2980905141
@jrmccluskey with the current implementation of `beam.Filter`, the output
pcollection type is not always the same as the input pcollection., e.g. the
following would fail:
```python
import apache_beam as beam
def is_even(num: float):
return num % 2 == 0.0
with beam.Pipeline() as pipeline:
even_nums = (
pipeline
| beam.Create([1, 2
]).with_output_types(int)
| beam.Filter(is_even))
assert even_nums.element_type is int
```
We could fix this by implementing `Filter` as a composite `PTransform`
and just set the output pcollection element_type manually in `Filter.expand`.
Would it be worth doing this? Not sure how many use cases rely specifically on
`beam.Filter` being a `ParDo` or whether that's something we need to guarantee
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
