sohailshk opened a new pull request, #41320:
URL: https://github.com/apache/beam/pull/41320
Fixes #41306
`util.Reify.Window` declared its output type as its input type `T`, and
`WindowInValue` and `TimestampInValue` declared `tuple[K, V]`, but they output
`(element, timestamp, window)`, `(key, (value, timestamp, window))` and `(key,
TimestampedValue(value, timestamp))`. When the input has an inferred element
type (for example the output of `Create`, also after a `Map(lambda x: x)`), the
coder is picked from the declared output type and the runner fails to encode
the reified elements. The existing tests did not see it because their inputs
are `TimestampedValue`/`WindowedValue` objects, whose element type has no
specific coder. The reified part of the output types is now `Any` (`tuple[K,
Any]` for the two KV transforms). For inputs that already worked (element type
`Any`, or a KV whose value type is `Any`) the output coder does not change; a
precise tuple hint for `Window` would have changed it from
`FastPrimitivesCoder` to a `TupleCoder`, which matters for streaming update
compatibili
ty.
Test: `test_typed_input` runs the three transforms on a `Create` of `('a',
1)` and `('b', 2)`; it fails on master (the bundle fails while encoding the
output) and passes with the change. `pytest apache_beam/transforms/util_test.py
-k Reify`: 5 passed (1 failed, 4 passed on master source). `Reify.Window` on
`Create([1, 2])` failed on master with `TypeError: '<' not supported between
instances of 'tuple' and 'int'` (`TypeError: an integer is required` with Beam
2.77.0 from PyPI) and returns `(1, MIN_TIMESTAMP, GlobalWindow())` and `(2,
MIN_TIMESTAMP, GlobalWindow())` after the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
