AmirF194 opened a new issue, #39942:
URL: https://github.com/apache/beam/issues/39942
### What happened?
`PickleCoder.as_deterministic_coder()` and
`_MemoizingPickleCoder.as_deterministic_coder()` raise `TypeError` on every
call instead of returning a working deterministic coder.
Both methods do:
```python
def as_deterministic_coder(self, step_label, error_message=None):
return FastPrimitivesCoder(self, requires_deterministic=step_label)
```
`FastPrimitivesCoder.__init__(self, fallback_coder=PickleCoder())` has never
accepted a `requires_deterministic` argument, so this raises:
```
TypeError: FastPrimitivesCoder.__init__() got an unexpected keyword argument
'requires_deterministic'
```
Repro on current `master`:
```python
from apache_beam.coders.coders import PickleCoder
PickleCoder().as_deterministic_coder('label')
# TypeError: FastPrimitivesCoder.__init__() got an unexpected keyword
argument 'requires_deterministic'
```
Root cause: commit 9be70c9861 (2021-03-08) changed
`PickleCoder.as_deterministic_coder` from the working
`DeterministicFastPrimitivesCoder(self, step_label)` to the broken
`FastPrimitivesCoder(self, requires_deterministic=...)` in the same diff that
added a `requires_deterministic` kwarg to a different class,
`coder_impl.FastPrimitivesCoderImpl` (the Cython impl, not the
`coders.FastPrimitivesCoder` wrapper referenced here). `_MemoizingPickleCoder`
(added later) copied the same already-broken line.
Any caller that resolves to one of these coders and asks for a deterministic
version hits this, for example `GroupByEncryptedKey.expand()` in
`apache_beam/transforms/util.py`, which wraps the call in `except ValueError`
and does not catch the `TypeError`, so it crashes instead of logging the
intended non-deterministic-key warning.
This is a narrower, separate defect from #28558's own headline claim (which
is about `pickle_library` being ignored, and was addressed by @tvalentyn as
working as intended for `PickleCoder`). @tvalentyn identified this specific
`as_deterministic_coder` bug in a comment on that thread in 2023 and invited a
PR for it, but #28558 itself is now closed and its title does not match this
fix, so filing it here as its own issue.
### Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
### Issue Components
- Component: Python SDK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
