Running with --no_pipeline_type_check also disables any type inference
for coders, so in this case (essentially) all your coders will be
PickleCoder. You're getting an error here because beam inferred the
output of MyDoFn to be a str and chose a (more efficient) str coder to
encode its outputs, but in fact it outputs lists.
It would be useful to see your code to see if this is a bug in Beam or
an erroneous type declaration.
On Fri, Jul 30, 2021 at 7:28 AM Evan Galpin <evan.gal...@gmail.com> wrote:
>
> Hi all,
>
> I wonder if anyone can shed some light on an issue I'm having with type
> checking and the Beam python SDK.
>
> Without changing any python code, I'm finding that running my pipeline with
> or without the option no_pipeline_type_check starts the pipeline without
> issue; no type check errors are raised and the pipeline begins processing.
>
> However, if I allow pipeline type checking, I get a coder error later at
> runtime:
>
> File "apache_beam/coders/coder_impl.py", line 271, in
> apache_beam.coders.coder_impl.CallbackCoderImpl.estimate_size
> File "/my/path/lib/python3.6/site-packages/apache_beam/coders/coders.py",
> line 189, in estimate_size
> return len(self.encode(value))
> File “/my/path/lib/python3.6/site-packages/apache_beam/coders/coders.py",
> line 411, in encode
> return value.encode('utf-8')
> AttributeError: 'list' object has no attribute 'encode' [while running
> 'ParDo(MyDoFn)']
>
> I do realize that the attribute error itself is a legitimate type of error,
> and list type does in fact have no "encode" attribute. But if I run my
> pipeline with --no_pipeline_type_check, this attr error does not arise and
> the pipeline completes without errors. What could be happening here?
>
> Thanks,
> Evan
