[ https://issues.apache.org/jira/browse/BEAM-11037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17549023#comment-17549023 ]
Danny McCormick commented on BEAM-11037: ---------------------------------------- This issue has been migrated to https://github.com/apache/beam/issues/20633 > Python DirectRunner InputStream Issues (RuntimeError: VarLong too long) > ----------------------------------------------------------------------- > > Key: BEAM-11037 > URL: https://issues.apache.org/jira/browse/BEAM-11037 > Project: Beam > Issue Type: Bug > Components: runner-direct, sdk-py-core > Affects Versions: 2.24.0 > Environment: Windows 10 64bit, Python 3.7.9 > Reporter: Michael Stauffer > Priority: P3 > > I have a Beam workflow that runs fine on Dataflow. However, when I run it on > my local direct runner (Windows 10, Python SDK), I run into some strange > behaviour if I have a negative number in my PCollection (see below for an > example). > Minimal code example to reproduce the exception: > {code:java} > from apache_beam.options.pipeline_options import PipelineOptions > from apache_beam.pipeline import Pipeline > import apache_beam > if __name__ == '__main__': > pipeline_options = PipelineOptions() > with Pipeline(options=pipeline_options) as p: > lines = ( > p > | apache_beam.Create([ > {"Test": -1} > ])) > lines | apache_beam.io.WriteToText('./lines') > {code} > Exception: > {code:java} > File "apache_beam\coders\coder_impl.py", line 222, in > apache_beam.coders.coder_impl.StreamCoderImpl.decode > File "apache_beam\coders\coder_impl.py", line 224, in > apache_beam.coders.coder_impl.StreamCoderImpl.decode > File "apache_beam\coders\coder_impl.py", line 470, in > apache_beam.coders.coder_impl.FastPrimitivesCoderImpl.decode_from_stream > File "apache_beam\coders\coder_impl.py", line 450, in > apache_beam.coders.coder_impl.FastPrimitivesCoderImpl.decode_from_stream > File "apache_beam\coders\stream.pyx", line 193, in > apache_beam.coders.stream.InputStream.read_var_int64 > RuntimeError: VarLong too long. [while running 'Create/Map(decode)'] > {code} > Any help is highly appreciated. > -- This message was sent by Atlassian Jira (v8.20.7#820007)