[
https://issues.apache.org/jira/browse/BEAM-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17546375#comment-17546375
]
Kenneth Knowles commented on BEAM-2294:
---------------------------------------
This issue has been migrated to https://github.com/apache/beam/issues/18247
> Initial size estimation fails for mobile gaming examples for DataflowRunner
> when run in Windows
> -----------------------------------------------------------------------------------------------
>
> Key: BEAM-2294
> URL: https://issues.apache.org/jira/browse/BEAM-2294
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Chamikara Madhusanka Jayalath
> Priority: P3
>
> This happens for pipelines user_score.py and hourly_team_score.py
> Jobs complete successfully but initial size estimation fails with following
> error.
> INFO:root:Could not estimate size of source
> <apache_beam.io.textio._TextSource object at 0x03E34FF0> due to an exception
> : Traceback (most recent call last):
> File
> "C:\Users\chamikara\pythontest1\rc4_test\env_rc4_1\lib\site-packages\apache_beam\runners\dataflow\dataflow_runner
> .py", line 554, in run_Read
> transform.source.estimate_size())
> File
> "C:\Users\chamikara\pythontest1\rc4_test\env_rc4_1\lib\site-packages\apache_beam\internal\gcp\json_value.py",
> lin
> e 59, in get_typed_value_descriptor
> raise TypeError('Cannot get a type descriptor for %s.' % repr(obj))
> TypeError: Cannot get a type descriptor for 23899840340L.
> Seems to be due to a combination of Windows returning Long for type name [1]
> and SDK not handling long type at [2].
> [1]
> http://stackoverflow.com/questions/22513445/python-handles-long-ints-differently-on-windows-and-unix
> [2]
> https://github.com/apache/beam/blob/master/sdks/python/apache_beam/internal/gcp/json_value.py#L35
> cc: [~altay]
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
