[
https://issues.apache.org/jira/browse/BEAM-7377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16864459#comment-16864459
]
Valentyn Tymofieiev commented on BEAM-7377:
-------------------------------------------
As of Beam 2.13.0, type inference on Python 3.6+ is limited, however the nature
of limitations is such that when a type cannot be correctly inferred, we
typeinference falls back to the least restrictive typehint "Any", this means
that a pipeline may miss run-time type-check validation, however pipeline
execution does not fail.
Following test snippet illustrates current limitations:
{noformat}
options = PipelineOptions([])
options.view_as(StandardOptions).streaming = True
p = TestPipeline(options=options)
pcoll = (p
| ReadFromPubSub('projects/project/topics/a_topic',
None, 'a_label', with_attributes=True,
timestamp_attribute='time')
| beam.Map(lambda x: x))
self.assertEqual(PubsubMessage, pcoll.element_type) # Fails on Python 3.6
since pcoll.element_type evaluates to Any
{noformat}
However users could help supply additional annotations via .with_output_types()
as a workaround, for example:
{noformat}
pcoll = (p
| ReadFromPubSub('projects/project/topics/a_topic',
None, 'a_label', with_attributes=True,
timestamp_attribute='time')
| beam.Map(lambda x: x).with_output_types(PubsubMessage))
self.assertEqual(PubsubMessage, pcoll.element_type) # Passes on Python
3.6.
{noformat}
That said, there is an outstanding https://github.com/apache/beam/pull/8856
that improves type inference for Python 3.6+ and addresses current limitations.
> Verify that Typehints annotations degrade gracefully on Py3.6+ until
> BEAM-2713 is resolved
> ------------------------------------------------------------------------------------------
>
> Key: BEAM-7377
> URL: https://issues.apache.org/jira/browse/BEAM-7377
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Affects Versions: 2.14.0
> Reporter: Valentyn Tymofieiev
> Assignee: Valentyn Tymofieiev
> Priority: Blocker
>
> cc: [~udim]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
