[ https://issues.apache.org/jira/browse/BEAM-9377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17068131#comment-17068131 ]
Udi Meiri commented on BEAM-9377: --------------------------------- Verified for myself that Nones returned from map_fn indeed appear in the PCollection: {code} def test_typed_map_optional(self): # Optional does affect output type (Nones are NOT ignored). def map_fn(element: int) -> typehints.Optional[int]: if element == 1: return None else: return element result = [1, 2, 3] | beam.Map(map_fn) self.assertCountEqual([None, 2, 3], result) {code} > Python typehints: Map wrapper prevents Optional stripping > --------------------------------------------------------- > > Key: BEAM-9377 > URL: https://issues.apache.org/jira/browse/BEAM-9377 > Project: Beam > Issue Type: New Feature > Components: sdk-py-core > Reporter: Udi Meiri > Priority: Major > > This existing test is wrong: > {code} > def test_map_wrapper_optional_output(self): > # Optional does affect output type (Nones are NOT ignored). > def map_fn(unused_element: int) -> typehints.Optional[int]: > return 1 > th = beam.Map(map_fn).get_type_hints() > self.assertEqual(th.input_types, ((int, ), {})) > self.assertEqual(th.output_types, ((typehints.Optional[int], ), {})) > {code} > The resulting output type should be int. > {code} > inital output hint: > Optional[int] > with wrapper: > Iterable[Optional[int]] > with DoFn.default_type_hints: > Optional[int] > {code} > However any Nones returned by a DoFn's process method are dropped, so the > actual element_type returned is plain int. -- This message was sent by Atlassian Jira (v8.3.4#803005)