[
https://issues.apache.org/jira/browse/FLINK-41104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18131063#comment-18131063
]
Sangkyoon Nam commented on FLINK-41104:
---------------------------------------
[~dianfu], I'd like to take this one as well, as a separate PR from
FLINK-41106. At {{e206faa4ca2}} I reproduced {{contains('x')}} returning
{{False}} after {{put('x', None)}}, both before and after commit, and the fix
is ready. I'll add regression coverage for those cases, missing keys, removal,
and putting a {{None}} value after {{clear()}}. Could you assign this to me?
> MapState.contains() returns false for an existing key whose value is None
> -------------------------------------------------------------------------
>
> Key: FLINK-41104
> URL: https://issues.apache.org/jira/browse/FLINK-41104
> Project: Flink
> Issue Type: Bug
> Components: API / Python
> Reporter: Dian Fu
> Priority: Major
>
> In PyFlink process mode, MapState.contains(key) incorrectly treats an
> existing mapping with a None value as an absent entry.
> Reproduction:
> {code:python}
> state.put(1, None)
> state.contains(1) # Returns False
> {code}
> Expected:
> contains(1) returns True while the mapping exists, and returns False after
> the key is removed.
> Actual:
> It returns False for the existing mapping, both before commit and after
> commit. The behavior also occurs with an uncached backend response.
> Root cause:
> InternalSynchronousMapRuntimeState.contains() determines membership using
> self.get(map_key) is not None. This loses the distinction between a missing
> entry and an existing null-valued entry, although the write cache and
> map-state protocol preserve an explicit existence flag.
> The affected implementation is in
> flink-python/pyflink/fn_execution/state_impl.py. Membership checks should use
> entry existence rather than value nullness.
> Regression tests should cover null-valued, non-null-valued, missing, and
> removed entries across buffered, cached, and uncached reads, including the
> "key in state" syntax.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
