[
https://issues.apache.org/jira/browse/FLINK-27836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642013#comment-17642013
]
Nico Kruber commented on FLINK-27836:
-------------------------------------
Unfortunately, it's not that simple because Flink doesn't have control over the
serializer that is used here. Just because Flink's own serializers are (by
chance) correct here, doesn't mean that an arbitrary serializer for user data
is. And serializer choice is at the user's convenience...
> RocksDBMapState iteration may stop too early for var-length prefixes
> --------------------------------------------------------------------
>
> Key: FLINK-27836
> URL: https://issues.apache.org/jira/browse/FLINK-27836
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.13.6, 1.14.4, 1.15.0
> Reporter: Nico Kruber
> Priority: Major
>
> A similar, yet orthogonal, issue to
> https://issues.apache.org/jira/browse/FLINK-11141 is that the iterators used
> in RocksDBMapState iterate over everything with a matching prefix of
> flink-key and namespace. With var-length serializers for either of them,
> however, it may return data for unrelated keys and/or namespaces.
> It looks like the built-in serializers of Flink are not affected though since
> they use a var-length encoding that is prefixed with the object's length and
> thus different lengths will not have the same prefix. More exotic
> serializers, e.g. relying on a terminating NUL character, may expose the
> above-mentioned behaviour, though.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
