[
https://issues.apache.org/jira/browse/FLINK-38137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009255#comment-18009255
]
Ramin Gharib commented on FLINK-38137:
--------------------------------------
[~nishita0905] This gets more complicated if you have a MapView of
{noformat}
MapView<String, Integer> s{noformat}
And then you
{noformat}
s.put("nullValue", null);{noformat}
The value passed to *_serializeValueNullSensitive_* is not null but is an
object of type *_TtlValue_* with a _*userValue*_ field of null. This causes
deserialization errors in some restore tests I am trying to implement for the
PTFs.
> RocksDB State Backend Null Serialization Causes NPE and Asymmetric
> (De)Serialization Logic
> ------------------------------------------------------------------------------------------
>
> Key: FLINK-38137
> URL: https://issues.apache.org/jira/browse/FLINK-38137
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Reporter: Ramin Gharib
> Priority: Major
>
> The RocksDB state backend has a critical flaw in its handling of null values,
> which can cause NullPointerExceptions and create unnecessary serialization
> overhead.
> h3. Problem Description
> In [AbstractRocksDBState, the
> serializeValueNullSensitive()|https://github.com/raminqaf/flink/blob/main/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/state/rocksdb/AbstractRocksDBState.java#L179-L179]
> method unconditionally calls the serializer even when the value is null:
> {code:java}
> <T> byte[] serializeValueNullSensitive(T value, TypeSerializer<T> serializer)
> throws IOException {
> dataOutputView.clear();
> dataOutputView.writeBoolean(value == null); // Write null flag
> return serializeValueInternal(value, serializer); // Always serialize,
> even if null
> }
> private <T> byte[] serializeValueInternal(T value, TypeSerializer<T>
> serializer)
> throws IOException {
> serializer.serialize(value, dataOutputView); // Can throw NPE if value
> is null
> return dataOutputView.getCopyOfBuffer();
> } {code}
> This design has two major flaws:
> # NPE Risk: The behavior becomes dependent on the TypeSerializer
> implementation. Serializers not designed to handle null inputs will throw
> NullPointerException.
> # Asymmetric Logic: There's a critical mismatch between serialization and
> deserialization:
> *
> ** Serialization: Writes null flag + always attempts to serialize the value
> object (even if null)
> ** Deserialization: Reads null flag + immediately returns null without
> attempting deserialization if flag is true
> h3. Evidence from Deserialization Code
> In
> [RocksDBMapState|https://github.com/apache/flink/blob/bf1cd860617f7b51ac91516814c0e931e5bba241/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/state/rocksdb/RocksDBMapState.java#L413],
> the deserialization logic shows this asymmetry:
> {code:java}
> private static <UV> UV deserializeValueNullSensitive(
> byte[] rawValueBytes,
> TypeSerializer<UV> valueSerializer)
> throws IOException {
> dataInputView.setBuffer(rawValueBytes);
> boolean isNull = dataInputView.readBoolean();
> return isNull ? null : valueSerializer.deserialize(dataInputView); //
> Never deserializes if null
> } {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
