[
https://issues.apache.org/jira/browse/FLINK-41063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-41063:
-----------------------------------
Labels: pull-request-available (was: )
> DataStream large-record sorting unnecessarily falls back to Kryo for payload
> serialization
> -------------------------------------------------------------------------------------------
>
> Key: FLINK-41063
> URL: https://issues.apache.org/jira/browse/FLINK-41063
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
> Affects Versions: 2.3.0, 2.2.1
> Environment: Observed on Flink 2.2.1; regression tests reproduce the
> issue and verify the fix on 2.3.0.
> Reporter: Dominik Dębowczyk
> Priority: Major
> Labels: pull-request-available
>
> With {{taskmanager.runtime.large-record-handler}} enabled, both
> {{FixedLengthByteKeyComparator}} and {{VariableLengthByteKeyComparator}}
> extract the entire record as a sort key, although comparison only uses the
> byte-array key and timestamp.
> {{LargeRecordHandler}} then infers a serializer for the complete record
> instead of reusing the supplied payload serializer. This can unexpectedly
> select Kryo or fail when generic types are disabled.
> We encountered this with the Apache Beam Flink Runner: {{{}Kryo
> initialization failed with a classloader conflict during spilling{}}},
> surfaced as “The sorter is closed already.” JFR revealed the underlying
> exception.
> The proposed fix is to extract only the byte-array key and timestamp and
> return matching flat comparators. This preserves ordering, avoids payload
> serializer inference, and eliminates redundant payload serialization in the
> key spill file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
