[
https://issues.apache.org/jira/browse/FLINK-28367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17561943#comment-17561943
]
Lim Qing Wei edited comment on FLINK-28367 at 7/3/22 9:30 PM:
--------------------------------------------------------------
I had a look, I think part of it is that we are using a very old version of
Kryo, this issue is fixed in the newer version.
One way to fix it easily is to add new kryo version to your deps, as documented
here: [https://github.com/EsotericSoftware/kryo#with-maven]
If you're using sbt, just add the following to your deps should suffice, it
fixes the issue for me (using your example, thanks for that)
{code:java}
"com.esotericsoftware" % "kryo" % "5.3.0"{code}
I think it might be a good idea to bump Flink's Kryo, but I dont know what it
entails, is it going to be a pain to upgrade because we depends on many
obselete API? Might be good to get contributor's input here, and have it
tracked in dedicated ticket
was (Author: JIRAUSER289353):
I had a look, I think part of it is that we are using a very old version of
Kryo, this issue is likely fixed in the newer version.
One way to fix it easily is to add new kryo version to your deps, as documented
here: [https://github.com/EsotericSoftware/kryo#with-maven]
I think it might be a good idea to bump Flink's Kryo, but I dont know what it
entails, as in is it going to be a pain to upgrade because we depends on a lot
of obselete API? Might be good to get contributor's input here
> OffsetDateTime does not work with keyBy
> ---------------------------------------
>
> Key: FLINK-28367
> URL: https://issues.apache.org/jira/browse/FLINK-28367
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream, API / Type Serialization System
> Affects Versions: 1.15.0
> Environment: * Java 1.8 (openjdk 1.8.0_322)
> * Scala 2.12.15
> * Flink 1.15.0
> Reporter: Henrik Laxhuber
> Priority: Minor
>
> Using keyBy incorrectly (de-)serializes java.time.OffsetDateTime types - the
> offset gets lost and becomes null.
> Here's a minimal non-working example:
>
> {code:java}
> import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment
> import java.time.OffsetDateTime
> object MWE {
> def main(args: Array[String]): Unit = {
> val env = StreamExecutionEnvironment.getExecutionEnvironment
> env
> .fromElements("2022-07-03T15:35:48.142Z", "2022-07-03T15:35:48.438Z")
> .map(OffsetDateTime.parse(_))
> .keyBy((t: OffsetDateTime) => t)
> .print()
> env.execute()
> }
> } {code}
>
> Expected Output:
> {code:java}
> 2022-07-03T15:35:48.438Z
> 2022-07-03T15:35:48.142Z{code}
> Actual Output:
> {code:java}
> 2022-07-03T15:35:48.438null
> 2022-07-03T15:35:48.142null{code}
> The issue arises whenever keyBy and OffsetDateTime are involved; I believe it
> could have something to do with the way that flink serializes the state.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
