[jira] [Comment Edited] (FLINK-16819) Got KryoException while using UDAF in flink1.9
[ https://issues.apache.org/jira/browse/FLINK-16819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086483#comment-17086483 ] Lsw_aka_laplace edited comment on FLINK-16819 at 4/18/20, 3:05 PM: --- [~jark] These jobs which I mentioned aboved has been running correctly for a few day after upgrading to 1.9.2. Your advice did help us to fix this problem Salute! was (Author: neighborhood): [~jark] These jobs which I mentioned aboved has been running correctly for a few day after upgrading to 1.9.2. Your advance did help us to fix this problem Salute! > Got KryoException while using UDAF in flink1.9 > -- > > Key: FLINK-16819 > URL: https://issues.apache.org/jira/browse/FLINK-16819 > Project: Flink > Issue Type: Bug > Components: API / Type Serialization System, Table SQL / Planner >Affects Versions: 1.9.1 > Environment: Flink1.9.1 > Apache hadoop 2.7.2 >Reporter: Xingxing Di >Priority: Major > Fix For: 1.9.2 > > > Recently, we are trying to upgrade online *sql jobs* from flink1.7 to > flink1.9 , most jobs works fine, but some jobs got KryoExceptions. > We found that UDAF will trigger this exception, btw ,we are using blink > planner. > *Here is the full stack traces:* > 2020-03-27 11:46:55 > com.esotericsoftware.kryo.KryoException: > java.lang.IndexOutOfBoundsException: Index: 104, Size: 2 > Serialization trace: > seed (java.util.Random) > gen (com.tdunning.math.stats.AVLTreeDigest) > at > com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125) > at > com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528) > at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > at > com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) > at > com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528) > at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > at > org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:346) > at > org.apache.flink.util.InstantiationUtil.deserializeFromByteArray(InstantiationUtil.java:536) > at > org.apache.flink.table.dataformat.BinaryGeneric.getJavaObjectFromBinaryGeneric(BinaryGeneric.java:86) > at > org.apache.flink.table.dataformat.DataFormatConverters$GenericConverter.toExternalImpl(DataFormatConverters.java:628) > at > org.apache.flink.table.dataformat.DataFormatConverters$GenericConverter.toExternalImpl(DataFormatConverters.java:633) > at > org.apache.flink.table.dataformat.DataFormatConverters$DataFormatConverter.toExternal(DataFormatConverters.java:320) > at > org.apache.flink.table.dataformat.DataFormatConverters$PojoConverter.toExternalImpl(DataFormatConverters.java:1293) > at > org.apache.flink.table.dataformat.DataFormatConverters$PojoConverter.toExternalImpl(DataFormatConverters.java:1257) > at > org.apache.flink.table.dataformat.DataFormatConverters$DataFormatConverter.toExternal(DataFormatConverters.java:302) > at GroupAggsHandler$71.setAccumulators(Unknown Source) > at > org.apache.flink.table.runtime.operators.aggregate.GroupAggFunction.processElement(GroupAggFunction.java:151) > at > org.apache.flink.table.runtime.operators.aggregate.GroupAggFunction.processElement(GroupAggFunction.java:43) > at > org.apache.flink.streaming.api.operators.KeyedProcessOperator.processElement(KeyedProcessOperator.java:85) > at > org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processElement(StreamOneInputProcessor.java:164) > at > org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:143) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:279) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.run(StreamTask.java:301) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:406) > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:530) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.IndexOutOfBoundsException: Index: 104, Size: 2 > at java.util.ArrayList.rangeCheck(ArrayList.java:657) > at java.util.ArrayList.get(ArrayList.java:433) > at > com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:42) > at com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:805) > at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:677) > at > com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) > ... 26
[jira] [Comment Edited] (FLINK-16819) Got KryoException while using UDAF in flink1.9
[ https://issues.apache.org/jira/browse/FLINK-16819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17068264#comment-17068264 ] Lsw_aka_laplace edited comment on FLINK-16819 at 3/27/20, 4:42 AM: --- also I can make sure that it happens when the job tries to do checkpointing was (Author: neighborhood): also I can make sure that it happend when the job tries to do checkpointing > Got KryoException while using UDAF in flink1.9 > -- > > Key: FLINK-16819 > URL: https://issues.apache.org/jira/browse/FLINK-16819 > Project: Flink > Issue Type: Bug > Components: API / Type Serialization System, Table SQL / Planner >Affects Versions: 1.9.1 > Environment: Flink1.9.1 > Apache hadoop 2.7.2 >Reporter: Xingxing Di >Priority: Major > > Recently, we are trying to upgrade online *sql jobs* from flink1.7 to > flink1.9 , most jobs works fine, but some jobs got KryoExceptions. > We found that UDAF will trigger this exception, btw ,we are using blink > planner. > *Here is the full stack traces:* > 2020-03-27 11:46:55 > com.esotericsoftware.kryo.KryoException: > java.lang.IndexOutOfBoundsException: Index: 104, Size: 2 > Serialization trace: > seed (java.util.Random) > gen (com.tdunning.math.stats.AVLTreeDigest) > at > com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125) > at > com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528) > at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > at > com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) > at > com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528) > at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > at > org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:346) > at > org.apache.flink.util.InstantiationUtil.deserializeFromByteArray(InstantiationUtil.java:536) > at > org.apache.flink.table.dataformat.BinaryGeneric.getJavaObjectFromBinaryGeneric(BinaryGeneric.java:86) > at > org.apache.flink.table.dataformat.DataFormatConverters$GenericConverter.toExternalImpl(DataFormatConverters.java:628) > at > org.apache.flink.table.dataformat.DataFormatConverters$GenericConverter.toExternalImpl(DataFormatConverters.java:633) > at > org.apache.flink.table.dataformat.DataFormatConverters$DataFormatConverter.toExternal(DataFormatConverters.java:320) > at > org.apache.flink.table.dataformat.DataFormatConverters$PojoConverter.toExternalImpl(DataFormatConverters.java:1293) > at > org.apache.flink.table.dataformat.DataFormatConverters$PojoConverter.toExternalImpl(DataFormatConverters.java:1257) > at > org.apache.flink.table.dataformat.DataFormatConverters$DataFormatConverter.toExternal(DataFormatConverters.java:302) > at GroupAggsHandler$71.setAccumulators(Unknown Source) > at > org.apache.flink.table.runtime.operators.aggregate.GroupAggFunction.processElement(GroupAggFunction.java:151) > at > org.apache.flink.table.runtime.operators.aggregate.GroupAggFunction.processElement(GroupAggFunction.java:43) > at > org.apache.flink.streaming.api.operators.KeyedProcessOperator.processElement(KeyedProcessOperator.java:85) > at > org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processElement(StreamOneInputProcessor.java:164) > at > org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:143) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:279) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.run(StreamTask.java:301) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:406) > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:530) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.IndexOutOfBoundsException: Index: 104, Size: 2 > at java.util.ArrayList.rangeCheck(ArrayList.java:657) > at java.util.ArrayList.get(ArrayList.java:433) > at > com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:42) > at com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:805) > at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:677) > at > com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106) > ... 26 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
