[jira] [Commented] (FLINK-22962) Key group is not in KeyGroupRange error while checkpointing
[ https://issues.apache.org/jira/browse/FLINK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360956#comment-17360956 ] Prateek Kohli commented on FLINK-22962: --- We have designed our keyby logic in such a way that we are using KeySelector i.e. our getKey() method is returning Serializable. Also, our KeyedProcessFunction uses Object type for key(KeyedProcessFunction) instead of any concrete class. Can this be the cause of any error? > Key group is not in KeyGroupRange error while checkpointing > --- > > Key: FLINK-22962 > URL: https://issues.apache.org/jira/browse/FLINK-22962 > Project: Flink > Issue Type: Bug > Components: Runtime / State Backends >Affects Versions: 1.12.1 > Environment: Linux >Reporter: Prateek Kohli >Priority: Major > > Hi, > > We are getting the below exception while using rocksdb as state backend at > the time of checkpointing: > 2021-06-10 12:05:13,933 INFO > org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable [] - > Aggregator (3/4)#0 - asynchronous part of checkpoint 2 could not be completed. > java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: > Key group 0 is not in KeyGroupRange\{startKeyGroup=5, endKeyGroup=7}. > at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_261] > at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_261] > at > org.apache.flink.runtime.concurrent.FutureUtils.runIfNotDoneAndGet(FutureUtils.java:621) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.streaming.api.operators.OperatorSnapshotFinalizer.(OperatorSnapshotFinalizer.java:54) > ~[flink-streaming-java_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable.run(AsyncCheckpointRunnable.java:122) > [flink-streaming-java_2.11-1.12.1.jar:1.12.1] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [?:1.8.0_261] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [?:1.8.0_261] > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261] > Caused by: java.lang.IllegalArgumentException: Key group 0 is not in > KeyGroupRange\{startKeyGroup=5, endKeyGroup=7}. > at > org.apache.flink.runtime.state.KeyGroupRangeOffsets.computeKeyGroupIndex(KeyGroupRangeOffsets.java:144) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.runtime.state.KeyGroupRangeOffsets.setKeyGroupOffset(KeyGroupRangeOffsets.java:106) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.contrib.streaming.state.snapshot.RocksFullSnapshotStrategy$SnapshotAsynchronousPartCallable.writeKVStateData(RocksFullSnapshotStrategy.java:333) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.contrib.streaming.state.snapshot.RocksFullSnapshotStrategy$SnapshotAsynchronousPartCallable.writeSnapshotToOutputStream(RocksFullSnapshotStrategy.java:264) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.contrib.streaming.state.snapshot.RocksFullSnapshotStrategy$SnapshotAsynchronousPartCallable.callInternal(RocksFullSnapshotStrategy.java:227) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.contrib.streaming.state.snapshot.RocksFullSnapshotStrategy$SnapshotAsynchronousPartCallable.callInternal(RocksFullSnapshotStrategy.java:180) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.runtime.state.AsyncSnapshotCallable.call(AsyncSnapshotCallable.java:78) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_261] > at > org.apache.flink.runtime.concurrent.FutureUtils.runIfNotDoneAndGet(FutureUtils.java:618) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > ... 5 more > > When we change the state backend to file or heap we do not get this error. > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22962) Key group is not in KeyGroupRange error while checkpointing
[ https://issues.apache.org/jira/browse/FLINK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454991#comment-17454991 ] Yun Tang commented on FLINK-22962: -- [~prateekkohli2112] After flink-1.13, we unify the format of savepoint of RocksDB and heap keyed state backends. Will you still face the problem? > Key group is not in KeyGroupRange error while checkpointing > --- > > Key: FLINK-22962 > URL: https://issues.apache.org/jira/browse/FLINK-22962 > Project: Flink > Issue Type: Bug > Components: Runtime / State Backends >Affects Versions: 1.12.1 > Environment: Linux >Reporter: Prateek Kohli >Priority: Major > > Hi, > > We are getting the below exception while using rocksdb as state backend at > the time of checkpointing: > 2021-06-10 12:05:13,933 INFO > org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable [] - > Aggregator (3/4)#0 - asynchronous part of checkpoint 2 could not be completed. > java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: > Key group 0 is not in KeyGroupRange\{startKeyGroup=5, endKeyGroup=7}. > at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_261] > at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_261] > at > org.apache.flink.runtime.concurrent.FutureUtils.runIfNotDoneAndGet(FutureUtils.java:621) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.streaming.api.operators.OperatorSnapshotFinalizer.(OperatorSnapshotFinalizer.java:54) > ~[flink-streaming-java_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable.run(AsyncCheckpointRunnable.java:122) > [flink-streaming-java_2.11-1.12.1.jar:1.12.1] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [?:1.8.0_261] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [?:1.8.0_261] > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261] > Caused by: java.lang.IllegalArgumentException: Key group 0 is not in > KeyGroupRange\{startKeyGroup=5, endKeyGroup=7}. > at > org.apache.flink.runtime.state.KeyGroupRangeOffsets.computeKeyGroupIndex(KeyGroupRangeOffsets.java:144) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.runtime.state.KeyGroupRangeOffsets.setKeyGroupOffset(KeyGroupRangeOffsets.java:106) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.contrib.streaming.state.snapshot.RocksFullSnapshotStrategy$SnapshotAsynchronousPartCallable.writeKVStateData(RocksFullSnapshotStrategy.java:333) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.contrib.streaming.state.snapshot.RocksFullSnapshotStrategy$SnapshotAsynchronousPartCallable.writeSnapshotToOutputStream(RocksFullSnapshotStrategy.java:264) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.contrib.streaming.state.snapshot.RocksFullSnapshotStrategy$SnapshotAsynchronousPartCallable.callInternal(RocksFullSnapshotStrategy.java:227) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.contrib.streaming.state.snapshot.RocksFullSnapshotStrategy$SnapshotAsynchronousPartCallable.callInternal(RocksFullSnapshotStrategy.java:180) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at > org.apache.flink.runtime.state.AsyncSnapshotCallable.call(AsyncSnapshotCallable.java:78) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_261] > at > org.apache.flink.runtime.concurrent.FutureUtils.runIfNotDoneAndGet(FutureUtils.java:618) > ~[flink-dist_2.11-1.12.1.jar:1.12.1] > ... 5 more > > When we change the state backend to file or heap we do not get this error. > > > > -- This message was sent by Atlassian Jira (v8.20.1#820001)