[jira] [Updated] (BEAM-4798) IndexOutOfBoundsException when Flink parallelism > 1
[ https://issues.apache.org/jira/browse/BEAM-4798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aljoscha Krettek updated BEAM-4798: --- Fix Version/s: 2.7.0 > IndexOutOfBoundsException when Flink parallelism > 1 > > > Key: BEAM-4798 > URL: https://issues.apache.org/jira/browse/BEAM-4798 > Project: Beam > Issue Type: Bug > Components: runner-flink >Affects Versions: 2.4.0, 2.5.0 >Reporter: Alexey Romanenko >Assignee: Alexey Romanenko >Priority: Major > Fix For: 2.7.0 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > Running job on Flink in streaming mode and get data from a Kafka topic with > parallelism > 1 causes an exception: > {noformat} > Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 > at java.util.ArrayList.rangeCheck(ArrayList.java:657) > at java.util.ArrayList.get(ArrayList.java:433) > at > org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapper.run(UnboundedSourceWrapper.java:277) > at > org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:87) > at > org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:56) > at > org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:99) > at > org.apache.flink.streaming.runtime.tasks.StoppableSourceStreamTask.run(StoppableSourceStreamTask.java:45) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:306) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703) > at java.lang.Thread.run(Thread.java:748) > {noformat} > It happens when number of Kafka topic partitions is less than value of > parallelism (number of task slots). > So, workaround for now can be to set parallelism <= number of topic > partitions, thus if parallelism=2 then number_partitions >= 2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-4798) IndexOutOfBoundsException when Flink parallelism > 1
[ https://issues.apache.org/jira/browse/BEAM-4798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Romanenko updated BEAM-4798: --- Affects Version/s: 2.4.0 > IndexOutOfBoundsException when Flink parallelism > 1 > > > Key: BEAM-4798 > URL: https://issues.apache.org/jira/browse/BEAM-4798 > Project: Beam > Issue Type: Bug > Components: runner-flink >Affects Versions: 2.4.0, 2.5.0 >Reporter: Alexey Romanenko >Assignee: Alexey Romanenko >Priority: Major > > Running job on Flink in streaming mode and get data from a Kafka topic with > parallelism > 1 causes an exception: > {noformat} > Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 > at java.util.ArrayList.rangeCheck(ArrayList.java:657) > at java.util.ArrayList.get(ArrayList.java:433) > at > org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapper.run(UnboundedSourceWrapper.java:277) > at > org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:87) > at > org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:56) > at > org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:99) > at > org.apache.flink.streaming.runtime.tasks.StoppableSourceStreamTask.run(StoppableSourceStreamTask.java:45) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:306) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703) > at java.lang.Thread.run(Thread.java:748) > {noformat} > It happens when number of Kafka topic partitions is less than value of > parallelism (number of task slots). > So, workaround for now can be to set parallelism <= number of topic > partitions, thus if parallelism=2 then number_partitions >= 2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)