[jira] [Updated] (BEAM-3605) Kinesis ShardReadersPoolTest shouldForgetClosedShardIterator failure
[ https://issues.apache.org/jira/browse/BEAM-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-3605: -- Labels: flake (was: ) > Kinesis ShardReadersPoolTest shouldForgetClosedShardIterator failure > > > Key: BEAM-3605 > URL: https://issues.apache.org/jira/browse/BEAM-3605 > Project: Beam > Issue Type: Bug > Components: sdk-java-extensions >Reporter: Kenneth Knowles >Assignee: Paweł Kaczmarczyk >Priority: Critical > Labels: flake > > Here's one: > https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/1758/testReport/junit/org.apache.beam.sdk.io.kinesis/ShardReadersPoolTest/shouldForgetClosedShardIterator/ > Filing all test failures as "Critical" so we can sickbay or fix. > The Jenkins build will get GC'd so here is the error: > {code} > java.lang.AssertionError: > Expecting: > <["shard1", "shard2"]> > to contain only: > <["shard2"]> > but the following elements were unexpected: > <["shard1"]> > at > org.apache.beam.sdk.io.kinesis.ShardReadersPoolTest.shouldForgetClosedShardIterator(ShardReadersPoolTest.java:270) > {code} > and stderr > {code} > Feb 01, 2018 11:24:16 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Shard iterator for shard1 shard is closed, finishing the read loop > org.apache.beam.sdk.io.kinesis.KinesisShardClosedException > Feb 01, 2018 11:24:16 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Kinesis Shard read loop has finished > Feb 01, 2018 11:24:16 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Shard iterator for shard1 shard is closed, finishing the read loop > org.apache.beam.sdk.io.kinesis.KinesisShardClosedException > Feb 01, 2018 11:24:16 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Kinesis Shard read loop has finished > Feb 01, 2018 11:24:19 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Shard iterator for shard1 shard is closed, finishing the read loop > org.apache.beam.sdk.io.kinesis.KinesisShardClosedException > Feb 01, 2018 11:24:19 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Kinesis Shard read loop has finished > Feb 01, 2018 11:24:23 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Shard iterator for shard1 shard is closed, finishing the read loop > org.apache.beam.sdk.io.kinesis.KinesisShardClosedException: Shard iterator > reached end of the shard: streamName=null, shardId=shard1 > at > org.apache.beam.sdk.io.kinesis.ShardRecordsIterator.readNextBatch(ShardRecordsIterator.java:70) > at > org.apache.beam.sdk.io.kinesis.ShardReadersPool.readLoop(ShardReadersPool.java:121) > at > org.apache.beam.sdk.io.kinesis.ShardReadersPool.lambda$startReadingShards$0(ShardReadersPool.java:112) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Feb 01, 2018 11:24:23 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Kinesis Shard read loop has finished > Feb 01, 2018 11:24:23 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Shard iterator for shard2 shard is closed, finishing the read loop > org.apache.beam.sdk.io.kinesis.KinesisShardClosedException: Shard iterator > reached end of the shard: streamName=null, shardId=shard2 > at > org.apache.beam.sdk.io.kinesis.ShardRecordsIterator.readNextBatch(ShardRecordsIterator.java:70) > at > org.apache.beam.sdk.io.kinesis.ShardReadersPool.readLoop(ShardReadersPool.java:121) > at > org.apache.beam.sdk.io.kinesis.ShardReadersPool.lambda$startReadingShards$0(ShardReadersPool.java:112) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Feb 01, 2018 11:24:23 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool stop > INFO: Closing shard iterators pool > Feb 01, 2018 11:24:24 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Kinesis Shard read loop has finished > Feb 01, 2018 11:24:24 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Shard iterator for shard1 shard is closed, finishing the read loop
[jira] [Updated] (BEAM-3605) Kinesis ShardReadersPoolTest shouldForgetClosedShardIterator failure
[ https://issues.apache.org/jira/browse/BEAM-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-3605: -- Labels: flake sickbay (was: flake) > Kinesis ShardReadersPoolTest shouldForgetClosedShardIterator failure > > > Key: BEAM-3605 > URL: https://issues.apache.org/jira/browse/BEAM-3605 > Project: Beam > Issue Type: Bug > Components: sdk-java-extensions >Reporter: Kenneth Knowles >Assignee: Kenneth Knowles >Priority: Critical > Labels: flake, sickbay > > Here's one: > https://builds.apache.org/job/beam_PreCommit_Java_GradleBuild/1758/testReport/junit/org.apache.beam.sdk.io.kinesis/ShardReadersPoolTest/shouldForgetClosedShardIterator/ > Filing all test failures as "Critical" so we can sickbay or fix. > The Jenkins build will get GC'd so here is the error: > {code} > java.lang.AssertionError: > Expecting: > <["shard1", "shard2"]> > to contain only: > <["shard2"]> > but the following elements were unexpected: > <["shard1"]> > at > org.apache.beam.sdk.io.kinesis.ShardReadersPoolTest.shouldForgetClosedShardIterator(ShardReadersPoolTest.java:270) > {code} > and stderr > {code} > Feb 01, 2018 11:24:16 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Shard iterator for shard1 shard is closed, finishing the read loop > org.apache.beam.sdk.io.kinesis.KinesisShardClosedException > Feb 01, 2018 11:24:16 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Kinesis Shard read loop has finished > Feb 01, 2018 11:24:16 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Shard iterator for shard1 shard is closed, finishing the read loop > org.apache.beam.sdk.io.kinesis.KinesisShardClosedException > Feb 01, 2018 11:24:16 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Kinesis Shard read loop has finished > Feb 01, 2018 11:24:19 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Shard iterator for shard1 shard is closed, finishing the read loop > org.apache.beam.sdk.io.kinesis.KinesisShardClosedException > Feb 01, 2018 11:24:19 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Kinesis Shard read loop has finished > Feb 01, 2018 11:24:23 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Shard iterator for shard1 shard is closed, finishing the read loop > org.apache.beam.sdk.io.kinesis.KinesisShardClosedException: Shard iterator > reached end of the shard: streamName=null, shardId=shard1 > at > org.apache.beam.sdk.io.kinesis.ShardRecordsIterator.readNextBatch(ShardRecordsIterator.java:70) > at > org.apache.beam.sdk.io.kinesis.ShardReadersPool.readLoop(ShardReadersPool.java:121) > at > org.apache.beam.sdk.io.kinesis.ShardReadersPool.lambda$startReadingShards$0(ShardReadersPool.java:112) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Feb 01, 2018 11:24:23 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Kinesis Shard read loop has finished > Feb 01, 2018 11:24:23 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Shard iterator for shard2 shard is closed, finishing the read loop > org.apache.beam.sdk.io.kinesis.KinesisShardClosedException: Shard iterator > reached end of the shard: streamName=null, shardId=shard2 > at > org.apache.beam.sdk.io.kinesis.ShardRecordsIterator.readNextBatch(ShardRecordsIterator.java:70) > at > org.apache.beam.sdk.io.kinesis.ShardReadersPool.readLoop(ShardReadersPool.java:121) > at > org.apache.beam.sdk.io.kinesis.ShardReadersPool.lambda$startReadingShards$0(ShardReadersPool.java:112) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Feb 01, 2018 11:24:23 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool stop > INFO: Closing shard iterators pool > Feb 01, 2018 11:24:24 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Kinesis Shard read loop has finished > Feb 01, 2018 11:24:24 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool > readLoop > INFO: Shard iterator for shard1 shard is closed, fini