Boyang Chen created KAFKA-9727:
----------------------------------
Summary: Flaky system test StreamsEOSTest.test_failure_and_recovery
Key: KAFKA-9727
URL: https://issues.apache.org/jira/browse/KAFKA-9727
Project: Kafka
Issue Type: Bug
Reporter: Boyang Chen
Assignee: Boyang Chen
Hits no lock available exceptions sometime after task revive:
[2020-03-13 05:40:50,224] ERROR stream-thread
[EosTest-46de8ee5-82a1-4bd4-af23-f4acd8515f0f-StreamThread-1] Encountered the
following exception during processing and the thread is going to shut down:
(org.apache.kafka.streams.processor.internals.StreamThread)
org.apache.kafka.streams.errors.ProcessorStateException: Error opening store
KSTREAM-AGGREGATE-STATE-STORE-0000000003 at location
/mnt/streams/EosTest/0_0/rocksdb/KSTREAM-AGGREGATE-STATE-STORE-0000000003
at
org.apache.kafka.streams.state.internals.RocksDBTimestampedStore.openRocksDB(RocksDBTimestampedStore.java:87)
at
org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:191)
at
org.apache.kafka.streams.state.internals.RocksDBStore.init(RocksDBStore.java:230)
at
org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:48)
at
org.apache.kafka.streams.state.internals.ChangeLoggingKeyValueBytesStore.init(ChangeLoggingKeyValueBytesStore.java:44)
at
org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:48)
at
org.apache.kafka.streams.state.internals.MeteredKeyValueStore.lambda$init$0(MeteredKeyValueStore.java:101)
at
org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.maybeMeasureLatency(StreamsMetricsImpl.java:806)
at
org.apache.kafka.streams.state.internals.MeteredKeyValueStore.init(MeteredKeyValueStore.java:101)
at
org.apache.kafka.streams.processor.internals.StateManagerUtil.registerStateStores(StateManagerUtil.java:81)
at
org.apache.kafka.streams.processor.internals.StandbyTask.initializeIfNeeded(StandbyTask.java:86)
at
org.apache.kafka.streams.processor.internals.TaskManager.tryToCompleteRestoration(TaskManager.java:275)
at
org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:583)
at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:498)
at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:472)
Caused by: org.rocksdb.RocksDBException: lock :
/mnt/streams/EosTest/0_0/rocksdb/KSTREAM-AGGREGATE-STATE-STORE-0000000003/LOCK:
No locks available
at org.rocksdb.RocksDB.open(Native Method)
at org.rocksdb.RocksDB.open(RocksDB.java:286)
at
org.apache.kafka.streams.state.internals.RocksDBTimestampedStore.openRocksDB(RocksDBTimestampedStore.java:75)
... 14 more
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
