Jiri Syrovy created KAFKA-21580:
-----------------------------------
Summary: TopologyTestDriver leaves state stores open when its
construction fails
Key: KAFKA-21580
URL: https://issues.apache.org/jira/browse/KAFKA-21580
Project: Kafka
Issue Type: Bug
Components: streams, streams-test-utils
Affects Versions: 4.3.2, 4.4.0
Reporter: Jiri Syrovy
The {{TopologyTestDriver}} constructor sets up the global task and the stream
task, which opens their state stores. If that setup throws, the constructor
does not close what it has opened. The caller gets no driver object, so it
cannot call {{close()}} either, and the stores stay open until the JVM exits.
For a persistent store this leaves the RocksDB instance open and its {{LOCK}}
file held.
h3. When it happens
Any failure after the stores are opened, for example:
* the task is treated as corrupted during initialization
({{TaskCorruptedException}} under exactly-once),
* a processor throws from {{init()}}.
h3. How to reproduce
{code:java}
final Topology topology = new Topology();
topology.addSource("source", "input");
topology.addProcessor("processor", () -> new Processor<String, Long, Void,
Void>() {
@Override
public void init(final ProcessorContext<Void, Void> context) {
store = context.getStateStore("store");
throw new IllegalStateException("failed to initialize");
}
@Override
public void process(final Record<String, Long> record) { }
}, "source");
topology.addStateStore(
Stores.keyValueStoreBuilder(Stores.persistentKeyValueStore("store"),
Serdes.String(), Serdes.Long()),
"processor");
assertThrows(StreamsException.class, () -> new
TopologyTestDriverBuilder(topology).withConfig(config).build());
store.isOpen(); // true, expected false
{code}
The same happens for a global store whose processor fails to initialize.
It was found through {{SegmentedStoreWithoutSegmentsTest}} (KAFKA-21492), which
expects a {{TaskCorruptedException}} from the constructor. After the exception,
opening the segment directory again in the same JVM fails with "lock hold by
current process", for each of the seven store types the test covers.
h3. Proposed fix
When the setup fails, the constructor closes the task dirty and closes the
global state manager, then rethrows the original failure.
It does not clean the state directory as {{close()}} does. Under exactly-once
the dirty close of the task deletes the task directory, as Kafka Streams does
for a corrupted task.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)