Nandini Singhal created KAFKA-19971:
---------------------------------------
Summary: Tiered Storage Fetches Fail with IllegalStateException
During Index Cache Eviction Window
Key: KAFKA-19971
URL: https://issues.apache.org/jira/browse/KAFKA-19971
Project: Kafka
Issue Type: Bug
Components: Tiered-Storage
Reporter: Nandini Singhal
Remote log fetches can fail with IllegalStateException: This entry is marked
for cleanup when the index cache is under pressure and evicting entries. A race
condition exists between entry eviction and fetch operations, causing requests
to fail even though the data could be re-fetched from remote storage.
When the remote index cache reaches its capacity and needs to evict entries, a
multi-step eviction process creates a window where in-flight fetch requests can
fail unexpectedly:
1. Cache eviction triggers removal of an index entry
2. Entry is marked for cleanup (markedForCleanup = true)
3. Index files are renamed to .deleted suffix
4. Cleanup is scheduled to run after a delay (default: 10 seconds)
5. After the delay, cleanup sets cleanStarted = true and deletes files
6. During this window, if a fetch request references this entry, it fails
with:
java.lang.IllegalStateException: This entry is marked for cleanup
Expected behavior: Should treat as cache miss and re-fetch from remote storage
(or handle gracefully).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
