betalb opened a new issue, #20752:
URL: https://github.com/apache/druid/issues/20752
### Affected Version
37.0.0, Coordinator/Overlord combined
### Description
Calling markUnused with an interval smaller than an existing segment causes
inconsistent handling:
- PostgreSQL preserves segments that overlap but are not fully contained
within the interval.
- The incremental metadata cache marks those segments unused and the
disappear.
The API can return numChangedSegments: 0 while removing segments from the
cache. Restart of the coordinator restores those segments.
Configuration
```properties
druid.coordinator.asOverlord.enabled=true
druid.manager.segments.useIncrementalCache=ifSynced
```
#### Reproduction
1. Ingest a DAY segment into test_ds covering:
```text
2026-10-24T00:00:00.000Z/2026-10-25T00:00:00.000Z
```
2. Verify that it is marked used=true in PostgreSQL and appears in the
Coordinator’s metadata (I've used druid sql metadata query).
3. Mark an hour within that day unused (note that requested interval lies
within our segment interval):
```http
POST /druid/coordinator/v1/datasources/test_ds/markUnused
Content-Type: application/json
{
"interval": "2026-10-24T01:00:00.000Z/2026-10-24T02:00:00.000Z"
}
```
4. The response is:
```json
{
"numChangedSegments": 0,
"segmentStateChanged": false
}
```
5. PostgreSQL still shows the DAY segment as used=true, with its original
used_status_last_updated.
6. After the cache snapshot refreshes, the DAY segment is absent from the
sql metadata query (sys.segments)
#### Expected behavior
PostgreSQL and the cache to be consistent.
#### Related code
SQL Implementation explicitly states that only intervals fully contained
within the requested interval should be eligible for marking unused
https://github.com/apache/druid/blob/master/server/src/main/java/org/apache/druid/metadata/SqlSegmentsMetadataQuery.java#L809
Doesn't look like
https://github.com/apache/druid/blob/master/server/src/main/java/org/apache/druid/metadata/segment/cache/HeapMemoryDatasourceSegmentCache.java#L557
is doing the same, it looks like it is doing an overlap check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
