kfaraz commented on code in PR #16990:
URL: https://github.com/apache/druid/pull/16990#discussion_r1741497586
##########
server/src/main/java/org/apache/druid/segment/metadata/CoordinatorSegmentMetadataCache.java:
##########
@@ -682,9 +682,16 @@ public RowSignature buildDataSourceRowSignature(final
String dataSource)
RowSignature rowSignature =
optionalSchema.get().getSchemaPayload().getRowSignature();
mergeRowSignature(columnTypes, rowSignature);
} else {
- // mark it for refresh, however, this case shouldn't arise by design
- markSegmentAsNeedRefresh(segmentId);
log.debug("SchemaMetadata for segmentId [%s] is absent.", segmentId);
+
+ ImmutableDruidDataSource druidDataSource =
+
sqlSegmentsMetadataManager.getImmutableDataSourceWithUsedSegments(segmentId.getDataSource());
+
+ if (druidDataSource != null && druidDataSource.getSegment(segmentId)
!= null) {
Review Comment:
Whenever `SqlSegmentsMetadataManager` polls the database and returns the
latest list of used segments, we should check that list and remove any segments
from cache which are not in that list.
This removal should not be triggered based on any callback from server
inventory view. Datasource should be the source of truth on whether a segment
is used or not.
Also, I was under the impression that this feature is already included. Can
you confirm, @findingrish ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org
