tarun11Mavani commented on code in PR #18467:
URL: https://github.com/apache/pinot/pull/18467#discussion_r3266044083
##########
pinot-server/src/main/java/org/apache/pinot/server/predownload/PredownloadTableInfo.java:
##########
@@ -64,44 +52,42 @@ public InstanceDataManagerConfig
getInstanceDataManagerConfig() {
}
/**
- * After loading segment metadata from ZK, try to load from local and check
if we are able to skip
- * the downloading
+ * Checks whether the segment already exists locally with a matching CRC,
and if so populates
+ * its local size. Reads CRC directly from {@code creation.meta} and size
from directory
+ * traversal — no mmap of index files is performed (segment loading).
+ *
+ * <p>Note: a CRC match does not guarantee segment integrity. If the local
segment directory is
+ * corrupted (e.g. truncated index files), the server startup path handles
recovery:
+ * {@link
org.apache.pinot.core.data.manager.BaseTableDataManager#addNewOnlineSegment}
calls
+ * {@link
org.apache.pinot.core.data.manager.BaseTableDataManager#tryLoadExistingSegment},
which
+ * performs a full segment load. If loading fails, it falls back to
+ * {@link
org.apache.pinot.core.data.manager.BaseTableDataManager#downloadAndLoadSegment}
to
+ * re-fetch the segment from deep store or peers.
*
- * @param predownloadSegmentInfo SegmentInfo of segment to be loaded
- * @return true if already presents, false if needs to be downloaded
+ * @param predownloadSegmentInfo SegmentInfo of segment to be checked
+ * @return true if the segment is present with matching CRC (download can be
skipped),
+ * false if it is missing or has a CRC mismatch
Review Comment:
Use md style doc for java doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
