noob-se7en commented on code in PR #18381:
URL: https://github.com/apache/pinot/pull/18381#discussion_r3247846483
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java:
##########
@@ -771,6 +817,31 @@ public ExecutorService getSegmentPreloadExecutor() {
return _segmentPreloadExecutor;
}
+ public ZkHelixPropertyStore<ZNRecord> getPropertyStore() {
+ return _propertyStore;
+ }
+
+ public ConcurrentMap<String, SegmentDataManager> getSegmentDataManagerMap() {
Review Comment:
returning the raw mutable map lets callers bypass `unregisterSegment`'s
`_recentlyDeletedSegments` tracking. consider `Collections.unmodifiableMap`
instead.
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java:
##########
@@ -1014,6 +1085,7 @@ public void reloadSegment(String segmentName,
IndexLoadingConfig indexLoadingCon
indexLoadingConfig.setSegmentTier(segmentTier);
indexLoadingConfig.setTableDataDir(_tableDataDir);
File indexDir = getSegmentDataDir(segmentName, segmentTier,
indexLoadingConfig.getTableConfig());
+ _segmentReloadSemaphore.acquire(segmentName, _logger);
Lock segmentLock = getSegmentLock(segmentName);
Review Comment:
Incase we get an exception here the above semaphore might remain acquired.
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java:
##########
@@ -1542,8 +1630,7 @@ public List<StaleSegment> getStaleSegments() {
return staleSegments;
}
- @VisibleForTesting
- StaleSegment isSegmentStale(IndexLoadingConfig indexLoadingConfig,
SegmentDataManager segmentDataManager) {
+ public StaleSegment isSegmentStale(IndexLoadingConfig indexLoadingConfig,
SegmentDataManager segmentDataManager) {
Review Comment:
PR body says `isSegmentStale` should be `protected` but the diff makes it
`public`. `protected` is enough for subclass-based test extensibility across
modules; same applies to
`registerSegment`/`unregisterSegment`/`downloadSegment`/`createBackup`/`removeBackup`.
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/data/manager/TableDataManager.java:
##########
@@ -194,6 +194,13 @@ void offloadSegment(String segmentName)
void offloadSegmentUnsafe(String segmentName)
throws Exception;
+ /**
+ * Deletes a segment from this table — offloads it if currently loaded, then
removes its on-disk data (the per-segment
+ * directory and any tier-specific artefacts).
+ */
+ void deleteSegment(String segmentName)
Review Comment:
abstract method on this interface compile-breaks external implementers.
consider making it `default` (throw `UnsupportedOperationException`, or
delegate to `BaseTableDataManager.deleteSegmentFilesFromDisk`).
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java:
##########
@@ -1410,6 +1483,22 @@ protected void removeBackup(File indexDir)
@Override
public boolean tryLoadExistingSegment(SegmentZKMetadata zkMetadata,
IndexLoadingConfig indexLoadingConfig) {
+ ImmutableSegment segment = tryLoadExistingSegmentInternal(zkMetadata,
indexLoadingConfig);
+ if (segment == null) {
+ return false;
+ }
+ addSegment(segment, zkMetadata);
+ return true;
+ }
+
+ /**
+ * Just Loads a segment from the existing on-disk copy without registering
it in {@code _segmentDataManagerMap} or
+ * invoking other hooks.
+ * Returns {@code null} when the on-disk copy is absent, has a stale CRC
under or fails to load
+ */
+ @Nullable
+ public ImmutableSegment tryLoadExistingSegmentInternal(SegmentZKMetadata
zkMetadata,
Review Comment:
internal name for public method seem to be an anti-pattern.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
