tarun11Mavani commented on PR #19776: URL: https://github.com/apache/pinot/pull/19776#issuecomment-6941662007
> [@tarun11Mavani](https://github.com/tarun11Mavani) could you take a look at these two remaining cases and address them in a follow-up PR? I reviewed the final revision `d9338330` after merge. The failed-invalidation fix and the segment-name collision fix are addressed; the first issue below is a different collision introduced by the new root. > > 1. **[P1] The new metadata archive root can collide with a live table and delete its full segments.** `Deleted_Segments_Metadata` is a valid raw table name, so its live deep-store directory is also the new archive root. The [unconditional sweep](https://github.com/apache/pinot/blob/d9338330bf12f45d07b985053ac16a721e5d13dd/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java#L466-L471) runs even with preservation disabled. On S3/GCS, listing that root returns live segment objects as supposed table directories. Listing each object as a directory searches `<segment>/` and returns empty; [the empty-directory cleanup](https://github.com/apache/pinot/blob/d9338330bf12f45d07b985053ac16a721e5d13dd/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/SegmentDeletionManager.java#L528-L535) then calls `delete(<segment>, false)`, which deletes the actual file object. For example, [S3PinotFS's file branch](https://github.com/apach e/pinot/blob/d9338330bf12f45d07b985053ac16a721e5d13dd/pinot-plugins/pinot-file-system/pinot-s3/src/main/java/org/apache/pinot/plugin/filesystem/S3PinotFS.java#L603-L609) deletes that exact key. This affects both OFFLINE and REALTIME and does not wait for segment retention. Please protect existing table-name collisions and reject non-directory entries before directory cleanup; add a regression for a **table** named `Deleted_Segments_Metadata`, not only a segment with that name. > 2. **[P2] Matching sidecar paths does not establish that the sidecar describes the incoming build.** Start with `seg` and its old `seg.metadata.tar.gz`, then stage a changed `seg.tar.gz` in the same directory. A metadata push with `preferMetadataTarGz=false` and `copyToDeepStoreForMetadataPush=true` extracts fresh request metadata from the tar. However, [this exemption](https://github.com/apache/pinot/blob/d9338330bf12f45d07b985053ac16a721e5d13dd/pinot-controller/src/main/java/org/apache/pinot/controller/api/upload/ZKOperator.java#L695-L698) skips invalidation solely because the computed sidecar paths match. The controller replaces the full segment while retaining its old sidecar. A later metadata push using the default preference can consume the old CRC/document counts, and deletion preservation archives a mismatched pair. Please validate the sidecar against the incoming metadata, or invalidate/rewrite it from the request. A same-directory test with an old sidecar and independe ntly supplied fresh request metadata would cover this. > > I would prioritize the first case because it can remove live deep-store segment files with the new feature flags off. These findings are based on source/control-flow inspection; I did not run a local reproducer. @Jackie-Jiang I have created a new PR to address 1. https://github.com/apache/pinot/pull/20284 Will share the PR for 2 soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
