danny0405 commented on PR #20651:
URL: https://github.com/apache/hudi/pull/20651#issuecomment-6845276625

   Reviewing head `300756f645b6963a63ef55ef4c728cecc841eda0`, I see a 
correctness concern with positive N and a separate configuration propagation 
gap.
   
   **1. A matching suffix can suppress a necessary refresh and accept stale 
file listings.**
   
   The shortened hash is used for both the refresh decision and final 
validation, so the relaxation applies even when no archival occurred. For 
example, with N = 1:
   
   ```text
   Server cached: [commit001,            clean003]
   Client fresh:  [commit001, commit002, clean003]
                             ^ completed late
   ```
   
   Both sides hash only `clean003`, and the client's last instant is also 
present on the server. `isLocalViewBehind()` therefore returns false: the 
server skips sync and passes final validation. If its partition view was warmed 
before `commit002` completed, it can return the base file from `001` even 
though the client expects the updated file from `002`.
   
   The existing 
`TestTimelineViewRefresh.testRealViewRefreshesOutOfOrderCompletion` already 
exercises essentially this real-file scenario with full hashing. Please extend 
it to send N = 1 and verify that the returned base file is still `002`. 
Increasing N moves this blind spot further back; it does not eliminate it.
   
   The same issue applies to pending-compaction transitions outside the window: 
a server can continue treating a completed compaction as pending, which affects 
base-file and file-slice selection. In the opposite direction, an older client 
can accept a server that has observed an unseen older replace commit. The new 
`testInstantsBeforeHashedInstantsAreNotCompared` explicitly permits that 
mismatch, but its mocked empty file listing does not establish that actual file 
selection remains correct.
   
   These are code-traced findings; I have not run a modified real-file test or 
reproduced end-to-end data corruption. The default N = 0 preserves existing 
behavior, but opt-in status and N <= the retention count do not establish 
correctness when enabled. I would want real-file late-completion, compaction, 
and restore coverage before treating this as a low-risk refresh optimization.
   
   **2. The new config is not copied into the embedded server's remote-view 
config.**
   
   
[`EmbeddedTimelineService#getRemoteFileSystemViewConfig()`](https://github.com/apache/hudi/blob/300756f645b6963a63ef55ef4c728cecc841eda0/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/embedded/EmbeddedTimelineService.java#L210-L225)
 builds a fresh `FileSystemViewStorageConfig` and copies selected client 
properties, but does not copy `getRemoteTimelineHashNumInstants()`. 
Consequently, remote views constructed from this returned config still receive 
the default 0, even when the writer configured a positive value. This is 
particularly relevant because #20650 reports the embedded timeline server path.
   
   Please propagate the setting through that builder and add a test through the 
embedded-service configuration path. The new test that directly constructs 
`RemoteHoodieTableFileSystemView` with a configured builder bypasses this gap.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to