vinothchandar commented on PR #19309: URL: https://github.com/apache/hudi/pull/19309#issuecomment-5273914045
Did a pass comparing this against my comments on #19107. Level setting on what carried over: ### Addressed: RFC number/handles/registry link — sorted. Posting block replacing per-vector records — this is the right reframe; MDT record count was never the concern, amplification was. IVF vs HNSW rationale, generation model, active generation pointer, generation-aware cleaning — all now spelled out. Stale entry handling is locked down to delta supersession + RLI arbitration instead of a menu of options. Good. Rebuild is replay-only with catch-up + short lock at cutover, so background build under live writes is covered. nprobe/refineFactor now explicit config with defaults; benchmark section added. ### Still open from my side: - LIMITATIONS section — We'd still want VECTOR schema evolution (dimension change → rebuild?) written down so we throw the right errors in the first cut. - Concurrency across OCC/NBCC still needs a table to clearly spell out the scenarios; today it's scattered across §5.5/§9.1 (?) - "Generation" is used in §2 before it's defined in §4.5 — introduce it in a line up front. - How do we detect a vector is unchanged? §5.3 assumes "non-vector update". we need to flesh this out - Re-rank cost varies by base file format — worth a line, and a recommendation. - Partition pushdown to bound the search is deferred as non-goal; fine for v1, but let's note it as a known gap since the partition dictionary is already in the block. - §7 still fully specs LIRE split/merge even though MVP is create/drop/rebuild — let's mark clearly what's in v1 vs later. is LIRE in the v1? - Minor: define "posting" for the average reader, and clarify "shard" here vs how we use it in RLI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
