Hi all, I'd like to share a bug fix introduced by the following PR:
- master: https://github.com/apache/iotdb/pull/18647 This PR fixes two related TVList iterator state issues that can make a MemChunk return wrong values when reads cross memtable page boundaries. In an aligned TVList, a duplicate-timestamp group prepared before a page boundary could be discarded while the iterator cursor did not move, so the group was later recomputed from its last physical row and earlier non-null values were lost (e.g. at time=100 the expected [2, 20] became [null, 20]). In a non-aligned TVList, nextBatch() advanced the cursor but left probeNext set, so a following point read skipped prepareNext() and could return an old duplicate version or an already-deleted point. Both only manifest when multiple physical rows need iterator normalization (duplicate timestamps or points hidden by deletion) and a batch page is followed by an empty or overlapping point page. The root cause is inconsistent probeNext handling across page switches: - Aligned path: skipToCurrentTimeRangeStartPosition() unconditionally cleared probeNext, even when the binary search did not move the cursor. A duplicate-timestamp group prepared before the page boundary was therefore thrown away although it was still valid. - Non-aligned path: nextBatch() advanced index without clearing probeNext, leaving the iterator in a state where index has advanced but probeNext is still true, which violates the iterator invariant the point reader relies on. The fix keeps the prepared state consistent with the cursor: - Aligned: clear probeNext only when the cursor actually advances (newIndex > index); when newIndex == index, the prepared duplicate-timestamp group stays valid for a later page. - Non-aligned: clear probeNext after nextBatch() builds and returns its block, matching the existing aligned batch behavior so any later point read performs prepareNext() from the current physical index. Main changes include: - TVList.java: moved probeNext = false into the cursor-advanced branch of skipToCurrentTimeRangeStartPosition(), and added probeNext = false after nextBatch(). - AlignedTVListIteratorTest.java: added regression tests for aligned ASC and DESC page switching over [1,33], [34,66], [67,100]. - NonAlignedTVListIteratorTest.java: added regression tests for non-aligned ASC batch-to-point duplicate timestamps and DESC batch-to-point deletion handling. These are iterator-level regression tests that verify the state transitions directly; the full SeriesScanUtil -> LazyMemVersionPageReader -> PriorityMergeReader integration path is not covered. Feedback and suggestions are welcome. 大家好， 我想分享以下 PR 中修复的一个 bug： - master：https://github.com/apache/iotdb/pull/18647 本 PR 修复了 TVList 迭代器中两个相关的状态问题，它们会导致跨 memtable 页边界读取时 MemChunk 返回错误的值。在对齐 TVList 中，页边界之前已准备好的重复时间戳分组会在迭代器游标未移动的情况下被丢弃，于是该分组随后从其最后一条物理行重新计算，较早的非空值被丢失（例如 time=100 处期望的 [2, 20] 变成了 [null, 20]）。在非对齐 TVList 中，nextBatch() 推进了游标但没有清除 probeNext，导致后续的点读跳过 prepareNext() ，可能返回旧的重复版本或已被删除的点。这两个问题都只在多条物理行需要迭代器归一化（重复时间戳或被删除点遮住）且批读页之后跟着空页或重叠点读页时出现。 根本原因是页切换时对 probeNext 的处理不一致： - 对齐路径：skipToCurrentTimeRangeStartPosition() 无条件清除 probeNext ，即使二分查找并未移动游标。于是页边界之前准备好的重复时间戳分组在仍然有效的情况下被丢弃。 - 非对齐路径：nextBatch() 推进了 index 却没有清除 probeNext，使迭代器处于 index 已推进但 probeNext 仍为 true 的状态，破坏了点读路径所依赖的迭代器不变式。 本次修复让准备好的状态与游标保持一致： - 对齐：仅当游标真正推进（newIndex > index）时才清除 probeNext；当 newIndex == index 时，已准备好的重复时间戳分组对后续页仍然有效。 - 非对齐：在 nextBatch() 构建并返回结果块后清除 probeNext，与已有的对齐批读行为一致，从而保证后续任何点读都会从当前物理索引执行 prepareNext()。 主要修改包括： - TVList.java：把 probeNext = false 移入 skipToCurrentTimeRangeStartPosition() 中游标推进的分支，并在 nextBatch() 之后增加 probeNext = false。 - AlignedTVListIteratorTest.java：新增对齐 ASC、DESC 在 [1,33]、[34,66]、[67,100] 三页切换下的回归测试。 - NonAlignedTVListIteratorTest.java：新增非对齐 ASC 批读转点读的重复时间戳、以及 DESC 批读转点读的删除处理回归测试。 这些是迭代器级别的回归测试，直接验证状态转换；完整的 SeriesScanUtil -> LazyMemVersionPageReader -> PriorityMergeReader 集成路径未覆盖。 欢迎大家提出反馈和建议。 Best regards, Wenwei Shu
