XiaoHongbo-Hope opened a new pull request, #10701: URL: https://github.com/apache/paimon/pull/10701
### Purpose LeRobot frame lookups still query the Python BTree scanner when native reading is enabled. Route batch predicates through the existing native planner so Rust can locate and read frame rows. Reuse `read.native.enabled` / `scan.native-plan.enabled`; keep the Python path when native is unavailable. Pin the reader's resolved snapshot across batches and worker serialization. Keep residual predicates for rows not yet covered by an index. No training API changes or Rust changes are required. ### Tests - 158 tests and 44 subtests passed, including unindexed appends, projection, missing/duplicate IDs, snapshot/tag stability, spawn workers and fallback. - Flake8 and `git diff --check` passed. - Local diagnostic: 100K rows, 256 random IDs per call, 3 paired rounds; median row-read latency 36.9 ms → 11.6 ms. This is not an end-to-end training benchmark; full DataLoader throughput remains to be measured. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
