linliu-code opened a new pull request, #677:
URL: https://github.com/apache/hudi-rs/pull/677
Foundation for landing a second merge implementation. **2 files, +251/−2.**
No read changes.
### What this establishes
`hoodie.read.merge.engine` selects `legacy` (the default) or `v2`.
`FileGroupReader::read_file_slice_from_paths` becomes the single decision
point — every entry
point, eager and streaming, already funnels through it:
```
read_file_slice_from_paths(base, logs, options)
├─ metadata table → existing reader (always, setting ignored)
├─ engine == legacy → existing reader (the default)
├─ v2 cannot serve this → existing reader (reason logged)
├─ engine == v2 → the new engine
└─ no logs / read-optimized → base-file read
│
all paths ────┴──→ apply_eager_options(filters,
projection)
```
`Table`, DataFusion, the Python binding and the cxx bridge are untouched.
### Selecting the engine is a request, not a guarantee
`v2_unsupported_reason()` runs first and names what the engine cannot serve.
Those reads are
served by the existing reader instead, so **the setting cannot turn a
working read into a failing
one** — and the engine can land incrementally behind a contract that removes
one reason at a
time, rather than needing full parity before it is reachable.
Every fall back is logged. A fall back nobody can observe is
indistinguishable from an engine
that is never used.
### The distinction that keeps fall back safe
The check is decided **from config before any I/O — never a catch-all on
error.** A read that
fails *inside* the engine propagates. Retrying it on the existing reader
would make a bug look
like a success, make results depend on which engine happened to win, and
leave differential
tests unable to see anything.
### Two cases error rather than fall back
- **`CUSTOM` record merge mode** needs a merger neither reader implements.
Falling back would use
the existing reader's own derivation, which drops deletes — wrong rows are
worse than a
refusal. Read by raw key (`hoodie.record.merge.mode`); this crate has no
typed config for it
yet, and adding one belongs with the reader that acts on it.
- **A metadata table** is routed to the existing reader *before* the check,
since its base files
and log blocks are HFile. The error behind that routing is an unreachable
guard, so a future
change to the routing fails loudly rather than reaching an engine that
cannot read HFile.
Erroring outright would make metadata tables unreadable whenever the
setting is applied
globally — and table listing reads one.
### An unrecognised value is an error
Not a fall back to the default. `get_or_default` would have swallowed a typo
and read with the
other engine, leaving a caller convinced they had exercised the one they
asked for. That is the
one outcome a switch like this must not produce.
### Testing
Five tests: the default is `legacy`; an unrecognised name errors and names
the value; asking for
`v2` falls back with a reason; **a real merge-on-read slice returns the same
rows whether or not
`v2` is selected**; a metadata table ignores the setting.
Every capability is unimplemented in this PR, so `v2` is never entered. If
it ever reports itself
capable before an engine is wired up, the read errors loudly rather than
silently doing nothing.
Full suite green: 715 core + 79 table-read + 39 datafusion. `cargo fmt
--check` and clippy clean.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
