codope opened a new pull request, #767:
URL: https://github.com/apache/hudi-rs/pull/767
## Description
`read_file_slice` and `read_file_slice_from_paths` built
`ReadOptions::new()` internally, so a C++ caller got no column projection, no
filters and no read-optimized mode even though `FileGroupReader` supports all
of them. A merge-on-read slice was always materialized with every column.
This adds two shared bridge structs and two entry points:
```cpp
struct HudiFilter { rust::String field; rust::String op; rust::String
value; };
struct HudiReadOptions { rust::Vec<rust::String> projection; // empty =
all columns
rust::Vec<HudiFilter> filters; // op: = != <
<= > >= IN "NOT IN"
rust::Vec<rust::String> hudi_options; //
"hoodie.read.*=value"
};
read_file_slice_with_options(slice, options);
read_file_slice_from_paths_with_options(base_file_path, log_file_paths,
options);
```
Filters use the same `(field, op, value)` grammar as the Python binding so
the two bindings stay in sync; `hudi_options` uses the same `key=value` shape
as the reader constructor, which gives `hoodie.read.use.read_optimized.mode`,
`hoodie.read.stream.batch_size` and `hoodie.read.as.of.timestamp` for free. The
existing two entry points stay as wrappers over default options.
Behavior change for C++ callers: a `key=value` option without `=` now throws
`rust::Error` from `new_file_group_reader_with_options` instead of being
silently dropped, so both option surfaces in the header behave the same. The
header doc says so.
Stacked on #766 (branch `cpp/catch-unwind-636`); review the last commit only
until that merges. `cargo test -p hudi-cpp --doc` fails with a pre-existing
`E0464: multiple candidates for rlib hudi` that is present on `main`; `make
test-rust` runs `--all-targets`, which skips doctests.
Closes #637. Related: apache/hudi#18308.
## How are the changes test-covered
- [ ] N/A
- [x] Automated tests (unit and/or integration tests)
- [ ] Manual tests
- [ ] Details are described below
Unit tests in `cpp/src/lib.rs`: option mapping (defaults, every field, bad
operator, option without `=`, and the `key=value` parser the constructor now
shares); a MOR slice with real log files read through
`read_file_slice_with_options` with a projection and an `id = <max>` filter;
the paths entry point driven through real `CxxString`/`CxxVector` arguments
with a projection and filter; and read-optimized mode proven by a slice whose
only log file does not exist, which reads with the option set and fails naming
the log file without it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
