Hi dev@, I would like to propose closing a feature-parity gap in the Java engine and get early feedback on the scope before I open the PRs.
*Motivation* Today, the Java engine (hudi-java-client) only ships HoodieJavaWriteClient and HoodieJavaTableServiceClient. There is no read client, and the write path cannot complete MERGE_ON_READ (MoR) flows. While delta commits exist, there is no way to run or schedule compaction, or read back MoR tables from a plain Java application. Spark and Flink both have full MoR support, but embedded Java users currently have none. *What I Have Working (on my fork, full CI matrix green)* 1. New HoodieJavaReadClient for the Java engine: Supports snapshot, read-optimized, time-travel, and completion-time-based incremental reads for both MERGE_ON_READ and COPY_ON_WRITE tables. It builds one HoodieFileGroupReader per file slice (base file merged with log files) behind a lazily-chained ClosableIterator to keep memory bounded. Every call works on a fresh meta-client snapshot, making it safe under non-blocking concurrency control (NBCC) with inflight writers. Known limitations (documented in the javadoc) include no archived-timeline incremental ranges, no populate.meta.fields=false tables for completion-time filtering, no InternalSchema evolution reconciliation, and no CDC. 2. MoR write-side completion for the Java engine: Wires in the delta commit executor and compaction action executor so Java writers can run compaction end-to-end. 3. Bucket index support for the Java engine: Includes JavaBucketIndexPartitioner, bulk-insert partitioner, index factory, and layout-config wiring. 4. Fix in BaseAvroPayload (hudi-common): Modified getRecord(schema) to resolve fields by name from the record's true writer schema when the requested schema is a projection. This fixes a pre-existing bug that silently nulled data fields on merges against persisted data. Renamed or added-column schemas retain the legacy positional decode that the Hive/Presto MoR readers rely on. Unresolvable fields with no default now fail loudly with an AvroTypeException instead of returning corrupted data. Since this touches the shared merge path for all engines, I plan to land it first as its own PR to call out the risk. *Testing* Includes ~2,900 lines of new tests covering MoR snapshot/incremental reads, NBCC concurrent reader/writer coverage, payload schema-mismatch regression tests, and bucket-index partitioner tests. The full fork CI matrix (all engine jobs) is green, along with local runs of the Hive MoR schema-evolution functional tests and the hudi-hadoop-mr realtime reader suites. *Proposed Plan* I plan to file a feature GitHub issue with sub-issues and open three PRs: 1. fix(common): BaseAvroPayload decode fix 2. feat(client): HoodieJavaReadClient + MoR write-side completion 3. feat(client): bucket index for the Java engine *Questions for the Community* - Is a feature issue and PRs the right vehicle here, or would the community prefer an RFC given the new public read API? - Do you have any concerns with the proposed HoodieJavaReadClient API surface (iterator-based, caller closes iterators) before I finalize it? - Are the documented v1 limitations acceptable, or are any of them considered must-haves for the first iteration? Early feedback is welcome. You can review the code on my fork here: https://github.com/apache/hudi/compare/master...ravjot28:hudi:master Thanks, Ravjot
