jianguotian opened a new pull request, #10516:
URL: https://github.com/apache/paimon/pull/10516
### Purpose
Mosaic reads Arrow batches internally, but currently exposes row iterators
to callers. Expose those batches through Paimon's existing vectorized reader
contract so downstream engines can process columns without unpacking rows.
- `ArrowBatchReader` and `MosaicRecordsReader` expose independent batch
wrappers and release Arrow buffers per consumed batch, preserving projection,
prefetch and row-tracking fallback.
- `MosaicRecordsWriter` exposes its conversion buffer, flush and
failed-state check to subclasses. Failed native writes reject further input and
are not retried during close.
- `MosaicWriterFactory` adds an overridable creation hook while retaining
option validation.
Existing bundle interfaces, column-to-Arrow conversion and native Mosaic
writing are reused. `MosaicRecordsWriter` continues to use the row fallback for
ordinary `VectorizedBundleRecords`; consumers can implement batch-write
adapters through the new hooks.
### Tests
Passed clean Maven verification on JDK 8 (linux/aarch64): **131 tests, zero
failures, errors or skips**. Checkstyle, Spotless and Enforcer checks were
enabled.
```bash
mvn -B -pl paimon-arrow,paimon-mosaic -am \
-Dtest=ArrowBatchReaderTest,Mosaic*Test -DwildcardSuites=none \
-DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false clean verify
```
Coverage includes prefetch, projection, row tracking across row groups,
deletion vectors, batch lifetime, native-write failures, external writer
subclass round trips and factory option validation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
