shatrughnaupadhyay opened a new pull request, #51700: URL: https://github.com/apache/arrow/pull/51700
A single binary or string value can exceed a gRPC message limit even when its record batch contains one row. This Python example sends IPC bytes through capped metadata-only DoExchange frames, with a private capability probe, sequence checks, a total byte limit and temporary-file reassembly. This is an experimental application protocol related to #34485 and #41840. It leaves standard FlightData unchanged and requires both peers to opt in. Reassembly uses a capped temporary file. Input batches, IPC copies, decoded output, native transport allocations and OS caching remain outside the frame-size bound; process RSS is not bounded. It does not resolve #34485. Validation: 41 tests passed with Python 3.12.12 and the PyArrow 25.0.1 wheel, including 4 MiB single-row values under a 128 KiB client gRPC message cap, malformed frames, cancellation and cleanup. Flake8, autopep8 and whitespace checks passed. A benchmark is included. Source-built Arrow and upstream CI have not been verified. Draft: design agreement and human author review are pending. AI assistance: Codex generated the example, tests, benchmark and documentation and ran the local checks. The author must review, understand and own the changes before marking this ready. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
