fornwall opened a new issue, #10854: URL: https://github.com/apache/arrow-rs/issues/10854
### Problem The Arrow C Stream interface lets `get_schema` and `get_next` return errno-compatible error codes. `FFI_ArrowArrayStream::new`, however, chooses the code only from the outer `ArrowError` variant: - `NotYetImplemented` -> `ENOSYS` - `MemoryError` -> `ENOMEM` - `IoError` -> `EIO` - everything else -> `EINVAL` A producer whose domain error has more precise semantics cannot preserve them. For example, an ADBC driver naturally wraps an `adbc_core::error::Error` in `ArrowError::ExternalError`; cancellation, timeout, missing-resource, authorization, and all other statuses then become `EINVAL`. Even an `IoError` containing `std::io::Error::from_raw_os_error(ECANCELED)` becomes `EIO` because the raw OS code is discarded. Producers must currently reimplement the unsafe Arrow C Stream callbacks to return the right code. ### Minimal reproduction 1. Implement a `RecordBatchReader` whose next item is an `ArrowError::ExternalError` representing cancellation. 2. Export it with `FFI_ArrowArrayStream::new`. 3. Invoke the exported stream's `get_next` callback. 4. Observe `EINVAL`; there is no semantically correct `ArrowError` that makes it return `ECANCELED`. ### Expected behavior Provide a supported way for producers to preserve or select an errno-compatible nonzero result when exporting an Arrow C Stream. Possible designs include preserving `std::io::Error::raw_os_error()` for `ArrowError::IoError`, adding a general error-code representation, or accepting a producer-supplied mapping. A mapping must not allow zero for an error, because that reports success while leaving the output uninitialized. The need is not ADBC-specific: any Rust producer with domain-specific failure codes encounters the same loss. ADBC makes it visible because its specification explicitly permits `ECANCELED` while consuming a result stream. ### Related work - #10300 proposed a custom error-code mapper and was closed in favor of handling the ADBC use case in Apache ADBC. - apache/arrow-adbc#4475 explored ADBC rich stream errors and was closed for the maintainer to revisit the broader design. - #10844 improved import diagnostics so the callback result and producer message are retained in the returned error text; it does not change the exporter mapping above. I maintain a Rust ADBC driver with hand-written, panic-contained C Stream callbacks, so that driver can map its statuses locally and is not blocked on this issue. This issue tracks the remaining general exporter limitation; I am intentionally not opening another PR without agreement on the desired API. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
