eitsupi opened a new pull request, #975:
URL: https://github.com/apache/arrow-nanoarrow/pull/975
Add `function_array_stream(fun, schema, validate = TRUE)` to create an Arrow
array stream that requests batches lazily from an R function. Construction and
schema access do not call the function; returning `NULL` ends the stream. The
explicit schema is copied at construction, and batches are converted and
optionally validated against it.
The stream retains callback state across export and garbage collection and
supports `array_stream_set_finalizer()`. Errors remain terminal, reentrant
calls are rejected, and callback evaluation is restricted to the R main thread.
Tests cover empty and multi-batch streams, coercion and validation, errors and
interrupts, abort restarts, warnings, export, finalizers, and release/GC
behavior.
Feedback requested on the evaluation boundary: the implementation uses
`R_ToplevelExec()` around `R_tryCatch()` for errors and interrupts.
`R_tryCatch()` alone does not contain non-local exits such as
`invokeRestart("abort")`; allowing those jumps across an Arrow C Stream
callback can bypass a C/C++ consumer's cleanup. The outer boundary converts
these exits to EIO.
This isolation also means the caller's calling handlers do not apply inside
the callback, and warnings are handled in the isolated top-level context unless
the callback handles them itself. These semantics are documented. Is this
boundary and its handler behavior appropriate for the R API, or would you
prefer another approach using public R APIs?
Validation on R 4.6.1 / Linux: the package test suite passes, and `R CMD
check --as-cran --no-manual` reports 0 errors, 0 warnings, and the same
compiler-flag NOTE as main. The check test suite reports 1718 passes and 7
skips for unavailable optional reticulate/lz4 support. An additional
installed-package lifecycle check completes 100 export/finalizer cycles and
collects 100 unreferenced streams with no remaining preserved R objects.
Related to #386.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
