nealrichardson commented on a change in pull request #11032:
URL: https://github.com/apache/arrow/pull/11032#discussion_r722330403
##########
File path: r/R/duckdb.R
##########
@@ -111,3 +111,18 @@ duckdb_disconnector <- function(con, tbl_name) {
})
environment()
}
+
+to_arrow <- function(.data) {
+ res <- DBI::dbSendQuery(dbplyr::remote_con(.data),
dbplyr::remote_query(.data), arrow = TRUE)
+
+ # transform the source ExecNode we get below into something dataset-like
that we can
+ # make arrow_dplyr_querys/datasets from
+ # Should be able to:
+ # * get the record batch reader from duckdb
+ # * produce the SourceNode
+ # * build an ExecPlan with that in place of the ScanNode you would have
gotten from ExecNode_Scan
+ plan <- ExecPlan$create()
Review comment:
I don't think this will work because you're making this node with an
ExecPlan here, but I think you'll be creating a different ExecPlan in
`collect()`.
Why not return the RecordBatchReader from
duckdb::duckdb_fetch_record_batch(res)? Then do the wrapping of that inside
`plan$Build` (or even `plan$Scan`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
