vmingchen commented on code in PR #66:
URL: https://github.com/apache/datafusion-ray/pull/66#discussion_r1973563759
##########
src/util.rs:
##########
@@ -397,6 +402,52 @@ fn print_node(plan: &Arc<dyn ExecutionPlan>, indent:
usize, output: &mut String)
}
}
+async fn exec_sql(query: String, tables: Vec<(String, String)>) ->
PyResult<RecordBatch> {
+ let ctx = SessionContext::new();
+ for (name, path) in tables {
+ if path.ends_with(".parquet") {
Review Comment:
Thanks for pointing this out! I looked into it, and it turned out
[`register_parquet` internally use `register_listing_table` as
well](https://github.com/apache/datafusion/blob/fc2fbb3d6b3aded73f1b0902168e008e580c89c1/datafusion/core/src/execution/context/parquet.rs#L55-L61).
The later is capable of register both a single file and a directory of files.
So I have changed the function to use `register_listing_table` only in
https://github.com/apache/datafusion-ray/pull/66/commits/b8e0c6b3c10321896f39e33dee94ea843d91505a;
the new commit also adds a unit test to check that it is working for both file
and directory and a doc pointing to the format of the URI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
