kmitchener commented on PR #3540:
URL:
https://github.com/apache/arrow-datafusion/pull/3540#issuecomment-1252211707
I think that rather than any options, the support for object_store should be
invisible to the user. So just running this SQL from datafusion-cli should be
enough to trigger the cli to register the s3 objectstore:
```
create external table unicorns stored as parquet location
's3://my_bucket/lineitem/';
```
You could do that by creating the logical plan from the user's given SQL,
then only registering an object store if they're creating an external location
that's in the cloud. Something like:
```rust
let logical_plan = ctx.create_logical_plan(sql)?;
match logical_plan {
LogicalPlan::CreateExternalTable(external) => {
if external.location.to_lowercase().starts_with("s3://") {
// use
https://docs.rs/object_store/latest/object_store/path/struct.Path.html
// parse the path, find the bucket
// register the object store
}
}
_ => {},
}
// continue on and execute the logical plan
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
