houqp commented on a change in pull request #1779:
URL: https://github.com/apache/arrow-datafusion/pull/1779#discussion_r801279141
##########
File path: datafusion/src/datasource/object_store/local.rs
##########
@@ -39,6 +39,11 @@ pub struct LocalFileSystem;
#[async_trait]
impl ObjectStore for LocalFileSystem {
async fn list_file(&self, prefix: &str) -> Result<FileMetaStream> {
+ let prefix = if let Some((_scheme, path)) = prefix.split_once("://") {
Review comment:
Got it. Shouldn't this vip host:port be managed as an attribute in the
hdfs object store instance? for example, when we create the object store
instance and registry it in the registry, we specify the vip, then we only need
to provide the object path in list and get object store calls. Internally when
the object store handles the list operation, it will perform the pass in the
vip host to the hdfs client. Basically my thinking on this is if we have an one
to one mapping between an objectstore instance and vip, then we shouldn't need
to pass in the vip as part of the object key in these method calls.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
