yahoNanJing commented on a change in pull request #1779:
URL: https://github.com/apache/arrow-datafusion/pull/1779#discussion_r801275370
##########
File path: datafusion/src/datasource/object_store/local.rs
##########
@@ -39,6 +39,11 @@ pub struct LocalFileSystem;
#[async_trait]
impl ObjectStore for LocalFileSystem {
async fn list_file(&self, prefix: &str) -> Result<FileMetaStream> {
+ let prefix = if let Some((_scheme, path)) = prefix.split_once("://") {
Review comment:
For hdfs, there's a HA mechanism to provide a kind of vip host name. The
hostname is not a name for a real host, by which ping will not work. However,
the dependent object store client is able to recognize that name to direct
requests to the real host.
I think this part should be the capability of remote object store, either by
the way of HDFS or providing a vip service.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
