Ted-Jiang opened a new pull request #1665:
URL: https://github.com/apache/arrow-datafusion/pull/1665
# Which issue does this PR close?
Closes #1648.
# Rationale for this change
We should filter out some path in list_file method,
like
https://github.com/apache/spark/blob/5b91381b9bcea27b7be6d9cef852efbd6c23d98a/core/src/main/scala/org/apache/spark/util/HadoopFSUtils.scala#L358
# Some question
I am not sure is this situation in same on S3 or other object store?
Should we add a method in struct `ObjectReader` like `should_filter_out `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
