Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11576#issuecomment-194108153 Yea I think evenutally it might make sense -- but one problem is that it is very expensive to detect this, especially when there are a very large number of files, which is also common.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org