Davies Liu created SPARK-15307:
----------------------------------
Summary: Super slow to load a partitioned table from local disks
Key: SPARK-15307
URL: https://issues.apache.org/jira/browse/SPARK-15307
Project: Spark
Issue Type: Bug
Components: SQL
Reporter: Davies Liu
It took 22 seconds to load an partitioned table with 3000 files on local disks,
because DeprecatedRawLocalFileStatus in RawLocalFileSystem use subprocess to
load permission information.
Right now, we did use these permission information, we could use another
constructor of LocatedFileStatus to avoid this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
