guichaoxian created SPARK-22089: ----------------------------------- Summary: There is no need for fileStatusCache to invalidateAll when InMemoryFileIndex refresh Key: SPARK-22089 URL: https://issues.apache.org/jira/browse/SPARK-22089 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 2.1.0 Reporter: guichaoxian
the fileStatusCache is globally shared cache,refresh is only for one table,so we do not need to invalidateAll entry in fileStatusCache {code:java} /** Globally shared (not exclusive to this table) cache for file statuses to speed up listing. */ private val fileStatusCache = FileStatusCache.getOrCreate(sparkSession) {code} {code:java} override def refresh(): Unit = { refresh0() fileStatusCache.invalidateAll() } {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org