Github user sujith71955 commented on a diff in the pull request: https://github.com/apache/spark/pull/22721#discussion_r231794961 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -183,13 +183,14 @@ case class InsertIntoHadoopFsRelationCommand( refreshUpdatedPartitions(updatedPartitionPaths) } - // refresh cached files in FileIndex - fileIndex.foreach(_.refresh()) - // refresh data cache if table is cached - sparkSession.catalog.refreshByPath(outputPath.toString) - if (catalogTable.nonEmpty) { + sparkSession.sessionState.catalog.refreshTable(catalogTable.get.identifier) --- End diff -- Here one more point i want to mention is in hive by default hive.stats.autogather is true, so the DML commands in hive tables will make sure the stats always updated, so i think if we initialize the stats in insert flow , the problems will be solved.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org