Repository: spark Updated Branches: refs/heads/branch-2.2 eef72d3f0 -> 38a0532cf
[SPARK-22637][SQL] Only refresh a logical plan once. ## What changes were proposed in this pull request? `CatalogImpl.refreshTable` uses `foreach(..)` to refresh all tables in a view. This traverses all nodes in the subtree and calls `LogicalPlan.refresh()` on these nodes. However `LogicalPlan.refresh()` is also refreshing its children, as a result refreshing a large view can be quite expensive. This PR just calls `LogicalPlan.refresh()` on the top node. ## How was this patch tested? Existing tests. Author: Herman van Hovell <hvanhov...@databricks.com> Closes #19837 from hvanhovell/SPARK-22637. (cherry picked from commit 475a29f11ef488e7cb19bf7e0696d9d099d77c92) Signed-off-by: gatorsmile <gatorsm...@gmail.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/38a0532c Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/38a0532c Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/38a0532c Branch: refs/heads/branch-2.2 Commit: 38a0532cfc59adbd182ad925da99d32c27a12f2b Parents: eef72d3 Author: Herman van Hovell <hvanhov...@databricks.com> Authored: Tue Nov 28 16:03:47 2017 -0800 Committer: gatorsmile <gatorsm...@gmail.com> Committed: Tue Nov 28 16:04:02 2017 -0800 ---------------------------------------------------------------------- .../src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/38a0532c/sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala ---------------------------------------------------------------------- diff --git a/sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala b/sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala index e3ed419..f5338ac 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala @@ -468,7 +468,7 @@ class CatalogImpl(sparkSession: SparkSession) extends Catalog { if (tableMetadata.tableType == CatalogTableType.VIEW) { // Temp or persistent views: refresh (or invalidate) any metadata/data cached // in the plan recursively. - table.queryExecution.analyzed.foreach(_.refresh()) + table.queryExecution.analyzed.refresh() } else { // Non-temp tables: refresh the metadata cache. sessionCatalog.refreshTable(tableIdent) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
