[spark] branch master updated: [SPARK-39181][SQL] `SessionCatalog.reset` should not drop temp functions twice

[dongjoon]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 3d9049533d8 [SPARK-39181][SQL] `SessionCatalog.reset` should not drop 
temp functions twice
3d9049533d8 is described below

commit 3d9049533d8bc75cc2c4832dc2fd6f8ac53efe21
Author: Wenchen Fan <wenc...@databricks.com>
AuthorDate: Fri May 13 10:21:15 2022 -0700

    [SPARK-39181][SQL] `SessionCatalog.reset` should not drop temp functions 
twice
    
    ### What changes were proposed in this pull request?
    
    `SessionCatalog.reset` is a test only API and it drops the temp functions 
twice currently:
    1. once with `listFunctions(DEFAULT_DATABASE)...` which list temp functions 
as well
    2. once with `functionRegistry.clear()`
    
    This PR changes `listFunctions` to `externalCatalog.listFunctions` which 
only list permanent functions.
    
    ### Why are the changes needed?
    
    code simplification and probably makes tests run faster
    
    ### Does this PR introduce _any_ user-facing change?
    
    no
    
    ### How was this patch tested?
    
    N/A
    
    Closes #36542 from cloud-fan/minor.
    
    Authored-by: Wenchen Fan <wenc...@databricks.com>
    Signed-off-by: Dongjoon Hyun <dongj...@apache.org>
---
 .../apache/spark/sql/catalyst/catalog/SessionCatalog.scala    | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
index 6b7f8a207d6..d6c80f98bf7 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
@@ -1818,13 +1818,10 @@ class SessionCatalog(
     listTables(DEFAULT_DATABASE).foreach { table =>
       dropTable(table, ignoreIfNotExists = false, purge = false)
     }
-    listFunctions(DEFAULT_DATABASE).map(_._1).foreach { func =>
-      if (func.database.isDefined) {
-        dropFunction(func, ignoreIfNotExists = false)
-      } else {
-        dropTempFunction(func.funcName, ignoreIfNotExists = false)
-      }
-    }
+    // Temp functions are dropped below, we only need to drop permanent 
functions here.
+    externalCatalog.listFunctions(DEFAULT_DATABASE, "*").map { f =>
+      FunctionIdentifier(f, Some(DEFAULT_DATABASE))
+    }.foreach(dropFunction(_, ignoreIfNotExists = false))
     clearTempTables()
     globalTempViewManager.clear()
     functionRegistry.clear()


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org