(spark) branch master updated: [SPARK-47285][SQL] AdaptiveSparkPlanExec should always use the context.session

[yao]

This is an automated email from the ASF dual-hosted git repository.

yao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 36299f1b2bff [SPARK-47285][SQL] AdaptiveSparkPlanExec should always 
use the context.session
36299f1b2bff is described below

commit 36299f1b2bff8c892624bc5c0e4c00ea3f261532
Author: ulysses-you <ulyssesyo...@gmail.com>
AuthorDate: Wed Mar 6 09:53:53 2024 +0800

    [SPARK-47285][SQL] AdaptiveSparkPlanExec should always use the 
context.session
    
    ### What changes were proposed in this pull request?
    
    Use `context.session` instead of `session` to avoid potential issue. For 
example, a cached plan may re-instance `AdaptiveSparkPlanExec` with a different 
active session.
    
    ### Why are the changes needed?
    
    avoid potential issue
    
    ### Does this PR introduce _any_ user-facing change?
    
    avoid potential issue
    
    ### How was this patch tested?
    
    avoid potential issue
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    no
    
    Closes #45388 from ulysses-you/aqe.
    
    Authored-by: ulysses-you <ulyssesyo...@gmail.com>
    Signed-off-by: Kent Yao <y...@apache.org>
---
 .../apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala  | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git 
a/sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala
 
b/sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala
index b2a1141f2f5b..2879aaca7215 100644
--- 
a/sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala
+++ 
b/sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala
@@ -87,7 +87,7 @@ case class AdaptiveSparkPlanExec(
 
   // The logical plan optimizer for re-optimizing the current logical plan.
   @transient private val optimizer = new AQEOptimizer(conf,
-    session.sessionState.adaptiveRulesHolder.runtimeOptimizerRules)
+    context.session.sessionState.adaptiveRulesHolder.runtimeOptimizerRules)
 
   // `EnsureRequirements` may remove user-specified repartition and assume the 
query plan won't
   // change its output partitioning. This assumption is not true in AQE. Here 
we check the
@@ -103,7 +103,8 @@ case class AdaptiveSparkPlanExec(
 
   @transient private val costEvaluator =
     conf.getConf(SQLConf.ADAPTIVE_CUSTOM_COST_EVALUATOR_CLASS) match {
-      case Some(className) => CostEvaluator.instantiate(className, 
session.sparkContext.getConf)
+      case Some(className) =>
+        CostEvaluator.instantiate(className, 
context.session.sparkContext.getConf)
       case _ => 
SimpleCostEvaluator(conf.getConf(SQLConf.ADAPTIVE_FORCE_OPTIMIZE_SKEWED_JOIN))
     }
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org