[GitHub] [spark] cloud-fan commented on a change in pull request #26983: [SPARK-30331][SQL] Set isFinalPlan to true before posting the final AdaptiveSparkPlan event
cloud-fan commented on a change in pull request #26983: [SPARK-30331][SQL] Set
isFinalPlan to true before posting the final AdaptiveSparkPlan event
URL: https://github.com/apache/spark/pull/26983#discussion_r361085709
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala
##
@@ -39,6 +40,25 @@ class AdaptiveQueryExecSuite
setupTestData()
private def runAdaptiveAndVerifyResult(query: String): (SparkPlan,
SparkPlan) = {
+var finalPlanExistsOnce: Option[Boolean] = None
+val listener = new SparkListener {
+ override def onOtherEvent(event: SparkListenerEvent): Unit = {
+event match {
+ case SparkListenerSQLAdaptiveExecutionUpdate(_, _, sparkPlanInfo) =>
+if (sparkPlanInfo.simpleString.startsWith(
+ "AdaptiveSparkPlan(isFinalPlan=true)")) {
+ if (finalPlanExistsOnce.isDefined) {
+finalPlanExistsOnce = Some(false)
Review comment:
isn't it much simpler to use an int count?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #26983: [SPARK-30331][SQL] Set isFinalPlan to true before posting the final AdaptiveSparkPlan event
cloud-fan commented on a change in pull request #26983: [SPARK-30331][SQL] Set
isFinalPlan to true before posting the final AdaptiveSparkPlan event
URL: https://github.com/apache/spark/pull/26983#discussion_r360857625
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala
##
@@ -39,6 +40,20 @@ class AdaptiveQueryExecSuite
setupTestData()
private def runAdaptiveAndVerifyResult(query: String): (SparkPlan,
SparkPlan) = {
+var finalPlanExists = false
Review comment:
should we count it and make sure the event appears once and only once?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #26983: [SPARK-30331][SQL] Set isFinalPlan to true before posting the final AdaptiveSparkPlan event
cloud-fan commented on a change in pull request #26983: [SPARK-30331][SQL] Set
isFinalPlan to true before posting the final AdaptiveSparkPlan event
URL: https://github.com/apache/spark/pull/26983#discussion_r360857528
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala
##
@@ -39,6 +40,20 @@ class AdaptiveQueryExecSuite
setupTestData()
private def runAdaptiveAndVerifyResult(query: String): (SparkPlan,
SparkPlan) = {
+var finalPlanExists = false
+val listener = new SparkListener {
+ override def onOtherEvent(event: SparkListenerEvent): Unit = {
+event match {
+ case SparkListenerSQLAdaptiveExecutionUpdate(_, _, sparkPlanInfo) =>
+if
(sparkPlanInfo.simpleString.startsWith("AdaptiveSparkPlan(isFinalPlan=true)")) {
Review comment:
can we make it more type safe? e.g. check if it's a `AdaptiveSparkPlanExec`,
and check the `isFinalPlan` flag.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
