This is an automated email from the ASF dual-hosted git repository.
changchen pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 0fbec927b4 [GLUTEN-11550][UT] Add
GlutenTestSetWithSystemPropertyTrait, enable 6 suites, remove 7
TestHiveSingleton suites, fix JobTagging flaky test (#11847)
0fbec927b4 is described below
commit 0fbec927b43d62beded4c7046ad299d2abba0a07
Author: Chang Chen <[email protected]>
AuthorDate: Fri Apr 10 11:14:16 2026 +0800
[GLUTEN-11550][UT] Add GlutenTestSetWithSystemPropertyTrait, enable 6
suites, remove 7 TestHiveSingleton suites, fix JobTagging flaky test (#11847)
* [GLUTEN-11550][UT] Add GlutenTestSetWithSystemPropertyTrait and enable 6
suites
Add a reusable trait for test suites whose parent creates per-test
SparkContext/SparkSession (via LocalSparkContext/LocalSparkSession).
It injects GlutenPlugin config via system properties so per-test
sessions inherit it, avoiding shared session conflicts.
Enable 6 suites by switching to this trait:
- GlutenSQLExecutionSuite (SQLExecutionSuite: new SparkContext internally)
- GlutenSQLJsonProtocolSuite (SQLJsonProtocolSuite with LocalSparkSession)
- GlutenShufflePartitionsUtilSuite (ShufflePartitionsUtilSuite with
LocalSparkContext)
- GlutenExternalAppendOnlyUnsafeRowArraySuite (with LocalSparkContext)
- GlutenUnsafeRowSerializerSuite (UnsafeRowSerializerSuite with
LocalSparkSession)
- GlutenSparkSessionJobTaggingAndCancellationSuite (with LocalSparkContext)
Co-authored-by: Copilot <[email protected]>
* [GLUTEN-11550][UT] Remove TestHiveSingleton suites that cannot load
GlutenPlugin
Remove 7 test suites (spark40 + spark41) that extend TestHiveSingleton.
TestHiveSingleton.spark (protected val) conflicts with Gluten test traits'
spark (implicit protected def), making it impossible to load GlutenPlugin.
The non-Hive variants (e.g. GlutenBucketedReadWithoutHiveSupportSuite)
are already enabled and cover the same functionality.
Removed suites:
- GlutenBucketedReadWithHiveSupportSuite
- GlutenBucketedWriteWithHiveSupportSuite
- GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite
- GlutenCommitFailureTestRelationSuite
- GlutenJsonHadoopFsRelationSuite
- GlutenParquetHadoopFsRelationSuite
- GlutenSimpleTextHadoopFsRelationSuite
Co-authored-by: Copilot <[email protected]>
* [GLUTEN-11550][UT] Fix JobTagging flaky test with fresh thread pool
The original test uses ExecutionContext.global whose ForkJoinPool reuses
threads created long before addTag(). InheritableThreadLocal (used by
managedJobTags) only copies values at thread creation time, so reused
threads see an empty tag map. This causes withSessionTagsApplied to not
attach the user tag to the job, making cancelJobsWithTagWithFuture
return empty.
Fix: testGluten override that creates a fresh SingleThreadExecutor AFTER
addTag(), ensuring the new thread inherits InheritableThreadLocal values.
Co-authored-by: Copilot <[email protected]>
---------
Co-authored-by: Copilot <[email protected]>
---
.../sql/GlutenTestSetWithSystemPropertyTrait.scala | 48 ++++++++++++++++
gluten-ut/excluded-spark-uts.md | 3 +-
.../gluten/utils/velox/VeloxTestSettings.scala | 18 ++----
...parkSessionJobTaggingAndCancellationSuite.scala | 67 +++++++++++++++++++++-
...utenExternalAppendOnlyUnsafeRowArraySuite.scala | 4 +-
.../sql/execution/GlutenSQLExecutionSuite.scala | 4 +-
.../sql/execution/GlutenSQLJsonProtocolSuite.scala | 6 +-
.../GlutenShufflePartitionsUtilSuite.scala | 6 +-
.../execution/GlutenUnsafeRowSerializerSuite.scala | 6 +-
.../GlutenBucketedReadWithHiveSupportSuite.scala | 23 --------
.../GlutenBucketedWriteWithHiveSupportSuite.scala | 23 --------
.../GlutenCommitFailureTestRelationSuite.scala | 23 --------
...necessaryBucketedScanWithHiveSupportSuite.scala | 23 --------
.../sources/GlutenJsonHadoopFsRelationSuite.scala | 23 --------
.../GlutenParquetHadoopFsRelationSuite.scala | 23 --------
.../GlutenSimpleTextHadoopFsRelationSuite.scala | 23 --------
.../gluten/utils/velox/VeloxTestSettings.scala | 18 ++----
...parkSessionJobTaggingAndCancellationSuite.scala | 67 +++++++++++++++++++++-
...utenExternalAppendOnlyUnsafeRowArraySuite.scala | 4 +-
.../sql/execution/GlutenSQLExecutionSuite.scala | 4 +-
.../sql/execution/GlutenSQLJsonProtocolSuite.scala | 6 +-
.../GlutenShufflePartitionsUtilSuite.scala | 6 +-
.../execution/GlutenUnsafeRowSerializerSuite.scala | 6 +-
.../GlutenBucketedReadWithHiveSupportSuite.scala | 23 --------
.../GlutenBucketedWriteWithHiveSupportSuite.scala | 23 --------
.../GlutenCommitFailureTestRelationSuite.scala | 23 --------
...necessaryBucketedScanWithHiveSupportSuite.scala | 23 --------
.../sources/GlutenJsonHadoopFsRelationSuite.scala | 23 --------
.../GlutenParquetHadoopFsRelationSuite.scala | 23 --------
.../GlutenSimpleTextHadoopFsRelationSuite.scala | 23 --------
30 files changed, 225 insertions(+), 370 deletions(-)
diff --git
a/gluten-ut/common/src/test/scala/org/apache/spark/sql/GlutenTestSetWithSystemPropertyTrait.scala
b/gluten-ut/common/src/test/scala/org/apache/spark/sql/GlutenTestSetWithSystemPropertyTrait.scala
new file mode 100644
index 0000000000..0b38f1c016
--- /dev/null
+++
b/gluten-ut/common/src/test/scala/org/apache/spark/sql/GlutenTestSetWithSystemPropertyTrait.scala
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql
+
+/**
+ * A test trait for suites whose parent creates per-test
SparkContext/SparkSession (via
+ * LocalSparkContext, LocalSparkSession, or direct `new SparkContext`).
+ *
+ * Injects GlutenPlugin config via system properties so per-test sessions
inherit it. SparkConf
+ * reads system properties prefixed with "spark." as defaults during
construction.
+ *
+ * Do NOT use GlutenTestsTrait or GlutenSQLTestsTrait for these suites -- they
create a shared
+ * SparkSession in beforeAll() that conflicts with per-test SparkContext
creation.
+ */
+trait GlutenTestSetWithSystemPropertyTrait extends GlutenTestsCommonTrait {
+
+ override def beforeAll(): Unit = {
+ System.setProperty("spark.plugins", "org.apache.gluten.GlutenPlugin")
+ System.setProperty("spark.memory.offHeap.enabled", "true")
+ System.setProperty("spark.memory.offHeap.size", "1024MB")
+ System.setProperty(
+ "spark.shuffle.manager",
+ "org.apache.spark.shuffle.sort.ColumnarShuffleManager")
+ super.beforeAll()
+ }
+
+ override def afterAll(): Unit = {
+ super.afterAll()
+ System.clearProperty("spark.plugins")
+ System.clearProperty("spark.memory.offHeap.enabled")
+ System.clearProperty("spark.memory.offHeap.size")
+ System.clearProperty("spark.shuffle.manager")
+ }
+}
diff --git a/gluten-ut/excluded-spark-uts.md b/gluten-ut/excluded-spark-uts.md
index 598eca6493..26527bcce1 100644
--- a/gluten-ut/excluded-spark-uts.md
+++ b/gluten-ut/excluded-spark-uts.md
@@ -3,4 +3,5 @@
| Suites
| Versions | Reason to exclude
[...]
|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[...]
-|
org.apache.spark.sql.SingleLevelAggregateHashMapSuite<br/>org.apache.spark.sql.TwoLevelAggregateHashMapSuite<br/>org.apache.spark.sql.TwoLevelAggregateHashMapWithVectorizedMapSuite
| 4.0, 4.1 | This UTs is similar to `DataFrameAggregateSuite`.<br/>The only
difference being that it contains variation for Spark codegen enabled and
disabled and enabling single level or two level aggregate hash-maps which is
also specific to Spark Aggregate implementation.<br/>We already run
`GlutenDataFra [...]
\ No newline at end of file
+|
org.apache.spark.sql.SingleLevelAggregateHashMapSuite<br/>org.apache.spark.sql.TwoLevelAggregateHashMapSuite<br/>org.apache.spark.sql.TwoLevelAggregateHashMapWithVectorizedMapSuite
| 4.0, 4.1 | This UTs is similar to `DataFrameAggregateSuite`.<br/>The only
difference being that it contains variation for Spark codegen enabled and
disabled and enabling single level or two level aggregate hash-maps which is
also specific to Spark Aggregate implementation.<br/>We already run
`GlutenDataFra [...]
+|
org.apache.spark.sql.sources.GlutenBucketedReadWithHiveSupportSuite<br/>org.apache.spark.sql.sources.GlutenBucketedWriteWithHiveSupportSuite<br/>org.apache.spark.sql.sources.GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite<br/>org.apache.spark.sql.sources.GlutenCommitFailureTestRelationSuite<br/>org.apache.spark.sql.sources.GlutenJsonHadoopFsRelationSuite<br/>org.apache.spark.sql.sources.GlutenParquetHadoopFsRelationSuite<br/>org.apache.spark.sql.sources.GlutenSimpleTextHadoopF
[...]
\ No newline at end of file
diff --git
a/gluten-ut/spark40/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
b/gluten-ut/spark40/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
index 80a33021d1..a70ec62871 100644
---
a/gluten-ut/spark40/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
+++
b/gluten-ut/spark40/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
@@ -680,7 +680,7 @@ class VeloxTestSettings extends BackendTestSettings {
// TODO: 4.x enableSuite[GlutenDataSourceScanExecRedactionSuite] // 2
failures
// TODO: 4.x enableSuite[GlutenDataSourceV2ScanExecRedactionSuite] // 2
failures
enableSuite[GlutenExecuteImmediateEndToEndSuite]
- // TODO: 4.x enableSuite[GlutenExternalAppendOnlyUnsafeRowArraySuite] // 14
failures
+ enableSuite[GlutenExternalAppendOnlyUnsafeRowArraySuite]
enableSuite[GlutenGlobalTempViewSuite]
enableSuite[GlutenGlobalTempViewTestSuite]
enableSuite[GlutenGroupedIteratorSuite]
@@ -696,10 +696,10 @@ class VeloxTestSettings extends BackendTestSettings {
// TODO: 4.x enableSuite[GlutenRemoveRedundantProjectsSuite] // 14 failures
// TODO: 4.x enableSuite[GlutenRemoveRedundantSortsSuite] // 1 failure
enableSuite[GlutenRowToColumnConverterSuite]
- // TODO: 4.x enableSuite[GlutenSQLExecutionSuite] // 1 failure
+ enableSuite[GlutenSQLExecutionSuite]
enableSuite[GlutenSQLFunctionSuite]
- // TODO: 4.x enableSuite[GlutenSQLJsonProtocolSuite] // 1 failure
- // TODO: 4.x enableSuite[GlutenShufflePartitionsUtilSuite] // 1 failure
+ enableSuite[GlutenSQLJsonProtocolSuite]
+ enableSuite[GlutenShufflePartitionsUtilSuite]
// TODO: 4.x enableSuite[GlutenSimpleSQLViewSuite] // 1 failure
// TODO: 4.x enableSuite[GlutenSparkPlanSuite] // 1 failure
enableSuite[GlutenSparkPlannerSuite]
@@ -707,7 +707,7 @@ class VeloxTestSettings extends BackendTestSettings {
enableSuite[GlutenSparkSqlParserSuite]
enableSuite[GlutenUnsafeFixedWidthAggregationMapSuite]
enableSuite[GlutenUnsafeKVExternalSorterSuite]
- // TODO: 4.x enableSuite[GlutenUnsafeRowSerializerSuite] // 1 failure
+ enableSuite[GlutenUnsafeRowSerializerSuite]
// TODO: 4.x enableSuite[GlutenWholeStageCodegenSparkSubmitSuite] // 1
failure
// TODO: 4.x enableSuite[GlutenWholeStageCodegenSuite] // 24 failures
enableSuite[GlutenBroadcastExchangeSuite]
@@ -790,14 +790,7 @@ class VeloxTestSettings extends BackendTestSettings {
enableSuite[GlutenSaveLoadSuite]
enableSuite[GlutenTableScanSuite]
// Generated suites for org.apache.spark.sql.sources
- // TODO: 4.x enableSuite[GlutenBucketedReadWithHiveSupportSuite] // 2
failures
- // TODO: 4.x enableSuite[GlutenBucketedWriteWithHiveSupportSuite] // 1
failure
- // TODO: 4.x enableSuite[GlutenCommitFailureTestRelationSuite] // 2 failures
enableSuite[GlutenDataSourceAnalysisSuite]
- // TODO: 4.x
enableSuite[GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite] // 2
failures
- // TODO: 4.x enableSuite[GlutenJsonHadoopFsRelationSuite] // 2 failures
- // TODO: 4.x enableSuite[GlutenParquetHadoopFsRelationSuite] // 2 failures
- // TODO: 4.x enableSuite[GlutenSimpleTextHadoopFsRelationSuite] // 2
failures
// Generated suites for org.apache.spark.sql
enableSuite[GlutenCacheManagerSuite]
enableSuite[GlutenDataFrameShowSuite]
@@ -823,6 +816,7 @@ class VeloxTestSettings extends BackendTestSettings {
// TODO: 4.x enableSuite[GlutenSetCommandSuite] // 1 failure
enableSuite[GlutenSparkSessionBuilderSuite]
enableSuite[GlutenSparkSessionJobTaggingAndCancellationSuite]
+ .exclude("Tags set from session are prefixed with session UUID")
enableSuite[GlutenTPCDSCollationQueryTestSuite]
enableSuite[GlutenTPCDSModifiedPlanStabilitySuite]
enableSuite[GlutenTPCDSModifiedPlanStabilityWithStatsSuite]
diff --git
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/GlutenSparkSessionJobTaggingAndCancellationSuite.scala
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/GlutenSparkSessionJobTaggingAndCancellationSuite.scala
index 8fbb53e24c..f761d6de36 100644
---
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/GlutenSparkSessionJobTaggingAndCancellationSuite.scala
+++
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/GlutenSparkSessionJobTaggingAndCancellationSuite.scala
@@ -16,6 +16,71 @@
*/
package org.apache.spark.sql
+import org.apache.spark.SparkContext
+import org.apache.spark.scheduler.{SparkListener, SparkListenerJobStart}
+import org.apache.spark.sql.execution.SQLExecution
+import org.apache.spark.util.ThreadUtils
+
+import java.util.concurrent.{Executors, Semaphore, TimeUnit}
+
+import scala.concurrent.{ExecutionContext, Future}
+import scala.concurrent.duration._
+
class GlutenSparkSessionJobTaggingAndCancellationSuite
extends SparkSessionJobTaggingAndCancellationSuite
- with GlutenTestsTrait {}
+ with GlutenTestSetWithSystemPropertyTrait {
+
+ // Override with testGluten because the original test uses
ExecutionContext.global whose
+ // ForkJoinPool reuses threads created before addTag("one").
InheritableThreadLocal (used by
+ // managedJobTags) only copies values at thread creation time, so reused
threads see an empty
+ // tag map. This causes withSessionTagsApplied to not attach the user tag to
the job, making
+ // cancelJobsWithTagWithFuture return empty. We fix this by creating a fresh
thread pool AFTER
+ // addTag so that new threads inherit the InheritableThreadLocal values.
+ testGluten("Tags set from session are prefixed with session UUID") {
+ sc = new SparkContext("local[2]", "test")
+ val session = classic.SparkSession.builder().sparkContext(sc).getOrCreate()
+ import session.implicits._
+
+ val sem = new Semaphore(0)
+ sc.addSparkListener(new SparkListener {
+ override def onJobStart(jobStart: SparkListenerJobStart): Unit = {
+ sem.release()
+ }
+ })
+
+ session.addTag("one")
+
+ // Use a fresh thread pool created AFTER addTag so that new threads inherit
+ // InheritableThreadLocal values (managedJobTags, threadUuid).
+ val threadPool = Executors.newSingleThreadExecutor()
+ implicit val ec: ExecutionContext =
ExecutionContext.fromExecutor(threadPool)
+ try {
+ Future {
+ session.range(1, 10000).map { i => Thread.sleep(100); i }.count()
+ }
+
+ assert(sem.tryAcquire(1, 1, TimeUnit.MINUTES))
+ val activeJobsFuture =
+ session.sparkContext.cancelJobsWithTagWithFuture(
+ session.managedJobTags.get()("one"),
+ "reason")
+ val activeJob = ThreadUtils.awaitResult(activeJobsFuture,
60.seconds).head
+ val actualTags = activeJob.properties
+ .getProperty(SparkContext.SPARK_JOB_TAGS)
+ .split(SparkContext.SPARK_JOB_TAGS_SEP)
+ assert(
+ actualTags.toSet == Set(
+ session.sessionJobTag,
+ s"${session.sessionJobTag}-thread-${session.threadUuid.get()}-one",
+ SQLExecution.executionIdJobTag(
+ session,
+ activeJob.properties
+ .get(SQLExecution.EXECUTION_ROOT_ID_KEY)
+ .asInstanceOf[String]
+ .toLong)
+ ))
+ } finally {
+ threadPool.shutdownNow()
+ }
+ }
+}
diff --git
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenExternalAppendOnlyUnsafeRowArraySuite.scala
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenExternalAppendOnlyUnsafeRowArraySuite.scala
index f947c4f406..4229733100 100644
---
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenExternalAppendOnlyUnsafeRowArraySuite.scala
+++
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenExternalAppendOnlyUnsafeRowArraySuite.scala
@@ -16,8 +16,8 @@
*/
package org.apache.spark.sql.execution
-import org.apache.spark.sql.GlutenTestsTrait
+import org.apache.spark.sql.GlutenTestSetWithSystemPropertyTrait
class GlutenExternalAppendOnlyUnsafeRowArraySuite
extends ExternalAppendOnlyUnsafeRowArraySuite
- with GlutenTestsTrait {}
+ with GlutenTestSetWithSystemPropertyTrait {}
diff --git
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenSQLExecutionSuite.scala
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenSQLExecutionSuite.scala
index 51c06b3b69..05b1d5fea1 100644
---
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenSQLExecutionSuite.scala
+++
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenSQLExecutionSuite.scala
@@ -16,6 +16,6 @@
*/
package org.apache.spark.sql.execution
-import org.apache.spark.sql.GlutenTestsTrait
+import org.apache.spark.sql.GlutenTestSetWithSystemPropertyTrait
-class GlutenSQLExecutionSuite extends SQLExecutionSuite with GlutenTestsTrait
{}
+class GlutenSQLExecutionSuite extends SQLExecutionSuite with
GlutenTestSetWithSystemPropertyTrait {}
diff --git
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenSQLJsonProtocolSuite.scala
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenSQLJsonProtocolSuite.scala
index 7ce28aae89..12af785acd 100644
---
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenSQLJsonProtocolSuite.scala
+++
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenSQLJsonProtocolSuite.scala
@@ -16,6 +16,8 @@
*/
package org.apache.spark.sql.execution
-import org.apache.spark.sql.GlutenTestsTrait
+import org.apache.spark.sql.GlutenTestSetWithSystemPropertyTrait
-class GlutenSQLJsonProtocolSuite extends SQLJsonProtocolSuite with
GlutenTestsTrait {}
+class GlutenSQLJsonProtocolSuite
+ extends SQLJsonProtocolSuite
+ with GlutenTestSetWithSystemPropertyTrait {}
diff --git
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenShufflePartitionsUtilSuite.scala
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenShufflePartitionsUtilSuite.scala
index 866d6cea54..0184af6774 100644
---
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenShufflePartitionsUtilSuite.scala
+++
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenShufflePartitionsUtilSuite.scala
@@ -16,6 +16,8 @@
*/
package org.apache.spark.sql.execution
-import org.apache.spark.sql.GlutenTestsTrait
+import org.apache.spark.sql.GlutenTestSetWithSystemPropertyTrait
-class GlutenShufflePartitionsUtilSuite extends ShufflePartitionsUtilSuite with
GlutenTestsTrait {}
+class GlutenShufflePartitionsUtilSuite
+ extends ShufflePartitionsUtilSuite
+ with GlutenTestSetWithSystemPropertyTrait {}
diff --git
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenUnsafeRowSerializerSuite.scala
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenUnsafeRowSerializerSuite.scala
index 39a52a35dd..0f76b5fc2a 100644
---
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenUnsafeRowSerializerSuite.scala
+++
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/GlutenUnsafeRowSerializerSuite.scala
@@ -16,6 +16,8 @@
*/
package org.apache.spark.sql.execution
-import org.apache.spark.sql.GlutenTestsTrait
+import org.apache.spark.sql.GlutenTestSetWithSystemPropertyTrait
-class GlutenUnsafeRowSerializerSuite extends UnsafeRowSerializerSuite with
GlutenTestsTrait {}
+class GlutenUnsafeRowSerializerSuite
+ extends UnsafeRowSerializerSuite
+ with GlutenTestSetWithSystemPropertyTrait {}
diff --git
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenBucketedReadWithHiveSupportSuite.scala
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenBucketedReadWithHiveSupportSuite.scala
deleted file mode 100644
index 115f933638..0000000000
---
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenBucketedReadWithHiveSupportSuite.scala
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.sources
-
-import org.apache.spark.sql.GlutenTestsCommonTrait
-
-class GlutenBucketedReadWithHiveSupportSuite
- extends BucketedReadWithHiveSupportSuite
- with GlutenTestsCommonTrait {}
diff --git
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenBucketedWriteWithHiveSupportSuite.scala
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenBucketedWriteWithHiveSupportSuite.scala
deleted file mode 100644
index 0ccbb1cf43..0000000000
---
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenBucketedWriteWithHiveSupportSuite.scala
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.sources
-
-import org.apache.spark.sql.GlutenTestsCommonTrait
-
-class GlutenBucketedWriteWithHiveSupportSuite
- extends BucketedWriteWithHiveSupportSuite
- with GlutenTestsCommonTrait {}
diff --git
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenCommitFailureTestRelationSuite.scala
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenCommitFailureTestRelationSuite.scala
deleted file mode 100644
index 0b7232772f..0000000000
---
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenCommitFailureTestRelationSuite.scala
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.sources
-
-import org.apache.spark.sql.GlutenTestsCommonTrait
-
-class GlutenCommitFailureTestRelationSuite
- extends CommitFailureTestRelationSuite
- with GlutenTestsCommonTrait {}
diff --git
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite.scala
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite.scala
deleted file mode 100644
index 3ce4d072bc..0000000000
---
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite.scala
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.sources
-
-import org.apache.spark.sql.GlutenTestsCommonTrait
-
-class GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite
- extends DisableUnnecessaryBucketedScanWithHiveSupportSuite
- with GlutenTestsCommonTrait {}
diff --git
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenJsonHadoopFsRelationSuite.scala
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenJsonHadoopFsRelationSuite.scala
deleted file mode 100644
index 4bc841c4ad..0000000000
---
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenJsonHadoopFsRelationSuite.scala
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.sources
-
-import org.apache.spark.sql.GlutenTestsCommonTrait
-
-class GlutenJsonHadoopFsRelationSuite
- extends JsonHadoopFsRelationSuite
- with GlutenTestsCommonTrait {}
diff --git
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenParquetHadoopFsRelationSuite.scala
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenParquetHadoopFsRelationSuite.scala
deleted file mode 100644
index 8aef43c990..0000000000
---
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenParquetHadoopFsRelationSuite.scala
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.sources
-
-import org.apache.spark.sql.GlutenTestsCommonTrait
-
-class GlutenParquetHadoopFsRelationSuite
- extends ParquetHadoopFsRelationSuite
- with GlutenTestsCommonTrait {}
diff --git
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenSimpleTextHadoopFsRelationSuite.scala
b/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenSimpleTextHadoopFsRelationSuite.scala
deleted file mode 100644
index 40a99eb003..0000000000
---
a/gluten-ut/spark40/src/test/scala/org/apache/spark/sql/sources/GlutenSimpleTextHadoopFsRelationSuite.scala
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.sources
-
-import org.apache.spark.sql.GlutenTestsCommonTrait
-
-class GlutenSimpleTextHadoopFsRelationSuite
- extends SimpleTextHadoopFsRelationSuite
- with GlutenTestsCommonTrait {}
diff --git
a/gluten-ut/spark41/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
b/gluten-ut/spark41/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
index be6e89bbd3..5a31f1d6f6 100644
---
a/gluten-ut/spark41/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
+++
b/gluten-ut/spark41/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
@@ -663,7 +663,7 @@ class VeloxTestSettings extends BackendTestSettings {
// TODO: 4.x enableSuite[GlutenDataSourceScanExecRedactionSuite] // 2
failures
// TODO: 4.x enableSuite[GlutenDataSourceV2ScanExecRedactionSuite] // 2
failures
enableSuite[GlutenExecuteImmediateEndToEndSuite]
- // TODO: 4.x enableSuite[GlutenExternalAppendOnlyUnsafeRowArraySuite] // 14
failures
+ enableSuite[GlutenExternalAppendOnlyUnsafeRowArraySuite]
enableSuite[GlutenGlobalTempViewSuite]
enableSuite[GlutenGlobalTempViewTestSuite]
enableSuite[GlutenGroupedIteratorSuite]
@@ -679,10 +679,10 @@ class VeloxTestSettings extends BackendTestSettings {
// TODO: 4.x enableSuite[GlutenRemoveRedundantProjectsSuite] // 14 failures
// TODO: 4.x enableSuite[GlutenRemoveRedundantSortsSuite] // 1 failure
enableSuite[GlutenRowToColumnConverterSuite]
- // TODO: 4.x enableSuite[GlutenSQLExecutionSuite] // 1 failure
+ enableSuite[GlutenSQLExecutionSuite]
enableSuite[GlutenSQLFunctionSuite]
- // TODO: 4.x enableSuite[GlutenSQLJsonProtocolSuite] // 1 failure
- // TODO: 4.x enableSuite[GlutenShufflePartitionsUtilSuite] // 1 failure
+ enableSuite[GlutenSQLJsonProtocolSuite]
+ enableSuite[GlutenShufflePartitionsUtilSuite]
// TODO: 4.x enableSuite[GlutenSimpleSQLViewSuite] // 2 failures
// TODO: 4.x enableSuite[GlutenSparkPlanSuite] // 1 failure
enableSuite[GlutenSparkPlannerSuite]
@@ -690,7 +690,7 @@ class VeloxTestSettings extends BackendTestSettings {
enableSuite[GlutenSparkSqlParserSuite]
enableSuite[GlutenUnsafeFixedWidthAggregationMapSuite]
enableSuite[GlutenUnsafeKVExternalSorterSuite]
- // TODO: 4.x enableSuite[GlutenUnsafeRowSerializerSuite] // 1 failure
+ enableSuite[GlutenUnsafeRowSerializerSuite]
// TODO: 4.x enableSuite[GlutenWholeStageCodegenSparkSubmitSuite] // 1
failure
// TODO: 4.x enableSuite[GlutenWholeStageCodegenSuite] // 24 failures
enableSuite[GlutenBroadcastExchangeSuite]
@@ -775,14 +775,7 @@ class VeloxTestSettings extends BackendTestSettings {
enableSuite[GlutenSaveLoadSuite]
enableSuite[GlutenTableScanSuite]
// Generated suites for org.apache.spark.sql.sources
- // TODO: 4.x enableSuite[GlutenBucketedReadWithHiveSupportSuite] // 2
failures
- // TODO: 4.x enableSuite[GlutenBucketedWriteWithHiveSupportSuite] // 2
failures
- // TODO: 4.x enableSuite[GlutenCommitFailureTestRelationSuite] // 2 failures
enableSuite[GlutenDataSourceAnalysisSuite]
- // TODO: 4.x
enableSuite[GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite] // 2
failures
- // TODO: 4.x enableSuite[GlutenJsonHadoopFsRelationSuite] // 2 failures
- // TODO: 4.x enableSuite[GlutenParquetHadoopFsRelationSuite] // 2 failures
- // TODO: 4.x enableSuite[GlutenSimpleTextHadoopFsRelationSuite] // 2
failures
// Generated suites for org.apache.spark.sql
enableSuite[GlutenCacheManagerSuite]
enableSuite[GlutenDataFrameShowSuite]
@@ -807,6 +800,7 @@ class VeloxTestSettings extends BackendTestSettings {
// TODO: 4.x enableSuite[GlutenSetCommandSuite] // 1 failure
enableSuite[GlutenSparkSessionBuilderSuite]
enableSuite[GlutenSparkSessionJobTaggingAndCancellationSuite]
+ .exclude("Tags set from session are prefixed with session UUID")
enableSuite[GlutenTPCDSCollationQueryTestSuite]
enableSuite[GlutenTPCDSModifiedPlanStabilitySuite]
enableSuite[GlutenTPCDSModifiedPlanStabilityWithStatsSuite]
diff --git
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/GlutenSparkSessionJobTaggingAndCancellationSuite.scala
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/GlutenSparkSessionJobTaggingAndCancellationSuite.scala
index 6befbbe213..f761d6de36 100644
---
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/GlutenSparkSessionJobTaggingAndCancellationSuite.scala
+++
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/GlutenSparkSessionJobTaggingAndCancellationSuite.scala
@@ -16,8 +16,71 @@
*/
package org.apache.spark.sql
-import org.apache.spark.sql.shim.GlutenTestsTrait
+import org.apache.spark.SparkContext
+import org.apache.spark.scheduler.{SparkListener, SparkListenerJobStart}
+import org.apache.spark.sql.execution.SQLExecution
+import org.apache.spark.util.ThreadUtils
+
+import java.util.concurrent.{Executors, Semaphore, TimeUnit}
+
+import scala.concurrent.{ExecutionContext, Future}
+import scala.concurrent.duration._
class GlutenSparkSessionJobTaggingAndCancellationSuite
extends SparkSessionJobTaggingAndCancellationSuite
- with GlutenTestsTrait {}
+ with GlutenTestSetWithSystemPropertyTrait {
+
+ // Override with testGluten because the original test uses
ExecutionContext.global whose
+ // ForkJoinPool reuses threads created before addTag("one").
InheritableThreadLocal (used by
+ // managedJobTags) only copies values at thread creation time, so reused
threads see an empty
+ // tag map. This causes withSessionTagsApplied to not attach the user tag to
the job, making
+ // cancelJobsWithTagWithFuture return empty. We fix this by creating a fresh
thread pool AFTER
+ // addTag so that new threads inherit the InheritableThreadLocal values.
+ testGluten("Tags set from session are prefixed with session UUID") {
+ sc = new SparkContext("local[2]", "test")
+ val session = classic.SparkSession.builder().sparkContext(sc).getOrCreate()
+ import session.implicits._
+
+ val sem = new Semaphore(0)
+ sc.addSparkListener(new SparkListener {
+ override def onJobStart(jobStart: SparkListenerJobStart): Unit = {
+ sem.release()
+ }
+ })
+
+ session.addTag("one")
+
+ // Use a fresh thread pool created AFTER addTag so that new threads inherit
+ // InheritableThreadLocal values (managedJobTags, threadUuid).
+ val threadPool = Executors.newSingleThreadExecutor()
+ implicit val ec: ExecutionContext =
ExecutionContext.fromExecutor(threadPool)
+ try {
+ Future {
+ session.range(1, 10000).map { i => Thread.sleep(100); i }.count()
+ }
+
+ assert(sem.tryAcquire(1, 1, TimeUnit.MINUTES))
+ val activeJobsFuture =
+ session.sparkContext.cancelJobsWithTagWithFuture(
+ session.managedJobTags.get()("one"),
+ "reason")
+ val activeJob = ThreadUtils.awaitResult(activeJobsFuture,
60.seconds).head
+ val actualTags = activeJob.properties
+ .getProperty(SparkContext.SPARK_JOB_TAGS)
+ .split(SparkContext.SPARK_JOB_TAGS_SEP)
+ assert(
+ actualTags.toSet == Set(
+ session.sessionJobTag,
+ s"${session.sessionJobTag}-thread-${session.threadUuid.get()}-one",
+ SQLExecution.executionIdJobTag(
+ session,
+ activeJob.properties
+ .get(SQLExecution.EXECUTION_ROOT_ID_KEY)
+ .asInstanceOf[String]
+ .toLong)
+ ))
+ } finally {
+ threadPool.shutdownNow()
+ }
+ }
+}
diff --git
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenExternalAppendOnlyUnsafeRowArraySuite.scala
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenExternalAppendOnlyUnsafeRowArraySuite.scala
index cd3aeb6438..4229733100 100644
---
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenExternalAppendOnlyUnsafeRowArraySuite.scala
+++
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenExternalAppendOnlyUnsafeRowArraySuite.scala
@@ -16,8 +16,8 @@
*/
package org.apache.spark.sql.execution
-import org.apache.spark.sql.shim.GlutenTestsTrait
+import org.apache.spark.sql.GlutenTestSetWithSystemPropertyTrait
class GlutenExternalAppendOnlyUnsafeRowArraySuite
extends ExternalAppendOnlyUnsafeRowArraySuite
- with GlutenTestsTrait {}
+ with GlutenTestSetWithSystemPropertyTrait {}
diff --git
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenSQLExecutionSuite.scala
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenSQLExecutionSuite.scala
index 27865af762..05b1d5fea1 100644
---
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenSQLExecutionSuite.scala
+++
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenSQLExecutionSuite.scala
@@ -16,6 +16,6 @@
*/
package org.apache.spark.sql.execution
-import org.apache.spark.sql.shim.GlutenTestsTrait
+import org.apache.spark.sql.GlutenTestSetWithSystemPropertyTrait
-class GlutenSQLExecutionSuite extends SQLExecutionSuite with GlutenTestsTrait
{}
+class GlutenSQLExecutionSuite extends SQLExecutionSuite with
GlutenTestSetWithSystemPropertyTrait {}
diff --git
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenSQLJsonProtocolSuite.scala
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenSQLJsonProtocolSuite.scala
index 2ca7b4380d..12af785acd 100644
---
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenSQLJsonProtocolSuite.scala
+++
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenSQLJsonProtocolSuite.scala
@@ -16,6 +16,8 @@
*/
package org.apache.spark.sql.execution
-import org.apache.spark.sql.shim.GlutenTestsTrait
+import org.apache.spark.sql.GlutenTestSetWithSystemPropertyTrait
-class GlutenSQLJsonProtocolSuite extends SQLJsonProtocolSuite with
GlutenTestsTrait {}
+class GlutenSQLJsonProtocolSuite
+ extends SQLJsonProtocolSuite
+ with GlutenTestSetWithSystemPropertyTrait {}
diff --git
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenShufflePartitionsUtilSuite.scala
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenShufflePartitionsUtilSuite.scala
index 5a6595bd9e..0184af6774 100644
---
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenShufflePartitionsUtilSuite.scala
+++
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenShufflePartitionsUtilSuite.scala
@@ -16,6 +16,8 @@
*/
package org.apache.spark.sql.execution
-import org.apache.spark.sql.shim.GlutenTestsTrait
+import org.apache.spark.sql.GlutenTestSetWithSystemPropertyTrait
-class GlutenShufflePartitionsUtilSuite extends ShufflePartitionsUtilSuite with
GlutenTestsTrait {}
+class GlutenShufflePartitionsUtilSuite
+ extends ShufflePartitionsUtilSuite
+ with GlutenTestSetWithSystemPropertyTrait {}
diff --git
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenUnsafeRowSerializerSuite.scala
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenUnsafeRowSerializerSuite.scala
index d81f8d58e7..0f76b5fc2a 100644
---
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenUnsafeRowSerializerSuite.scala
+++
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/GlutenUnsafeRowSerializerSuite.scala
@@ -16,6 +16,8 @@
*/
package org.apache.spark.sql.execution
-import org.apache.spark.sql.shim.GlutenTestsTrait
+import org.apache.spark.sql.GlutenTestSetWithSystemPropertyTrait
-class GlutenUnsafeRowSerializerSuite extends UnsafeRowSerializerSuite with
GlutenTestsTrait {}
+class GlutenUnsafeRowSerializerSuite
+ extends UnsafeRowSerializerSuite
+ with GlutenTestSetWithSystemPropertyTrait {}
diff --git
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenBucketedReadWithHiveSupportSuite.scala
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenBucketedReadWithHiveSupportSuite.scala
deleted file mode 100644
index 115f933638..0000000000
---
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenBucketedReadWithHiveSupportSuite.scala
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.sources
-
-import org.apache.spark.sql.GlutenTestsCommonTrait
-
-class GlutenBucketedReadWithHiveSupportSuite
- extends BucketedReadWithHiveSupportSuite
- with GlutenTestsCommonTrait {}
diff --git
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenBucketedWriteWithHiveSupportSuite.scala
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenBucketedWriteWithHiveSupportSuite.scala
deleted file mode 100644
index 0ccbb1cf43..0000000000
---
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenBucketedWriteWithHiveSupportSuite.scala
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.sources
-
-import org.apache.spark.sql.GlutenTestsCommonTrait
-
-class GlutenBucketedWriteWithHiveSupportSuite
- extends BucketedWriteWithHiveSupportSuite
- with GlutenTestsCommonTrait {}
diff --git
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenCommitFailureTestRelationSuite.scala
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenCommitFailureTestRelationSuite.scala
deleted file mode 100644
index 0b7232772f..0000000000
---
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenCommitFailureTestRelationSuite.scala
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.sources
-
-import org.apache.spark.sql.GlutenTestsCommonTrait
-
-class GlutenCommitFailureTestRelationSuite
- extends CommitFailureTestRelationSuite
- with GlutenTestsCommonTrait {}
diff --git
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite.scala
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite.scala
deleted file mode 100644
index 3ce4d072bc..0000000000
---
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite.scala
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.sources
-
-import org.apache.spark.sql.GlutenTestsCommonTrait
-
-class GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite
- extends DisableUnnecessaryBucketedScanWithHiveSupportSuite
- with GlutenTestsCommonTrait {}
diff --git
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenJsonHadoopFsRelationSuite.scala
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenJsonHadoopFsRelationSuite.scala
deleted file mode 100644
index 4bc841c4ad..0000000000
---
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenJsonHadoopFsRelationSuite.scala
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.sources
-
-import org.apache.spark.sql.GlutenTestsCommonTrait
-
-class GlutenJsonHadoopFsRelationSuite
- extends JsonHadoopFsRelationSuite
- with GlutenTestsCommonTrait {}
diff --git
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenParquetHadoopFsRelationSuite.scala
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenParquetHadoopFsRelationSuite.scala
deleted file mode 100644
index 8aef43c990..0000000000
---
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenParquetHadoopFsRelationSuite.scala
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.sources
-
-import org.apache.spark.sql.GlutenTestsCommonTrait
-
-class GlutenParquetHadoopFsRelationSuite
- extends ParquetHadoopFsRelationSuite
- with GlutenTestsCommonTrait {}
diff --git
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenSimpleTextHadoopFsRelationSuite.scala
b/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenSimpleTextHadoopFsRelationSuite.scala
deleted file mode 100644
index 40a99eb003..0000000000
---
a/gluten-ut/spark41/src/test/scala/org/apache/spark/sql/sources/GlutenSimpleTextHadoopFsRelationSuite.scala
+++ /dev/null
@@ -1,23 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.spark.sql.sources
-
-import org.apache.spark.sql.GlutenTestsCommonTrait
-
-class GlutenSimpleTextHadoopFsRelationSuite
- extends SimpleTextHadoopFsRelationSuite
- with GlutenTestsCommonTrait {}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
