felipepessoto opened a new issue, #13496: URL: https://github.com/apache/gluten/issues/13496
### Description #### Problem The Delta Spark UT workflow (`.github/workflows/delta_spark_ut.yml`), introduced in [#12388](https://github.com/apache/gluten/pull/12388), runs upstream Delta tests against a Gluten Velox bundle. The pipeline now supplies JVM-wide Gluten defaults through `java-test-args.sh`, including `spark.plugins=org.apache.gluten.GlutenPlugin` ([#13131](https://github.com/apache/gluten/pull/13131)). However, supplying defaults is not the same as verifying the actual test environment. A regression in JVM options, plugin registration, or suite/session setup could cause tests to run with vanilla Spark and pass without validating the intended Gluten-enabled environment. This is a missing safeguard, not a claim that the current tests are running without Gluten. The known-failures gate may detect such a change indirectly through now-passing tests, but it should not be the mechanism that proves Gluten is active. #### Coverage snapshot For the [successful October 28 run](https://github.com/apache/gluten/actions/runs/40756222035), matching the eight shards' executed-test manifests to transitive fixture inheritance in Delta 4.4.1 / Spark 4.1 gives: | Fixture reach | Suites | Executed tests | |---|---:|---:| | Inherit `DeltaSQLCommandTest` | 712 | 21,792 | | Do not inherit `DeltaSQLCommandTest` | 70 | 779 | | Total | 782 | 22,571 | This excludes 5,160 skipped tests. At least 518 tests across 43 of the suites outside `DeltaSQLCommandTest` use other Spark-backed fixtures or direct context/session creation. These include plain `SharedSparkSession`, Hive, Java/JUnit, and the newer session-test fixture. The remaining tests include helpers that may not require Spark. Therefore, adding an assertion only to `DeltaSQLCommandTest` would leave a meaningful coverage gap. These counts describe one run's fixture reach, not guaranteed runtime coverage or fixed thresholds for future runs. #### Proposed approach Start with CI-only fixture validation and a native-execution smoke test, without introducing a Java agent or changing production Gluten behavior: 1. Add runtime checks to `DeltaSQLCommandTest` and the identified alternative Spark fixture/session-creation paths. Validate the actual context's plugin configuration, installed Gluten SQL rules, and effective enablement, including the thread-local disable setting. Check the default state before test execution and its restoration after scoped overrides. 2. Execute a small, supported query in every forked JVM that runs Spark-backed tests. Verify both a Gluten/native execution plan and successful execution. Use the inherited Gluten configuration rather than explicitly re-enabling Gluten inside the smoke test. 3. Record validation evidence and enforce it independently of the known-failures comparison. Missing required evidence or a validation violation must fail the shard, including during baseline refresh and when a test otherwise matches a known-failure or flaky entry. 4. Report coverage and exceptions explicitly. Spark-free tests can be classified as not applicable. Intentional disabling is allowed, but a disabled setting must not automatically exempt a test from validation. #### Acceptance criteria - [ ] Removing Gluten from the inherited plugin configuration, preventing its SQL-rule installation, or disabling it by default causes CI to fail with an actionable diagnostic. - [ ] Missing required guard or per-JVM smoke-test evidence fails CI rather than silently reducing coverage. - [ ] Validation failures cannot be accepted by `known-failures.txt`, flaky quarantine, `update_baseline=true`, or `fail_on_fixed=false`. - [ ] Scoped intentional disabling, such as `withSQLConf("spark.gluten.enabled" -> "false")`, remains supported when the default is restored. Tests intentionally starting with Gluten disabled have explicit, documented exemptions. - [ ] Coverage includes the identified alternative Spark fixture paths; any remaining uncovered Spark-backed suites are reported explicitly. - [ ] Legitimate Spark fallback and tests that execute no native query remain valid. Individual test queries are not required to contain Gluten operators. - [ ] Positive and negative checks exercise the guard and independent gate, including missing instrumentation and intentional opt-outs. #### Scope and remaining limitation Fixture-level validation does not prove that every additional session created inside a test is correctly configured. For example, [the session-isolation test in `DeltaSuite.scala`](https://github.com/delta-io/delta/blob/v4.4.1/spark/src/test/scala/org/apache/spark/sql/delta/DeltaSuite.scala#L1852-L1862) creates another session with `newSession()`. The initial implementation should document this limitation and any uncovered session paths. Runtime-level validation of every additional session can be a follow-up; this issue should not be described as providing that stronger guarantee. Drafted with assistance from GitHub Copilot (GPT-6 Astra). ### Gluten version main branch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
