zhengruifeng opened a new pull request, #55870:
URL: https://github.com/apache/spark/pull/55870
### What changes were proposed in this pull request?
Fix the base class of `ArrowInterfaceTests` in
`python/pyspark/pandas/tests/test_arrow_interface.py` so the test is actually
discovered by `unittest`.
The class was declared as:
```python
class ArrowInterfaceTests(ArrowInterfaceTestsMixin, SQLTestUtils):
pass
```
`SQLTestUtils` is a plain mixin that does not extend `unittest.TestCase`, so
this class itself is not a `TestCase` subclass and `unittest`'s discovery
(`TestLoader.loadTestsFromModule`) silently skips it.
Verified with `loader.loadTestsFromModule(...).countTestCases()`:
| | tests collected | `issubclass(TestCase)` |
| --- | --- | --- |
| Before fix | 0 | `False` |
| After fix | 3 | `True` |
The sibling Spark Connect parity test `ArrowInterfaceParityTests` in
`tests/connect/test_parity_arrow_interface.py` extends `ReusedConnectTestCase`
(which does chain to `TestCase`) and so was unaffected, masking the issue.
Fix: switch the base to `PandasOnSparkTestCase`, matching the other
`pyspark.pandas.tests.test_*` classes (e.g. `NamespaceTests`, `UtilsTests`,
`InternalFrameTests`, `SQLTests`). `PandasOnSparkTestCase` chains through
`ReusedSQLTestCase -> ReusedPySparkTestCase -> PySparkBaseTestCase ->
unittest.TestCase`.
### Why are the changes needed?
`test_spark_arrow_c_streamer_arrow_consumer` (added in SPARK-55124) has not
actually been running in CI since it was introduced; it is a latent gap in
coverage of the pandas-on-Spark Arrow C-stream interface.
### Does this PR introduce _any_ user-facing change?
No. Test-only change.
### How was this patch tested?
- Discovery check (above): `unittest.TestLoader.loadTestsFromModule` now
collects the test.
- Ran the test locally:
```
$ python -m unittest -v pyspark.pandas.tests.test_arrow_interface
test_assert_classic_mode (...ArrowInterfaceTests) ... ok
test_spark_arrow_c_streamer_arrow_consumer (...ArrowInterfaceTests) ... ok
test_assert_classic_mode (...PandasOnSparkTestCase) ... ok
----------------------------------------------------------------------
Ran 3 tests in 10.683s
OK
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude opus-4-7
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
