longvu-db opened a new pull request, #55573:
URL: https://github.com/apache/spark/pull/55573
### What changes were proposed in this pull request?
Add 6 tests in a new `DataSourceV2RepeatedSQLConnectSuite` that verify DSv2
tables reflect the latest state when accessed repeatedly via `sql()` in Spark
Connect mode (without CACHE TABLE). Each `sql("SELECT * FROM t")` call creates
a fresh plan that is re-analyzed on the server, so it always sees the most
recent data, schema, and table identity.
The tests use an in-process Connect server (`SparkConnectServerTest`) and
cover three scenarios, each with a session-write and an external-write variant:
- **Scenario 1 (external writes)**: After a writer adds rows (via session
SQL or catalog API), a subsequent `sql()` query sees the new data.
- **Scenario 2 (external schema changes)**: After a writer adds a column and
inserts data with the new schema (via session SQL or catalog API), a subsequent
`sql()` query reflects the updated schema.
- **Scenario 3 (drop/recreate)**: After a writer drops and recreates the
table (via session SQL or catalog API), a subsequent `sql()` query sees the
empty recreated table.
External writes use `getServerSession()` to access the server-side catalog
API (`loadTable` with write privileges, `alterTable`,
`dropTable`/`createTable`), matching the pattern used by the classic-mode tests
in #55462.
### Why are the changes needed?
These tests document and lock down the expected behavior: repeated `sql()`
access in Connect mode always sees the latest table state. This prevents
regressions if internal resolution or caching logic changes. This is the
Connect companion to the classic-mode tests in #55462.
### Does this PR introduce _any_ user-facing change?
No. This PR is test-only.
### How was this patch tested?
6 new tests in `DataSourceV2RepeatedSQLConnectSuite`, all passing:
```
build/sbt 'connect/testOnly *DataSourceV2RepeatedSQLConnectSuite'
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-opus-4-6)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
