cloud-fan opened a new pull request, #55593: URL: https://github.com/apache/spark/pull/55593
### What changes were proposed in this pull request? Follow-up to [SPARK-52729](https://issues.apache.org/jira/browse/SPARK-52729) (which added `MetadataOnlyTable` and CREATE / ALTER VIEW … AS support for DS v2 catalogs). This PR closes out the *Remaining work* section of that PR's description and lands a few API/test cleanups along the way. **1. Renames (commit `[SQL] Rename v2 metadata-table API ...`).** Mechanical rename of the v2 view-API surface introduced by the parent PR: | Before | After | |---------------------|--------------------| | `MetadataOnlyTable` | `MetadataTable` | | `RelationCatalog` | `TableViewCatalog` | | `loadRelation()` | `loadTableOrView()` | | `DataSourceV2MetadataOnly{Table,View}Suite` | `DataSourceV2Metadata{Table,View}Suite` | The unrelated v2 helpers `CatalogV2Util.loadRelation` and `RelationResolution`'s private `loadRelation(V2TableReference)` predate the parent PR and are intentionally not renamed. **2. New view DDL execs on a non-session v2 `ViewCatalog`** (`AlterV2ViewExec.scala`): - `AlterV2ViewSetPropertiesExec`, `AlterV2ViewUnsetPropertiesExec` — merge / drop user TBLPROPERTIES on the existing view; dispatch to `ViewCatalog.replaceView`. - `AlterV2ViewSchemaBindingExec` — rewrites the schema-binding mode field; dispatch to `replaceView`. - `RenameV2ViewExec` — dispatches to a new `ViewCatalog.renameView(oldIdent, newIdent)` (added to `ViewCatalog`, contracted on `TableViewCatalog`). A shared `V2ViewMetadataMutation.builderFrom(existing)` helper seeds a `ViewInfo.Builder` from the existing view so that mutating execs override only the field they're changing — schema, queryText, captured resolution context, captured SQL configs, and queryColumnNames flow through unchanged. **3. New view inspection execs** (`V2ViewInspectionExecs.scala`): - `ShowCreateV2ViewExec` — reconstructs the `CREATE VIEW …` DDL from `ViewInfo` (column list, comment, collation, TBLPROPERTIES, schema-binding clause, view text). - `ShowV2ViewPropertiesExec`, `ShowV2ViewColumnsExec`, `DescribeV2ViewExec`, `DescribeV2ViewColumnExec` — produce output rows directly from `ViewInfo` fields. `DESCRIBE TABLE EXTENDED` emits a v2-native `# Detailed View Information` block (Catalog / Identifier / View Text / View Current Catalog / View Current Namespace / View Schema Mode / View Query Output Columns / Properties). **4. v1-parity for `SHOW TABLES` on a `TableViewCatalog`** (`ShowTablesExec.scala`): when the catalog is a `TableViewCatalog`, route through `listRelationSummaries` so views appear alongside tables. Pure `TableCatalog` catalogs continue to use `listTables` and return tables only. **5. Wiring + analysis-time payload (`DataSourceV2Strategy`, `Analyzer`, `ResolvedPersistentView`).** All `UNSUPPORTED_FEATURE.TABLE_OPERATION` pins from the parent PR for these commands are replaced with real strategy cases. To match the v2-table convention (where `ResolvedTable` carries the resolved `Table` and the exec reads from it), `ResolvedPersistentView` gains `viewInfo: Option[ViewInfo]`. The two v2 paths in `Analyzer.lookupTableOrView` (TableViewCatalog single-RPC + pure ViewCatalog fallback) populate it; v2 execs read from it directly without re-loading at runtime. The session-catalog v1 path leaves it `None` and continues to be rewritten to v1 commands by `ResolveSessionCatalog` before the strategy fires. **6. Tests — per-catalog command triplets.** Mirror the DROP TABLE layout from `sql/core/src/test/.../command/{,v1/,v2/}` for every new view command: - `command/<Cmd>SuiteBase.scala` — unified tests parameterized by `\$catalog` - `command/v1/<Cmd>Suite.scala` — extends Base + `ViewCommandSuiteBase` (pins `\$catalog` to `spark_catalog`) - `command/v2/<Cmd>Suite.scala` — extends Base + `ViewCommandSuiteBase` (pins `\$catalog` to a fresh `test_view_catalog` backed by a new general-purpose `InMemoryTableViewCatalog` test fixture), plus catalog-state assertions Triplets cover: SET TBLPROPERTIES, UNSET TBLPROPERTIES, RENAME TO, WITH SCHEMA BINDING, SHOW CREATE TABLE, SHOW TBLPROPERTIES, SHOW COLUMNS, DESCRIBE TABLE, DESCRIBE TABLE COLUMN, DROP VIEW. ### Why are the changes needed? The parent PR shipped CREATE VIEW + ALTER VIEW … AS through the v2 surface but pinned the rest of the view DDL/inspection family with `UNSUPPORTED_FEATURE.TABLE_OPERATION`. Third-party v2 catalogs that host views still couldn't run `ALTER VIEW SET TBLPROPERTIES`, `ALTER VIEW … RENAME TO`, `ALTER VIEW … WITH SCHEMA <mode>`, `DESCRIBE`, `SHOW CREATE TABLE`, `SHOW TBLPROPERTIES`, `SHOW COLUMNS` against their views — full v1 parity for non-session view catalogs requires this set. The rename to `MetadataTable` / `TableViewCatalog` / `loadTableOrView` was discussed during the parent PR's review as a clarity improvement; doing it now (before the API ships in a release) avoids deprecation churn later. Threading `ViewInfo` onto `ResolvedPersistentView` brings v2 view command flow in line with the existing v2 table command convention (resolved metadata attached at analysis time, not re-loaded at exec) — looking up database objects at runtime is the anti-pattern this PR removes. ### Does this PR introduce _any_ user-facing change? Yes for **connector developers**: - The rename is a source-incompatible change on the still-`@Evolving` v2 view API surface. Connectors implementing the parent PR's `RelationCatalog` / overriding `loadRelation` / referencing `MetadataOnlyTable` need to update to `TableViewCatalog` / `loadTableOrView` / `MetadataTable`. - `ViewCatalog` gains a new abstract `renameView(oldIdent, newIdent)` method. Existing `ViewCatalog` implementations need to add it (the parent PR has not yet released, so this is still pre-release breakage). Yes for **end users on a non-session v2 view catalog**: the listed DDL / inspection commands now succeed against a `ViewCatalog` instead of erroring with `UNSUPPORTED_FEATURE.TABLE_OPERATION`. `SHOW TABLES` on a `TableViewCatalog` now returns both tables and views, matching v1 SHOW TABLES output. No user-facing change on the session catalog path — those plans are still rewritten to the v1 commands by `ResolveSessionCatalog` and behave exactly as before. ### How was this patch tested? - 10 new per-catalog `*SuiteBase` triplets under `sql/core/src/test/.../command/{,v1/,v2/}` exercise each command against both a v1 (session) catalog and a fresh v2 `InMemoryTableViewCatalog` fixture. - The pre-existing `DataSourceV2MetadataViewSuite` was updated: the parent PR's `UNSUPPORTED_FEATURE.TABLE_OPERATION` pins for the implemented-here commands are replaced with positive smoke tests that exercise the new functional paths and verify the catalog-side state. `REFRESH TABLE` / `ANALYZE TABLE` / `ANALYZE TABLE … FOR COLUMNS` on a v2 view remain pinned (no v2 implementation here). - `SHOW TABLES on a TableViewCatalog` now asserts both tables and views appear in the result set. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude (Anthropic) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
