This is an automated email from the ASF dual-hosted git repository.
szehon-ho pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 274f6ca38411 [SPARK-57516][DOCS] Update Data Source V2 docs for
ViewCatalog and TableViewCatalog
274f6ca38411 is described below
commit 274f6ca3841105a3bc0db712b421ff3f88be910b
Author: Szehon Ho <[email protected]>
AuthorDate: Mon Jun 22 14:40:58 2026 -0700
[SPARK-57516][DOCS] Update Data Source V2 docs for ViewCatalog and
TableViewCatalog
### What changes were proposed in this pull request?
Update the Data Source V2 documentation (`docs/sql-data-sources-v2.md`) to
reflect the current state of DSV2 view support:
- Remove the stale "work in progress — not yet integrated into query
resolution" note on `ViewCatalog`, since view DDL
(`CREATE`/`REPLACE`/`ALTER`/`DROP`/`SHOW VIEWS`) and reading from a view
(`SELECT`) are now resolved and executed against the catalog.
- Correct the `ViewCatalog` method table to match the actual API
(`createView(ident, info)`, `replaceView`, `createOrReplaceView`, `viewExists`,
`invalidateView`); remove the nonexistent `alterView(ident, changes...)`.
- Describe `ViewInfo` as the carrier of a view's metadata.
- Add a `TableViewCatalog` section for connectors that expose both tables
and views in a single shared namespace, including `loadTableOrView` and
`listTableAndViewSummaries`.
- Note that the built-in session catalog (`V2SessionCatalog`) is not a
`ViewCatalog`; DSV2 view support applies to custom catalogs registered via
`spark.sql.catalog.<name>`.
### Why are the changes needed?
The documentation was out of date with the implementation. `ViewCatalog` is
now wired into analysis, DDL execution, and query reads (since 4.2.0), and
`TableViewCatalog` was added, but the docs still described the feature as
incomplete and listed incorrect method signatures.
### Does this PR introduce _any_ user-facing change?
No (documentation-only).
### How was this patch tested?
Documentation-only change. Verified the documented method signatures
against `ViewCatalog.java` and `TableViewCatalog.java`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Cursor (Claude Opus 4.8)
Closes #56580 from szehon-ho/docs-dsv2-viewcatalog.
Authored-by: Szehon Ho <[email protected]>
Signed-off-by: Szehon Ho <[email protected]>
(cherry picked from commit 857e5c6929b0189e7888d404ef4aee994095e187)
Signed-off-by: Szehon Ho <[email protected]>
---
docs/sql-data-sources-v2.md | 52 +++++++++++++++++++++++++++++++++++++--------
1 file changed, 43 insertions(+), 9 deletions(-)
diff --git a/docs/sql-data-sources-v2.md b/docs/sql-data-sources-v2.md
index 1a8e75f34efc..4b4d0f35cb4e 100644
--- a/docs/sql-data-sources-v2.md
+++ b/docs/sql-data-sources-v2.md
@@ -94,7 +94,7 @@ A catalog adds capabilities by mixing in additional
| `TableCatalog` | List, load, create, alter, and drop tables |
| `StagingTableCatalog` | Atomic create-table-as-select /
replace-table-as-select |
| `SupportsNamespaces` | Create, alter, drop, and list namespaces |
-| `ViewCatalog` | List, load, create, alter, and drop views *(work in progress
— not yet integrated into query resolution)* |
+| `ViewCatalog` | List, load, create, replace, rename, and drop views |
| `FunctionCatalog` | List and load functions |
| `ProcedureCatalog` | Load and list stored procedures |
@@ -141,22 +141,31 @@ adds namespace (database/schema) management:
### ViewCatalog
-> **Note:** `ViewCatalog` is a work in progress. The interface is defined but
is not yet
-> integrated into Spark's query resolution or planning.
-
[`ViewCatalog`](api/java/org/apache/spark/sql/connector/catalog/ViewCatalog.html)
extends
-`CatalogPlugin` and provides methods for view lifecycle management:
+`CatalogPlugin` and provides methods for view lifecycle management. View DDL
+(`CREATE`/`REPLACE`/`ALTER`/`DROP`/`SHOW VIEWS`) and reading from a view
(`SELECT`) are resolved
+and executed against the catalog. Connectors that expose both
+tables and views in a single shared namespace implement
+[`TableViewCatalog`](api/java/org/apache/spark/sql/connector/catalog/TableViewCatalog.html),
+which extends both `TableCatalog` and `ViewCatalog`. A view's metadata is
represented by
+[`ViewInfo`](#view).
| Method | Description |
|--------|-------------|
| `listViews(namespace)` | List views in a namespace |
-| `loadView(ident)` | Load a view by identifier |
-| `createView(viewInfo)` | Create a new view |
-| `replaceView(viewInfo, orCreate)` | Replace (or create) a view |
-| `alterView(ident, changes...)` | Alter a view's properties or schema |
+| `loadView(ident)` | Load a view's `ViewInfo` by identifier |
+| `viewExists(ident)` | Test whether a view exists (has a default
implementation) |
+| `invalidateView(ident)` | Invalidate cached metadata for a view (has a
default implementation) |
+| `createView(ident, info)` | Create a new view |
+| `replaceView(ident, info)` | Atomically replace an existing view (`ALTER
VIEW ... AS`) |
+| `createOrReplaceView(ident, info)` | Create a view, or atomically replace it
if it exists (`CREATE OR REPLACE VIEW`; has a default implementation) |
| `dropView(ident)` | Drop a view |
| `renameView(oldIdent, newIdent)` | Rename a view |
+> **Note:** The built-in session catalog (`V2SessionCatalog`) is not a
`ViewCatalog`; views in
+> the session catalog continue to use the v1 (Hive-compatible) code path. DSV2
view support
+> applies to custom catalogs registered via `spark.sql.catalog.<name>`.
+
### FunctionCatalog
[`FunctionCatalog`](api/java/org/apache/spark/sql/connector/catalog/FunctionCatalog.html)
@@ -217,6 +226,31 @@ Additional mix-ins enable further capabilities:
| `SupportsPartitionManagement` | Partition DDL (`ADD`/`DROP`/`RENAME
PARTITION`) |
| `SupportsMetadataColumns` | Expose hidden metadata columns (e.g. file
name, row position) |
+## View
+
+[`ViewInfo`](api/java/org/apache/spark/sql/connector/catalog/ViewInfo.html) is
the metadata
+abstraction for a view — a named SQL query managed through a
[`ViewCatalog`](#viewcatalog).
+Unlike a [`Table`](#table), a view has no data of its own: it stores a SQL
query along with the
+context needed to resolve and expand that query consistently whenever the view
is referenced.
+
+A `ViewInfo` provides:
+
+| Method | Description |
+|--------|-------------|
+| `queryText()` | The SQL text of the view |
+| `columns()` | The view's output columns |
+| `currentCatalog()` | The current catalog captured at creation time, used to
resolve unqualified identifiers in the query text |
+| `currentNamespace()` | The current namespace captured at creation time, used
alongside `currentCatalog()` |
+| `sqlConfigs()` | The SQL configs captured at creation time, applied when
parsing and analyzing the view body |
+| `schemaMode()` | The view's [schema binding
mode](sql-ref-syntax-ddl-create-view.html) (`BINDING`, `COMPENSATION`, `TYPE
EVOLUTION`, or `EVOLUTION`) |
+| `queryColumnNames()` | Output column names of the query, used to map the
query output to the view's declared columns |
+| `viewDependencies()` | The structured list of objects (source tables and
functions) the view depends on |
+| `properties()` | A string map of view properties |
+
+When a view is read (`SELECT`), Spark resolves and expands `queryText()` using
the captured
+catalog, namespace, and SQL configs, so the view behaves consistently
regardless of the session
+state at query time.
+
## Read Path
The read path follows a builder pattern that separates logical planning from
physical execution:
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
