morningman opened a new pull request, #70560:
URL: https://github.com/apache/doris/pull/70560
### Background
Since #61382, a table stream (`CREATE STREAM s ON TABLE t`) is listed with
the tables and views of its database in `SHOW FULL TABLES` and
`information_schema.tables`, with `ENGINE` `Stream`. Incremental refresh
(#62606) creates one for each base table of an incrementally refreshed
materialized view, named `__doris_ivm_stream_<mv id>_<hash>`.
`TableIf.TableType.toMysqlType` gives a stream the table type `VIEW`.
### What problem does this PR solve?
Issue Number: None
Related PR: #61382, #62606
PR Type: Enhancement
Problem Summary: Metadata clients take a `VIEW` row for a view: they read
its definition and parse it for lineage. A stream has no view definition; `SHOW
CREATE TABLE` prints `CREATE STREAM ... ON TABLE ...`. OpenMetadata, for one,
ingests every stream as a view whose definition is that statement, and the
streams incremental refresh creates appear as views nobody created. This PR
lists a stream as `BASE TABLE`: it is read with SELECT like a table, and MySQL
defines no other table type that fits. `ENGINE` stays `Stream`, so clients can
still tell streams apart. Table streams and incremental refresh are only on
master, so no released version changes its output.
### Details
- **Before and after:**
| | Before | After |
|---|---|---|
| `Table_type` of a stream in `SHOW FULL TABLES` | `VIEW` | `BASE TABLE` |
| `TABLE_TYPE`, `ENGINE` of a stream in `information_schema.tables` |
`VIEW`, `Stream` | `BASE TABLE`, `Stream` |
| the `__doris_ivm_stream_*` streams of an incrementally refreshed MV |
listed, `VIEW` | listed, `BASE TABLE` |
Views keep `VIEW` and system tables `SYSTEM VIEW`.
`information_schema.views`, `SHOW STREAMS`, `SHOW CREATE TABLE` and
`information_schema.table_streams` are unchanged: they select streams and views
by `TableType`, not by this string.
- **Configs and defaults:** None.
- **Impact on existing users:** None on released versions: table streams and
incremental refresh are on neither branch-4.1 nor branch-4.2. On master, `SHOW
FULL TABLES ... WHERE Table_type = 'VIEW'` and JDBC `getTables` with the type
`VIEW` no longer return streams, and the type `TABLE` now does. Nothing is
persisted: the type is computed whenever a listing is served.
### Testing
- Regression test (new): `query_p0/schema_table/test_stream_table_type`
creates a table, a stream on it and a view, and checks the stream and the view
in `SHOW FULL TABLES`, all three in `information_schema.tables`, and that
`information_schema.views` lists only the view. The `.out` is generated. On the
code before this PR it fails at its first check: `Expect cell is: BASE TABLE`,
`But real is: VIEW`.
- FE unit test (new): `TableTypeTest` checks that a stream maps to `BASE
TABLE` with the engine `Stream`, and that `VIEW`, `INLINE_VIEW`, `SCHEMA`,
`OLAP` and `MATERIALIZED_VIEW` keep their types. `./run-fe-ut.sh --run
org.apache.doris.catalog.TableTypeTest`: 2 tests, 0 failures.
- Manual: an incrementally refreshed materialized view over a merge-on-write
table with changelog; its `__doris_ivm_stream_*` stream is listed as `BASE
TABLE` / `Stream` in both listings, and as `VIEW` / `Stream` before this PR.
- Environment: a local cluster whose FE ran this branch's fe-core jar (and,
for the before runs, a jar of the same base without this change), with a BE
from an earlier master build (`6991e766329`); the change is FE-only.
### Notes for Reviewers
- Why `BASE TABLE` and not a new value such as `STREAM`: MySQL defines `BASE
TABLE`, `VIEW` and `SYSTEM VIEW`, and clients map and filter by them (MySQL
Connector/J returns `BASE TABLE` rows as `TABLE` from `getTables`), so a new
value would drop streams from client listings or break their type mapping.
- Why the streams incremental refresh creates stay listed: they are real
objects that `SHOW STREAMS` and `information_schema.table_streams` show and
that can be queried; hiding them by their name prefix in some listings would
make the listings disagree.
- The incremental-refresh case is checked by hand rather than in the suite:
it needs a merge-on-write base table with changelog, and a materialized view's
`ENGINE` in `information_schema.tables` is about to change (#70523), which the
`.out` would then depend on.
### Release note
None
### Check List (For Author)
- Behavior changed: Yes. Table streams are listed as `BASE TABLE` instead of
`VIEW`; table streams are not released yet.
- Does this need documentation: No. doris-website has no page on table
streams yet.
- Pre-submission status: Ready
### AI Disclosure
- Tool: Claude Code (Claude Opus 5.5)
- Generated: the code change, both tests and this description
- Human review: not stated
- Prompts: While integrating OpenMetadata with Doris, asked to fix in Doris
that table streams are listed as views; chose to list every stream as BASE
TABLE and not to hide the streams that incremental refresh creates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
