morningman opened a new pull request, #70522:
URL: https://github.com/apache/doris/pull/70522
### Background
`information_schema.table_options` describes each internal OLAP table: its
key model (`TABLE_MODEL`, `TABLE_MODEL_KEY`), distribution, bucket and
partition counts, and, since #56122, its partitioning (`PARTITION_METHOD`,
`PARTITION_EXPRESSION`, `PARTITION_KEY`, `RANGE`). #56122 made the FE emit one
row per partition, so a table with N partitions has N rows that repeat the
table-level columns, and an unpartitioned table, which always has its one
implicit partition, has one row. A RANGE or LIST partitioned table can have no
partition at all: an AUTO PARTITION table before its first load, or a table
created with `PARTITION BY RANGE(k) ()` before any `ADD PARTITION`.
### What problem does this PR solve?
Issue Number: None
Related PR: #56122 (introduced the bug)
PR Type: Bugfix
Problem Summary:
On master, a partitioned OLAP table with no partition has no row in
`information_schema.table_options`, although `information_schema.tables` lists
it. Clients that read the key model from this table lose such tables; the
OpenMetadata Doris connector, for example, builds primary keys from
`TABLE_MODEL` and `TABLE_MODEL_KEY` and finds none for an AUTO PARTITION table
that has not been loaded yet. This PR keeps one row per partition and adds one
table-level row, with `PARTITION_NUM` 0, for a table without partitions.
### Details
- **Frequency:** Always. `CREATE TABLE t (dt DATETIME NOT NULL, k INT NOT
NULL) UNIQUE KEY(dt, k) AUTO PARTITION BY RANGE (date_trunc(dt, 'day')) ()
DISTRIBUTED BY HASH(k) BUCKETS 1 PROPERTIES ("replication_num" = "1"); SELECT *
FROM information_schema.table_options WHERE TABLE_NAME = 't';` returns an empty
set until a load creates the first partition.
- **Root cause:** `MetadataGenerator.tableOptionsForInternalCatalog` builds
every row inside the loop over `OlapTable.getAllPartitions()` (#56122). For a
table with no partition the loop body never runs, so the table contributes no
row.
- **Why existing tests missed it:** Unit tests: no FE unit test read
`table_options`. Regression: `query_p0/system/test_table_options` (P0 and cloud
P0) has RANGE and LIST tables with three partitions each and unpartitioned
tables, which always have one partition; none of its tables has zero
partitions, so the `.out` that #56122 regenerated could not show a missing
table.
- **Fix:** when `getAllPartitions()` is empty, emit one row before the
per-partition loop. Its `PARTITION_EXPRESSION` is the partition columns, the
value the table's rows carry once partitions exist, rather than the string
`NULL` that marks an unpartitioned table; `PARTITION_KEY` and `RANGE` are
empty, as for unpartitioned tables.
- **Other paths and branches:** `information_schema.partitions` lists
partitions, so a table without partitions has no row there by design.
`table_properties`, `information_schema.tables`, `SHOW TABLE STATUS`, and the
external-catalog rows of `table_options` are one row per table: not affected.
Cloud mode runs the same code. branch-3.1, branch-4.0, branch-4.1, and
branch-4.2 lack #56122 and emit one row per table, so only master has the bug.
### Testing
- Regression `query_p0/system/test_table_options`: three tables without
partitions were added (AUTO RANGE, AUTO LIST, and `PARTITION BY RANGE(k1) ()`);
`select_check_1` and `select_check_2` list them, and the regenerated `.out`
adds only their six rows. On the unfixed FE the suite fails (`Check tag
'select_check_1' failed, line 2 ... Expect cell is:
auto_list_table_no_partition But real is: duplicate_table`); on the fixed FE it
passes. The privilege checks (`select_check_3` to `select_check_5`) are
unchanged.
- FE UT `TableOptionsMetadataGeneratorTest` checks the single row of an AUTO
RANGE table without partitions (`PARTITION_NUM` 0, `PARTITION_EXPRESSION` `dt`,
empty `PARTITION_KEY` and `RANGE`) and that a LIST table with two partitions
keeps one row per partition. With the fix: 2 tests, 0 failures; with
`MetadataGenerator` reverted to master, `testTableWithoutPartitionHasOneRow`
fails with `expected: <1> but was: <0>`.
- Both ran locally. The regression suite ran on a single-FE, single-BE
cluster: an FE built from this PR and a BE built from 6991e766329 (the change
is FE-only); the unfixed FE is the same jar with `MetadataGenerator` compiled
from master.
### Notes for Reviewers
Most of the `MetadataGenerator` diff moves the table-level columns,
`TABLE_CATALOG` through `PARTITION_METHOD`, into `newTableOptionsRow`, which
the new row and the per-partition rows both use. The moved lines are unchanged
except that the comment `key columTyp` now reads `TABLE_MODEL_KEY`; `git diff
--color-moved` shows the move.
### Release note
None
### Check List (For Author)
- Behavior changed: No
- Does this need documentation: No
- Pre-submission status: Ready
### AI Disclosure
- Tool: Claude Code (Claude Opus 5.5)
- Generated: code, tests, this description
- Human review: not stated
- Prompts: As part of the OpenMetadata integration work, asked that Doris
keep the per-partition rows of #56122 and only add a row for tables without
partitions, so the connector can read every table's key model; asked for full
local verification (an FE build, the regression suite and the FE unit test,
each before and after the fix).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
