raghav-reglobe opened a new pull request, #70835:
URL: https://github.com/apache/doris/pull/70835
### Background
Doris has column privileges: `GRANT SELECT_PRIV(c1, c2) ON db.t TO u` lets a
user read those columns and no others. The planner checks them in the
`CheckPrivileges` rewrite rule, which reuses column pruning to find the columns
a statement reads from each table and view, and hands them to
`UserAuthentication.checkPermission`. When a statement reads no column of an
internal table, that method asks for SELECT on the whole table (the rule of
#35465).
### What problem does this PR solve?
Issue Number: close #70831
Related PR: #35465 (introduced the table-level rule for a statement that
reads no column), #45478 (added the regression suite whose two expectations
change here)
PR Type: Feature
Problem Summary:
A user who was granted some columns of a table, and not the table, cannot
run `COUNT(*)`, `COUNT(1)` or `SELECT 1 FROM t` on it, although the same user
can fetch a granted column for every row. MySQL and PostgreSQL both let these
statements run once one column is granted; the issue has the comparison. BI
tools and JDBC clients send them on their own, so they fail for every
column-limited role. With this PR, under the built-in privilege model, a
statement that reads rows of a table and none of its columns runs for a user
who holds SELECT on at least one column the table has now. Most of the diff is
tests. After an upgrade, statements that were refused for a holder of a column
start to pass; nothing that passed is refused.
### Details
- **Change map:** 287 changed lines of production code in 5 files, 420 lines
of unit tests in 3 files, 22 lines in one regression suite.
- `CheckPrivileges` (163 lines): the rule. On a refusal, for a user who
holds a column of the table, it decides whether the statement reads any column
of that table. See Design.
- `UserAuthentication` (45 lines): `findSelectableColumn`, which names a
column the user holds. `checkPermission` decides as before; its only edit moves
a condition into a helper.
- `AccessControllerManager`, `Auth`, `Role` (79 lines): the new question,
"which of these columns does the user hold SELECT on by a grant on the column",
answered by the built-in model only.
- **Scope:** `COUNT(*)`, `COUNT(1)`, `SELECT 1 FROM t` and any other
statement that reads no visible column of a table or view, under the built-in
privilege model. Not in scope: catalogs governed by another authorization
source (Ranger, a custom access controller), which are asked exactly what they
were asked before; the HTTP row-count API, which keeps asking for SELECT on the
table.
- **Design:**
- The existing check is not changed. A bare internal table nothing is read
from is still asked for SELECT on the table. A table written with an alias, a
view and a subquery sit under a project; when nothing is read from that
project, column pruning keeps one column in it (the narrowest type) so the plan
still has a column to produce, and the check is still made on that column.
- When that check refuses, `CheckPrivileges` asks whether the user holds
SELECT on some visible column the table has now (`findSelectableColumn`). If
not, the refusal stands at once. A grant left on a dropped or renamed column
does not count. A sys table and the cluster snapshot tables never qualify,
because their check is not about their own grants.
- If the user holds a column, the plan is walked a second time. In this
walk a project that nothing reads asks its child only for the columns its own
expressions are computed from, where column pruning would keep one output. A
column the project merely passes through is not read; a column an expression is
computed from is, because the statement may still evaluate that expression and
an expression can fail. A table that then needs no visible column passes; a
table that needs one keeps its refusal.
- The second walk is used only for a table reached through project,
aggregate, filter, join, sort, top-n, limit and sink nodes (`isPlainNode`).
These are the nodes the change was written for and is tested under; each states
every column it reads from its children in its own expressions. Under any other
node the refusal stays as it was. An unlisted node costs a refusal, never a
grant.
- For the SQL cache, the table is recorded with the held column, so a
cache hit replays "does the user still hold that column".
- Statements that pass the existing check never take the second walk.
- **Usage:** nothing to set.
```sql
GRANT SELECT_PRIV(name) ON db.t TO u;
-- as u
SELECT COUNT(*) FROM db.t; -- now allowed
SELECT COUNT(*) FROM db.t a; -- now allowed
SELECT COUNT(*) FROM (SELECT id FROM db.t) q; -- now allowed: id is
never read
SELECT COUNT(id) FROM db.t; -- still denied
SELECT COUNT(*) FROM db.t WHERE id = 1; -- still denied
```
- **Default behavior:** on, with no switch. The change only allows
statements that were refused, for users who already hold a column of the table.
- **Compatibility:** no metadata, edit-log or protocol change. The
authorization plugin API is not touched: its recorded surface and the behaviour
baseline (`access-control-behavior-baseline.txt`) are unchanged. In a rolling
upgrade an old FE refuses these statements and a new one allows them; both
answers are safe.
- **Behavior change:** under the built-in privilege model, a holder of
SELECT on at least one column of a table or view may run statements that read
no column of it. Two expectations of `auth_p0/test_select_count_auth.groovy`
change from denied to allowed for that reason.
### Testing
FE unit tests and checkstyle ran on this head (one commit on master
d556f32bf8c9) in a build pod: 25 test classes, 317 tests, 0 failures, 0 errors;
checkstyle 0 violations. Besides the three classes below, the run covered
`AccessControllerManagerTest`, `AccessTranslationTest`,
`LegacyAccessControllerPluginTest`, `AccessControlBehaviorBaselineTest` (the
baseline file is unchanged), `AuthorizationPluginFromDirectoryTest`,
`AuthTest`, `ColumnPrivTest`, `GrantTablePrivilegeCommandTest`,
`ColumnPruningTest`, `SqlCacheTest`, `SqlCacheContextPluginTableTest`,
`OlapTableSqlCacheVersionTest`, `StatementContextTest`, `ViewTest`,
`CreateViewTest`, `AnalyzeCTETest`, `CheckRowPolicyTest`, `SetOperationTest`,
`PluginDrivenSysExternalTableTest`, `PluginDrivenSysTableTest`,
`DorisFlightSqlSchemaTest` and `InferPredicatesTest`.
- `SelectWithoutColumnsPrivilegeTest` (new, real `GRANT` statements on
internal tables and a view): the statements that now pass, for a direct grant
and for a grant held through a role; the statements that must stay refused (a
column read by `COUNT(col)`, a predicate, a join condition, `DISTINCT`, `GROUP
BY`, `ORDER BY`, or an expression computed in a subquery that nothing reads);
set operations and a CTE that is not inlined keep their refusal; a user with no
privilege, a user with LOAD only and a user with a column of another table are
refused; the check the SQL cache replays passes. The granted column is never
the narrowest one, so the column the planner keeps is one the user does not
hold.
- `AnyColumnPrivilegeTest` (new): the new question at the manager. A grant
on a column name the table no longer has, another table, another privilege, a
revoked grant, a holder of the table itself and a non-built-in source all get
no column.
- `TestCheckPrivileges` (extended): under a custom access controller a
holder of one column is still refused `COUNT(*)` when the column the planner
keeps is another one, as on master.
- The same planner tests were also run on unmodified master: all fifteen
statements of the "now pass" list are refused there, and the SQL cache test
fails with them; the other four tests of `SelectWithoutColumnsPrivilegeTest`
and the extended `TestCheckPrivileges` pass on master as they do here.
- `auth_p0/test_select_count_auth.groovy` is updated and was not run locally
(no cluster for regression suites here).
### Notes for Reviewers
- The new question is answered in
`AccessControllerManager.findColumnWithPriv` from the built-in model directly
and is not a method of `AuthorizationPlugin`, because that surface is frozen
and an addition is a major version change. The per-column question the API does
have was not used either: a source would be asked about every column of the
table, and one that audits would record a refusal per column the statement
never read. If you prefer a plugin question, I will rework it.
- `Auth.findColWithPriv` takes the read lock around the role walk.
- The plain-node list is short on purpose: it is what was tested. Adding a
node to it is a claim that the node states every column it reads in
`getExpressions()`.
- Start reading at `CheckPrivileges.checkColumnPrivileges`, then
`readsNoColumnOf`.
### Release note
Under the built-in privilege model, a user who holds SELECT on at least one
column of a table or view can run statements that read no column of it, such as
`SELECT COUNT(*)`, `SELECT COUNT(1)` and `SELECT 1 FROM t`. This matches MySQL
and PostgreSQL. Statements that read a column still need that column's
privilege.
### Check List (For Author)
- Behavior changed: Yes (see Details)
- Does this need documentation: Yes. A sentence in the column-privilege
section of the privilege guide in apache/doris-website; I will send it once the
rule is agreed.
- Pre-submission status: Partially verified: FE unit tests and checkstyle
ran in a build pod; the regression suite did not run.
### AI Disclosure
- Tool: Claude Code (Claude Fable 5.2)
- Generated: code, tests and this description
- Human review: the requirement and the design direction are mine, and I
read the comparison with MySQL and PostgreSQL and the test results. I did not
read the generated code line by line. A local review of this head with the
project's review flow (`doris-repo-review`) is running; I will add its result
here.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]