FrankChen021 commented on code in PR #20183:
URL: https://github.com/apache/druid/pull/20183#discussion_r3964277056
##########
docs/querying/sql-metadata-tables.md:
##########
@@ -135,10 +135,58 @@ WHERE "IS_AGGREGATOR" = 'YES'
The "sys" schema provides visibility into Druid segments, servers and tasks.
:::info
- Note: "sys" tables do not currently support Druid-specific functions like
`TIME_PARSE` and
- `APPROX_QUANTILE_DS`. Only standard SQL functions can be used.
+ By default, "sys" tables use the SQL-layer execution path and support only
standard SQL functions. You can enable
+ [native query execution](#native-query-execution) for supported system
tables, which enables expressions and
+ aggregations that the native SQL engine can translate.
:::
+### Native query execution
+
+The native SQL engine can plan supported system tables as native datasources.
To enable this behavior for a query, set
+`useNativeQueryForSystemTables` to `true` in the SQL query context:
+
+```json
+{
+ "query": "SELECT COUNT(DISTINCT task_id) FROM sys.tasks",
+ "context": {
+ "useNativeQueryForSystemTables": true
+ }
+}
+```
+
+For clients that support setting query context parameters with SQL statements,
you can enable native system-table
+execution with `SET`:
+
+```sql
+SET useNativeQueryForSystemTables = 'true';
+SELECT COUNT(DISTINCT datasource) FROM sys.tasks;
+```
+
+Native system-table execution is available when the resolved SQL engine is
`native`. You don't need to explicitly set
+the `engine` context parameter when `native` is already the default engine.
The following tables support native query
+execution:
+
+|Table|Source of rows|
+|-----|--------------|
+|[`sys.tasks`](#tasks-table)|The Overlord that owns task state. Supported
filters are pushed into task storage when the configured task storage
implementation supports filter pushdown.|
+|[`sys.server_properties`](#server_properties-table)|The Druid server
processes discovered in the cluster. Filters on `server` and `service_name` can
avoid reading properties from nodes that don't match.|
+
+After Druid retrieves the system-table rows, the native engine applies the
remaining filters, expressions,
+aggregations, sorting, and result processing. A system table that doesn't
advertise native query support continues to
+use its existing SQL-layer execution path, even when
`useNativeQueryForSystemTables` is `true`.
+
+Native system-table queries sent to the Router use distributed Broker
execution by default. To execute a native
+system-table Scan against only the contacted node, set the HTTP header
+`X-Druid-Native-Query-Route: local`. Local execution uses the authenticated
request identity and applies the table's
+authorization rules. The Broker uses the same header for remote node fan-out
requests. If the Broker itself is
+one of the selected nodes, it executes that node Scan in-process without an
HTTP request. The header
+controls routing and doesn't grant additional permissions.
Review Comment:
it's needed as the broker will send native query to router or itself to
retrieve data. for router, we need to tell whether it acts as a proxy or
execute the native query.
another way is by using query context, but I don't think the query context
is better way because this value here is just an internal flag, not user-facing
context parameter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
