FrankChen021 commented on code in PR #20183:
URL: https://github.com/apache/druid/pull/20183#discussion_r4361099644
##########
docs/querying/sql-metadata-tables.md:
##########
@@ -135,10 +135,61 @@ WHERE "IS_AGGREGATOR" = 'YES'
The "sys" schema provides visibility into Druid segments, servers and tasks.
:::info
- Note: "sys" tables do not currently support Druid-specific functions like
`TIME_PARSE` and
- `APPROX_QUANTILE_DS`. Only standard SQL functions can be used.
+ By default, "sys" tables use the SQL-layer execution path and support only
standard SQL functions. You can enable
Review Comment:
Reworded to "the Broker's built-in SQL interpreter".
##########
docs/querying/sql-metadata-tables.md:
##########
@@ -135,10 +135,61 @@ WHERE "IS_AGGREGATOR" = 'YES'
The "sys" schema provides visibility into Druid segments, servers and tasks.
:::info
- Note: "sys" tables do not currently support Druid-specific functions like
`TIME_PARSE` and
- `APPROX_QUANTILE_DS`. Only standard SQL functions can be used.
+ By default, "sys" tables use the SQL-layer execution path and support only
standard SQL functions. You can enable
+ experimental [native query execution](#native-query-execution) for supported
system tables, which enables expressions and
+ aggregations that the native SQL engine can translate.
+:::
+
+### Native query execution
+
+:::caution Experimental
+Native query execution for system tables is an experimental, opt-in feature.
Its behavior and interfaces may change.
:::
+The native SQL engine can plan supported system tables as native datasources.
To enable this behavior for a query, set
+`allowNativeFunctionsForSystemTables` to `true` in the SQL query context:
+
+```json
+{
+ "query": "SELECT COUNT(DISTINCT server) FROM sys.server_properties",
+ "context": {
+ "allowNativeFunctionsForSystemTables": true
+ }
+}
+```
+
+For clients that support setting query context parameters with SQL statements,
you can enable native system-table
+execution with `SET`:
+
+```sql
+SET allowNativeFunctionsForSystemTables = 'true';
+SELECT COUNT(DISTINCT server) FROM sys.server_properties;
+```
+
+Native system-table execution is available when the resolved SQL engine is
`native`. You don't need to explicitly set
+the `engine` context parameter when `native` is already the default engine.
The following tables support native query
+execution:
+
+|Table|Source of rows|
+|-----|--------------|
+|[`sys.server_properties`](#server_properties-table)|The Druid server
processes discovered in the cluster. Filters on `server` and `service_name` can
avoid reading properties from nodes that don't match.|
+
+After Druid retrieves the system-table rows, the native engine applies the
remaining filters, expressions,
+aggregations, sorting, and result processing. A system table that doesn't
advertise native query support continues to
+use its existing SQL-layer execution path, even when
`allowNativeFunctionsForSystemTables` is `true`.
+
+The parameter defaults to `false`. During a rolling upgrade, leave it disabled
until the Broker and all nodes
+that serve the native system tables have been upgraded. After the upgrade, you
can enable it by query or set
+`druid.query.default.context.allowNativeFunctionsForSystemTables=true` on
Brokers as the cluster-wide default. For more
+information, see [SQL query context](sql-query-context.md).
+
+Native system-table queries sent to the Router use distributed Broker
execution by default. To execute a native
Review Comment:
Removed.
##########
server/src/main/java/org/apache/druid/client/DirectDruidClient.java:
##########
@@ -785,7 +785,8 @@ private void checkTotalBytesLimit(long bytes)
.setHeader(
HttpHeaders.Names.CONTENT_TYPE,
isSmile ? SmileMediaTypes.APPLICATION_JACKSON_SMILE :
MediaType.APPLICATION_JSON
- ),
+ )
+ .setHeader(QueryResource.HEADER_NATIVE_QUERY_ROUTE,
QueryResource.NATIVE_QUERY_ROUTE_LOCAL),
Review Comment:
Now set only for `SystemTableDataSource` queries (POST and cancel DELETE).
##########
processing/src/main/java/org/apache/druid/query/SystemTableDataSource.java:
##########
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.query;
+
+import com.fasterxml.jackson.annotation.JsonCreator;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import com.google.common.base.Preconditions;
+
+import java.util.Collections;
+import java.util.Objects;
+import java.util.Set;
+
+/**
+ * Identifies a Druid system table whose rows are supplied by a service rather
than by segments.
+ */
+public class SystemTableDataSource extends LeafDataSource
Review Comment:
Overridden: returns itself, and throws if a policy is ever returned for a
system table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
