yashmayya opened a new pull request, #19766:
URL: https://github.com/apache/pinot/pull/19766
## The bug
With `explainAskingServers` (or
`pinot.query.multistage.explain.include.segment.plan`), the broker gets the
plan of each leaf stage from the servers. Then `PlanNodeToRelConverter`
converts the plan nodes back to Calcite nodes.
A leaf stage can run multi-stage operators above the single-stage plan. For
example, with `is_partitioned_by_group_by_keys`, both aggregates of
`SUM(AVG(x))` run in one leaf stage, and `AVG(x)` becomes `$SUM0(x) /
COUNT(x)`. EXPLAIN then shows:
```
PinotLogicalExchange(distribution=[hash[0]])
UnknownAggregate
UnknownProject
LeafStageCombineOperator(table=[mytable])
```
The broker logs `No operator found for function: DIVIDE` and `No agg
operator found for function: SUM0`.
`RexExpressionUtils` looks up the function names in `RexBuilder.getOpTab()`.
Calcite always returns `SqlStdOperatorTable` there, not `PinotOperatorTable`.
Also, plan nodes name most standard operators by `SqlKind` (see
`getFunctionName()`), for example `DIVIDE` for `/` and `SUM0` for `$SUM0`. So
the lookup fails for these operators, for comparisons, `AND` and `OR`, and for
all Pinot functions.
## The fix
- Look up the names in `PinotOperatorTable`. If no operator has the name,
use the operators whose `SqlKind` name is equal to it, for example `$SUM0` for
`SUM0` and `NOT IN` for `NOT_IN`.
- Create `IN` with `RelBuilder.in()`, which creates a `SEARCH`. Calcite
asserts that a `RexCall` is not `IN`.
- Remove the unused `RelOptCluster` parameter of `getAggFunction()`.
Only the EXPLAIN output changes. Query execution does not use this
conversion.
## Limits
`CAST` still shows the type of its second operand, for example `CAST($2,
'DOUBLE'):CHAR(6)`. This existed before. It was hidden when the node showed as
`Unknown`.
## Testing
- `RexExpressionUtilsTest.testSqlKindFunctionNames` converts `DIVIDE`, `IN`
and `NOT_IN` calls and back, and resolves `SUM0`. Without this change, it fails
with `No operator found for function: DIVIDE`.
- I ran 105 EXPLAIN queries of this type on a local cluster with the default
planner rules: nested aggregates with `is_partitioned_by_group_by_keys`, with
`CASE ... IN`, `FILTER (WHERE ... IN)`, `HAVING ... IN` and `HAVING ... NOT
IN`. Before this change, 97 of them showed `Unknown` nodes. After it, none did.
- All 2658 `pinot-query-planner` tests pass. `spotless`, `checkstyle` and
`license` are clean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
