FrankChen021 opened a new pull request, #20579:
URL: https://github.com/apache/druid/pull/20579
### Description
Follow-up to #20555. Migrates the remaining query context keys to typed
`QueryContextParameter` descriptors. Task-context keys (`Tasks`, compaction,
lookup and broadcast loading) are out of scope.
#### Docs generator
- Rows are generated wherever a managed doc has a marker, so a parameter can
appear in several tables (MSQ and Dart share some).
- Columns follow the header of the enclosing table.
`multi-stage-query/reference.md` rows get the statement-type prefix.
- New managed docs: `groupbyquery.md`, `searchquery.md`, `dart.md`,
`multi-stage-query/reference.md`.
- Every `PUBLIC` descriptor needs a marker, and every marker must name a
`PUBLIC` descriptor.
#### Migrated keys
- **MSQ and Dart (44):** `MultiStageQueryContext`, `DartControllerContext`,
`ClientMSQContext`. Keys whose values are MSQ types are string descriptors,
converted where read.
- **GroupBy (29):** `GroupByQueryConfig`, `GroupByQuery`, `GroupingEngine`,
`GroupByMergingQueryRunner`, moving-average `sortByDimsFirst`;
`timestampResultField` is shared with timeseries. Adds `floatParameter` and
`humanReadableBytesParameter`.
- **Others (18):** scan, search, timeseries `grandTotal`, topN, window
`unravel`, `serialization`, `postProcessing`, `skipIncrementalSegment`, SQL DML
granularity keys, remaining `PlannerConfig`/validator/reverse-lookup keys,
`approxQuantileDsMaxStreamLength`, and `segmentAcquireMode` (added in #20562).
- **Shared with task context (4):** `maxParseExceptions`,
`storeCompactionState`, `taskLockType`, `useConcurrentLocks` get query context
descriptors, separate from the task context keys in `Tasks`. MSQ reads them
from the query context; `MSQControllerTask` still lets the query context take
precedence over the task context when choosing the lock type.
- **System (7):** `queryFailTime`, `maxTimeArrayOutputName`,
`minTimeArrayOutputName`, `__exportFileFormat`, `__user`, `__resultFormat`,
`__timeColumn`.
After this PR, every query context key is declared in
`QueryContextParameters`; task context keys are out of scope.
Documented keys are `PUBLIC`, undocumented user-settable keys `HIDDEN`, and
keys only Druid sets `SYSTEM`. This PR renames the `Visibility.INTERNAL` level
added in #20555 to `SYSTEM`, which says more plainly that Druid owns these
values; `Visibility` has not been released yet. `taskLockType`,
`useConcurrentLocks`, `maxNumSegments`, `maxRowsMaterializedInWindow`,
`grandTotal` and `resultAsArray` were already described in the docs as
user-settable, so they are `PUBLIC` and get generated rows in the MSQ
reference, timeseries and groupBy tables. The public key constants they replace
are kept as `@Deprecated(since = "39.0.0", forRemoval = true)` aliases.
#### Behavior changes
- **Fix:** the SQL validator wrote a catalog table's `targetSegmentRows` to
`msqRowsPerSegment`, which nothing reads, so it had no effect. It now sets
MSQ's `rowsPerSegment`. The same fix is proposed separately for master in
apache/druid#20561.
- Integer and long values use exact-number parsing (e.g. `12.5` is rejected
instead of truncated), matching earlier migrations.
- `maxNumTasks` now declares its documented minimum of 2 as a constraint, so
a smaller value fails with `BadQueryContextException` ("must be within the
range [2, 2147483647]") wherever it is read. This replaces the hand-written
checks in `MSQTaskQueryMaker` and `ClientCompactionRunnerInfo`. Compaction
config validation still returns a validation failure rather than throwing, and
compaction slot accounting falls back to the default for a running task with an
invalid value, since that task fails validation when it starts.
- Some invalid values now fail with `BadQueryContextException` and the
common message format (`maxSegmentPartitionsOrderedInMemory`,
`deferExpressionDimensions`).
- An explicit `null` counts as unset where code used `containsKey`.
- **Docs:** the `maxNumTasks` description no longer says it "may also be
provided as `numTasks`". That alias has been documented since the MSQ docs were
added (apache/druid#12983) but was never supported in code.
- Doc fixes from regeneration: `arrayIngestMode` default is `array`,
`applyLimitPushDownToSegment` default comes from config, and the
`maxSelectorDictionarySize` description names the right property.
This PR and the SQL query context migration (replacing #20556) both touch
`QueryContextParameters` and `QueryContexts`; whichever merges second will be
rebased.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added documentation for new or modified features or behaviors.
- [x] added unit tests or modified existing tests to cover new code paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
