vvysotskyi opened a new pull request #1907: DRILL-7450: Improve performance for ANALYZE command URL: https://github.com/apache/drill/pull/1907 Jira - [DRILL-7450](https://issues.apache.org/jira/browse/DRILL-7450). This pull request contains several commits that may be reviewed one-by-one. The first commit "Improve performance for `ANALYZE` command" contains changes for two-phase metadata stream aggregation support. The main logic is placed in `MetadataAggPrule` - it converts the lowest `MetadataAggRel` into the following plan: ``` MetadataAggPrel(PHASE_2of2) \ HashToMergeExchangePrel \ MetadataAggPrel(PHASE_1of2) ``` Also, this commit introduces `ConvertMetadataAggregateToDirectScanRule` rule which replaces the lowest `MetadataAggRel` with `DrillDirectScanRel` populated from row groups metadata if required metadata is present: ``` MetadataAggRel(metadataLevel=ROW_GROUP) \ DrillScanRel ``` is converted into ``` DrillDirectScanRel ``` The last commit contains changes to use hash aggregation for `PHASE_1of2` for `ANALYZE` to reduce memory usage and avoid sorting non-aggregated data.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
