FrankChen021 opened a new pull request, #20307: URL: https://github.com/apache/druid/pull/20307
## 1. Outcome - `NativeCompactionTaskRunTest`: 22m10s -> 9m51s, saving 12m19s (55.6%), with all 760 active cases preserved. - Native + MSQ compaction tests: 24m27s -> 12m03s, saving 12m24s (50.7%), with active coverage unchanged. - Slowest main unit-test shard: 33m49s -> 27m25s, saving 6m24s (18.9%). - Shard spread: 11m49s -> 5m40s, a 52% reduction. Measured in [master CI](https://github.com/apache/druid/actions/runs/34300307586) and [the updated CI](https://github.com/FrankChen021/druid/actions/runs/34356971689), using the test `Execute` step without queue or setup time. ## 2. Description Class-level parameterization created the full configuration/test Cartesian product before applicability checks. Native discovered 2,016 cases and skipped 1,256 during execution; MSQ discovered 208 and skipped 60. This paid fixture/setup cost for cases that never ran. The existing shard layout also assumed `N*` was one of the heaviest groups. Optimizing Native compaction removed most of that weight and left the original layout significantly imbalanced. ## 3. Change - Select applicable compaction configurations before creating each test invocation. - Keep 760 Native and 148 MSQ active cases unchanged. - Use reusable per-test selections for Native and MSQ scenarios. - Rebalance the main unit tests into nine letter buckets. - Exclude separately sharded `QTest` reports from the main-shard balancing input. ## 4. Why Resharding With the compaction optimization but the original shard layout, `N*` fell from 33m49s to 20m30s, while the slowest shard still took 34m22s. The resulting 13m52s spread meant the saved compaction time did not reduce the CI critical path. The final layout reduces the slowest shard to 27m25s and the spread to 5m40s. The intermediate result with the original layout is available in [this workflow](https://github.com/FrankChen021/druid/actions/runs/34224378443). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
