xiangfu0 opened a new pull request, #20015: URL: https://github.com/apache/pinot/pull/20015
`WindowFunnelTest.testFunnelCompleteCountGroupByQueries` (multi-stage) intermittently fails in Integration Test Set 1 lane-b with `expected [40] but found [36]`: 13 times in failed runs since 2026-08-15, three of them on master. ## Root cause The test table has ten segments, each holding four users, so 36 groups is exactly one missing segment. The response carries no exception. The broker routes a new segment as optional until it sees the segment ONLINE in the external view. The single-stage engine sends optional segments to the servers, which query them once loaded, so the single-stage `count(*)` in `waitForAllDocsLoaded` can reach the expected count while the broker still routes the last uploaded segment as optional. The multi-stage engine skips optional segments (`WorkerManager.assignWorkersToNonPartitionedLeafFragment`, `TODO: support optional segments for multi-stage engine`), so the first multi-stage query after `setUp` can miss it. Reproduced locally by recreating the table 40 times and polling both engines right after the uploads: in 26 setups the single-stage `count(*)` returned 160 while the multi-stage one returned 144 for about 10 ms, and the broker routing showed the last segment under `optionalSegments`. ## Fix `CustomDataQueryClusterIntegrationTest.setUp` now also waits until the broker routes no optional segment for the table (`/debug/routingTableWithOptionalSegments`). The engine gap itself (multi-stage queries skipping optional segments) is out of scope here; see #18223 and #19144. ## Testing - Scratch loop (not committed), 80 table setups, querying the multi-stage engine as soon as the single-stage `count(*)` reaches 160: without the new wait, 34 of 40 returned exactly 36 groups with no exception; with `waitForNoOptionalSegments` first, 0 of 40. - `CustomClusterSuite`: 859 tests, 0 failures (includes `WindowFunnelTest`, 38 tests). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
