yaooqinn commented on PR #12142:
URL: https://github.com/apache/gluten/pull/12142#issuecomment-4570003937
Following up on my earlier +1 — looked deeper after the ClickHouse CI
failures and I think the registration is at the wrong layer.
**CH CI is PR-tied (not infra flake).** 5 tests fail in
`GlutenBitmapExpressionsQuerySuite` (CH backend) with:
```
GlutenException: Unknown aggregate function bitmap_construct_agg
at AggregateFunctionParserFactory::get
(cpp-ch/local-engine/Parser/AggregateFunctionParser.cpp:255)
```
Root cause: `Sig[BitmapConstructAgg]` is registered in
`shims/spark35,40,41/SparkXShims.aggregateExpressionMappings`, which is
**all-backend** — so the CH planner also accepts `bitmap_construct_agg` and
pushes it down to CH native, where the parser isn't registered, hence the
failure.
**Sibling pattern — `BloomFilterAgg`:**
- `backends-velox/.../VeloxSparkPlanExecApi.scala:1120` —
`Sig[VeloxBloomFilterAggregate](ExpressionNames.BLOOM_FILTER_AGG)` is
registered in `extraExpressionMappings`, which is **Velox-only**.
`BLOOM_FILTER_AGG` does **not** appear in any
`SparkXShims.aggregateExpressionMappings`.
Suggest moving `Sig[BitmapConstructAgg]` from the three `SparkXShims` files
to `VeloxSparkPlanExecApi.extraExpressionMappings`. `BITMAP_CONSTRUCT_AGG`
constant in `shims/common/ExpressionNames.scala` can stay (it's just a string),
or move it with the Sig — either is fine.
That should keep CH on the vanilla fallback path and let Velox continue to
route natively.
(Sorry for the late catch — my +1 cited `bitmap_bit_position` registration
as a fallback guard, but that doesn't protect against the outer
`bitmap_construct_agg` Sig being visible to non-Velox backends. The
shim-vs-backend layering is the actual gate.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
