iyils opened a new pull request, #68379:
URL: https://github.com/apache/doris/pull/68379
### What problem does this PR solve?
Issue Number: close #68309
Problem Summary:
BE serializes a bitmap holding at most 32 values as `BitmapTypeCode::SET`
(flag 5) whenever `enable_set_in_bitmap_value` is on, which has been the
default since #35730. The FE reader in `fe-common`'s
`BitmapValue.deserialize()` only knew the type codes 0 through 4, so any
BE-produced SET bitmap reaching an FE read path hit the default branch and
failed with `RuntimeException: unknown bitmap type 5`. Affected paths are the
ones that route BE bytes through `fe-common`, for example the Hive UDF binary
bitmap argument.
Reproduction: with `enable_set_in_bitmap_value=true`, `SELECT
bitmap_to_base64(bitmap_from_array([1, 3, 5]))` on BE returns
`BQMBAAAAAAAAAAMAAAAAAAAABQAAAAAAAAA=` (type 5, count 3, then little-endian 1,
3, 5). Feeding those bytes to `BitmapValue.deserialize()` throws instead of
yielding `{1, 3, 5}`.
Fix: add a `SET` case that reads the count as an unsigned byte, rejects a
count above `SET_TYPE_THRESHOLD` (32, matching BE's own reader and inline SET
capacity), and reads each value as a little-endian uint64. Instead of
introducing a distinct in-memory SET representation, the values are folded
through `add()`, which produces exactly the same set semantics in the existing
single/bitmap representation and leaves every downstream consumer of
`BitmapValue` unchanged.
Scope note: this fixes flag 5, the code BE emits on the default path. The FE
reader still does not know BE's `SET_V2` (10), `BITMAP32_V2` (12) or
`BITMAP64_V2` (13) codes, which require `bitmap_serialize_version` other than 1
on both sides; supporting those is deliberately out of scope here.
### Release note
Fixed the FE `BitmapValue` reader so it can deserialize SET-format bitmaps
(flag 5) produced by the BE, removing the `unknown bitmap type 5` failure on FE
paths that read BE-serialized bitmaps.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [ ] No.
- [x] Yes. The FE reader now accepts flag 5 (SET) instead of throwing
`RuntimeException`.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
