Hi everyone,
I've made some updates to this FLIP, including: 1. Removed BITMAP_LONG_CARDINALITY. 2. Changed the result type of BITMAP_CARDINALITY from INT to BIGINT. 3. Added 4 cardinality agg functions to optimize performance by avoiding unnecessary bitmap clones. 4. Added a work item, "Add a rewrite rule to eliminate unnecessary bitmap clones for cardinality calculation". 5. Renamed the internal implementation class from RoaringBitmap32Data to RoaringBitmapData for consistency with the BITMAP type name. Please review the updated FLIP and share your feedback. Looking forward to your thoughts! -- Best regards, dylanhz At 2025-11-21 11:07:22, "dylanhz" <[email protected]> wrote: Hi everyone, I would like to start a discussion about FLIP-556 Introduce BITMAP Data Type[1]. Flink currently has no native, compressed data type for large integer sets, forcing users to rely on external libraries like RoaringBitmap via UDFs. This limits performance, maintainability, and integration with Flink’s type system and SQL engine. We propose adding a built‑in BITMAP type based on RoaringBitmap to provide compact storage, exact deduplication, and efficient set operations (AND, OR, XOR) directly within Flink. I have had some initial discussions with @Lincoln Lee and @Jark Wu regarding this FLIP. Looking forward to your feedback and suggestions. [1] https://docs.google.com/document/d/1YNgIt93iFboogHMoKbDD4LjP5UrfqtF65hitGKRtKMs/edit?usp=sharing -- Best regards, dylanhz
