[
https://issues.apache.org/jira/browse/KYLIN-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dayue Gao updated KYLIN-2386:
-----------------------------
Description:
In KYLIN-2349 and KYLIN-2353, we changed the storage format of BitmapCounter
for better performance. In the new format, cardinality and serialized size are
recorded in the header part. This enables us to retrieve those information
without deserialize the whole data.
However, cardinality and serialized size can be quickly calculated just from
the header of [roaring
format|https://github.com/RoaringBitmap/RoaringFormatSpec/]. Performance tests
show that we could achieve the same performance gain without the format change.
The benefits are
* there is no need for user to rebuild existing cube to get better performance
* there is no need for developer to maintain two formats and deal with
compatibility issues
was:
In KYLIN-2349 and KYLIN-2353, we optimized performance of BitmapCounter by
changing its storage format. Cardinality and serialized size are recorded in
the header of the new format, enables us to retrieve those information without
deserialize the data.
In fact, cardinality and serialized size can be quickly calculated just from
the header of [roaring
format|https://github.com/RoaringBitmap/RoaringFormatSpec/]. Performance tests
show that we could achieve the same performance boost without the format change
of BitmapCounter. The benefits are
* there is no need for user to rebuild existing cube to get better performance
* there is no need for developer to maintain two formats and deal with
compatibility issues
> Revert KYLIN-2349 and KYLIN-2353
> --------------------------------
>
> Key: KYLIN-2386
> URL: https://issues.apache.org/jira/browse/KYLIN-2386
> Project: Kylin
> Issue Type: Task
> Components: Metadata
> Affects Versions: v2.0.0
> Reporter: Dayue Gao
> Assignee: Dayue Gao
>
> In KYLIN-2349 and KYLIN-2353, we changed the storage format of BitmapCounter
> for better performance. In the new format, cardinality and serialized size
> are recorded in the header part. This enables us to retrieve those
> information without deserialize the whole data.
> However, cardinality and serialized size can be quickly calculated just from
> the header of [roaring
> format|https://github.com/RoaringBitmap/RoaringFormatSpec/]. Performance
> tests show that we could achieve the same performance gain without the format
> change. The benefits are
> * there is no need for user to rebuild existing cube to get better performance
> * there is no need for developer to maintain two formats and deal with
> compatibility issues
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
