This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/paimon.git
The following commit(s) were added to refs/heads/master by this push:
new 2c2a0a8d17 [core] Make the default version of bitmap index to 2 (#5405)
2c2a0a8d17 is described below
commit 2c2a0a8d1798175fb53f73d89c09e605a96161b9
Author: Jingsong Lee <[email protected]>
AuthorDate: Mon Apr 7 10:00:30 2025 +0800
[core] Make the default version of bitmap index to 2 (#5405)
---
docs/content/concepts/spec/fileindex.md | 124 +++------------------
.../paimon/fileindex/bitmap/BitmapFileIndex.java | 4 +-
2 files changed, 19 insertions(+), 109 deletions(-)
diff --git a/docs/content/concepts/spec/fileindex.md
b/docs/content/concepts/spec/fileindex.md
index 117d90b206..2dcd7dbfb7 100644
--- a/docs/content/concepts/spec/fileindex.md
+++ b/docs/content/concepts/spec/fileindex.md
@@ -97,9 +97,11 @@ This class use (64-bits) long hash. Store the num hash
function (one integer) an
## Index: Bitmap
* `file-index.bitmap.columns`: specify the columns that need bitmap index.
-* `file-index.bitmap.version`: specify the bitmap index format version,
default version is 1, latest version is 2.
* `file-index.bitmap.<column_name>.index-block-size`: to config secondary
index block size, default value is 16kb.
+{{< tabs "bitmap" >}}
+
+{{< tab "V2" >}}
Bitmap file index format (V2):
@@ -162,8 +164,14 @@ length: 4 bytes int
</pre>
+{{< /tab >}}
+
+{{< tab "V1 (Legacy)" >}}
+
(Legacy) Bitmap file index format (V1):
+You can configure `file-index.bitmap.version` to use legacy bitmap version 1.
+
<pre>
Bitmap file index format (V1)
@@ -200,70 +208,14 @@ offset: 4 bytes int (when it is
negative, it represents t
and its position is the inverse of the
negative value)
</pre>
+{{< /tab >}}
+
+{{< /tabs >}}
+
Integers are all BIG_ENDIAN.
-Integer are all BIG_ENDIAN. In the paimon version that supports v2, the bitmap
index version defaults to v2.
-
-Bitmap only support the following data type:
-
-<table class="table table-bordered">
- <thead>
- <tr>
- <th class="text-left" style="width: 10%">Paimon Data Type</th>
- <th class="text-left" style="width: 5%">Supported</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td><code>TinyIntType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>SmallIntType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>IntType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>BigIntType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>DateType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>TimeType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>LocalZonedTimestampType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>TimestampType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>CharType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>VarCharType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>StringType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>BooleanType</code></td>
- <td>true</td>
- </tr>
- </tbody>
-</table>
+Bitmap only support the following data type: TinyIntType, SmallIntType,
IntType, BigIntType, DateType, TimeType,
+LocalZonedTimestampType, TimestampType, CharType, VarCharType, StringType,
BooleanType.
## Index: Bit-Slice Index Bitmap
@@ -314,47 +266,5 @@ BSI serialized format (V1)
+-------------------------------------------------+
</pre>
-BSI only support the following data type:
-
-<table class="table table-bordered">
- <thead>
- <tr>
- <th class="text-left" style="width: 10%">Paimon Data Type</th>
- <th class="text-left" style="width: 5%">Supported</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td><code>TinyIntType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>SmallIntType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>IntType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>BigIntType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>DateType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>LocalZonedTimestamp</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>TimestampType</code></td>
- <td>true</td>
- </tr>
- <tr>
- <td><code>DecimalType(precision, scale)</code></td>
- <td>true</td>
- </tr>
- </tbody>
-</table>
+BSI only support the following data type: TinyIntType, SmallIntType, IntType,
BigIntType, DateType, LocalZonedTimestamp,
+TimestampType, DecimalType.
diff --git
a/paimon-common/src/main/java/org/apache/paimon/fileindex/bitmap/BitmapFileIndex.java
b/paimon-common/src/main/java/org/apache/paimon/fileindex/bitmap/BitmapFileIndex.java
index 3678f05ded..5ee4994e07 100644
---
a/paimon-common/src/main/java/org/apache/paimon/fileindex/bitmap/BitmapFileIndex.java
+++
b/paimon-common/src/main/java/org/apache/paimon/fileindex/bitmap/BitmapFileIndex.java
@@ -45,7 +45,7 @@ import java.util.Map;
import java.util.function.Function;
import java.util.stream.Collectors;
-/** implementation of bitmap file index. */
+/** The implementation of bitmap file index. */
public class BitmapFileIndex implements FileIndexer {
public static final int VERSION_1 = 1;
@@ -88,7 +88,7 @@ public class BitmapFileIndex implements FileIndexer {
private final Options options;
public Writer(DataType dataType, Options options) {
- this.version = options.getInteger(VERSION, VERSION_1);
+ this.version = options.getInteger(VERSION, VERSION_2);
this.dataType = dataType;
this.valueMapper = getValueMapper(dataType);
this.options = options;
