gaodayue commented on a change in pull request #1694: optimize V2
BinaryPlainPage format (#1648)
URL: https://github.com/apache/incubator-doris/pull/1694#discussion_r317603892
##########
File path: be/src/olap/rowset/segment_v2/binary_plain_page.h
##########
@@ -81,24 +81,21 @@ class BinaryPlainPageBuilder : public PageBuilder {
Slice finish() override {
_finished = true;
- size_t offsets_pos = _buffer.size();
-
- // Set up the header
- encode_fixed32_le(&_buffer[0], _offsets.size());
- encode_fixed32_le(&_buffer[4], offsets_pos);
-
- _buffer.append(&_offsets[0], _offsets.size() * sizeof(uint32_t));
+ // Set up trailer
+ for (int i = 0; i < _offsets.size(); i++) {
+ put_fixed32_le(&_buffer, _offsets[i]);
Review comment:
Because offset bytes(4) is usually small compared to data bytes, so use
varint to store offsets won't save too much space. And use fixed-size offsets
enable us to avoid allocating offsets array and decoding varints during read.
AFAIK, store offsets in plain old little endian ints are common in data stores
such as LevelDB, Parquet, ORC, Elasticsearch, etc.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org
