[
https://issues.apache.org/jira/browse/IGNITE-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17135814#comment-17135814
]
Stanilovsky Evgeny commented on IGNITE-6499:
--------------------------------------------
[~steve.hostett...@gmail.com] i found you have no appropriate branch i.e.
ignite-6499, plz make it and further PR from this branch.
thanks !
> Compact NULL fields binary representation
> -----------------------------------------
>
> Key: IGNITE-6499
> URL: https://issues.apache.org/jira/browse/IGNITE-6499
> Project: Ignite
> Issue Type: Improvement
> Components: binary
> Affects Versions: 2.8
> Reporter: Alexandr Kuramshin
> Assignee: Steve Hostettler
> Priority: Major
> Labels: iep-2
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Current compact footer implementation writes offset for the every field in
> schema. Depending on serialized size of an object offset may be 1, 2 or 4
> bytes.
> Imagine an object with some 100 fields are null. It takes from 100 to 400
> bytes overhead. For middle-sized objects (about 260 bytes) it doubles the
> memory usage. For a small-sized objects (about 40 bytes) the memory usage
> increased by factor 3 or 4.
> Proposed two optimizations, the both should be implemented, the most optimal
> implementation should be selected dynamically upon object marshalling.
> 1. Write field index and offset for the only non-null fields in footer.
> Should be used for objects with a few non-null fields.
> 2. Write footer header then field offsets for the only non-null fields as
> follows
> [0] bit mask for first 8 fields, 0 - field is null, 1 - field is non-null
> [1] cumulative sum of "1" bits
> [2] bit mask for the next 8 fields
> [3] cumulative sum of "1" bits
> ... and so on
> [N1...N2] offset of first non-null field
> [N3...N4] offset of next non-null field
> ... and so on
> If we want to read fields from 0 to 7, then we read first footer byte, step
> through bits and find the offset index for non-null field or find that field
> is null.
> If we want to read fields from 8, then we read two footer bytes, take start
> offset index from the first byte, and then step through bits and find the
> offset index for non-null field or find that field is null.
> This supports up to 255 non-null fields per binary object.
> Overhead would be only 24 bytes per 100 null fields instead of 200 bytes for
> the middle-sized object.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
