[ https://issues.apache.org/jira/browse/IGNITE-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17135814#comment-17135814 ]
Stanilovsky Evgeny commented on IGNITE-6499: -------------------------------------------- [~steve.hostett...@gmail.com] i found you have no appropriate branch i.e. ignite-6499, plz make it and further PR from this branch. thanks ! > Compact NULL fields binary representation > ----------------------------------------- > > Key: IGNITE-6499 > URL: https://issues.apache.org/jira/browse/IGNITE-6499 > Project: Ignite > Issue Type: Improvement > Components: binary > Affects Versions: 2.8 > Reporter: Alexandr Kuramshin > Assignee: Steve Hostettler > Priority: Major > Labels: iep-2 > Time Spent: 10m > Remaining Estimate: 0h > > Current compact footer implementation writes offset for the every field in > schema. Depending on serialized size of an object offset may be 1, 2 or 4 > bytes. > Imagine an object with some 100 fields are null. It takes from 100 to 400 > bytes overhead. For middle-sized objects (about 260 bytes) it doubles the > memory usage. For a small-sized objects (about 40 bytes) the memory usage > increased by factor 3 or 4. > Proposed two optimizations, the both should be implemented, the most optimal > implementation should be selected dynamically upon object marshalling. > 1. Write field index and offset for the only non-null fields in footer. > Should be used for objects with a few non-null fields. > 2. Write footer header then field offsets for the only non-null fields as > follows > [0] bit mask for first 8 fields, 0 - field is null, 1 - field is non-null > [1] cumulative sum of "1" bits > [2] bit mask for the next 8 fields > [3] cumulative sum of "1" bits > ... and so on > [N1...N2] offset of first non-null field > [N3...N4] offset of next non-null field > ... and so on > If we want to read fields from 0 to 7, then we read first footer byte, step > through bits and find the offset index for non-null field or find that field > is null. > If we want to read fields from 8, then we read two footer bytes, take start > offset index from the first byte, and then step through bits and find the > offset index for non-null field or find that field is null. > This supports up to 255 non-null fields per binary object. > Overhead would be only 24 bytes per 100 null fields instead of 200 bytes for > the middle-sized object. -- This message was sent by Atlassian Jira (v8.3.4#803005)