xiangfu0 opened a new pull request, #19478:
URL: https://github.com/apache/pinot/pull/19478
## What
`ColumnMetadataImpl` held min and max as `Comparable` references, so a
numeric column retained two boxes it never needed. They are now stored as raw
bits in two `long` words and boxed only inside `getMinValue()`/`getMaxValue()`;
columns with a variable-width stored type keep object references and reuse the
same two words for their three element-length values, which is what makes the
change shrink the object rather than grow it. `FLOAT`/`DOUBLE` go through
`floatToIntBits`/`doubleToLongBits` so `NaN` compares equal and `-0.0` stays
distinct from `0.0`, matching `Float.equals`/`Double.equals`; a value whose
class does not match its stored type falls back to the reference so nothing can
be mistranslated. `bitsPerElement` moves into the flags word, with values
outside the encodable range falling back to the existing lazily allocated
holder.
Measured with 500k instances sharing one field spec, used-heap delta after
forced GC: a nullable INT column goes from 104 to 72 bytes, a STRING column
stays at 72, and nothing regresses for multi-value or mixed-shape columns.
Worth knowing: `getMinValue()`/`getMaxValue()` on a fixed-width column now
return a freshly boxed value per call rather than the same instance. Values
stay `equal()` and of the same class; only reference identity across calls is
no longer guaranteed, and no production code depended on it.
## Tests
`ColumnMetadataImplTest`: min and max round-trip for INT, LONG, FLOAT,
DOUBLE, BOOLEAN, TIMESTAMP, STRING, JSON, BYTES and BIG_DECIMAL including the
min-max-invalid flag and null cases; `NaN` is canonicalized but stays equal;
`bitsPerElement` round-trips across 11 values including both sides of the
encodable boundary and both overflow signs, asserting the flags do not bleed
into it.
## Why
A server keeps one metadata object graph per (segment, column) for as long
as the segment is loaded, so on wide tables the per-column footprint decides
how many segments a server can hold. Measured end to end on a 1000-column
segment, this series takes the heap retained at load from **4.08 MB to 0.175 MB
per segment** (4,080 to 174 bytes per column), with a fully compacting
collector on both sides. No on-disk format change, the
`/tables/{table}/segments/{segment}/metadata` JSON stays byte-identical, and
every public and SPI signature keeps working.
## Stack
Part 8 of 9, based on `xiangfu0/data-3221-6-lazy-materialization`. Review
only this part's own commits; the rest of the diff belongs to the parts below
it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
