[
https://issues.apache.org/jira/browse/IGNITE-29206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Werner updated IGNITE-29206:
-----------------------------------
Description:
*Motivation*
DirectMessageWriter serializes @Compress and pending raw fields by accumulating
the whole field in a reusable heap buffer (tmpBuf), which doubles until the
field fits and is never shrunk. Writers are long-lived (per NIO session for
communication, per node for discovery), so each one retains a buffer up to ~2x
the largest field it has ever serialized. Growing it also copies the data and
allocates humongous arrays.
The exchange coordinator is hit hardest: compressed fields of
GridDhtPartitionsFullMessage can reach tens of megabytes, and after a PME every
communication session keeps such a buffer, which can add up to hundreds of
megabytes of heap on large clusters. The retention was introduced in
IGNITE-28836.
Also, if serialization throws, tmpWriter keeps stale nested state that a later
field could continue from. Current callers drop the writer after an error, so
this is hardening.
*Changes*
The temporary buffer has a fixed size of 10 KB. The field is serialized into it
piece by piece, like into a socket buffer, and each piece is handed to a
consumer.
@Compress fields are deflated on the fly by the new
CompressedMessage.Compressor, so the raw data is never held in one buffer. The
wire format is unchanged.
Raw fields are collected piece by piece and joined into a single byte array.
If serialization fails, tmpWriter is dropped, as DirectMessageReader already
does with tmpReader.
Effect: each writer retains a constant 10 KB instead of the largest field ever
sent. Small fields (the vast majority) fit in a single piece, as before.
was:
*Motivation*
DirectMessageWriter serializes @Compress and pending raw fields by accumulating
the whole field in a reusable heap buffer (tmpBuf), which doubles until the
field fits and is never shrunk. Writers are long-lived (per NIO session for
communication, per node for discovery), so each one retains a buffer up to ~2x
the largest field it has ever serialized. Growing it also copies the data and
allocates humongous arrays.
The exchange coordinator is hit hardest: compressed fields of
GridDhtPartitionsFullMessage can reach tens of megabytes, and after a PME every
communication session keeps such a buffer, which can add up to hundreds of
megabytes of heap on large clusters. The retention was introduced in
IGNITE-28836.
Also, if serialization throws, tmpWriter keeps stale nested state that a later
field could continue from. Current callers drop the writer after an error, so
this is hardening.
*Changes*
The temporary buffer has a fixed size of 10 KB. The field is serialized
into it piece by piece, like into a socket buffer, and each piece is handed to
a consumer.
@Compress fields are deflated on the fly by the new
CompressedMessage.Compressor, so the raw data is never held in one buffer. The
wire format is unchanged.
Raw fields are collected piece by piece and joined into a single byte array.
If serialization fails, tmpWriter is dropped, as DirectMessageReader
already does with tmpReader.
Effect: each writer retains a constant 10 KB instead of the largest field ever
sent. Small fields (the vast majority) fit in a single piece, as before.
> Serialize compressed and raw fields in DirectMessageWriter through a
> fixed-size buffer
> --------------------------------------------------------------------------------------
>
> Key: IGNITE-29206
> URL: https://issues.apache.org/jira/browse/IGNITE-29206
> Project: Ignite
> Issue Type: Improvement
> Reporter: Dmitry Werner
> Assignee: Dmitry Werner
> Priority: Major
> Labels: ise
> Time Spent: 20m
> Remaining Estimate: 0h
>
> *Motivation*
> DirectMessageWriter serializes @Compress and pending raw fields by
> accumulating the whole field in a reusable heap buffer (tmpBuf), which
> doubles until the field fits and is never shrunk. Writers are long-lived (per
> NIO session for communication, per node for discovery), so each one retains a
> buffer up to ~2x the largest field it has ever serialized. Growing it also
> copies the data and allocates humongous arrays.
> The exchange coordinator is hit hardest: compressed fields of
> GridDhtPartitionsFullMessage can reach tens of megabytes, and after a PME
> every communication session keeps such a buffer, which can add up to hundreds
> of megabytes of heap on large clusters. The retention was introduced in
> IGNITE-28836.
> Also, if serialization throws, tmpWriter keeps stale nested state that a
> later field could continue from. Current callers drop the writer after an
> error, so this is hardening.
> *Changes*
> The temporary buffer has a fixed size of 10 KB. The field is serialized into
> it piece by piece, like into a socket buffer, and each piece is handed to a
> consumer.
> @Compress fields are deflated on the fly by the new
> CompressedMessage.Compressor, so the raw data is never held in one buffer.
> The wire format is unchanged.
> Raw fields are collected piece by piece and joined into a single byte array.
> If serialization fails, tmpWriter is dropped, as DirectMessageReader already
> does with tmpReader.
> Effect: each writer retains a constant 10 KB instead of the largest field
> ever sent. Small fields (the vast majority) fit in a single piece, as before.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
