[
https://issues.apache.org/jira/browse/IGNITE-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849788#comment-15849788
]
Vladimir Ozerov commented on IGNITE-4646:
-----------------------------------------
[~yzhdanov],
Setting hard limit to max buffer size is bad idea because modern high-end NICs
can have buffers of hundreds KBs and more. I do not see a reason for economy in
several bytes, let's just dedicate {{INT}} for chunk size. It will add 1-2%
overhead to message - negligible IMO.
Another important implication of this changes is that we can get rid of all
complex logic in reader. No "available data" checks, no boundary checks. Just
go through array with {{Unsafe}}.
Bad news is that I already tried what you desribed (remember my coding on
Joker? :)) and it didn't give any improvement. However, it was done before
striped pool optimization.
> Try to unmarshall direct messages in striped pool
> -------------------------------------------------
>
> Key: IGNITE-4646
> URL: https://issues.apache.org/jira/browse/IGNITE-4646
> Project: Ignite
> Issue Type: Improvement
> Reporter: Yakov Zhdanov
> Assignee: Igor Seliverstov
> Fix For: 2.0
>
>
> During marshalling in NIO thread the following should be added to the write
> buffer and sent to peer:
> 1. chunk size - 16 bits (this probably puts limitation on max write buffer
> size of 64k or will require some changes to direct writer)
> 2. last chunk - 1 bit
> 3. pool policy - 8 bits
> Here is the scheme to explain how this should work.
> {noformat}
> [chunk size] [pool policy] [partition] [last flag] [chunk data] X <-- no more
> space in write buffer
> [next chunk size] [last flag] [chunk data] <<-- we write next chunk once some
> space is available in write buffer, but we skip partition and policy flags
> and maybe others that should be sent only once.
> ...
> ...
> [next chunk size] [last flag] [chunk data] <<-- last flag is true here
> {noformat}
> Examples
> Write buffer - 64k
> Message - 84k
> # sender reserves space for chunk size
> # reserves space for policy and last chunk flag
> # marshalls message to buffer while it has free space (64k - SPACE will be
> written to buffer)
> # puts size and flags to reserved space in the beginning
> # sends buffer or part of it which makes some space available to further
> writes
> # reserves space for next chunk size and flags
> # marshalls message to buffer while it has free space (lets assume the rest
> of message fits)
> # puts size and last=true to the reserved space and sends
> Receiver:
> # reads chunk size, stores the target pool and partition
> # allocates heap buffer and copies chunk data to it from read buffer
> # once all message chunks are fully read message should be submitted to a
> pool where it will be unmarshalled and processed
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
