Thanks, Alex.
GridGain automatically compresses all the internal types. Somehow it looks
like the GridLongList may have been mixed. Can you please file a ticket for
2.5 release?
D.
On Mon, Mar 26, 2018 at 4:55 AM, Александр Меньшиков <sharple...@gmail.com>
wrote:
> I investigated network loading and found that a big part of internal data
> inside messages is `GridLongList`.
> It is a part of `GridDhtTxFinishRequest`,
> `GridDhtAtomicDeferredUpdateResponse`, `GridDhtAtomicUpdateRequest`,
> `GridNearAtomicFullUpdateRequest` and `NearCacheUpdates`.
>
> So I think it has the sense to optimize `GridLongList` serialization.
>
>
> Here we serialize all elements and don't take into account `idx` value:
>
> ```
>
> @Override public boolean writeTo(ByteBuffer buf, MessageWriter writer) {
>
> writer.setBuffer(buf);
>
>
>
> if (!writer.isHeaderWritten()) {
>
> if (!writer.writeHeader(directType(), fieldsCount()))
>
> return false;
>
>
>
> writer.onHeaderWritten();
>
> }
>
>
>
> switch (writer.state()) {
>
> case 0:
>
> if (!writer.writeLongArray("arr", arr))
>
> return false;
>
>
>
> writer.incrementState();
>
>
>
> case 1:
>
> if (!writer.writeInt("idx", idx))
>
> return false;
>
>
>
> writer.incrementState();
>
>
>
> }
>
>
>
> return true;
>
> }
>
> ```
>
>
>
> Which is not happening in another serialization method in the same class:
>
>
>
> ```
>
> public static void writeTo(DataOutput out, @Nullable GridLongList list)
> throws IOException {
>
> out.writeInt(list != null ? list.idx : -1);
>
>
>
> if (list != null) {
>
> for (int i = 0; i < list.idx; i++)
>
> out.writeLong(list.arr[i]);
>
> }
>
> }
>
> ```
>
>
> So, we can simply reduce messages size by sending only a valuable part of
> the array.
> If you don't mind I will create an issue in Jira for this.
>
>
> By the way, `long` is a huge type. As I see in most cases `GridLongList`
> uses for counters.
> And I have checked the possibility of compress `long` into smaller types as
> `int`, `short` or `byte` in test
> `GridCacheInterceptorAtomicRebalanceTest` (took it by random).
> And found out that all `long` in`GridLongList` can be cast to `int` and 70%
> of them to shorts.
> Such conversion is quite fast about 1.1 (ns) per element (I have checked it
> by JMH test).
>
>
>
> Of course, there are a lot of ways to compress data,
> but I know proprietary GridGain plug-in has different `MessageWriter`
> implementation.
> So maybe it is unnecessary and some compression already exists in this
> proprietary plug-in.
> Does someone know something about it?
>
