I investigated network loading and found that a big part of internal data
inside messages is `GridLongList`.
It is a part of `GridDhtTxFinishRequest`,
`GridDhtAtomicDeferredUpdateResponse`, `GridDhtAtomicUpdateRequest`,
`GridNearAtomicFullUpdateRequest` and `NearCacheUpdates`.

So I think it has the sense to optimize `GridLongList` serialization.


Here we serialize all elements and don't take into account `idx` value:

```

@Override public boolean writeTo(ByteBuffer buf, MessageWriter writer) {

        writer.setBuffer(buf);



        if (!writer.isHeaderWritten()) {

            if (!writer.writeHeader(directType(), fieldsCount()))

                return false;



            writer.onHeaderWritten();

        }



        switch (writer.state()) {

            case 0:

                if (!writer.writeLongArray("arr", arr))

                    return false;



                writer.incrementState();



            case 1:

                if (!writer.writeInt("idx", idx))

                    return false;



                writer.incrementState();



        }



        return true;

    }

```



Which is not happening in another serialization method in the same class:



```

public static void writeTo(DataOutput out, @Nullable GridLongList list)
throws IOException {

        out.writeInt(list != null ? list.idx : -1);



        if (list != null) {

            for (int i = 0; i < list.idx; i++)

                out.writeLong(list.arr[i]);

        }

    }

```


So, we can simply reduce messages size by sending only a valuable part of
the array.
If you don't mind I will create an issue in Jira for this.


By the way, `long` is a huge type. As I see in most cases `GridLongList`
uses for counters.
And I have checked the possibility of compress `long` into smaller types as
`int`, `short` or `byte` in test
`GridCacheInterceptorAtomicRebalanceTest` (took it by random).
And found out that all `long` in`GridLongList` can be cast to `int` and 70%
of them to shorts.
Such conversion is quite fast about 1.1 (ns) per element (I have checked it
by JMH test).



Of course, there are a lot of ways to compress data,
but I know proprietary GridGain plug-in has different `MessageWriter`
implementation.
So maybe it is unnecessary and some compression already exists in this
proprietary plug-in.
Does someone know something about it?

Reply via email to