[ https://issues.apache.org/jira/browse/CASSANDRA-14485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496825#comment-16496825 ]
Jason Brown commented on CASSANDRA-14485: ----------------------------------------- Patch below has the following optimizations: - remove {{IPAddressAndPort}} from each message sent - change paramSize (which is currently the number of parameters in the header), encoded as a 4-byte int, to represent the count of bytes in the parameters as a whole, encoded as a vint. - each individual parameter value is preceeded by a length, which is encoded as a 4-byte integer. Now encoding as a vint. - change payloadSize to a vint ||14485|| |[branch|https://github.com/jasobrown/cassandra/tree/14485]| |[utests & dtests|https://circleci.com/gh/jasobrown/workflows/cassandra/tree/14485]| Of course, we need to be backward compatible for versions < 4.0 (during cluster upgrade), so there is branching logic to support previous and current versions. The biggest gains in terms of size will be on small or tiny messages (think, {{Verb.REQUEST_RESPONSE}}), where the the size is reduced by about 40%. As ACKs to mutation messages are arguably ~25% of our message load, this is optimization will reduce network bandwidth. I've also included a jmh microbench, which shows that the optimzations, with and without parameters, reduce the time taken to serialize/deserialize by ~50%. Note, again, this these gains will be seen on messages with no/tiny payload. {noformat} [java] Benchmark (withParams) Mode Cnt Score Error Units [java] MessageOutBench.serialize40 true sample 925307 106.132 ± 14.263 ns/op [java] MessageOutBench.serialize40:serialize40·p0.00 true sample ≈ 0 ns/op [java] MessageOutBench.serialize40:serialize40·p0.50 true sample 75.000 ns/op [java] MessageOutBench.serialize40:serialize40·p0.90 true sample 88.000 ns/op [java] MessageOutBench.serialize40:serialize40·p0.95 true sample 112.000 ns/op [java] MessageOutBench.serialize40:serialize40·p0.99 true sample 245.000 ns/op [java] MessageOutBench.serialize40:serialize40·p0.999 true sample 9056.000 ns/op [java] MessageOutBench.serialize40:serialize40·p0.9999 true sample 21263.014 ns/op [java] MessageOutBench.serialize40:serialize40·p1.00 true sample 2025472.000 ns/op [java] MessageOutBench.serialize40 false sample 926698 119.026 ± 42.941 ns/op [java] MessageOutBench.serialize40:serialize40·p0.00 false sample ≈ 0 ns/op [java] MessageOutBench.serialize40:serialize40·p0.50 false sample 74.000 ns/op [java] MessageOutBench.serialize40:serialize40·p0.90 false sample 80.000 ns/op [java] MessageOutBench.serialize40:serialize40·p0.95 false sample 104.000 ns/op [java] MessageOutBench.serialize40:serialize40·p0.99 false sample 238.000 ns/op [java] MessageOutBench.serialize40:serialize40·p0.999 false sample 8804.816 ns/op [java] MessageOutBench.serialize40:serialize40·p0.9999 false sample 16704.000 ns/op [java] MessageOutBench.serialize40:serialize40·p1.00 false sample 9404416.000 ns/op [java] MessageOutBench.serializePre40 true sample 756186 224.933 ± 27.104 ns/op [java] MessageOutBench.serializePre40:serializePre40·p0.00 true sample ≈ 0 ns/op [java] MessageOutBench.serializePre40:serializePre40·p0.50 true sample 161.000 ns/op [java] MessageOutBench.serializePre40:serializePre40·p0.90 true sample 179.000 ns/op [java] MessageOutBench.serializePre40:serializePre40·p0.95 true sample 228.000 ns/op [java] MessageOutBench.serializePre40:serializePre40·p0.99 true sample 437.000 ns/op [java] MessageOutBench.serializePre40:serializePre40·p0.999 true sample 10368.000 ns/op [java] MessageOutBench.serializePre40:serializePre40·p0.9999 true sample 24192.000 ns/op [java] MessageOutBench.serializePre40:serializePre40·p1.00 true sample 1953792.000 ns/op [java] MessageOutBench.serializePre40 false sample 1163481 260.046 ± 49.837 ns/op [java] MessageOutBench.serializePre40:serializePre40·p0.00 false sample ≈ 0 ns/op [java] MessageOutBench.serializePre40:serializePre40·p0.50 false sample 201.000 ns/op [java] MessageOutBench.serializePre40:serializePre40·p0.90 false sample 236.000 ns/op [java] MessageOutBench.serializePre40:serializePre40·p0.95 false sample 274.000 ns/op [java] MessageOutBench.serializePre40:serializePre40·p0.99 false sample 489.000 ns/op [java] MessageOutBench.serializePre40:serializePre40·p0.999 false sample 9872.000 ns/op [java] MessageOutBench.serializePre40:serializePre40·p0.9999 false sample 22345.715 ns/op [java] MessageOutBench.serializePre40:serializePre40·p1.00 false sample 16596992.000 ns/op {noformat} > Optimize internode messaging protocol > ------------------------------------- > > Key: CASSANDRA-14485 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14485 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging > Reporter: Jason Brown > Assignee: Jason Brown > Priority: Major > Fix For: 4.0.x > > > There's some dead wood and places for optimization in the internode messaging > protocol. Currently, we include the sender's \{{IPAddressAndPort}} in *every* > internode message, even though we already sent that in the handshake that > established the connection/session. Further, there are several places where > we can use vints instead of a fixed, 4-byte integer value- especially as > those values will almost always be less than one byte. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org