[
https://issues.apache.org/jira/browse/RATIS-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Andika updated RATIS-2509:
-------------------------------
Description:
Recently, we did a benchmark comparing the OM leader-read that does not go
through Ratis and OM leader read that go through Ratis (through
submitClientRequestAsync). We saw that there is up to 25% decrease in read
throughput although the we make the raft.server.read.option to be DEFAULT which
should return immediately (235020 QPS -> 180433 QPS or 24% reduction in
throughput for pure reads with 100 threads).
The overheads seem to be because of request/response proto conversion,
RaftClientRequest construction, future chaining, .get() blocking, Ratis
metrics/reply building, and parsing the Ratis response back into OMResponse.
This means that if we submit linearizable read to follower, we incurs these
overhead on top of the linearizable read overhead (e.g. ReadIndex, etc).
We can try to find a way to reduce this overhead
was:
Recently, we did a benchmark comparing the OM leader-read that does not go
through Ratis and OM leader read that go through Ratis (through
submitClientRequestAsync). We saw that there is up to 25% decrease in read
throughput although the we make the raft.server.read.option to be DEFAULT which
should return immediately (235020 QPS -> 180433 QPS for pure reads with 100
threads).
The overheads seem to be because of request/response proto conversion,
RaftClientRequest construction, future chaining, .get() blocking, Ratis
metrics/reply building, and parsing the Ratis response back into OMResponse.
This means that if we submit linearizable read to follower, we incurs these
overhead on top of the linearizable read overhead (e.g. ReadIndex, etc).
We can try to find a way to reduce this overhead
> Optimization to reduce serde cost in Ratis request and response
> ---------------------------------------------------------------
>
> Key: RATIS-2509
> URL: https://issues.apache.org/jira/browse/RATIS-2509
> Project: Ratis
> Issue Type: Improvement
> Reporter: Ivan Andika
> Priority: Major
> Attachments: om-benchmark-leader-read-all-ratis.html
>
>
> Recently, we did a benchmark comparing the OM leader-read that does not go
> through Ratis and OM leader read that go through Ratis (through
> submitClientRequestAsync). We saw that there is up to 25% decrease in read
> throughput although the we make the raft.server.read.option to be DEFAULT
> which should return immediately (235020 QPS -> 180433 QPS or 24% reduction in
> throughput for pure reads with 100 threads).
> The overheads seem to be because of request/response proto conversion,
> RaftClientRequest construction, future chaining, .get() blocking, Ratis
> metrics/reply building, and parsing the Ratis response back into OMResponse.
> This means that if we submit linearizable read to follower, we incurs these
> overhead on top of the linearizable read overhead (e.g. ReadIndex, etc).
> We can try to find a way to reduce this overhead
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
