[
https://issues.apache.org/jira/browse/HDDS-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18078267#comment-18078267
]
Jheng-Sing Chen edited comment on HDDS-14647 at 5/4/26 5:43 PM:
----------------------------------------------------------------
I measured the SCMRatisRequest/SCMRatisResponse codec performance locally with
temporary JMH benchmark patches.
Per HDDS-10021, these benchmark patches are not intended to be merged into
Apache Ozone. They are shared only for reproducibility:
- Before benchmark patch: [https://github.com/Russole/ozone/pull/1]
- After benchmark patch: [https://github.com/Russole/ozone/pull/2]
I also attached the full JMH JSON results:
- before.json
- after.json
Benchmark environment:
- JMH: 1.37
- JDK: OpenJDK 17.0.18
- Mode: Throughput, ops/s
- Warmup: 5 iterations, 1 second each
- Measurement: 10 iterations, 1 second each
- Forks: 3
- Input size: 1000 pre-generated objects
Request results:
|Benchmark|Payload|Before ops/s|After ops/s|Speedup|
|—|—|---:|---:|---:|
|decodeRequest|PROTO|1,833,128.846|4,189,793.896|2.29x|
|decodeRequest|LONG|3,663,170.973|7,503,085.739|2.05x|
|decodeRequest|LIST_PROTO|603,870.857|2,112,250.569|3.50x|
|encodeRequest|PROTO|2,251,279.519|5,457,258.901|2.42x|
|encodeRequest|LONG|3,781,621.455|10,817,847.630|2.86x|
|encodeRequest|LIST_PROTO|833,376.936|2,729,654.338|3.28x|
Response results:
`requestPayloadType` does not affect response benchmarks, so the response
numbers are averaged across the repeated runs.
|Benchmark|Before avg ops/s|After avg ops/s|Speedup|
|—|---:|---:|---:|
|decodeResponse|5,909,073.082|15,371,080.770|2.60x|
|encodeResponse|7,543,301.707|21,087,906.501|2.80x|
Summary:
- SCMRatisRequest codec paths improved by about 2.05x to 3.50x.
- SCMRatisResponse codec paths improved by about 2.60x to 2.80x on average.
- The largest improvement is LIST_PROTO request decode, about 3.50x faster.
was (Author: JIRAUSER311294):
I measured the SCMRatisRequest/SCMRatisResponse codec performance locally with
temporary JMH benchmark patches.
Per HDDS-10021, these benchmark patches are not intended to be merged into
Apache Ozone. They are shared only for reproducibility:
- Before benchmark patch: https://github.com/Russole/ozone/pull/1
- After benchmark patch: https://github.com/Russole/ozone/pull/2
I also attached the full JMH JSON results:
- before.json
- after.json
Benchmark environment:
- JMH: 1.37
- JDK: OpenJDK 17.0.18
- Mode: Throughput, ops/s
- Warmup: 5 iterations, 1 second each
- Measurement: 10 iterations, 1 second each
- Forks: 3
- Input size: 1000 pre-generated objects
Request results:
| Benchmark | Payload | Before ops/s | After ops/s | Speedup |
|---|---|---:|---:|---:|
| decodeRequest | PROTO | 1,833,128.846 | 4,189,793.896 | 2.29x |
| decodeRequest | LONG | 3,663,170.973 | 7,503,085.739 | 2.05x |
| decodeRequest | LIST_PROTO | 603,870.857 | 2,112,250.569 | 3.50x |
| encodeRequest | PROTO | 2,251,279.519 | 5,457,258.901 | 2.42x |
| encodeRequest | LONG | 3,781,621.455 | 10,817,847.630 | 2.86x |
| encodeRequest | LIST_PROTO | 833,376.936 | 2,729,654.338 | 3.28x |
Response results:
`requestPayloadType` does not affect response benchmarks, so the response
numbers are averaged across the repeated runs.
| Benchmark | Before avg ops/s | After avg ops/s | Speedup |
|---|---:|---:|---:|
| decodeResponse | 5,909,073.082 | 15,371,080.770 | 2.60x |
| encodeResponse | 7,543,301.707 | 21,087,906.501 | 2.80x |
Summary:
- SCMRatisRequest codec paths improved by about 2.05x to 3.50x.
- SCMRatisResponse codec paths improved by about 2.60x to 2.80x on average.
- The largest improvement is LIST_PROTO request decode, about 3.50x faster.
> Umbrella: Improve SCM codec
> ---------------------------
>
> Key: HDDS-14647
> URL: https://issues.apache.org/jira/browse/HDDS-14647
> Project: Apache Ozone
> Issue Type: Improvement
> Components: SCM HA
> Reporter: Tsz-wo Sze
> Assignee: Jheng-Sing Chen
> Priority: Major
> Attachments: after.json
>
>
> The SCM Codec interface and the subclass implements are for
> serialize/deserialize Java objects to/from protobuf ByteString. The current
> implementation is inefficient since the Codec interface it not type specific
> and the implementations use Java reflection to handle the objects. In other
> words,
> - the static object type is available in the beginning
> - but, since the parameter is declared by Object, the type is forgotten,
> - then the Codec implementations use reflection to find out the type in
> runtime.
> A better way to use generic type in the Codec interface and a Codec
> implementation only has to handle a specific class. This idea is similar to
> org.apache.hadoop.hdds.utils.db.Codec<T>.
> BTW, we should rename classes in org.apache.hadoop.hdds.scm.ha.io.* to avoid
> confusion with the classes in org.apache.hadoop.hdds.utils.db.*.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
