AnDiXL commented on PR #3559:
URL: https://github.com/apache/brpc/pull/3559#issuecomment-6465153561
Thanks for the detailed review, @wwbmmm, and for checking the Copilot
comments. Below are the benchmark results for point 1 and proposals for points
2–4.
### 1. Motivation and benchmark
`fb_rpc` targets services whose payloads are already FlatBuffers, or where
per-request parse/serialize cost dominates server CPU. With `baidu_std` every
request and response is fully parsed and re-serialized. With `fb_rpc` the
handler reads fields directly from the received bytes without a parse step
(they are copied into one contiguous block only when they span several `IOBuf`
blocks), and the builder output is appended to the socket `IOBuf` without
another serialization pass. This transport has been used in the internal
storage network of Baidu AI Cloud for several years; this PR upstreams it.
To measure it I extended `example/benchmark_fb` with a timed mode and a
protobuf control service that has the same fields and is served over
`baidu_std` by the same server binary, so both protocols run with identical
client and server code.
| Item | Value |
|---|---|
| Hosts | 2 × AMD EPYC 7W83 (2 × 64C), client and server on separate hosts |
| Network | bond0, 200 Gb/s, TCP, RTT ≈ 0.02 ms, single connection |
| Build | Ubuntu 22.04 container, GCC 11, Release, Protobuf 3.12.4,
FlatBuffers 25.2.10 |
| CPU | both processes on the 112 NIC-local cores of NUMA node 0 |
| Load | closed loop, 1 / 16 / 64 bthread callers, one synchronous RPC in
flight per caller; echo of a string field of the given size, no attachment |
| Method | 3 s warm-up + 30 s, 5 rounds, protocol order alternated per
round; Δ is the median of the per-round fb/pb ratios |
| Server CPU | utime + stime of the server process from `/proc`, sampled
every second over the measured window |
Summary (fb_rpc vs baidu_std; positive QPS is better, negative CPU is
better):
| Payload | QPS, 1 caller | QPS, 16 callers | QPS, 64 callers | Server
CPU/req, 1 | Server CPU/req, 16 | Server CPU/req, 64 |
|---|---|---|---|---|---|---|
| 0 B | +12% | +7% | +4% | -22% | -15% | -16% |
| 32 B | +13% | +8% | +9% | -19% | -16% | -19% |
| 256 B | +16% | +8% | +6% | -13% | -15% | -18% |
| 1 KB | +13% | +9% | +12% | -21% | -17% | -18% |
| 4 KB | +14% | +9% | +9% | -19% | -18% | -17% |
| 8 KB | +11% | +9% | +7% | -18% | -18% | -19% |
| 16 KB | +15% | +8% | +9% | -15% | -19% | -19% |
| 64 KB | +18% | +6% | +0% | -14% | -19% | -13% |
| 256 KB | +21% | +1% | +0% | -16% | -10% | -12% |
| 1 MB | +30% | +4% | +5% | -18% | -17% | -24% |
- `fb_rpc` had higher QPS in 143 of 150 paired rounds; the worst round was
−2.3%.
- 1 caller: +11% to +30% QPS, p50 latency 8% to 24% lower.
- 16 and 64 callers: +4% to +12% up to 16 KB. At 64 KB and above both
protocols reach the limit of the single TCP connection (≈ 2.4 GB/s per
direction), so QPS converges. In a single 20 s run with
`connection_type=pooled` at 256 KB, both rise about 4.9× and remain equal
(+0.5%), so large-message throughput is bound by kernel copies and the network
rather than by the protocol.
- Server CPU per request is lower in every case, by 10% to 24%. This is the
most stable result.
Full results are in the collapsed section at the end.
**Builder sizing.** `MessageBuilder` starts at 1 KB and grows by
reallocate-and-copy. Without reserving the size, a 1 MB reply doubled server
CPU in our first runs (repeated glibc trim and page faults). The updated
example reserves the expected size, and the guide will say so for large
messages.
**Not covered:** RDMA, attachments, the full matrix with pooled connections,
and newer Protobuf releases.
**Missing features.** Authentication, compression, checksums and streaming
are out of scope for this PR. Because the metadata carries an explicit length,
fields such as log-id and user fields can be appended later without breaking
the known prefix.
### 2. ABI
Agreed that the layout should not depend on `BRPC_WITH_FLATBUFFERS`.
Proposal:
- `Channel` stops inheriting `flatbuffers::RpcChannel`. An internal adapter,
built only when the option is ON, implements `RpcChannel` and forwards to
`Channel`.
- Generated stubs gain a constructor that takes a `brpc::Channel*` and owns
the adapter, so `XXX_Stub stub(&channel);` stays as it is. The existing
`RpcChannel*` constructor is kept.
- `Controller::_flatbuffers_method` and `Server::_flatbuffers_services` are
declared unconditionally, with forward-declared types, and stay null when the
option is OFF. The cost is one pointer in `Controller` and one in `Server`.
- The ON validation checks that `sizeof` and member offsets of `Channel`,
`Controller` and `Server` match between ON and OFF builds.
With this, a release has one layout whatever the option is, and
protobuf-only users are not affected. A consumer built with ON against a
library built with OFF fails at link time instead of at run time. A separate
component would still need protocol registration and dispatch hooks in
`Server`, so I don't think it would reduce the coupling much.
### 3. Service registry
Proposal:
- Keep FlatBuffers methods out of `_method_map`. `baidu_std`, HTTP, sofa,
mongo and the nshead adaptor look methods up there by name and expect a
protobuf descriptor.
- Reuse `MethodProperty` for FlatBuffers methods (two extra pointer fields)
and keep a full-name index for them. This replaces the O(services × methods)
lookups.
- Add one internal iteration over both maps and use it for bvar exposure,
concurrency limiter setup, `MaxConcurrencyOf()` / `ignore_eovercrowded`,
`ClearServices()` and `/status`. A new per-method option is then added in one
place and applies to both kinds of service.
- Status: add `ListFlatBuffersServices()` and keep the signature of
`ListServices()`; document `user_service_count` as the sum of both; show
FlatBuffers methods in `/status`. `/list` and `/protobufs` stay protobuf-only,
because they return protobuf descriptors.
- A request over another protocol that names a FlatBuffers method gets
`ENOMETHOD`.
A shared registry abstraction for both kinds of service would touch the
protobuf paths much more, so I'd prefer to do it in a follow-up PR.
### 4. Wire format
Agreed. Compatibility with the earlier experimental FRPC formats is not a
goal, so the header and the fixed metadata will both be big-endian (network
order), using the butil byte-order helpers, with a test that fixes the byte
layout of request and response frames. The FlatBuffers payload stays
little-endian as FlatBuffers defines it; it is not part of the frame. I'll keep
the `FRPC` magic, because protocol detection on a shared port relies on it in
the same way as `PRPC`, and document both choices.
---
The benchmark changes to `example/benchmark_fb` (timed mode, `baidu_std`
control service, `--use_bthread`, builder sizing) are in a separate commit,
358a58a0. If the proposals for 2 and 3 look right to you, I'll implement them
in this PR together with 4.
<details>
<summary>Full results (1 / 16 / 64 callers, 10 payload sizes, medians of 5
rounds)</summary>
**1 caller**
| Payload | QPS fb_rpc / baidu_std | Δ QPS, median (min ~ max) | p50 µs |
p99 µs | Server CPU µs/req |
|---|---|---|---|---|---|
| 0 B | 28,282 / 25,303 | +11.8% (+4.7 ~ +24.7) | 35 / 38 | 43 / 51 | 15.9 /
20.4 |
| 32 B | 29,006 / 24,198 | +13.4% (+10.7 ~ +29.9) | 33 / 40 | 43 / 49 | 15.7
/ 19.4 |
| 256 B | 27,556 / 23,943 | +16.0% (-1.9 ~ +19.5) | 36 / 41 | 45 / 52 | 17.5
/ 20.2 |
| 1 KB | 26,330 / 23,436 | +12.7% (+5.0 ~ +25.6) | 37 / 42 | 46 / 52 | 16.6
/ 21.1 |
| 4 KB | 24,468 / 20,817 | +14.0% (+9.0 ~ +23.3) | 40 / 47 | 51 / 59 | 19.4
/ 23.9 |
| 8 KB | 21,338 / 18,744 | +11.4% (+2.0 ~ +20.1) | 46 / 52 | 57 / 66 | 20.8
/ 25.5 |
| 16 KB | 19,094 / 16,319 | +15.3% (+10.3 ~ +21.2) | 51 / 60 | 66 / 75 |
25.8 / 30.5 |
| 64 KB | 7,892 / 6,687 | +17.5% (+15.2 ~ +26.7) | 122 / 146 | 187 / 226 |
61.2 / 70.8 |
| 256 KB | 2,938 / 2,420 | +20.6% (+12.1 ~ +26.5) | 333 / 413 | 488 / 538 |
182.7 / 217.8 |
| 1 MB | 821 / 654 | +29.7% (+19.6 ~ +38.8) | 1207 / 1597 | 1867 / 2258 |
993.2 / 1208.1 |
**16 callers**
| Payload | QPS fb_rpc / baidu_std | Δ QPS, median (min ~ max) | p50 µs |
p99 µs | Server CPU µs/req |
|---|---|---|---|---|---|
| 0 B | 151,965 / 140,095 | +6.7% (+2.9 ~ +10.2) | 100 / 107 | 200 / 219 |
11.7 / 13.8 |
| 32 B | 151,451 / 138,715 | +8.1% (+4.4 ~ +14.1) | 99 / 109 | 200 / 218 |
11.9 / 14.1 |
| 256 B | 151,119 / 141,678 | +8.5% (-2.3 ~ +10.5) | 100 / 106 | 195 / 213 |
12.3 / 14.5 |
| 1 KB | 142,284 / 131,145 | +9.0% (+5.6 ~ +13.7) | 106 / 115 | 209 / 224 |
13.0 / 15.7 |
| 4 KB | 105,336 / 97,449 | +8.7% (+2.0 ~ +19.3) | 141 / 152 | 292 / 316 |
16.1 / 19.7 |
| 8 KB | 87,068 / 80,145 | +8.8% (+0.4 ~ +13.1) | 170 / 184 | 354 / 382 |
19.5 / 23.8 |
| 16 KB | 57,694 / 53,246 | +8.0% (+1.8 ~ +12.7) | 260 / 282 | 520 / 559 |
28.3 / 35.0 |
| 64 KB | 24,376 / 22,574 | +5.7% (+2.2 ~ +16.5) | 669 / 722 | 855 / 945 |
58.7 / 72.4 |
| 256 KB | 9,178 / 9,117 | +0.8% (-1.1 ~ +1.6) | 1739 / 1749 | 2109 / 2146 |
200.1 / 223.4 |
| 1 MB | 2,316 / 2,235 | +4.4% (+1.3 ~ +6.9) | 6899 / 7144 | 8178 / 8395 |
639.2 / 771.8 |
**64 callers**
| Payload | QPS fb_rpc / baidu_std | Δ QPS, median (min ~ max) | p50 µs |
p99 µs | Server CPU µs/req |
|---|---|---|---|---|---|
| 0 B | 353,581 / 337,546 | +4.2% (+1.0 ~ +11.4) | 163 / 171 | 363 / 390 |
7.4 / 8.8 |
| 32 B | 362,863 / 329,893 | +9.0% (+5.9 ~ +17.8) | 158 / 175 | 349 / 391 |
7.3 / 9.0 |
| 256 B | 349,073 / 331,986 | +5.6% (+2.9 ~ +7.9) | 165 / 173 | 353 / 377 |
7.8 / 9.5 |
| 1 KB | 280,872 / 254,466 | +11.7% (+0.6 ~ +14.6) | 211 / 242 | 426 / 458 |
8.3 / 10.2 |
| 4 KB | 181,194 / 168,392 | +9.3% (+0.0 ~ +13.8) | 350 / 376 | 606 / 649 |
11.2 / 13.6 |
| 8 KB | 136,591 / 128,839 | +6.9% (+4.2 ~ +8.5) | 459 / 491 | 807 / 855 |
14.0 / 17.4 |
| 16 KB | 95,390 / 87,490 | +9.0% (+8.1 ~ +13.2) | 671 / 736 | 898 / 1000 |
23.0 / 28.3 |
| 64 KB | 36,368 / 36,195 | +0.5% (-1.3 ~ +2.5) | 1753 / 1765 | 2004 / 2022
| 52.1 / 59.8 |
| 256 KB | 9,301 / 9,189 | +0.3% (-1.2 ~ +3.3) | 6876 / 6933 | 7448 / 7828 |
197.3 / 225.3 |
| 1 MB | 2,355 / 2,195 | +4.8% (+4.3 ~ +10.2) | 27180 / 29080 | 29206 /
30947 | 659.4 / 867.9 |
For each case the two protocols ran back to back in the same round on the
same server process. 15 of the 30 cases were measured in a second session (0 B,
256 B, 4 KB and 16 KB at all caller counts, and every size at 16 callers);
results from the two sessions differ by a few percent on their own, so compare
cases within a table row rather than across sessions.
Reproduce with the example (server and client hosts separate):
```sh
benchmark_fb_server --listen_addr=SERVER_IP:8901 --allow_remote
--duration_s=3600 --max_concurrency=4096
for p in fb_rpc baidu_std; do
benchmark_fb_client --server=SERVER_IP:8901 --allow_remote --protocol=$p
--use_bthread=true \
--duration_s=30 --warmup_s=3 --thread_num=64 --request_size=1024
done
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
