bzs1118 opened a new pull request, #3480: URL: https://github.com/apache/brpc/pull/3480
The server-side RDMA socket's _read_buf is accessed by two independent bthreads: PollCq (CQ socket) writes RDMA data via HandleCompletion, and OnNewMessages (main socket) reads TCP data for handshake / fallback. Since IOBuf is not thread-safe, concurrent access corrupts internal state and causes intermittent core dumps. Three fixes: 1. Switch edge trigger to `OnNewDataFromTcp`in ALL ExecuteServerHandshake end paths (ESTABLISHED + 5 failure paths). `OnNewDataFromTcp`checks the RDMA state: in ESTABLISHED it only reads 1 byte for EOF detection without touching _read_buf; in FALLBACK_TCP it delegates to `OnNewMessages`for TCP data. This prevents post-handshake races. 2. Guard `HandleCompletion`(IBV_WC_RECV) with a state check: skip writing to _read_buf and re-post the recv WR if the state is not ESTABLISHED. This prevents races during the handshake (after BringUpQp puts the QP into RTS, the client may start sending RDMA data before the server finishes processing the ACK). 3. Remove the source->size() > HELLO_ACK_LEN check in Phase 2. When a client falls back to TCP, the 4-byte ACK and the first RPC request may arrive in the same readv() call. Use cutn() to drain the 4-byte ACK and let remaining data be processed by other parsers, matching FallbackServerHandshake's behavior. 4. Return NOT_ENOUGH_DATA (not TRY_OTHERS) from the ESTABLISHED path so `OnNewMessages`stops processing _read_buf before PollCq starts writing. 5. Restore edge trigger to `OnNewMessages`in `RdmaTransport::Reset()`for health-check re-handshake. ### What problem does this PR solve? Issue Number: #3479 Problem Summary: ### What is changed and the side effects? Changed: Side effects: - Performance effects: - Breaking backward compatibility: --- ### Check List: - Please make sure your changes are compilable. - When providing us with a new feature, it is best to add related tests. - Please follow [Contributor Covenant Code of Conduct](https://github.com/apache/brpc/blob/master/CODE_OF_CONDUCT.md). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
