Todd Lipcon has submitted this change and it was merged. Change subject: KUDU-2065, KUDU-2011: Release sidecars on cancellation or timeout ......................................................................
KUDU-2065, KUDU-2011: Release sidecars on cancellation or timeout Previously, when an outbound call times out after transmission of the request has begun but not finished, the RPC layer will continue to send the entire payload (including any sidecars) to the remote destination until the promised number of bytes specified in the header are sent. This is problematic for the users of the RPC layer as there is no well defined point in which the sidecars are no longer referenced by the RPC layer. The only model in which this would work is for the caller to either transfer or share ownership of the sidecars with the RPC layer. This has caused some life cycle issues with row batches in Impala before (e.g. IMPALA-5093). This change fixes the problem above by modifying the RPC protocol to allow a mid-transmission RPC call to be cancelled. Specifically, a new footer is added to all outbound call requests. It contains a flag, when true, indicates the outbound call was cancelled after it has started but before it finished. The server will inspect this flag and ignore inbound calls with this flag set. This footer enables the caller to relinquish references to the sidecars early when an outbound call is cancelled or timed-out. Once the call is cancelled or timed-out, the RPC layer will send the remainder of the bytes for the request with some dummy bytes. Since the server always ignores cancelled call requests, it's okay to send random bytes. To avoid breaking compatibility, a new RPC feature flag REQUEST_FOOTERS is introduced in this change. During connection negotiation, the client will always include this flag in its feature set. A server which supports parsing footer will include REQUEST_FOOTERS in its feature set if it sees the client also supports it. A client will only send the footer if the remote server has the RPC feature flag REQUEST_FOOTERS. Change-Id: I5c8e294090279649082eebc4f6cfb6fe858bbbfc Reviewed-on: http://gerrit.cloudera.org:8080/7599 Reviewed-by: Todd Lipcon <t...@apache.org> Tested-by: Kudu Jenkins --- M docs/design-docs/rpc.md M src/kudu/rpc/blocking_ops.cc M src/kudu/rpc/connection.cc M src/kudu/rpc/connection.h M src/kudu/rpc/constants.cc M src/kudu/rpc/inbound_call.cc M src/kudu/rpc/inbound_call.h M src/kudu/rpc/outbound_call.cc M src/kudu/rpc/outbound_call.h M src/kudu/rpc/reactor.cc M src/kudu/rpc/rpc-test-base.h M src/kudu/rpc/rpc-test.cc M src/kudu/rpc/rpc_header.proto M src/kudu/rpc/rtest.proto M src/kudu/rpc/serialization.cc M src/kudu/rpc/serialization.h M src/kudu/rpc/server_negotiation.cc M src/kudu/rpc/transfer.cc M src/kudu/rpc/transfer.h 19 files changed, 532 insertions(+), 159 deletions(-) Approvals: Todd Lipcon: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/7599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I5c8e294090279649082eebc4f6cfb6fe858bbbfc Gerrit-PatchSet: 6 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon <t...@apache.org>