Sailesh Mukil has uploaded a new patch set (#2). Change subject: IMPALA-3977: TransmitData() should not block ......................................................................
IMPALA-3977: TransmitData() should not block TransmitData() calls effectively block until an error occurs (e.g. datastream_sender_timeout_ms expires) or the RPC is serviced by the receiver. This is bad for a variety of reasons, not least that it makes it impossible to cancel the sender without also cancelling the recipient (who might not even be running!). With our current RPC stack, this holds up the connection for as long as the receiver takes to get ready (or until it times out), which means that that connection is unusable by other threads. When we move to KuduRPC, we will have a different problem if this RPC remains blocking, in that it will hold up a service thread on the receiver side potentially blocking progress for other tasks on that host that require a service thread. This patch introduces a EAGAIN like error code that the receiver sends back to the sender if the DataStreamReceiver object is not yet ready to receive data. The sender tries again up to 5 times with intervals in between. Change-Id: I47f9d6c1de95a72ab73471dcd2a3118ef13e7db0 --- M be/src/runtime/data-stream-mgr.cc M be/src/runtime/data-stream-mgr.h M be/src/runtime/data-stream-sender.cc M common/thrift/generate_error_codes.py 4 files changed, 85 insertions(+), 139 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/5491/2 -- To view, visit http://gerrit.cloudera.org:8080/5491 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I47f9d6c1de95a72ab73471dcd2a3118ef13e7db0 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Sailesh Mukil <sail...@cloudera.com> Gerrit-Reviewer: Henry Robinson <he...@cloudera.com> Gerrit-Reviewer: Lars Volker <l...@cloudera.com> Gerrit-Reviewer: Sailesh Mukil <sail...@cloudera.com>
