Tim Armstrong created IMPALA-9134: ------------------------------------- Summary: Parallelise flush in data stream sender Key: IMPALA-9134 URL: https://issues.apache.org/jira/browse/IMPALA-9134 Project: IMPALA Issue Type: Improvement Components: Distributed Exec Reporter: Tim Armstrong
The data stream sender currently does a synchronous RPC to close each channel https://github.com/apache/impala/blob/d4648e8/be/src/runtime/krpc-data-stream-sender.cc#L565. This is suboptimal because it serializes the network round-trips and takes sum(RTT) over all the destinations in the best case, where no data needs to be flushed or 2 * sum(RTT) in the worst case if all channels need to flush data. If the RPCs were done asynchronously and overlapped with each other, we could get this down to 2 * max(RTT). -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org