[ https://issues.apache.org/jira/browse/IMPALA-9134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on IMPALA-9134 started by Tim Armstrong. --------------------------------------------- > Parallelise flush in data stream sender > --------------------------------------- > > Key: IMPALA-9134 > URL: https://issues.apache.org/jira/browse/IMPALA-9134 > Project: IMPALA > Issue Type: Sub-task > Components: Distributed Exec > Reporter: Tim Armstrong > Assignee: Tim Armstrong > Priority: Major > Labels: perf, scalability > > The data stream sender currently does a synchronous RPC to close each channel > https://github.com/apache/impala/blob/d4648e8/be/src/runtime/krpc-data-stream-sender.cc#L565. > This is suboptimal because it serializes the network round-trips and takes > sum(RTT) over all the destinations in the best case, where no data needs to > be flushed or 2 * sum(RTT) in the worst case if all channels need to flush > data. > If the RPCs were done asynchronously and overlapped with each other, we could > get this down to 2 * max(RTT). -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org