Matthew Jacobs has uploaded a new patch set (#2). Change subject: IMPALA-4633: Change broken gflag default for Kudu client mem ......................................................................
IMPALA-4633: Change broken gflag default for Kudu client mem We discovered that the current Kudu client defaults in the KuduTableSink are causing a large number of queries to timeout, failing the query. The current default value of the 'mutation buffer size' is 100MB which results in higher write throughput than Kudu can currently handle on large clusters. By decreasing the value of this flag, more RPCs will be sent for the same amount of data, i.e. throttling the load on Kudu. We found tests to be more successful on 200 nodes with a 10MB buffer size than the previous 100MB value where most queries couldn't complete due to timeouts. These queries were not timing out with the 10MB value. This appears to work well on 9 node stress tests as well. Change-Id: I0b3544f9a93c82e347f6e97540d6b561c30d09fd --- M be/src/exec/kudu-table-sink.cc M be/src/exec/kudu-table-sink.h 2 files changed, 5 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/5503/2 -- To view, visit http://gerrit.cloudera.org:8080/5503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I0b3544f9a93c82e347f6e97540d6b561c30d09fd Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Matthew Jacobs <m...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Matthew Jacobs <m...@cloudera.com>
