Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4633: Change broken gflag default for Kudu client mem ......................................................................
IMPALA-4633: Change broken gflag default for Kudu client mem We discovered that the current Kudu client defaults in the KuduTableSink are causing a large number of queries to timeout, failing the query. The current default value of the 'mutation buffer size' is 100MB which results in higher write throughput than Kudu can currently handle on large clusters. By decreasing the value of this flag, more RPCs will be sent for the same amount of data, i.e. throttling the load on Kudu. We found tests to be more successful on 200 nodes with a 10MB buffer size than the previous 100MB value where most queries couldn't complete due to timeouts. These queries were not timing out with the 10MB value. This appears to work well on 9 node stress tests as well. Change-Id: I0b3544f9a93c82e347f6e97540d6b561c30d09fd Reviewed-on: http://gerrit.cloudera.org:8080/5503 Reviewed-by: Dan Hecht <dhe...@cloudera.com> Tested-by: Internal Jenkins --- M be/src/exec/kudu-table-sink.cc M be/src/exec/kudu-table-sink.h 2 files changed, 5 insertions(+), 4 deletions(-) Approvals: Internal Jenkins: Verified Dan Hecht: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I0b3544f9a93c82e347f6e97540d6b561c30d09fd Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Matthew Jacobs <m...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Matthew Jacobs <m...@cloudera.com>
