Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/9337 )
Change subject: IMPALA-4874: Increase maximum KRPC message size ...................................................................... Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/9337/1/be/src/kudu/rpc/transfer.cc File be/src/kudu/rpc/transfer.cc: http://gerrit.cloudera.org:8080/#/c/9337/1/be/src/kudu/rpc/transfer.cc@38 PS1, Line 38: INT_MAX How close did you get to this in tests? Are we confident that values very close to INT_MAX would still work, i.e. there's no hidden constant overhead that will exceed some limit? http://gerrit.cloudera.org:8080/#/c/9337/1/testdata/workloads/functional-query/queries/QueryTest/large_strings.test File testdata/workloads/functional-query/queries/QueryTest/large_strings.test: http://gerrit.cloudera.org:8080/#/c/9337/1/testdata/workloads/functional-query/queries/QueryTest/large_strings.test@220 PS1, Line 220: select cast(fnv_hash(l_comment) as string) as h from tpch_parquet.lineitem union all Did you check how well these compress? Each hash seems to occur 5 times which could allow for up to 80% size reduction. Can you salt the comments with some random values before hashing them, e.g. fnv_hash(concat(l_comment), "a")) ... union ... fnv_hash(concat(l_comment, "b")), ... -- To view, visit http://gerrit.cloudera.org:8080/9337 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I876bba0536e1d85e41eacd9c0aeccfe5c2126e58 Gerrit-Change-Number: 9337 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Lars Volker <l...@cloudera.com> Gerrit-Reviewer: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Sailesh Mukil <sail...@cloudera.com> Gerrit-Comment-Date: Thu, 15 Feb 2018 22:01:05 +0000 Gerrit-HasComments: Yes
