[Impala-ASF-CR] IMPALA-12366: Use 2GB as the default for thrift rpc max message size
Michael Smith has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/20394 )
Change subject: IMPALA-12366: Use 2GB as the default for
thrift_rpc_max_message_size
..
IMPALA-12366: Use 2GB as the default for thrift_rpc_max_message_size
Thrift 0.16 implemented a limit on the max message size. In IMPALA-11669,
we added the thrift_rpc_max_message_size parameter and set the default
size to 1GB. Some existing clusters have needed to tune this parameter
higher because their workloads use message sizes larger than 1GB (e.g.
for metadata updates).
Historically, Impala has been able to send and receive 2GB messages,
so this changes the default value for thrift_rpc_max_message_size
to 2GB (INT_MAX). This can be reduced in future when Impala can guarantee
that messages work properly when split up into smaller batches.
TestGracefulShutdown::test_shutdown_idle started failing with this
change, because it is producing a different error message for one
of the negative tests. ClientRequestState::ExecShutdownRequest()
appends some extra explanation when it sees a "Network error" KRPC error,
and the test expects that extra explanation. This modifies
ClientRequestState::ExecShutdownRequest() to provide the extra explanation
for the new error ("Timed out") as well.
Testing:
- Ran GVO
Change-Id: Ib624201b683966a9feefb8fe45985f3d52d869fc
Reviewed-on: http://gerrit.cloudera.org:8080/20394
Tested-by: Impala Public Jenkins
Reviewed-by: Riza Suminto
Reviewed-by: Michael Smith
---
M be/src/rpc/thrift-util.cc
M be/src/service/client-request-state.cc
2 files changed, 10 insertions(+), 5 deletions(-)
Approvals:
Impala Public Jenkins: Verified
Riza Suminto: Looks good to me, approved
Michael Smith: Looks good to me, but someone else must approve
--
To view, visit http://gerrit.cloudera.org:8080/20394
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib624201b683966a9feefb8fe45985f3d52d869fc
Gerrit-Change-Number: 20394
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Smith
Gerrit-Reviewer: Riza Suminto
[Impala-ASF-CR] IMPALA-12366: Use 2GB as the default for thrift rpc max message size
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/20394 ) Change subject: IMPALA-12366: Use 2GB as the default for thrift_rpc_max_message_size .. Patch Set 2: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/20394 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib624201b683966a9feefb8fe45985f3d52d869fc Gerrit-Change-Number: 20394 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Thu, 24 Aug 2023 16:25:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12366: Use 2GB as the default for thrift rpc max message size
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/20394 ) Change subject: IMPALA-12366: Use 2GB as the default for thrift_rpc_max_message_size .. Patch Set 2: Code-Review+2 Looks good to me. Thank you for watching over this. -- To view, visit http://gerrit.cloudera.org:8080/20394 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib624201b683966a9feefb8fe45985f3d52d869fc Gerrit-Change-Number: 20394 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Riza Suminto Gerrit-Comment-Date: Thu, 24 Aug 2023 16:20:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12366: Use 2GB as the default for thrift rpc max message size
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20394 ) Change subject: IMPALA-12366: Use 2GB as the default for thrift_rpc_max_message_size .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/13828/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/20394 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib624201b683966a9feefb8fe45985f3d52d869fc Gerrit-Change-Number: 20394 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 24 Aug 2023 08:06:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12366: Use 2GB as the default for thrift rpc max message size
Joe McDonnell has uploaded this change for review. (
http://gerrit.cloudera.org:8080/20394
Change subject: IMPALA-12366: Use 2GB as the default for
thrift_rpc_max_message_size
..
IMPALA-12366: Use 2GB as the default for thrift_rpc_max_message_size
Thrift 0.16 implemented a limit on the max message size. In IMPALA-11669,
we added the thrift_rpc_max_message_size parameter and set the default
size to 1GB. Some existing clusters have needed to tune this parameter
higher because their workloads use message sizes larger than 1GB (e.g.
for metadata updates).
Historically, Impala has been able to send and receive 2GB messages,
so this changes the default value for thrift_rpc_max_message_size
to 2GB (INT_MAX). This can be reduced in future when Impala can guarantee
that messages work properly when split up into smaller batches.
TestGracefulShutdown::test_shutdown_idle started failing with this
change, because it is producing a different error message for one
of the negative tests. ClientRequestState::ExecShutdownRequest()
appends some extra explanation when it sees a "Network error" KRPC error,
and the test expects that extra explanation. This modifies
ClientRequestState::ExecShutdownRequest() to provide the extra explanation
for the new error ("Timed out") as well.
Testing:
- Ran GVO
Change-Id: Ib624201b683966a9feefb8fe45985f3d52d869fc
---
M be/src/rpc/thrift-util.cc
M be/src/service/client-request-state.cc
2 files changed, 10 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/20394/2
--
To view, visit http://gerrit.cloudera.org:8080/20394
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib624201b683966a9feefb8fe45985f3d52d869fc
Gerrit-Change-Number: 20394
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell
Gerrit-Reviewer: Impala Public Jenkins
