Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21420
Change subject: IMPALA-13020 (part 2): Split out external vs internal Thrift max message size ...................................................................... IMPALA-13020 (part 2): Split out external vs internal Thrift max message size The Thrift max message size is designed to protect against malicious messages that consume a lot of memory on the receiver. This is an important security measure for externally facing services, but it can interfere with internal communication within the cluster. Currently, the max message size is controlled by a single startup flag for both. This puts tensions between having a low value to protect against malicious messages versus having a high value to avoid issues with internal communication (e.g. large statestore updates). This introduces a new flag thrift_external_rpc_max_message_size to specify the limit for externally-facing services. The current thrift_rpc_max_message_size now applies only for internal services. Splitting them apart allows setting a much higher value for internal services (64GB) while leaving the externally facing services using the current 2GB limit. This modifies various code locations that wrap a Thrift transport to pass in the original transport's TConfiguration. This also adds DCHECKs to make sure that the new transport inherits the max message size. This limits the locations where we actually need to set max message size. ThriftServer/ThriftServerBuilder have a setting "is_external_facing" which can be specified on each ThriftServer. This modifies statestore and catalog to set is_external_facing to false. All other servers stay with the default of true. Testing: - This adds a test case to verify that is_external_facing uses the higher limit. Change-Id: Ib9a649ef49a8a99c7bd9a1b73c37c4c621661311 --- M be/src/catalog/catalogd-main.cc M be/src/common/init.cc M be/src/rpc/TAcceptQueueServer.cpp M be/src/rpc/TAcceptQueueServer.h M be/src/rpc/authentication.cc M be/src/rpc/thrift-client.cc M be/src/rpc/thrift-server-test.cc M be/src/rpc/thrift-server.cc M be/src/rpc/thrift-server.h M be/src/rpc/thrift-util.cc M be/src/rpc/thrift-util.h M be/src/statestore/statestore-subscriber.cc M be/src/statestore/statestore.cc M be/src/transport/THttpTransport.cpp M be/src/transport/TSaslServerTransport.cpp M be/src/transport/TSaslTransport.cpp M be/src/util/parquet-reader.cc 17 files changed, 157 insertions(+), 49 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/21420/1 -- To view, visit http://gerrit.cloudera.org:8080/21420 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib9a649ef49a8a99c7bd9a1b73c37c4c621661311 Gerrit-Change-Number: 21420 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell <joemcdonn...@cloudera.com>
