[ https://issues.apache.org/jira/browse/IMPALA-13202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joe McDonnell updated IMPALA-13202: ----------------------------------- Summary: Impala workloads can exceed Kudu client's rpc_max_message_size limit (was: KRPC flags used by libkudu_client.so can't be configured) > Impala workloads can exceed Kudu client's rpc_max_message_size limit > -------------------------------------------------------------------- > > Key: IMPALA-13202 > URL: https://issues.apache.org/jira/browse/IMPALA-13202 > Project: IMPALA > Issue Type: Bug > Components: Backend > Reporter: Quanlong Huang > Priority: Critical > Attachments: data.parquet > > > The way Impala integrates with KRPC is porting the KRPC codes into the Impala > code base. Flags and methods of KRPC are defined as GLOBAL in the impalad > executable. libkudu_client.so also compiles from the same KRPC codes and have > duplicate flags and methods defined as HIDDEN. > To be specifit, both the impalad executable and libkudu_client.so have the > symbol for kudu::rpc::InboundTransfer::ReceiveBuffer() > {noformat} > $ readelf -s --wide be/build/latest/service/impalad | grep ReceiveBuffer > 11118: 00000000022f5c88 1936 FUNC GLOBAL DEFAULT 13 > _ZN4kudu3rpc15InboundTransfer13ReceiveBufferEPNS_6SocketEPNS_10faststringE > 81380: 00000000022f5c88 1936 FUNC GLOBAL DEFAULT 13 > _ZN4kudu3rpc15InboundTransfer13ReceiveBufferEPNS_6SocketEPNS_10faststringE > $ readelf -s --wide > toolchain/toolchain-packages-gcc10.4.0/kudu-e742f86f6d/debug/lib/libkudu_client.so > | grep ReceiveBuffer > 1601: 0000000000086e4a 108 FUNC LOCAL DEFAULT 12 > _ZN4kudu3rpc15InboundTransfer13ReceiveBufferEPNS_6SocketEPNS_10faststringE.cold > 11905: 00000000001fec60 2076 FUNC LOCAL HIDDEN 12 > _ZN4kudu3rpc15InboundTransfer13ReceiveBufferEPNS_6SocketEPNS_10faststringE > $ c++filt > _ZN4kudu3rpc15InboundTransfer13ReceiveBufferEPNS_6SocketEPNS_10faststringE > kudu::rpc::InboundTransfer::ReceiveBuffer(kudu::Socket*, kudu::faststring*) > {noformat} > KRPC flags like rpc_max_message_size are also defined in both the impalad > executable and libkudu_client.so: > {noformat} > $ readelf -s --wide be/build/latest/service/impalad | grep > FLAGS_rpc_max_message_size > 14380: 0000000006006738 8 OBJECT GLOBAL DEFAULT 30 > _ZN5fLI6426FLAGS_rpc_max_message_sizeE > 80396: 0000000006006741 1 OBJECT GLOBAL DEFAULT 30 > _ZN3fLB44FLAGS_rpc_max_message_size_enable_validationE > 81399: 0000000006006741 1 OBJECT GLOBAL DEFAULT 30 > _ZN3fLB44FLAGS_rpc_max_message_size_enable_validationE > 117873: 0000000006006738 8 OBJECT GLOBAL DEFAULT 30 > _ZN5fLI6426FLAGS_rpc_max_message_sizeE > $ readelf -s --wide > toolchain/toolchain-packages-gcc10.4.0/kudu-e742f86f6d/debug/lib/libkudu_client.so > | grep FLAGS_rpc_max_message_size > 11882: 00000000008d61e1 1 OBJECT LOCAL HIDDEN 27 > _ZN3fLB44FLAGS_rpc_max_message_size_enable_validationE > 11906: 00000000008d61d8 8 OBJECT LOCAL DEFAULT 27 > _ZN5fLI6426FLAGS_rpc_max_message_sizeE > $ c++filt _ZN5fLI6426FLAGS_rpc_max_message_sizeE > fLI64::FLAGS_rpc_max_message_size {noformat} > libkudu_client.so uses its own methods and flags. The flags are HIDDEN so > can't be modified by Impala codes. E.g. IMPALA-4874 bumps > FLAGS_rpc_max_message_size to 2GB in RpcMgr::Init(), but the HIDDEN variable > FLAGS_rpc_max_message_size used in libkudu_client.so is still the default > value 50MB (52428800). We've seen error messages like this in the master > branch: > {code:java} > I0708 10:23:31.784974 2943 meta_cache.cc:294] > c243bda4702a5ab9:0ba93d2400000001] tablet 0c8f3446538449ee9d3df5056afe775e: > replica e0e1db54dab74f208e37ea1b975595e5 (127.0.0.1:31202) has failed: > Network error: TS failed: RPC frame had a length of 53477464, but we only > support messages up to 52428800 bytes long.{code} > CC [~joemcdonnell] [~wzhou] [~aserbin] -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org