[ https://issues.apache.org/jira/browse/KUDU-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393584#comment-16393584 ]
Michael Ho edited comment on KUDU-2305 at 3/9/18 11:19 PM: ----------------------------------------------------------- It appears that there are still few places in the code which may overflow if the combined size of the entire message exceeds 2GB. {{addition_size}} in {{SerializeMessage()}} is a 32-bit signed integer. However, the caller in {{OutboundCall::SetRequestPayload()}} may be passing a value larger than UINT_MAX. In which case, we will be truncating the value., leading to incorrect {{total_size}} being computed in {{SerializeMessage()}} {noformat} void SerializeMessage(const MessageLite& message, faststring* param_buf, int additional_size, bool use_cached_size) { {noformat} Along the same line, even when the {{total_size}} is larger than {{FLAGS_max_message_size}}, it appears that the client may keep sending the payload anyway and the receiver will reject it in {{InboundTransfer::ReceiveBuffer(}} path. It's possible that the sender may try to send the entire message before the receiver gets a chance to reject it and close the connection. In that sense, it's possible for {{total_size}} of a message to be larger than INT_MAX and some local variables in {{OutboundTransfer::SendBuffer}}, {{TlsSocket::Writev()}}, {{Socket::Writev()}} may overflow too. So, we may need to fix those places too. was (Author: kwho): It appears that there are still few places in the code which may overflow if the combined size of the entire message exceeds 2GB. {{addition_size}} in {{SerializeMessage()}} is a 32-bit signed integer. However, the caller in {{OutboundCall::SetRequestPayload()}} may be passing a value larger than INT_MAX. In which case, we will be truncating the value., leading to incorrect {{total_size}} being computed in {{SerializeMessage()}} {noformat} void SerializeMessage(const MessageLite& message, faststring* param_buf, int additional_size, bool use_cached_size) { {noformat} Along the same line, even when the {{total_size}} is larger than {{FLAGS_max_message_size}}, it appears that the client may keep sending the payload anyway and the receiver will reject it in {{InboundTransfer::ReceiveBuffer(}} path. It's possible that the sender may try to send the entire message before the receiver gets a chance to reject it and close the connection. In that sense, it's possible for {{total_size}} of a message to be larger than INT_MAX and some local variables in {{OutboundTransfer::SendBuffer}}, {{TlsSocket::Writev()}}, {{Socket::Writev()}} may overflow too. So, we may need to fix those places too. > Local variables can overflow when serializing a 2GB message > ----------------------------------------------------------- > > Key: KUDU-2305 > URL: https://issues.apache.org/jira/browse/KUDU-2305 > Project: Kudu > Issue Type: Bug > Components: rpc > Affects Versions: 1.6.0 > Reporter: Joe McDonnell > Assignee: Joe McDonnell > Priority: Major > Fix For: 1.7.0 > > > When rpc_max_message_size is set to its maximum of INT_MAX (2147483647), > certain local variables in SerializeMessage can overflow as messages approach > this size. Specifically, recorded_size, size_with_delim, and total_size are 4 > byte signed integers and could overflow when additional_size becomes large. > Since INT_MAX is the largest allowable value for rpc_max_message_size (a 4 > byte signed integer), these variables will not overflow if changed to 4 byte > unsigned integers. This would eliminate the potential problem for > serialization. > A similar problem exists in the InboundTransfer::ReceiveBuffer() and similar > codepaths. Changing those variables to unsigned integers should resolve the > issue. > This does not impact existing systems, because the default value of > rpc_max_message_size is 50MB. -- This message was sent by Atlassian JIRA (v7.6.3#76005)