SteNicholas opened a new pull request #679: HBASE-23113 IPC Netty Optimization URL: https://github.com/apache/hbase/pull/679 Netty options in IPC Server/Client optimization: 1.SO_BACKLOG setting:Two queues are maintained in the Linux system kernel: syns queue and accept queue. The first is a semi-join queue that saves the connections to the synrecv state after receiving the client syn. The default netty is 128,io.netty.util.NetUtil#SOMAXCONN , and then read /proc/sys/net/core /somaxconn to continue to determine, and then there are some system level coverage logic.In some scenarios, if the client is far redundant to the server and the connection is established, it may not be enough. This value should not be too large, otherwise it will not prevent SYN-Flood attacks. The current value has been changed to 1024. After setting, the value set by yourself is equivalent to setting the upper limit because of the setting of the system and the size of the system. If some settings of the Linux system operation and maintenance are wrong, it can be avoided at the code level.At present, our Linux level is usually set to 128, and the final calculation will be set to 128. 2.WRITE_BUFFER_WATER_MARK setting:After WRITEBUFFERWATERMARK sets the maximum and minimum Buffer that can be temporarily stored on a connection, isWritable returns unwritable if the amount of data waiting to be sent for the connection is greater than the set value. In this way, the client can no longer send, preventing this amount of continuous backlog, and eventually the client may hang. If this happens, it is usually caused by slow processing on the server side. This value can effectively protect the client. At this point the data was not sent. 3.SO_REUSEADDR - Port multiplexing (allowing multiple sockets to listen on the same IP+ port): For time-wait links, it ensures that the server restarts successfully. In the case where some servers start up very quickly, it can prevent startup failure. Netty decoder in IPC Server optimization: Netty provides a convenient decoding tool class ByteToMessageDecoder, as shown in the top half of the figure, this class has accumulate bulk unpacking capability, can read bytes from the socket as much as possible, and then synchronously call the decode method to decode the business object. And compose a List. Finally, the traversal traverses the List and submits it to ChannelPipeline for processing. Here we made a small change, as shown in the bottom half of the figure, the content to be submitted is changed from a single command to the entire List, which reduces the number of pipeline executions and improves throughput. This mode has no advantage in low-concurrency scenarios, and has a significant performance boost in boost throughput in high-concurrency scenarios.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
