Tian Jiang created IOTDB-1856: --------------------------------- Summary: Improve raft log size check Key: IOTDB-1856 URL: https://issues.apache.org/jira/browse/IOTDB-1856 Project: Apache IoTDB Issue Type: Improvement Components: Core/Cluster Reporter: Tian Jiang Attachments: image-2021-10-18-09-51-15-583.png
To avoid log-append failures caused by oversized log entries, the size of each entry is checked to ensure that it will fit the flush buffer before it is appended to the local log. However, the current implementation has two drawbacks: # The size check falls in a synchronized block, which means concurrent requests cannot be checked in parallel, and other requests will be significantly blocked if the request is rather large. # The size check directly serializes an entry into a ByteBuffer and uses it to get the size of the entry, but the buffer is discarded immediately after the check, so both CPU computation and memory allocation are wasted during the serialization are wasted. !image-2021-10-18-09-51-15-583.png|thumbnail! To resolve the two aforementioned problems, we need: # Move the size check out of the synchronized block. # Use a serialization-free way to get the size of an entry, or save the serialization result and reuse it during log persistency and log replication. If the second one is adopted, the serialization result should be freed in time to avoid a large memory footprint. -- This message was sent by Atlassian Jira (v8.3.4#803005)