[
https://issues.apache.org/jira/browse/RATIS-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz-wo Sze resolved RATIS-2093.
-------------------------------
Fix Version/s: 3.1.0
Resolution: Fixed
The pull request is now merged. Thanks, [~duongnguyen]!
> Decouple metadata and configuration entries from appendEntries buffer for
> stateMachineCache
> -------------------------------------------------------------------------------------------
>
> Key: RATIS-2093
> URL: https://issues.apache.org/jira/browse/RATIS-2093
> Project: Ratis
> Issue Type: Sub-task
> Reporter: Duong
> Assignee: Duong
> Priority: Major
> Fix For: 3.1.0
>
> Attachments: cache_reference.png,
> direct_mem_util_before_and_after.png, unreleased_messages_before_and_after.png
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When testing zero-copy in Ozone (stateMachineCache enabled), we saw hundreds
> of thousands of ServerProtocol messages trapped unclosed although the number
> of entries cached in Ozone StateMachine was small (<500). Also, the
> utilization of direct memory by Netty is high and doesn't go down after the
> test run is done.
>
> Turns out, an appendEntries request can contain multiple log entries. Some of
> them can be metadata or configuration entries whose size is small (~10-20
> bytes). Some of them can be StateMachine entries whose size is much bigger
> (4mb).
> Today, when stateMachineCache is enabled, the StateMachine entities stored in
> LogCache don't have a reference count to the original appendEntries, but
> metadata and configuration entries do. Because the size of metadata and
> configuration is small, they will almost never fill up the LogCache to
> trigger a cacheEvict. Their references to the original appendEntries request
> prevent the request buffer from being released when StateMachine cache evicts
> the StateMachine entries.
> !cache_reference.png|width=496,height=310!
>
> When stateMachineCache enabled, the metadata and config entries should not
> hold a reference to the original appendEntries.
> I did a quick test and compared the direct mem util and number of unreleased
> message before and after making the change.
> !direct_mem_util_before_and_after.png|width=545,height=251!
> !unreleased_messages_before_and_after.png|width=548,height=254!
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
