[
https://issues.apache.org/jira/browse/HDDS-2309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17142316#comment-17142316
]
Bharat Viswanadham edited comment on HDDS-2309 at 6/22/20, 6:06 PM:
--------------------------------------------------------------------
HDDS-3615 is taking care of calling cleanup cache only for tables that are
affected by the double buffer entries.
And regarding the other point, as clients increases, we will see the batching
increasing. As right now the logic is, if we have atleast one entry in double
buffer, background flush threads pick them, during this time all the
transactions are added to current buffer. Currently, this is by design. Need to
revisit this if it is causing any issues.
was (Author: bharatviswa):
HDDS-3615 is taking care of calling cleanup cache only for tables that are
affected by the double buffer entries.
And regarding the other point, as clients increases, we will see the batching
increasing. As right now the logic is, if we have atleast one entry in double
buffer, background flush threads pick them, during this time all the
transactions are added to current buffer. Currently, this is by design.
> Optimise OzoneManagerDoubleBuffer::flushTransactions to flush in batches
> ------------------------------------------------------------------------
>
> Key: HDDS-2309
> URL: https://issues.apache.org/jira/browse/HDDS-2309
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Manager
> Reporter: Rajesh Balamohan
> Assignee: Bharat Viswanadham
> Priority: Major
> Labels: TriagePending, performance
> Attachments: Screenshot 2019-10-15 at 4.19.13 PM.png
>
>
> When running a write heavy benchmark,
> {{{color:#000000}org/apache/hadoop/ozone/om/ratis/OzoneManagerDoubleBuffer.flushTransactions{color}}}
> was invoked for pretty much every write.
> This forces {{cleanupCache}} to be invoked which ends up choking in single
> thread executor. Attaching the profiler information which gives more details.
> Ideally, {{flushTransactions}} should batch up the work to reduce load on
> rocksDB.
>
> [https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerDoubleBuffer.java#L130]
>
> [https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerDoubleBuffer.java#L322]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
