[
https://issues.apache.org/jira/browse/ACCUMULO-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Park updated ACCUMULO-2889:
------------------------------------
Affects Version/s: 1.6.0
Status: Patch Available (was: In Progress)
First pass at batching metadata updates for new WALs. I'll attach a screenshot
of its affects as well.
> Batch metadata table updates for new walogs
> -------------------------------------------
>
> Key: ACCUMULO-2889
> URL: https://issues.apache.org/jira/browse/ACCUMULO-2889
> Project: Accumulo
> Issue Type: Bug
> Affects Versions: 1.6.0, 1.5.1
> Reporter: Jonathan Park
> Assignee: Jonathan Park
> Attachments: ACCUMULO-2889.0.patch.txt
>
>
> Currently, when we update the Metadata table with new loggers, we will update
> the metadata for each tablet serially. We could optimize this to instead use
> a batchwriter to send all metadata updates for all tablets in a batch.
> A few special cases include:
> - What if the !METADATA tablet was included in the batch?
> - What about the root tablet?
> Benefit:
> In one of our clusters, we're experiencing particularly slow HDFS operations
> leading to large oscillations in ingest performance. We haven't isolated the
> cause in HDFS but when we profile the tservers, we noticed that they were
> waiting for metadata table operations to complete. This would target the
> waiting.
> Potential downsides:
> Given the existing locking scheme, it looks like we may have to lock a tablet
> for slightly longer (we'll lock for the duration of the batch).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
