[
https://issues.apache.org/jira/browse/OAK-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Dulceanu updated OAK-7262:
---------------------------------
Description:
With OAK-4732, we slightly prioritised reads, allowing callers of
{{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time on
the commit semaphore, in order to fetch the latest head. The amount of time to
wait for was computed as the median of the latest 1000 commits, but the
implementation was flexible enough to replace the 0.5 quantile with a different
value.
The initial implementation used {{SynchronizedDescriptiveStatistics}} from
{{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} backed by
a {{SlindingWindowReservoir}} from the dropwizard metric library
({{io.dropwizard.metrics}}).
One of the problems observed with the current implementation is that, under
heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}}
decreases due to lock contention in {{commitTimeHistogram}} implementation.
Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It
copies an array of 1000 elements at each invocation of
{{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for
each element in the array.
was:
With OAK-4732, we slightly prioritised reads, allowing callers of
{{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time on
the commit semaphore, in order to fetch the latest head. The amount of time to
wait for was computed as the median of the latest 1000 commits, but the
implementation was flexible enough to replace the 0.5 quantile with a different
value.
The initial implementation used {{SynchronizedDescriptiveStatistics}} from
{{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} from the
dropwizard metric library (\{{io.dropwizard.metrics}}).
One of the problems observed with the current implementation is that, under
heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}}
decreases due to lock contention in {{commitTimeHistogram}} implementation.
Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It
copies an array of 1000 elements at each invocation of
{{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for
each element in the array.
> LockBasedScheduler#getHeadNodeState poor performance due to lock contention
> in commitTimeHistogram implementation
> -----------------------------------------------------------------------------------------------------------------
>
> Key: OAK-7262
> URL: https://issues.apache.org/jira/browse/OAK-7262
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
> Reporter: Andrei Dulceanu
> Assignee: Andrei Dulceanu
> Priority: Major
> Fix For: 1.9.0, 1.10
>
>
> With OAK-4732, we slightly prioritised reads, allowing callers of
> {{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time
> on the commit semaphore, in order to fetch the latest head. The amount of
> time to wait for was computed as the median of the latest 1000 commits, but
> the implementation was flexible enough to replace the 0.5 quantile with a
> different value.
> The initial implementation used {{SynchronizedDescriptiveStatistics}} from
> {{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} backed
> by a {{SlindingWindowReservoir}} from the dropwizard metric library
> ({{io.dropwizard.metrics}}).
> One of the problems observed with the current implementation is that, under
> heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}}
> decreases due to lock contention in {{commitTimeHistogram}} implementation.
> Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It
> copies an array of 1000 elements at each invocation of
> {{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for
> each element in the array.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
