[ https://issues.apache.org/jira/browse/OAK-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Dulceanu updated OAK-7262: --------------------------------- Description: With OAK-4732, we slightly prioritised reads, allowing callers of {{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time on the commit semaphore, in order to fetch the latest head. The amount of time to wait for was computed as the median of the latest 1000 commits, but the implementation was flexible enough to replace the 0.5 quantile with a different value. The initial implementation used {{SynchronizedDescriptiveStatistics}} from {{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} backed by a {{SlindingWindowReservoir}} from the dropwizard metric library ({{io.dropwizard.metrics}}). One of the problems observed with the current implementation is that, under heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}} decreases due to lock contention in {{commitTimeHistogram}} implementation. Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It copies an array of 1000 elements at each invocation of {{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for each element in the array. was: With OAK-4732, we slightly prioritised reads, allowing callers of {{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time on the commit semaphore, in order to fetch the latest head. The amount of time to wait for was computed as the median of the latest 1000 commits, but the implementation was flexible enough to replace the 0.5 quantile with a different value. The initial implementation used {{SynchronizedDescriptiveStatistics}} from {{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} from the dropwizard metric library (\{{io.dropwizard.metrics}}). One of the problems observed with the current implementation is that, under heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}} decreases due to lock contention in {{commitTimeHistogram}} implementation. Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It copies an array of 1000 elements at each invocation of {{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for each element in the array. > LockBasedScheduler#getHeadNodeState poor performance due to lock contention > in commitTimeHistogram implementation > ----------------------------------------------------------------------------------------------------------------- > > Key: OAK-7262 > URL: https://issues.apache.org/jira/browse/OAK-7262 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar > Reporter: Andrei Dulceanu > Assignee: Andrei Dulceanu > Priority: Major > Fix For: 1.9.0, 1.10 > > > With OAK-4732, we slightly prioritised reads, allowing callers of > {{LockBasedScheduler#getHeadNodeState}} to wait for a certain amount of time > on the commit semaphore, in order to fetch the latest head. The amount of > time to wait for was computed as the median of the latest 1000 commits, but > the implementation was flexible enough to replace the 0.5 quantile with a > different value. > The initial implementation used {{SynchronizedDescriptiveStatistics}} from > {{commons-math3}}. In OAK-6430, we replaced that with a {{Histogram}} backed > by a {{SlindingWindowReservoir}} from the dropwizard metric library > ({{io.dropwizard.metrics}}). > One of the problems observed with the current implementation is that, under > heavy loading, the performance of {{LockBasedScheduler#getHeadNodeState}} > decreases due to lock contention in {{commitTimeHistogram}} implementation. > Namely, {{SlidingWindowReservoir#getSnapshot}} seems to be a bottleneck. It > copies an array of 1000 elements at each invocation of > {{LockBasedScheduler#getHeadNodeState}}, acquiring and releasing a lock for > each element in the array. -- This message was sent by Atlassian JIRA (v7.6.3#76005)