[jira] [Commented] (OAK-4097) Add metric for FileStore journal writes
[ https://issues.apache.org/jira/browse/OAK-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15411963#comment-15411963 ] Michael Dürig commented on OAK-4097: Thanks for the patch, good catch! I applied it at http://svn.apache.org/viewvc?rev=1755514&view=rev Re. the integration test: I'm a bit concerned about the dependency on timing. These are the kind of tests prone to failure on CIs. Could it be done without depending on timings? I.e. disable the flush thread and invoke flush manually a couple of times? > Add metric for FileStore journal writes > --- > > Key: OAK-4097 > URL: https://issues.apache.org/jira/browse/OAK-4097 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Chetan Mehrotra >Assignee: Andrei Dulceanu >Priority: Minor > Fix For: Segment Tar 0.0.10 > > Attachments: OAK-4097-01.patch, OAK-4097-02.patch > > > TarMK flush thread should run every 5 secs and flush the current root head to > journal.log. It would be good to have a metric to capture the number of runs > per minute > This would help in confirming if flush is working at expected frequency or > delay in acquiring locks is causing some delays -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4097) Add metric for FileStore journal writes
[ https://issues.apache.org/jira/browse/OAK-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409349#comment-15409349 ] Andrei Dulceanu commented on OAK-4097: -- [~chetanm] Sure, I will think about/create a test and attach it to the issue when it's ready. > Add metric for FileStore journal writes > --- > > Key: OAK-4097 > URL: https://issues.apache.org/jira/browse/OAK-4097 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Chetan Mehrotra >Assignee: Andrei Dulceanu >Priority: Minor > Fix For: Segment Tar 0.0.10 > > Attachments: OAK-4097-01.patch > > > TarMK flush thread should run every 5 secs and flush the current root head to > journal.log. It would be good to have a metric to capture the number of runs > per minute > This would help in confirming if flush is working at expected frequency or > delay in acquiring locks is causing some delays -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4097) Add metric for FileStore journal writes
[ https://issues.apache.org/jira/browse/OAK-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409327#comment-15409327 ] Chetan Mehrotra commented on OAK-4097: -- Ack. Applied the patch in 1755288 > Add metric for FileStore journal writes > --- > > Key: OAK-4097 > URL: https://issues.apache.org/jira/browse/OAK-4097 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Chetan Mehrotra >Assignee: Andrei Dulceanu >Priority: Minor > Fix For: Segment Tar 0.0.10 > > Attachments: OAK-4097-01.patch > > > TarMK flush thread should run every 5 secs and flush the current root head to > journal.log. It would be good to have a metric to capture the number of runs > per minute > This would help in confirming if flush is working at expected frequency or > delay in acquiring locks is causing some delays -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4097) Add metric for FileStore journal writes
[ https://issues.apache.org/jira/browse/OAK-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409323#comment-15409323 ] Michael Dürig commented on OAK-4097: [~chetanm], agreed with the test concern. But let's apply the patch anyway and keep the issue open until we have a test (if feasible) to avoid the patch getting stale. > Add metric for FileStore journal writes > --- > > Key: OAK-4097 > URL: https://issues.apache.org/jira/browse/OAK-4097 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Chetan Mehrotra >Assignee: Andrei Dulceanu >Priority: Minor > Fix For: Segment Tar 0.0.10 > > Attachments: OAK-4097-01.patch > > > TarMK flush thread should run every 5 secs and flush the current root head to > journal.log. It would be good to have a metric to capture the number of runs > per minute > This would help in confirming if flush is working at expected frequency or > delay in acquiring locks is causing some delays -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4097) Add metric for FileStore journal writes
[ https://issues.apache.org/jira/browse/OAK-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409318#comment-15409318 ] Chetan Mehrotra commented on OAK-4097: -- [~dulceanu] Would it be possible to also add a test for this in {{FileStoreStatsTest}} such that future refactorings does not break the stats collection. If it turns out to be tricky then we can just apply the current patch. > Add metric for FileStore journal writes > --- > > Key: OAK-4097 > URL: https://issues.apache.org/jira/browse/OAK-4097 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Chetan Mehrotra >Assignee: Andrei Dulceanu >Priority: Minor > Fix For: Segment Tar 0.0.10 > > Attachments: OAK-4097-01.patch > > > TarMK flush thread should run every 5 secs and flush the current root head to > journal.log. It would be good to have a metric to capture the number of runs > per minute > This would help in confirming if flush is working at expected frequency or > delay in acquiring locks is causing some delays -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4097) Add metric for FileStore journal writes
[ https://issues.apache.org/jira/browse/OAK-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15406126#comment-15406126 ] Michael Dürig commented on OAK-4097: +1 for the patch from my side. [~chetanm], could you give the metrics API specific parts a quick review. Feel free to apply it. > Add metric for FileStore journal writes > --- > > Key: OAK-4097 > URL: https://issues.apache.org/jira/browse/OAK-4097 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Chetan Mehrotra >Assignee: Andrei Dulceanu >Priority: Minor > Fix For: Segment Tar 1.0.0 > > Attachments: OAK-4097-01.patch > > > TarMK flush thread should run every 5 secs and flush the current root head to > journal.log. It would be good to have a metric to capture the number of runs > per minute > This would help in confirming if flush is working at expected frequency or > delay in acquiring locks is causing some delays -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4097) Add metric for FileStore journal writes
[ https://issues.apache.org/jira/browse/OAK-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383852#comment-15383852 ] Michael Dürig commented on OAK-4097: I would actually prefer to loop this into {{FileStore.flush()}}. As {{FileStore}} already has a {{FileStoreMonitor}} member there is no need to track another dependency. Also this would cover all flushes of the journal either explicit and implicit by the flush thread. And it does not cover flushes of the segment buffers alone (i.e. without the journal). > Add metric for FileStore journal writes > --- > > Key: OAK-4097 > URL: https://issues.apache.org/jira/browse/OAK-4097 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Chetan Mehrotra >Assignee: Andrei Dulceanu >Priority: Minor > Fix For: 1.6, Segment Tar 0.0.6 > > > TarMK flush thread should run every 5 secs and flush the current root head to > journal.log. It would be good to have a metric to capture the number of runs > per minute > This would help in confirming if flush is working at expected frequency or > delay in acquiring locks is causing some delays -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4097) Add metric for FileStore journal writes
[ https://issues.apache.org/jira/browse/OAK-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383749#comment-15383749 ] Andrei Dulceanu commented on OAK-4097: -- I'd like some advice regarding how to approach this issue. One way would be to add a {{{FileStoreMonitor}}} instance variable in {{{TarRevisions}}} and have a new {{{FileStoreMonitor::flushed()}}} method being called at each {{{TarRevisions::flush()}}} invocation. IMO this would satisfy the requirement, since {{{TarRevisions::flush()}}} is the only place where a journal write happens. On the other hand, as suggested by [~frm], we could intercept calls to {{{org.apache.jackrabbit.oak.segment.WriteOperationHandler::flush()}}} "since this method guarantees that pending changes in in-memory segments are persisted on disk via a write." I think this is a more general approach, but since {{{WriteOperationHandler::flush()}}} is called in {{{SegmentWriter::flush()}}} which in turn is called in numerous places (including {{{FileStore::flush()}}} which causes also the journal write), without necessarily modifying the journal file, I'd assume the metric collected would be one related to {{{FileStore}}} writes, not to journal writes. Any ideas on how to move this forward? > Add metric for FileStore journal writes > --- > > Key: OAK-4097 > URL: https://issues.apache.org/jira/browse/OAK-4097 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Chetan Mehrotra >Assignee: Andrei Dulceanu >Priority: Minor > Fix For: 1.6, Segment Tar 0.0.6 > > > TarMK flush thread should run every 5 secs and flush the current root head to > journal.log. It would be good to have a metric to capture the number of runs > per minute > This would help in confirming if flush is working at expected frequency or > delay in acquiring locks is causing some delays -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4097) Add metric for FileStore journal writes
[ https://issues.apache.org/jira/browse/OAK-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15185546#comment-15185546 ] Michael Dürig commented on OAK-4097: My preferred solution would be OAK-4103. Until then I'm fine with such metric though. > Add metric for FileStore journal writes > --- > > Key: OAK-4097 > URL: https://issues.apache.org/jira/browse/OAK-4097 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segmentmk >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.6 > > > TarMK flush thread should run every 5 secs and flush the current root head to > journal.log. It would be good to have a metric to capture the number of runs > per minute > This would help in confirming if flush is working at expected frequency or > delay in acquiring locks is causing some delays -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4097) Add metric for FileStore journal writes
[ https://issues.apache.org/jira/browse/OAK-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184470#comment-15184470 ] Chetan Mehrotra commented on OAK-4097: -- Such a metric can easily be hooked into {{org.apache.jackrabbit.oak.plugins.segment.file.BackgroundThread}} which already collects the timing data. Just need to back them via a metric > Add metric for FileStore journal writes > --- > > Key: OAK-4097 > URL: https://issues.apache.org/jira/browse/OAK-4097 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segmentmk >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.6 > > > TarMK flush thread should run every 5 secs and flush the current root head to > journal.log. It would be good to have a metric to capture the number of runs > per minute > This would help in confirming if flush is working at expected frequency or > delay in acquiring locks is causing some delays -- This message was sent by Atlassian JIRA (v6.3.4#6332)