Re: [DISCUSS] Which I/O statistics should the FileStore expose?

2017-02-14 Thread Michael Tarantino
> It is also unclear to me if it's really appropriate to digest some > statistics before exposing them to the outside world. In example, how > the methods in DocumentStoreStats returning time series as > CompositeData play with other monitoring solutions like > ElasticSearch/Kibana? How easy is to

Re: [DISCUSS] Which I/O statistics should the FileStore expose?

2017-02-14 Thread Michael Dürig
On 13.2.17 6:21 , Francesco Mari wrote: Is it appropriate for IOMonitorMBean to perform some kind of aggregation, like sum and average? Should richer data be returned from the MBean, like tabular data? IMO aggregation should in general be left to the monitoring infrastructure. Our MBeans sho

Re: [DISCUSS] Which I/O statistics should the FileStore expose?

2017-02-14 Thread Marcel Reutegger
On 14/02/17 08:45, Francesco Mari wrote: What could be gained by adding Metrics to the trivial implementation of IOMonitorMBean described above? Primarily ease of use or ease of integration with a monitoring tool. An operations team only needs to integrate one type of MBeans (Metrics) into the

Re: [DISCUSS] Which I/O statistics should the FileStore expose?

2017-02-14 Thread Francesco Mari
2017-02-14 9:15 GMT+01:00 Chetan Mehrotra : > On Tue, Feb 14, 2017 at 1:15 PM, Francesco Mari > wrote: >> What could be gained >> by adding Metrics to the trivial implementation of IOMonitorMBean >> described above? > > The metrics created here are automatically registered in JMX (see > MetricStat

Re: [DISCUSS] Which I/O statistics should the FileStore expose?

2017-02-14 Thread Chetan Mehrotra
On Tue, Feb 14, 2017 at 1:15 PM, Francesco Mari wrote: > What could be gained > by adding Metrics to the trivial implementation of IOMonitorMBean > described above? The metrics created here are automatically registered in JMX (see MetricStatisticsProvider) and also accessible over web ui for exa

Re: [DISCUSS] Which I/O statistics should the FileStore expose?

2017-02-13 Thread Francesco Mari
Thanks Chetan and Julian for the pointers. On one hand I would like to do something similar - in spirit - to DocumentStoreStats. On the other hand, I don't see the advantage of the Metrics framework over a hand-rolled IOMonitorMBean with two AtomicLong fields and two volatile variables. IIUC there

Re: [DISCUSS] Which I/O statistics should the FileStore expose?

2017-02-13 Thread Chetan Mehrotra
Hi Francesco, As Julian mentioned it would be good to collects stats as Metrics. Have a look at DocumentStoreStats which collects some stats around operations being performed by DocumentStore implementations Chetan Mehrotra On Tue, Feb 14, 2017 at 12:37 AM, Julian Sedding wrote: > Hi Francesco

Re: [DISCUSS] Which I/O statistics should the FileStore expose?

2017-02-13 Thread Julian Sedding
Hi Francesco I believe you should implement an IOMonitor using the metrics in the org.apache.jackrabbit.oak.stats package. These can be backed by swappable StatisticsProvider implementations. I believe by default it's a NOOP implementation. However, I believe that if the MetricStatisticsProvider i

[DISCUSS] Which I/O statistics should the FileStore expose?

2017-02-13 Thread Francesco Mari
Hi all, The recently introduced IOMonitor allows the FileStore to trigger I/O events. Callback methods from IOMonitor can be implemented to receive information about segment reads and writes. A trivial implementation of IOMonitor is able to track the following raw data. - The number of segments