Sergey, thanks, now it looks more vivid.
> public interface PersistenceMemoryMetrics extends MemoryMetrics {
> public double getPageEvictionRate();
>
> public double getPageMissRate();
>
> public long getDirtyPages();
>
> public long getPagesOnDisk();
> }
What if we just add these 4 methods directly to MemoryMetrics interface?
Otherwise, we’ll have hard times describing the difference between all the
variety of metrics interfaces we have.
If the store is turned off the methods can throw a meaningful
IllegalStateException or return a negative value. I’m voting for the exception.
> public interface PersistenceStoreMetrics {
> //WAL-related metrics
> public double getWalLoggingRate();
>
> public int getWalArchiveSegments();
>
> public double getWalFsyncAverage();
>
> //checkpoint-related metrics
> public double getCpTime();
>
> public double getCpFsyncTime();
>
> public long getCpTotalPages();
>
> public long[] getCpPagesByType();
>
> public long getCpCopiedOnWritePages();
> }
I’m fine with these interface. Please do minor changes:
- rename Persistence to Persistent so that the final name is
“PersistentStoreMetrics”.
- I wouldn’t shorten the checkpointing related methods. Let’s write
“getCheckpointing…”
—
Denis
> On May 29, 2017, at 9:24 AM, Sergey Chugunov <sergey.chugu...@gmail.com>
> wrote:
>
> Hello Denis,
>
> I have an idea of how we could clarify proposed above approach to
> collecting metrics for PDS.
>
> I suggest to split all metrics into two major groups (disk-related and
> memory-related) and group them in the following interfaces:
>
> public interface PersistenceMemoryMetrics extends MemoryMetrics {
> public double getPageEvictionRate();
>
> public double getPageMissRate();
>
> public long getDirtyPages();
>
> public long getPagesOnDisk();
> }
>
> public interface PersistenceStoreMetrics {
> //WAL-related metrics
> public double getWalLoggingRate();
>
> public int getWalArchiveSegments();
>
> public double getWalFsyncAverage();
>
> //checkpoint-related metrics
> public double getCpTime();
>
> public double getCpFsyncTime();
>
> public long getCpTotalPages();
>
> public long[] getCpPagesByType();
>
> public long getCpCopiedOnWritePages();
> }
>
> From my standpoint it is important to distinguish between PDS metrics that
> are related mostly to memory management (from the first interface) and
> metrics bound to disk operations (from the second interface).
>
> First group of metrics is useful when collected per Memory Policy. It
> doesn't provide a lot of information when collected across many Memory
> Policies.
>
> Second group is now global (this may change in the future if Persistence
> Policies are introduced) and allows to easily spot any high-level issues
> with durable storage configuration or performance. More specific questions
> like "how this Memory Policy loads a durable storage subsystem?" may be
> deduced from PersistenceMemoryMetrics indicators.
>
> Denis, does it make sense for you?
>
> Any other ideas/questions/improvements or metrics to add to these
> interfaces?
>
> I would like to discuss all the details here before creating any tickets
> for the change.
>
> Thanks,
> Sergey.
>
>
> On Fri, May 19, 2017 at 9:26 PM, Denis Magda <dma...@apache.org> wrote:
>
>> Dmitriy,
>>
>> I have some high level concerns.
>>
>> Presently, Ignite already exposes memory/data related metrics via
>> MemoryMetrics and CacheMetrics interface.
>>
>> Looking at PersistentStoreMetrics interface I can’t get why methods like
>> getMemorySize(), getPagesInMemory(), getPagesSizeInMemory(), etc. are NOT
>> in MemoryMetrics interface.
>>
>> Next, I wouldn’t make out PersistentStore*Cache*Metrics interface that
>> contains a big subset of the methods from PersistentStoreMetric. There just
>> should be some API call that either returns global metrics or cache
>> specific ones. Plus, some of the methods might be added to existing
>> CacheInterface. Why not.
>>
>> I prefer to address this first. Guys, any other thoughts on this?
>>
>> —
>> Denis
>>
>>> On May 17, 2017, at 4:16 AM, Dmitriy Govorukhin <
>> dmitriy.govoruk...@gmail.com> wrote:
>>>
>>> Folk,
>>>
>>> As you know, ignite 2.1 will contain new module (pds), it will be
>>> provide ability to store data on disk. Let's discuss what type of
>>> metrics we need for this?
>>> I think it must be metrics per memory policy, per cache, checkpoint,
>>> and global metrics which will be aggregate all metrics.
>>>
>>> I did sketch.
>>>
>>> PersistentStoreMetrics.java
>>>
>>> public interface PersistentStoreMetrics {
>>>
>>> // Global metrics.
>>>
>>> public long getMemorySize();
>>>
>>> public long getDiskSize();
>>>
>>> public long getPagesInMemory();
>>>
>>> public long getPagesSizeInMemory();
>>>
>>> public long getPagesOnDisk();
>>>
>>> public long getPagesSizeOnDisk();
>>>
>>> public long getFreePages();
>>>
>>> public long getFreePagesSize();
>>>
>>> public long getDirtyPages();
>>>
>>> public long getDirtyPagesSize();
>>>
>>> public long walLog();
>>>
>>> public long walLogSize();
>>>
>>> // Frequency.
>>>
>>> public long getPagesRead();
>>>
>>> public long getPagesWrite();
>>>
>>> public long getFsync();
>>>
>>> public long getWal();
>>>
>>> public long getAverageWalFsyncTime();
>>>
>>> // Per cache.
>>>
>>> public PersistentStoreCacheMetrics cache(String name);
>>>
>>> public PersistentStoreCacheMetrics cache(int cacheId);
>>>
>>> // For last checkpoint.
>>>
>>> public PersistentStoreCheckpointMetrics getLastCheckPoint();
>>> }
>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>
>>> PersistentStoreCacheMetrics.java
>>>
>>> public interface PersistentStoreCacheMetrics {
>>>
>>> public String name();
>>>
>>> public double getFillFactor();
>>>
>>> public double getFillFactor(int part);
>>>
>>> public long getMemorySize();
>>>
>>> public long getDiskSize();
>>>
>>> public long getPagesInMemory();
>>>
>>> public long getPagesSizeInMemory();
>>>
>>> public long getPagesOnDisk();
>>>
>>> public long getPagesSizeOnDisk();
>>>
>>> public long getFreePages();
>>>
>>> public long getFreePagesSize();
>>>
>>> public long getDirtyPages();
>>>
>>> public long getDirtyPagesSize();
>>>
>>> public long getPagesRead();
>>>
>>> public long getPagesWritten();
>>> }
>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>
>>> PersistentStoreCheckpointMetrics.java
>>>
>>> public interface PersistentStoreCheckpointMetrics {
>>>
>>> public long getTotalPages();
>>>
>>> //TODO Page type is internal?
>>> public long[] pagesType();
>>>
>>> public long getExecutingTime();
>>>
>>> public long getFsyncTime();
>>>
>>> public long getPagesCopyOnWrite();
>>> }
>>
>>
