Denis,
Here is a version of CacheMetrics interface with all changes how I see them
(pretty long list :)).
public interface CacheMetrics {
public long getCacheHits();
public float getCacheHitPercentage();
public long getCacheMisses();
public float getCacheMissPercentage();
public long getCacheGets();
public long getCachePuts();
public long getCacheRemovals();
public long getCacheEvictions();
public float getAverageGetTime();
public float getAveragePutTime();
public float getAverageRemoveTime();
public float getAverageTxCommitTime();
public float getAverageTxRollbackTime();
public long getCacheTxCommits();
public long getCacheTxRollbacks();
public String name();
public long getOverflowSize();
public long getOffHeapGets();
public long getOffHeapPuts();//removing as it duplicates cachePuts
public long getOffHeapRemovals();
public long getOffHeapEvictions();
public long getOffHeapHits();
public float getOffHeapHitPercentage();
public long getOffHeapMisses();//removing as it duplicates cacheMisses
public float getOffHeapMissPercentage();//removing as it duplicates
cacheMissPercentage
public long getOffHeapEntriesCount();
public long getOffHeapPrimaryEntriesCount();
public long getOffHeapBackupEntriesCount();
public long getOffHeapAllocatedSize();
public long getOffHeapMaxSize();
public long getSwapGets();
public long getSwapPuts();
public long getSwapRemovals();
public long getSwapHits();
public long getSwapMisses();
public long getSwapEntriesCount();
public long getSwapSize();
public float getSwapHitPercentage();
public float getSwapMissPercentage();
public int getSize();
public int getKeySize();
public boolean isEmpty();
public int getDhtEvictQueueCurrentSize();
public int getTxThreadMapSize();
public int getTxXidMapSize();
public int getTxCommitQueueSize();
public int getTxPrepareQueueSize();
public int getTxStartVersionCountsSize();
public int getTxCommittedVersionsSize();
public int getTxRolledbackVersionsSize();
public int getTxDhtThreadMapSize();
public int getTxDhtXidMapSize();
public int getTxDhtCommitQueueSize();
public int getTxDhtPrepareQueueSize();
public int getTxDhtStartVersionCountsSize();
public int getTxDhtCommittedVersionsSize();
public int getTxDhtRolledbackVersionsSize();
public boolean isWriteBehindEnabled();
public int getWriteBehindFlushSize();
public int getWriteBehindFlushThreadCount();
public long getWriteBehindFlushFrequency();
public int getWriteBehindStoreBatchSize();
public int getWriteBehindTotalCriticalOverflowCount();
public int getWriteBehindCriticalOverflowCount();
public int getWriteBehindErrorRetryCount();
public int getWriteBehindBufferSize();
public String getKeyType();
public String getValueType();
public boolean isStoreByValue();
public boolean isStatisticsEnabled();
public boolean isManagementEnabled();
public boolean isReadThrough();
public boolean isWriteThrough();
public long getTotalAllocatedPages();
public long getTotalEvictedPages();
}
Also I suggest to introduce new interface for MemoryPolicy metrics and make
it available through *IgniteCacheDatabaseSharedManager*:
public interface IgniteMemoryPolicyMetrics {
/**
* @return Memory policy name.
*/
public String getName();
/**
* @return Total number of allocated pages.
*/
public long getTotalAllocatedPages();
/**
* @return Amount (in bytes) of not yet allocated space in PageMemory.
*/
public long getAvailableSpace();
/**
* @return Number of allocated pages per second within PageMemory.
*/
public float getAllocationRate();
/**
* @return Number of evicted pages per second within PageMemory.
*/
public float getEvictionRate();
/**
* Large entities bigger than page are split into fragments so each
fragment can fit into a page.
*
* @return Percentage of pages fully occupied by large entities.
*/
public long getLargeEntriesPagesPercentage();
//---FreeList-related metrics
/**
* @return Free space to overall size ratio across all pages in
FreeList.
*/
public float getPagesFillFactor();
/**
* @return Percentage of pages in FreeList with free space >= 8 and <
16 bytes
*/
public float getPagesPercentage_8_16_freeBytes();
/**
* @return Percentage of pages in FreeList with free space >= 16 and <
64 bytes
*/
public float getPagesPercentage_16_64_freeBytes();
/**
* @return Percentage of pages in FreeList with free space >= 64 and <
256 bytes
*/
public float getPagesPercentage_64_256_freeBytes();
}
In my mind last three methods provide some kind of hist to give an insight
about memory fragmentation.
If there are a lot of pages with relatively big free chunks and less with a
smaller chunks it may indicate that memory is fragmented and it may be
reasonable to adjust page sizes.
Thanks,
Sergey.
On Thu, Mar 16, 2017 at 1:29 AM, Denis Magda <dma...@apache.org> wrote:
> Hi Sergey,
>
> >> In memory management scheme based on MemoryPolicies it may be useful
> (and
> >> easier) to collect some metrics not for individual caches but for whole
> >> MemoryPolicies where several caches may reside.
> >>
>
> I would collect the metrics for every single MemoryPolicy as well as for
> individual caches. It makes sense to expose which cache contributes more to
> memory utilization.
>
> >> - free space / used space tracking;
> >> - allocation / eviction rate;
>
> Please consider this as well:
> - total number of pages;
> - total number of enters (how hard to support?).
>
> >> - metrics to track memory fragmentation: e.g. % of pages with only 8
> >> bytes free, 16 bytes free and so on;
> >> - % of big fragmented entries in cache: may be useful to adjust page
> >> size.
>
> >
> How do you see this in the metrics interface?
>
>
> > 3. Useful, not going to remove:
> > getOffHeapGets //useful as there still may be deserialized entries
> > residing on-heap
> > getOffHeapHitPercentage
> > getOffHeapHits //overall hits include offheap and onheap
> > getOffHeapMisses //I think in new model is the same as getCacheMisses
> > getOffHeapMissPercentage //same as above
> > getOffHeapPuts //same as above
> > getOffHeapRemovals //same as above
>
> Could you please prepare an updated version of the cache metrics adding
> new methods and renaming existing ones (only if necessary)? It will be
> simpler to keep up the discussion relying on this updated interface.
>
> —
> Denis
>
> > On Mar 15, 2017, at 8:32 AM, Sergey Chugunov <sergey.chugu...@gmail.com>
> wrote:
> >
> > Also I looked through current set of metrics available on
> > *CacheMetrics *interface
> > and suggest following changes:
> >
> >
> > 1. All methods related to tracking swap space (including
> > *getOverflowSize*) to be removed.
> >
> > 2. Useless/hard to calculate in new memory management approach:
> > getOffHeapAllocatedSize //max size is constrained by MemoryPolicy
> config
> > getOffHeapEntriesCount //all cache entries live offheap
> > getOffHeapEvictions //will be captured on MemoryPolicyMetrics level;
> > getOffHeapMaxSize //same as the first one
> >
> > 3. Useful, not going to remove:
> > getOffHeapGets //useful as there still may be deserialized entries
> > residing on-heap
> > getOffHeapHitPercentage
> > getOffHeapHits //overall hits include offheap and onheap
> > getOffHeapMisses //I think in new model is the same as getCacheMisses
> > getOffHeapMissPercentage //same as above
> > getOffHeapPuts //same as above
> > getOffHeapRemovals //same as above
> >
> > Please share your thought if I miss something here.
> >
> > Thanks,
> > Sergey Chugunov.
> >
> > On Wed, Mar 15, 2017 at 4:51 PM, Sergey Chugunov <
> sergey.chugu...@gmail.com>
> > wrote:
> >
> >> Hello Igniters,
> >>
> >> As part of [1] cache metrics need to be updated as some of them like
> swap
> >> hits are not applicable anymore.
> >>
> >> In memory management scheme based on MemoryPolicies it may be useful
> (and
> >> easier) to collect some metrics not for individual caches but for whole
> >> MemoryPolicies where several caches may reside.
> >>
> >> I suggest the following list of new metrics to collect for each
> >> MemoryPolicy:
> >>
> >> - free space / used space tracking;
> >> - allocation / eviction rate;
> >> - metrics to track memory fragmentation: e.g. % of pages with only 8
> >> bytes free, 16 bytes free and so on;
> >> - % of big fragmented entries in cache: may be useful to adjust page
> >> size.
> >>
> >>
> >> Please suggest any other metrics that may be worth tracking.
> >>
> >> [1] https://issues.apache.org/jira/browse/IGNITE-3477
> >>
> >> Thanks,
> >> Sergey Chugunov.
> >>
>
>
