Pavel, I think that point 1 is the correct way to calculate the committed size for a log. It is already calculated regardless of the metricsEnabled flag.
In addition, I suggest more readable log format for data regions in issue comments [1]. [1] https://issues.apache.org/jira/browse/IGNITE-9305 вт, 21 авг. 2018 г. в 12:15, Pavel Pereslegin <xxt...@gmail.com>: > > Hello, Igniters. > > I assigned ticket [1] created by Denis and want to clarify how to log > committed size. > The metric offHeapSize (in DataRegionMetricsImpl) is always > calculated, but getOffHeapSize returns zero if memory metrics are > disabled for this data region. > > So I see the following options: > 1. Modify method getOffHeapSize so that it always returns actual value > offHeapSize. > 2. Add another offHeapSize() method. > 3. Output to log max size instead of committed (change "comm" to "max" > in log output). > 4. Don't bother about disabling metrics and output to log value > returned by getOffHeapSize. > > Any thoughts? > > [1] https://issues.apache.org/jira/browse/IGNITE-9305 > сб, 18 авг. 2018 г. в 3:17, Denis Magda <dma...@apache.org>: > > > > Vova, the things are even simpler - we have this > > > > ignite.dataRegionMetrics().getPhysicalMemorySize() that returns the > > number equal/comparabel to pageNumber X pageSize. > > > > > > Igniters, if you believe that we need to do more work here then let's > > do it iteratively. Let's fix the off-heap occupied size the way above > > (just print out getPhysicalMemorySize() for every data region). Then > > do the rest. This needs to be fixed in 2.7. > > > > > > -- > > > > Denis > > > > > > On Fri, Aug 17, 2018 at 10:20 AM Vladimir Ozerov <voze...@gridgain.com> > > wrote: > > > > > Folks, > > > > > > We already have this: > > > >>> PageMemory [pages=6997377] > > > > > > Then we can multiply it by page size and get occupied memory. Am I wrong? > > > > > > On Fri, Aug 17, 2018 at 12:56 PM Dmitriy Pavlov <dpavlov....@gmail.com> > > > wrote: > > > > > > > Hi Maxim, > > > > > > > > thank you for stepping in and for finding these issues. Yes, these > > > tickets > > > > are correct. > > > > > > > > I can move https://issues.apache.org/jira/browse/IGNITE-5583 to > > > unassigned > > > > if someone would like to implement this change. I will not have enough > > > time > > > > to complete it in 1 month (before 2.7 release). > > > > > > > > Sincerely, > > > > Dmitriy Pavlov > > > > > > > > пт, 17 авг. 2018 г. в 11:04, Maxim Muzafarov <maxmu...@gmail.com>: > > > > > > > > > Igniters, > > > > > > > > > > Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch non-heap > > > > memory > > > > > metrics > > > > > to new page memory semantics` and related previous disscustions to it > > > > [4]. > > > > > > > > > > Also we have some additional improvements to CacheMetrics: > > > > > IGNITE-5490 [2] - `Implement replacement for obsolete > > > > > CacheMetrics#getOffHeapAllocatedSize` > > > > > IGNITE-5765 [3] - `CacheMetrics interface cleanup, documentation and > > > > fixes` > > > > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5583 > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-5490 > > > > > [3] https://issues.apache.org/jira/browse/IGNITE-5765 > > > > > [4] > > > > > > > > > > > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Negative-non-heap-memory-maximum-td17990.html > > > > > > > > > > On Fri, 17 Aug 2018 at 10:14 Dmitriy Pavlov <dpavlov....@gmail.com> > > > > wrote: > > > > > > > > > > > Hi Igniters, > > > > > > > > > > > > It is not an easy fix, so I'm not sure it is possible to do in 2.7. > > > > > > > > > > > > Offheap size is not reported by VM (it returns -1). To implement it > > > we > > > > > need > > > > > > totally migrate off-heap memory metrics to durable memory data. > > > > > > > > > > > > I think this issue was reported and I'll find the duplicate. > > > > > > > > > > > > Sincerely, > > > > > > Dmitriy Pavlov > > > > > > > > > > > > пт, 17 авг. 2018 г. в 6:10, Denis Magda <dma...@apache.org>: > > > > > > > > > > > > > Yes, it was at the end of my wordy email :) > > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305 > > > > > > > > > > > > > > -- > > > > > > > Denis > > > > > > > > > > > > > > On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan < > > > > > > dsetrak...@apache.org> > > > > > > > wrote: > > > > > > > > > > > > > > > Is there a blocker ticket for 2.7? > > > > > > > > > > > > > > > > On Thu, Aug 16, 2018, 19:59 Denis Magda <dma...@apache.org> > > > wrote: > > > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > Was troubleshooting an Ignite deployment today and couldn't > > > find > > > > > out > > > > > > > from > > > > > > > > > the logs what was the actual off-heap space used. > > > > > > > > > > > > > > > > > > Those were the given memory resoures (Ignite 2.6): > > > > > > > > > > > > > > > > > > [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager] > > > > > Topology > > > > > > > > > snapshot [ver=1, servers=1, clients=0, CPUs=64, > > > *offheap=30.0GB*, > > > > > > > > > heap=24.0GB] > > > > > > > > > > > > > > > > > > And that weird stuff was reported by the node (pay attention > > > > > > > > > to > > > > the > > > > > > > last > > > > > > > > > line): > > > > > > > > > > > > > > > > > > [2018-08-16 15:45:50,211][INFO > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017] > > > > > > > > > Metrics for local node (to disable set 'metricsLogFrequency' > > > > > > > > > to > > > > 0) > > > > > > > > > ^-- Node [id=c033026e, name=cluster_31-Dec-2017, > > > > > > > uptime=00:38:00.257] > > > > > > > > > ^-- H/N/C [hosts=1, nodes=1, CPUs=64] > > > > > > > > > ^-- CPU [cur=0.03%, avg=5.54%, GC=0%] > > > > > > > > > ^-- PageMemory [pages=6997377] > > > > > > > > > ^-- Heap [used=9706MB, free=61.18%, comm=22384MB] > > > > > > > > > * ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this > > > line > > > > is > > > > > > > > always > > > > > > > > > the same!* > > > > > > > > > > > > > > > > > > Had to change the code by using > > > > dataRegion.getPhysicalMemoryPages() > > > > > > to > > > > > > > > find > > > > > > > > > out that actual off-heap usage size was > > > > > > > > > >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB > > > > > > > > > > > > > > > > > > Let's fix this issue in 2.7, I proposed a new format. Please > > > > review > > > > > > and > > > > > > > > > share your thoughts: > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305 > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > -- > > > > > Maxim Muzafarov > > > > > > > > > > > > -- Best wishes, Amelchev Nikita
