Hi,
Thank you for the hint. I just wrote a small program to allocate 1 GB
memory and to free the same memory using UNSAFE api. I took the native
memory tracking report 3 times, before allocating 1 GB memory, after
allocating 1 GB memory and after freeing 1 GB memory.
Here is the snippet of reports. From the result I can conclude that
allocated offheap memory using UNSAFE api is listed under "Internal"
category of the report.
NOTE: DirectByteBuffer internally uses unsafe apis.
1) Before allocating 1 GB memory (off heap).
- Internal (reserved=9782KB, committed=9782KB)
(malloc=9718KB #3360)
(mmap: reserved=64KB, committed=64KB)
2) After allocating 1 GB memory (off heap).
- Internal (reserved=1058386KB, committed=1058386KB)
(malloc=1058322KB #3415)
(mmap: reserved=64KB, committed=64KB)
3) After freeing up 1GB memory
- Internal (reserved=9798KB, committed=9798KB)
(malloc=9734KB #3299)
(mmap: reserved=64KB, committed=64KB)
Sample program:
public static void main(String[] args) throws NoSuchFieldException,
IllegalAccessException, InterruptedException {
Field f = Unsafe.class.getDeclaredField("theUnsafe");
f.setAccessible(true);
final Unsafe unsafe = (Unsafe) f.get(null);
System.out.println("1)Now going to sleep");
Thread.sleep(60000);
System.out.println("Now allocating 1GB off-heap.....");
long address = unsafe.allocateMemory(1024 * 1024 * 1024);
System.out.println("Allocated 1GB off-heap.....");
System.out.println("2)Now going to sleep");
Thread.sleep(60000);
unsafe.freeMemory(address);
System.out.println("3)Now going to sleep");
Thread.sleep(60000);
System.out.println("Exited.....");
}
Thanks.,
Prasad
On Mon, Feb 18, 2019 at 7:56 PM Павлухин Иван <vololo...@gmail.com> wrote:
> Prasad,
>
> Someone has already posted a snippet [1].
>
> [1] https://gist.github.com/prasanthj/48e7063cac88eb396bc9961fb3149b58
>
> пн, 18 февр. 2019 г. в 17:23, Павлухин Иван <vololo...@gmail.com>:
> >
> > Hi Prasad,
> >
> > As far as I remember offheap memory allocated with use of Unsafe is
> > not reflected in Native Memory Tracking report. You are right that
> > documentation is not verbose about reported categories [1]. It might
> > be the case that memory allocated by ByteBuffer.allocateDirect falls
> > into "internal" category. You can check it out by writing an example
> > application using ByteBuffer.allocateDirect.
> >
> > [1]
> https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr022.html#BABHIFJC
> >
> > пн, 18 февр. 2019 г. в 16:04, Dmitriy Pavlov <dpav...@apache.org>:
> > >
> > > Hi.
> > >
> > > Please don't use posting to both dev/user lists simultaneously.
> > >
> > > If your question is not related to any contribution you are planning
> to do, then the user list is a better place to ask, because a possible
> answer may be interesting to all Ignite users.
> > >
> > > If you are going to fix any issue and would like to discuss a
> proposal, please use dev list.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > пн, 18 февр. 2019 г. в 16:00, Prasad Bhalerao <
> prasadbhalerao1...@gmail.com>:
> > >>
> > >> Hi,
> > >>
> > >> I have set the off heap size to 500 MB and max heap size to 512 MB.
> > >>
> > >> My process is taking around 1.7 GB on Windows 10 as per the task
> manager. So I decided to track the memory distribution using jcmd to find
> out if there are any memory leaks in non-heap space.
> > >>
> > >> After pushing the data to cache I took the native memory summary
> using jcmd tool.
> > >>
> > >> I am trying to understand in which of the following section,
> allocated off heap memory goes?
> > >>
> > >> Does off heap come under "Internal" category? Can any ignite memory
> expert help me with this?
> > >>
> > >> Oracle documentation does not clearly talks about it. I am also
> attaching the native memory detail file.
> > >>
> > >> C:\Java64\jdk1.8.0_144\bin>jcmd.exe 16956 VM.native_memory summary
> > >>
> > >> 16956:
> > >>
> > >> Total: reserved=3513712KB, committed=2249108KB
> > >>
> > >> - Java Heap (reserved=524288KB, committed=524288KB)
> > >>
> > >> (mmap: reserved=524288KB,
> committed=524288KB)
> > >>
> > >>
> > >>
> > >> - Class (reserved=1127107KB, committed=86507KB)
> > >>
> > >> (classes #13259)
> > >>
> > >> (malloc=10947KB #17120)
> > >>
> > >> (mmap: reserved=1116160KB,
> committed=75560KB)
> > >>
> > >>
> > >>
> > >> - Thread (reserved=89748KB, committed=89748KB)
> > >>
> > >> (thread #88)
> > >>
> > >> (stack: reserved=89088KB,
> committed=89088KB)
> > >>
> > >> (malloc=270KB #454)
> > >>
> > >> (arena=391KB #175)
> > >>
> > >>
> > >>
> > >> - Code (reserved=254854KB, committed=30930KB)
> > >>
> > >> (malloc=5254KB #8013)
> > >>
> > >> (mmap: reserved=249600KB,
> committed=25676KB)
> > >>
> > >>
> > >>
> > >> - GC (reserved=29656KB, committed=29576KB)
> > >>
> > >> (malloc=10392KB #385)
> > >>
> > >> (mmap: reserved=19264KB,
> committed=19184KB)
> > >>
> > >>
> > >>
> > >> - Compiler (reserved=188KB, committed=188KB)
> > >>
> > >> (malloc=57KB #243)
> > >>
> > >> (arena=131KB #3)
> > >>
> > >>
> > >>
> > >> - Internal (reserved=1464736KB, committed=1464736KB)
> > >>
> > >> (malloc=1464672KB #40848)
> > >>
> > >> (mmap: reserved=64KB, committed=64KB)
> > >>
> > >>
> > >>
> > >> - Symbol (reserved=18973KB, committed=18973KB)
> > >>
> > >> (malloc=15353KB #152350)
> > >>
> > >> (arena=3620KB #1)
> > >>
> > >>
> > >>
> > >> - Native Memory Tracking (reserved=3450KB, committed=3450KB)
> > >>
> > >> (malloc=14KB #167)
> > >>
> > >> (tracking overhead=3436KB)
> > >>
> > >>
> > >>
> > >> - Arena Chunk (reserved=712KB, committed=712KB)
> > >>
> > >> (malloc=712KB)
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>
