Thanks for quick reply. I mistakenly write 20 million to earlier but objects I am putting in cache are 2 million which means .9 GB approx.
Sorry for putting wrong number of objects earlier. I am starting 2 server nodes of 5 GB each and a client node of 5 gb to load data into cache. Apart from this cache I am having 2 more caches whose size combined is less than 400 MB. On 17 Feb 2016 10:44, "vkulichenko [via Apache Ignite Users]" < ml-node+s70518n3050...@n6.nabble.com> wrote: > Hi vidhu, > > First of all, can you please subscribe to the mailing list so that > community receives email notifications? Please follow this simple > instruction: > http://apache-ignite-users.70518.x6.nabble.com/mailing_list/MailingListOptions.jtp?forum=1 > > vidhu wrote > I am facing a problem where my heap reaches to about 2.8 gb when I am > storing 20 million objects. Size of each object is approx. 462 bytes. > Structure of my object is a simple pojo having 101 fields each of int > type. > > I tried using following 3 marshalling techniques: > > 1.) Default marshaller of Ignite. (Size of heap ~ 2.8 gb, looking with > Visual VM find that most of objects on heap are byte array.) > > 2.) Optimized marshaller. (Again size was ~ 2.8 gb, looking with Visual VM > find that this time size of objects on heap of byte array is half + there > are lot of objects of class that I put in cache. ) > > 3.) Tried implementing Binarylizable in my class that I am storing in > cache. I used rawWriter. This time size reduced considerably but I was not > able to read values out of cache, (getting null on read). > > I am implementing this in java version 1.8. > > 20,000,000 * 462 bytes is already around 8.6GB of memory, so 2.8 is not > enough even without Ignite's overhead. Did you try to allocate more heap > memory and/or start more nodes? With your sizings I would start with two > nodes, 8GB of memory per node. > > As for options you tried, I would recommend to use default binary > marshaller because it's much more compact. Also you should take a look at > offheap memory [1]. It can be useful if you're going to have a lot of data. > > [1] https://apacheignite.readme.io/docs/off-heap-memory > > -Val > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-ignite-users.70518.x6.nabble.com/Huge-heap-size-for-ignite-cache-when-storing-20-million-objects-of-small-size-tp3049p3050.html > To unsubscribe from Huge heap size for ignite cache when storing 20 > million objects of small size, click here > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3049&code=bWV2aWRodXNoYXJtYUBnbWFpbC5jb218MzA0OXwtMTAzMjgwMDMwOQ==> > . > NAML > <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Huge-heap-size-for-ignite-cache-when-storing-20-million-objects-of-small-size-tp3049p3051.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
