Hi All, I have been looking around why a large number of heap is being consumed by MemoryIdempotentRepository. I saw one JIRA as well regarding inProgressRepository not getting cleared. Can that be the reason here?
Also,I am not using consumerTemplate to consume 1000o files. Does that also use inProgressRepository? And when is it cleared from the memory? Any help is much appreciated. Thanks! Chirag Dewan ________________________________ From: Chirag Dewan <chirag.dewa...@yahoo.in> To: "users@camel.apache.org" <users@camel.apache.org>; Chirag Dewan <chirag.dewa...@yahoo.in> Sent: Wednesday, 12 February 2014 6:31 PM Subject: Re: OOM issue due to MemoryIdempotentRepository Hi, And I am using readLock=changed. So that can be a reason too? Chirag Dewan ________________________________ From: Chirag Dewan <chirag.dewa...@yahoo.in> To: "users@camel.apache.org" <users@camel.apache.org> Sent: Wednesday, 12 February 2014 6:07 PM Subject: Re: OOM issue due to MemoryIdempotentRepository Hi Claus, Thanks for the quick reply. It might well be. But there is one thing I would like to get some insight into. If idempotent=false(default) is used,will the LRU Cache still store the file referrence? Because that is what I can see in my memory dump. org.apache.camel.util.LRUCache >org.apache.camel.com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap I maybe doing something wrong here. Is there anything else I need to do to make sure that file is deleted after it has been processed and not stored in the cache? Thanks. Chirag Dewan ________________________________ From: Claus Ibsen <claus.ib...@gmail.com> To: "users@camel.apache.org" <users@camel.apache.org> Sent: Wednesday, 12 February 2014 5:42 PM Subject: Re: OOM issue due to MemoryIdempotentRepository Hi Some days ago someone else posted about a OOME issue when using Storm. It smells like a Storm + Camel issue somewhere. On Wed, Feb 12, 2014 at 11:50 AM, Chirag Dewan <chirag.dewa...@yahoo.in> wrote: > Hi All, > > I am using Camel 2.12.1 with JDK 1.7.0_45. I have a FTP consumer route in my > application. > > > Now I intent to delete the file after consuming it. So I have delete=true in > my route. After consuming around 6000 files(out of 10000,as my test case) JVM > goes Out of memory. > > > In my heap dump I can see the following trace as the major memory consumers: > > java.util.ArrayList > >java.lang.Object > > >>org.apache.camel.management.DefaultManagementLifecycleStrategy$PreRegisterService > > >org.apache.camel.component.file.FileEndpoint > > >org.apache.camel.processor.idempotent.MemoryIdempotentRepository > >org.apache.camel.util.LRUCache > >>org.apache.camel.com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap > > > So if I am not using indempotent,should it still store the files in the > memory? Is this memory repository used,even if I am deleting the files and > not storing in memory? Is there a way I can avoid that? > > I am testing this scenario under Storm as execution environment and with 2GB > max heap space. > > Thanks in advance > > Chirag Dewan -- Claus Ibsen ----------------- Red Hat, Inc. Email: cib...@redhat.com Twitter: davsclaus Blog: http://davsclaus.com Author of Camel in Action: http://www.manning.com/ibsen Make your Camel applications look hawt, try: http://hawt.io
