Thanks,
Claus Ibsen-2 wrote: > > On Mon, Feb 22, 2010 at 1:01 PM, Claus Ibsen <[email protected]> > wrote: >> On Mon, Feb 22, 2010 at 12:57 PM, wmoussel <[email protected]> wrote: >>> >>> I considered decreasing the poll interval as well, but couldn't get cpu >>> usage >>> down enough for acceptable interval... >>> >>> I'm trying PollingConsumerPollStrategy but begin method is a void not a >>> boolean, isn't it? >>> >> >> Ah yeah it is. I can see we haven't made that a boolean to allow you >> to deny polling. >> You can create a ticket in JIRA so we can enhance this so you can do >> that in Camel 2.3. >> > > I have created the ticket > https://issues.apache.org/activemq/browse/CAMEL-2492 > >>> >>> >>> Claus Ibsen-2 wrote: >>>> >>>> On Mon, Feb 22, 2010 at 9:04 AM, wmoussel <[email protected]> wrote: >>>>> >>>>> Hi, >>>>> >>>>> Here's my initial issue. I have to poll for new files in a directory >>>>> containing a lot of files (5000 to 20000) (and I can't move them after >>>>> processing). >>>>> >>>>> When using idempotent option it takes way too much cpu (like 9% even >>>>> when >>>>> they aren't new files). >>>>> >>>> >>>> How often are you gonna poll in those files? You can probably change >>>> the delay to be less frequent. >>>> >>>> In any case somehow you gotta match all those 20000 files whether or >>>> not you have processed them before. >>>> >>>> >>>>> So i tried adding a filter so that the consumer would only take files >>>>> less >>>>> than 2 days old. It doesn't change much in the end. When I looked >>>>> into >>>>> how >>>>> filter works in Java I got why it's still cpu consuming... >>>>> >>>>> So then I thought i would get recent file names with a unix command >>>>> like >>>>> find . -mtime -1 and use pollenrich to poll specific files. The way I >>>>> understand how poll enrich works, it will poll the whole directory and >>>>> then >>>>> try to match filename afterwards. That is also cpu consuming since it >>>>> gets >>>>> all the files as exchanges prior to the match. Am I wrong ? >>>>> >>>> >>>> pollEnrich is not suitable for this as its meant for polling and >>>> aggregating 1 resource at a time. >>>> >>>>> So I'm basically trying to invoke a FileName to GenericFileExchange >>>>> processor in the middle of my route. Is there a way to do this? >>>>> >>>>> I'm open to any suggestion. >>>>> >>>> >>>> The file component supports using a custom PollingConsumerPollStrategy >>>> http://camel.apache.org/polling-consumer.html >>>> >>>> You can then implement your own logic and return false in the being >>>> method where there are no new files since last time. >>>> >>>> >>>>> Thanks in advance :) >>>>> >>>>> Wandrille >>>>> -- >>>>> View this message in context: >>>>> http://old.nabble.com/File-Consumer-with-dynamic-list-of-files-to-poll-tp27683938p27683938.html >>>>> Sent from the Camel - Users mailing list archive at Nabble.com. >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Claus Ibsen >>>> Apache Camel Committer >>>> >>>> Author of Camel in Action: http://www.manning.com/ibsen/ >>>> Open Source Integration: http://fusesource.com >>>> Blog: http://davsclaus.blogspot.com/ >>>> Twitter: http://twitter.com/davsclaus >>>> >>>> >>> >>> -- >>> View this message in context: >>> http://old.nabble.com/File-Consumer-with-dynamic-list-of-files-to-poll-tp27683938p27686162.html >>> Sent from the Camel - Users mailing list archive at Nabble.com. >>> >>> >> >> >> >> -- >> Claus Ibsen >> Apache Camel Committer >> >> Author of Camel in Action: http://www.manning.com/ibsen/ >> Open Source Integration: http://fusesource.com >> Blog: http://davsclaus.blogspot.com/ >> Twitter: http://twitter.com/davsclaus >> > > > > -- > Claus Ibsen > Apache Camel Committer > > Author of Camel in Action: http://www.manning.com/ibsen/ > Open Source Integration: http://fusesource.com > Blog: http://davsclaus.blogspot.com/ > Twitter: http://twitter.com/davsclaus > > -- View this message in context: http://old.nabble.com/File-Consumer-with-dynamic-list-of-files-to-poll-tp27683938p27686496.html Sent from the Camel - Users mailing list archive at Nabble.com.
