Thanks for the changes and suggestions - for my case just setting the timeout to 5 or 10 seconds should suffice, but at least I got some insight about async options and the reason file component does work sequentially.
Claus Ibsen-2 wrote: > > BTW if you want to speed up the file routing you can use the async > concurrency. A bit on this blog > http://davsclaus.blogspot.com/2009/05/on-road-to-camel-20-concurrency-with.html > > Note the DSL was renamed from async to threads > > And then use a single file consumer to avoid race conditions for > grabbing new files. > > > On Mon, Oct 12, 2009 at 12:56 PM, Claus Ibsen <claus.ib...@gmail.com> > wrote: >> On Mon, Oct 12, 2009 at 12:27 PM, Vladimir Okhotnikov >> <vokhotni...@gmail.com> wrote: >>> >>> Sorry for the poor English - I know about the timeout option now - I saw >>> it >>> immediately AFTER I found the reason. I obviously saw the option before, >>> among others, but did not pay enough attention to realize I should not >>> use >>> the default. >>> >> >> You english is good. Just as poor/good as mine :) >> >> You have a good point about the default timeout not being good. I have >> just changed it to 10 seconds and also Camel will now log at WARN >> level if the lock could not be acquired within that period. Before it >> was just DEBUG level. >> >>> And I did have the camel DEBUG level enabled - was the first thing I >>> have >>> done to debug the problem - and it did not help me at all - there were >>> no >>> relevant messages at all - and the DEBUG logging did work - I get a log >>> of >>> all routes on camel start etc. >>> >> >> You can enable DEBUG for only the file component >> org.apache.camel.component.file=DEBUG >> >> >>> Claus, I'm not trying to blame or rant or something. I still think the >>> framework is awesome. I apologize if I somehow made you think I'm >>> feeling >>> different. I just truly believe that infinite waits by default should be >>> ruthlessly eliminated, and doing this will benefit the Camel and its >>> users >>> a great deal. >>> >> >> None taken. I may have well started this, sorry as well. Its just that >> there is alot of free based support which does take much time. >> But also gives great feedback on the Camel. All together helps improve >> the community, which is ultimate the best for us and Apache. >> >> In you case you can set the timeout to a low value such as 1, 100, >> 500, 1000 or 2000 etc to let Camel quickly try the next file so you >> get it to process files quicker. >> >> >> >>> >>> Claus Ibsen-2 wrote: >>>> >>>> On Mon, Oct 12, 2009 at 10:15 AM, Vladimir Okhotnikov >>>> <vokhotni...@gmail.com> wrote: >>>>> >>>>> Yes, I known about the readLockTimeout option. The problem with this >>>>> option >>>>> is that by a) default it waits forever and b) while it waits for one >>>>> file >>>>> it >>>>> does not process others. That's what made me spend 2 days wondering >>>>> about >>>>> the behavior described in the thread almost without clues, and it will >>>>> made >>>>> others. >>>> >>>> You did not mention that you knew this option and why have you not >>>> tried using it? >>>> >>>> Camel does TRACE / DEBUG logging which you could have enabled. >>>> >>>> Which is also written on the file component wiki page >>>> http://camel.apache.org/file2.html >>>> >>>> Debug logging >>>> This component has log level TRACE that can be helpful if you have >>>> problems. >>>> >>>> So if you have done that you would NOT have been in the dark for 2 >>>> days. >>>> >>>> >>>> >>>> The file consumer is NOT concurrent. So yes it will wait. >>>> >>>> If you want concurrency then define multiple routes with a file >>>> consumer. Which btw is not fun as file system is not transactional and >>>> having multiple consumers racing for the same files is not fun. >>>> >>>> >>>> Good idea about the default timeout which I have created a ticket for >>>> https://issues.apache.org/activemq/browse/CAMEL-2062 >>>> >>>> >>>> >>>>> >>>>> As is pretty well explained in the "Release It" book, infinite waits >>>>> are >>>>> bad. Infinite waits by default are very bad, especially in integration >>>>> software - it virtually guarantees that at least one will slip into >>>>> your >>>>> live system, then it's just a matter of time until it blocks critical >>>>> function or critical amount of resources - and you're waken in the >>>>> middle >>>>> of >>>>> the night to deal with it. >>>>> >>>>> What do you think? >>>>> >>>>> >>>>> >>>>> Claus Ibsen-2 wrote: >>>>>> >>>>>> >>>>>> You can already do this by the readLockTimeout option. See more >>>>>> details >>>>>> here: >>>>>> http://camel.apache.org/file2.html >>>>>> >>>>>> For example you can set it to 10000 as 10 seconds and if Camel cannot >>>>>> acquire the lock within that period it will skip this file and try >>>>>> the >>>>>> next one. >>>>>> >>>>>> -- >>>>>> Claus Ibsen >>>>>> Apache Camel Committer >>>>>> >>>>>> Open Source Integration: http://fusesource.com >>>>>> Blog: http://davsclaus.blogspot.com/ >>>>>> Twitter: http://twitter.com/davsclaus >>>>>> >>>>>> >>>>> >>>>> -- >>>>> View this message in context: >>>>> http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25851978.html >>>>> Sent from the Camel - Users mailing list archive at Nabble.com. >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Claus Ibsen >>>> Apache Camel Committer >>>> >>>> Open Source Integration: http://fusesource.com >>>> Blog: http://davsclaus.blogspot.com/ >>>> Twitter: http://twitter.com/davsclaus >>>> >>>> >>> >>> -- >>> View this message in context: >>> http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25853488.html >>> Sent from the Camel - Users mailing list archive at Nabble.com. >>> >>> >> >> >> >> -- >> Claus Ibsen >> Apache Camel Committer >> >> Open Source Integration: http://fusesource.com >> Blog: http://davsclaus.blogspot.com/ >> Twitter: http://twitter.com/davsclaus >> > > > > -- > Claus Ibsen > Apache Camel Committer > > Open Source Integration: http://fusesource.com > Blog: http://davsclaus.blogspot.com/ > Twitter: http://twitter.com/davsclaus > > -- View this message in context: http://www.nabble.com/File-component-blocked-by-existing-files-tp25803233p25854546.html Sent from the Camel - Users mailing list archive at Nabble.com.
