Wau... thanks for that. I don't understand how I could miss it, cause i searched a lot for such a feature.
Another thing that I have experienced in 2.0, is that the "file" component is unable to move/copy files larger than 2Gigs, such large files are cut around 2 Gigs limit. Now, I'm not sure if it has anything to do with the OS or the filesystems or what. I have seen it both in Linux (debian) and Mac OS X. And I don't understand either, why the file component does not use the NIO api or at least we don't have an option to tell it use the NIO api instead. Anyway, for now I've solved the problem by using a smiple component instead of the "file" comp. to move/copy files using the NIO api. It is much faster and it has no problem with very large files. Bests, Farhad S. Claus Ibsen-2 wrote: > > On Fri, Nov 27, 2009 at 1:12 PM, shirazi <m...@farhad.eu> wrote: >> >> Hi, >> Ok, I guess I should rather provide an AggregationStrategy. >> The reason why I use a timer is that when I use the ftp endpoint, it >> starts >> to fetch everything that is on the ftp in the first call and that's not >> what >> i want, because I'm processing files over 3 Gigs and there are many of >> them. >> Then I figured that by using a timer, i could have ftp to download one >> file >> at a time!! >> What I basically need is to somehow tell the ftp, how many files I want >> to >> fetch at a time. This is probably doable now in camel-2.1 by using the >> "ThrottlingInflightRoutePolicy", which i havn't tried yet. >> Do you have any suggestions? > > Yeah the FTP component extends the File component so you got all the > options from the file component as well > http://camel.apache.org/file2.html > > And there is a maxMessagesPerPoll you can use to tell how many files > you want at most to poll. For example = 1 in your use. > > And that option is avail in 2.0 also. > > In fact all these are as they are batch polling consumers > http://camel.apache.org/batch-consumer.html > > > >> >> Bests, >> Farhad S. >> >> >> >> Claus Ibsen-2 wrote: >>> >>> Hi >>> >>> BTW why are you using a timer -> ftp when the ftp have a build in timer? >>> >>> >>> >>> On Fri, Nov 27, 2009 at 12:55 PM, shirazi <m...@farhad.eu> wrote: >>>> >>>> Hi, >>>> Please consider the following route: >>>> >>>> from("timer://someTrigger?delay=30000&fixedRate=true&period=300000"). >>>> pollEnrich("ftp://cmoreFtp?binary=true&localWorkDirectory=/tmp&idempotent=true&idempotentRepository=#caStreams";, >>>> 10000). >>>> to("file:///tmp/vw/cmore?tempPrefix=.inprogress"); >>>> >>>> The pollEnrich cast a NullPointerException in camel-2.0, if there is >>>> nothing >>>> to fetch from the ftp. I got rid of the exception by a >>>> "try-catch(NullpointerExcpetion.class).stop()" around the route. Now in >>>> camel-2.1-SNAPSHOT, I get the following exception on the "to" endpoint: >>>> ================= >>>> 12:37:06,899 INFO [Tracer] >>> --> >>>> file:///tmp/vw/cmore?tempPrefix=.inprogress_, Pattern:InOnly, >>>> Headers:{firedTime=Fri Nov 27 12:36:56 CET 2009}, >>>> Exception:org.apache.camel.component.file.GenericFileOperationFailedException: >>>> Cannot store file: >>>> /tmp/vw/cmore/.inprogress_02216f38-8ffc-4aec-b638-27229d857075 >>>> 12:37:06,899 INFO [Tracer] >>> --> >>>> file:///tmp/vw/cmore?tempPrefix=.inprogress_, Pattern:InOnly, >>>> Headers:{firedTime=Fri Nov 27 12:36:56 CET 2009}, >>>> Exception:org.apache.camel.component.file.GenericFileOperationFailedException: >>>> Cannot store file: >>>> /tmp/vw/cmore/.inprogress_02216f38-8ffc-4aec-b638-27229d857075 >>>> 12:37:06,899 ERROR [TimerConsumer] Cannot store file: >>>> /tmp/vw/cmore/.inprogress_02216f38-8ffc-4aec-b638-27229d857075 >>>> org.apache.camel.component.file.GenericFileOperationFailedException: >>>> Cannot >>>> store file: >>>> /tmp/vw/cmore/.inprogress_02216f38-8ffc-4aec-b638-27229d857075 >>>> at >>>> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:204) >>>> at >>>> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:184) >>>> at >>>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:126) >>>> at >>>> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:57) >>>> at >>>> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97) >>>> at >>>> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95) >>>> at >>>> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146) >>>> at >>>> org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94) >>>> at >>>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82) >>>> at >>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>>> at >>>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53) >>>> at >>>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82) >>>> at >>>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:162) >>>> at >>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>>> at >>>> org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:206) >>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74) >>>> at >>>> org.apache.camel.processor.TryProcessor.process(TryProcessor.java:63) >>>> at >>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>>> at >>>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53) >>>> at >>>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82) >>>> at >>>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:162) >>>> at >>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>>> at >>>> org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:206) >>>> at >>>> org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:54) >>>> at >>>> org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48) >>>> at >>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>>> at >>>> org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:103) >>>> at >>>> org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:50) >>>> at java.util.TimerThread.mainLoop(Timer.java:512) >>>> at java.util.TimerThread.run(Timer.java:462) >>>> Caused by: org.apache.camel.InvalidPayloadException: No body available >>>> of >>>> type: java.io.InputStream on: Message: [Body is null]. Caused by: No >>>> type >>>> converter available to convert from type: null to the required type: >>>> java.io.InputStream with value null on the exchange: Exchange[Message: >>>> [Body >>>> is null]] >>>> at >>>> org.apache.camel.impl.MessageSupport.getMandatoryBody(MessageSupport.java:103) >>>> at >>>> org.apache.camel.util.ExchangeHelper.getMandatoryInBody(ExchangeHelper.java:116) >>>> at >>>> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:198) >>>> ... 29 more >>>> Caused by: org.apache.camel.NoTypeConversionAvailableException: No type >>>> converter available to convert from type: null to the required type: >>>> java.io.InputStream with value null >>>> at >>>> org.apache.camel.impl.converter.DefaultTypeConverter.mandatoryConvertTo(DefaultTypeConverter.java:126) >>>> at >>>> org.apache.camel.impl.MessageSupport.getMandatoryBody(MessageSupport.java:101) >>>> ... 31 more >>>> ============ >>>> Now, I can remedy the situation by a try-catch block or an interceptor, >>>> but >>>> in my opinion this should be considered a bug in camel and it should >>>> silently ignore if there was nothing to fetch from the ftp. >>>> >>>> Bests, >>>> Farhad Shiraz >>>> >>>> -- >>>> View this message in context: >>>> http://old.nabble.com/camel-casts-exception-when-there-is-nothing-to-fetch-from-the-ftp-endpoint-tp26541447p26541447.html >>>> Sent from the Camel - Users mailing list archive at Nabble.com. >>>> >>>> >>> >>> >>> >>> -- >>> Claus Ibsen >>> Apache Camel Committer >>> >>> Author of Camel in Action: http://www.manning.com/ibsen/ >>> Open Source Integration: http://fusesource.com >>> Blog: http://davsclaus.blogspot.com/ >>> Twitter: http://twitter.com/davsclaus >>> >>> >> >> -- >> View this message in context: >> http://old.nabble.com/camel-casts-exception-when-there-is-nothing-to-fetch-from-the-ftp-endpoint-tp26541447p26541650.html >> Sent from the Camel - Users mailing list archive at Nabble.com. >> >> > > > > -- > Claus Ibsen > Apache Camel Committer > > Author of Camel in Action: http://www.manning.com/ibsen/ > Open Source Integration: http://fusesource.com > Blog: http://davsclaus.blogspot.com/ > Twitter: http://twitter.com/davsclaus > > -- View this message in context: http://old.nabble.com/camel-casts-exception-when-there-is-nothing-to-fetch-from-the-ftp-endpoint-tp26541447p26541877.html Sent from the Camel - Users mailing list archive at Nabble.com.
