On Fri, Nov 27, 2009 at 1:12 PM, shirazi <m...@farhad.eu> wrote: > > Hi, > Ok, I guess I should rather provide an AggregationStrategy. > The reason why I use a timer is that when I use the ftp endpoint, it starts > to fetch everything that is on the ftp in the first call and that's not what > i want, because I'm processing files over 3 Gigs and there are many of them. > Then I figured that by using a timer, i could have ftp to download one file > at a time!! > What I basically need is to somehow tell the ftp, how many files I want to > fetch at a time. This is probably doable now in camel-2.1 by using the > "ThrottlingInflightRoutePolicy", which i havn't tried yet. > Do you have any suggestions?
Yeah the FTP component extends the File component so you got all the options from the file component as well http://camel.apache.org/file2.html And there is a maxMessagesPerPoll you can use to tell how many files you want at most to poll. For example = 1 in your use. And that option is avail in 2.0 also. In fact all these are as they are batch polling consumers http://camel.apache.org/batch-consumer.html > > Bests, > Farhad S. > > > > Claus Ibsen-2 wrote: >> >> Hi >> >> BTW why are you using a timer -> ftp when the ftp have a build in timer? >> >> >> >> On Fri, Nov 27, 2009 at 12:55 PM, shirazi <m...@farhad.eu> wrote: >>> >>> Hi, >>> Please consider the following route: >>> >>> from("timer://someTrigger?delay=30000&fixedRate=true&period=300000"). >>> pollEnrich("ftp://cmoreFtp?binary=true&localWorkDirectory=/tmp&idempotent=true&idempotentRepository=#caStreams";, >>> 10000). >>> to("file:///tmp/vw/cmore?tempPrefix=.inprogress"); >>> >>> The pollEnrich cast a NullPointerException in camel-2.0, if there is >>> nothing >>> to fetch from the ftp. I got rid of the exception by a >>> "try-catch(NullpointerExcpetion.class).stop()" around the route. Now in >>> camel-2.1-SNAPSHOT, I get the following exception on the "to" endpoint: >>> ================= >>> 12:37:06,899 INFO [Tracer] >>> --> >>> file:///tmp/vw/cmore?tempPrefix=.inprogress_, Pattern:InOnly, >>> Headers:{firedTime=Fri Nov 27 12:36:56 CET 2009}, >>> Exception:org.apache.camel.component.file.GenericFileOperationFailedException: >>> Cannot store file: >>> /tmp/vw/cmore/.inprogress_02216f38-8ffc-4aec-b638-27229d857075 >>> 12:37:06,899 INFO [Tracer] >>> --> >>> file:///tmp/vw/cmore?tempPrefix=.inprogress_, Pattern:InOnly, >>> Headers:{firedTime=Fri Nov 27 12:36:56 CET 2009}, >>> Exception:org.apache.camel.component.file.GenericFileOperationFailedException: >>> Cannot store file: >>> /tmp/vw/cmore/.inprogress_02216f38-8ffc-4aec-b638-27229d857075 >>> 12:37:06,899 ERROR [TimerConsumer] Cannot store file: >>> /tmp/vw/cmore/.inprogress_02216f38-8ffc-4aec-b638-27229d857075 >>> org.apache.camel.component.file.GenericFileOperationFailedException: >>> Cannot >>> store file: >>> /tmp/vw/cmore/.inprogress_02216f38-8ffc-4aec-b638-27229d857075 >>> at >>> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:204) >>> at >>> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:184) >>> at >>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:126) >>> at >>> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:57) >>> at >>> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97) >>> at >>> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95) >>> at >>> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146) >>> at >>> org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94) >>> at >>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82) >>> at >>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>> at >>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53) >>> at >>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82) >>> at >>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:162) >>> at >>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>> at >>> org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:206) >>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74) >>> at >>> org.apache.camel.processor.TryProcessor.process(TryProcessor.java:63) >>> at >>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>> at >>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53) >>> at >>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82) >>> at >>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:162) >>> at >>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>> at >>> org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:206) >>> at >>> org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:54) >>> at >>> org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48) >>> at >>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>> at >>> org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:103) >>> at >>> org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:50) >>> at java.util.TimerThread.mainLoop(Timer.java:512) >>> at java.util.TimerThread.run(Timer.java:462) >>> Caused by: org.apache.camel.InvalidPayloadException: No body available of >>> type: java.io.InputStream on: Message: [Body is null]. Caused by: No type >>> converter available to convert from type: null to the required type: >>> java.io.InputStream with value null on the exchange: Exchange[Message: >>> [Body >>> is null]] >>> at >>> org.apache.camel.impl.MessageSupport.getMandatoryBody(MessageSupport.java:103) >>> at >>> org.apache.camel.util.ExchangeHelper.getMandatoryInBody(ExchangeHelper.java:116) >>> at >>> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:198) >>> ... 29 more >>> Caused by: org.apache.camel.NoTypeConversionAvailableException: No type >>> converter available to convert from type: null to the required type: >>> java.io.InputStream with value null >>> at >>> org.apache.camel.impl.converter.DefaultTypeConverter.mandatoryConvertTo(DefaultTypeConverter.java:126) >>> at >>> org.apache.camel.impl.MessageSupport.getMandatoryBody(MessageSupport.java:101) >>> ... 31 more >>> ============ >>> Now, I can remedy the situation by a try-catch block or an interceptor, >>> but >>> in my opinion this should be considered a bug in camel and it should >>> silently ignore if there was nothing to fetch from the ftp. >>> >>> Bests, >>> Farhad Shiraz >>> >>> -- >>> View this message in context: >>> http://old.nabble.com/camel-casts-exception-when-there-is-nothing-to-fetch-from-the-ftp-endpoint-tp26541447p26541447.html >>> Sent from the Camel - Users mailing list archive at Nabble.com. >>> >>> >> >> >> >> -- >> Claus Ibsen >> Apache Camel Committer >> >> Author of Camel in Action: http://www.manning.com/ibsen/ >> Open Source Integration: http://fusesource.com >> Blog: http://davsclaus.blogspot.com/ >> Twitter: http://twitter.com/davsclaus >> >> > > -- > View this message in context: > http://old.nabble.com/camel-casts-exception-when-there-is-nothing-to-fetch-from-the-ftp-endpoint-tp26541447p26541650.html > Sent from the Camel - Users mailing list archive at Nabble.com. > > -- Claus Ibsen Apache Camel Committer Author of Camel in Action: http://www.manning.com/ibsen/ Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus
