On Fri, Nov 27, 2009 at 1:37 PM, shirazi <m...@farhad.eu> wrote: > > Wau... thanks for that. I don't understand how I could miss it, cause i > searched a lot for such a feature. > > Another thing that I have experienced in 2.0, is that the "file" component > is unable to move/copy files larger than 2Gigs, such large files are cut > around 2 Gigs limit. Now, I'm not sure if it has anything to do with the OS > or the filesystems or what. I have seen it both in Linux (debian) and Mac OS > X. And I don't understand either, why the file component does not use the > NIO api or at least we don't have an option to tell it use the NIO api > instead. > Anyway, for now I've solved the problem by using a smiple component instead > of the "file" comp. to move/copy files using the NIO api. It is much faster > and it has no problem with very large files. >
For move it uses java.io.File rename and are you saying its not optimized for the underlying OS? I would assume that it could move a 2gb file in no time if the file is on the same share. Can you share your code that you are using? > Bests, > Farhad S. > > > Claus Ibsen-2 wrote: >> >> On Fri, Nov 27, 2009 at 1:12 PM, shirazi <m...@farhad.eu> wrote: >>> >>> Hi, >>> Ok, I guess I should rather provide an AggregationStrategy. >>> The reason why I use a timer is that when I use the ftp endpoint, it >>> starts >>> to fetch everything that is on the ftp in the first call and that's not >>> what >>> i want, because I'm processing files over 3 Gigs and there are many of >>> them. >>> Then I figured that by using a timer, i could have ftp to download one >>> file >>> at a time!! >>> What I basically need is to somehow tell the ftp, how many files I want >>> to >>> fetch at a time. This is probably doable now in camel-2.1 by using the >>> "ThrottlingInflightRoutePolicy", which i havn't tried yet. >>> Do you have any suggestions? >> >> Yeah the FTP component extends the File component so you got all the >> options from the file component as well >> http://camel.apache.org/file2.html >> >> And there is a maxMessagesPerPoll you can use to tell how many files >> you want at most to poll. For example = 1 in your use. >> >> And that option is avail in 2.0 also. >> >> In fact all these are as they are batch polling consumers >> http://camel.apache.org/batch-consumer.html >> >> >> >>> >>> Bests, >>> Farhad S. >>> >>> >>> >>> Claus Ibsen-2 wrote: >>>> >>>> Hi >>>> >>>> BTW why are you using a timer -> ftp when the ftp have a build in timer? >>>> >>>> >>>> >>>> On Fri, Nov 27, 2009 at 12:55 PM, shirazi <m...@farhad.eu> wrote: >>>>> >>>>> Hi, >>>>> Please consider the following route: >>>>> >>>>> from("timer://someTrigger?delay=30000&fixedRate=true&period=300000"). >>>>> pollEnrich("ftp://cmoreFtp?binary=true&localWorkDirectory=/tmp&idempotent=true&idempotentRepository=#caStreams";, >>>>> 10000). >>>>> to("file:///tmp/vw/cmore?tempPrefix=.inprogress"); >>>>> >>>>> The pollEnrich cast a NullPointerException in camel-2.0, if there is >>>>> nothing >>>>> to fetch from the ftp. I got rid of the exception by a >>>>> "try-catch(NullpointerExcpetion.class).stop()" around the route. Now in >>>>> camel-2.1-SNAPSHOT, I get the following exception on the "to" endpoint: >>>>> ================= >>>>> 12:37:06,899 INFO [Tracer] >>> --> >>>>> file:///tmp/vw/cmore?tempPrefix=.inprogress_, Pattern:InOnly, >>>>> Headers:{firedTime=Fri Nov 27 12:36:56 CET 2009}, >>>>> Exception:org.apache.camel.component.file.GenericFileOperationFailedException: >>>>> Cannot store file: >>>>> /tmp/vw/cmore/.inprogress_02216f38-8ffc-4aec-b638-27229d857075 >>>>> 12:37:06,899 INFO [Tracer] >>> --> >>>>> file:///tmp/vw/cmore?tempPrefix=.inprogress_, Pattern:InOnly, >>>>> Headers:{firedTime=Fri Nov 27 12:36:56 CET 2009}, >>>>> Exception:org.apache.camel.component.file.GenericFileOperationFailedException: >>>>> Cannot store file: >>>>> /tmp/vw/cmore/.inprogress_02216f38-8ffc-4aec-b638-27229d857075 >>>>> 12:37:06,899 ERROR [TimerConsumer] Cannot store file: >>>>> /tmp/vw/cmore/.inprogress_02216f38-8ffc-4aec-b638-27229d857075 >>>>> org.apache.camel.component.file.GenericFileOperationFailedException: >>>>> Cannot >>>>> store file: >>>>> /tmp/vw/cmore/.inprogress_02216f38-8ffc-4aec-b638-27229d857075 >>>>> at >>>>> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:204) >>>>> at >>>>> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:184) >>>>> at >>>>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:126) >>>>> at >>>>> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:57) >>>>> at >>>>> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:97) >>>>> at >>>>> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:95) >>>>> at >>>>> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:146) >>>>> at >>>>> org.apache.camel.processor.SendProcessor.doProcess(SendProcessor.java:94) >>>>> at >>>>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:82) >>>>> at >>>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>>>> at >>>>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53) >>>>> at >>>>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82) >>>>> at >>>>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:162) >>>>> at >>>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>>>> at >>>>> org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:206) >>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:74) >>>>> at >>>>> org.apache.camel.processor.TryProcessor.process(TryProcessor.java:63) >>>>> at >>>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>>>> at >>>>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:53) >>>>> at >>>>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:82) >>>>> at >>>>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:162) >>>>> at >>>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>>>> at >>>>> org.apache.camel.processor.DefaultChannel.process(DefaultChannel.java:206) >>>>> at >>>>> org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:54) >>>>> at >>>>> org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48) >>>>> at >>>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67) >>>>> at >>>>> org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:103) >>>>> at >>>>> org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:50) >>>>> at java.util.TimerThread.mainLoop(Timer.java:512) >>>>> at java.util.TimerThread.run(Timer.java:462) >>>>> Caused by: org.apache.camel.InvalidPayloadException: No body available >>>>> of >>>>> type: java.io.InputStream on: Message: [Body is null]. Caused by: No >>>>> type >>>>> converter available to convert from type: null to the required type: >>>>> java.io.InputStream with value null on the exchange: Exchange[Message: >>>>> [Body >>>>> is null]] >>>>> at >>>>> org.apache.camel.impl.MessageSupport.getMandatoryBody(MessageSupport.java:103) >>>>> at >>>>> org.apache.camel.util.ExchangeHelper.getMandatoryInBody(ExchangeHelper.java:116) >>>>> at >>>>> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:198) >>>>> ... 29 more >>>>> Caused by: org.apache.camel.NoTypeConversionAvailableException: No type >>>>> converter available to convert from type: null to the required type: >>>>> java.io.InputStream with value null >>>>> at >>>>> org.apache.camel.impl.converter.DefaultTypeConverter.mandatoryConvertTo(DefaultTypeConverter.java:126) >>>>> at >>>>> org.apache.camel.impl.MessageSupport.getMandatoryBody(MessageSupport.java:101) >>>>> ... 31 more >>>>> ============ >>>>> Now, I can remedy the situation by a try-catch block or an interceptor, >>>>> but >>>>> in my opinion this should be considered a bug in camel and it should >>>>> silently ignore if there was nothing to fetch from the ftp. >>>>> >>>>> Bests, >>>>> Farhad Shiraz >>>>> >>>>> -- >>>>> View this message in context: >>>>> http://old.nabble.com/camel-casts-exception-when-there-is-nothing-to-fetch-from-the-ftp-endpoint-tp26541447p26541447.html >>>>> Sent from the Camel - Users mailing list archive at Nabble.com. >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Claus Ibsen >>>> Apache Camel Committer >>>> >>>> Author of Camel in Action: http://www.manning.com/ibsen/ >>>> Open Source Integration: http://fusesource.com >>>> Blog: http://davsclaus.blogspot.com/ >>>> Twitter: http://twitter.com/davsclaus >>>> >>>> >>> >>> -- >>> View this message in context: >>> http://old.nabble.com/camel-casts-exception-when-there-is-nothing-to-fetch-from-the-ftp-endpoint-tp26541447p26541650.html >>> Sent from the Camel - Users mailing list archive at Nabble.com. >>> >>> >> >> >> >> -- >> Claus Ibsen >> Apache Camel Committer >> >> Author of Camel in Action: http://www.manning.com/ibsen/ >> Open Source Integration: http://fusesource.com >> Blog: http://davsclaus.blogspot.com/ >> Twitter: http://twitter.com/davsclaus >> >> > > -- > View this message in context: > http://old.nabble.com/camel-casts-exception-when-there-is-nothing-to-fetch-from-the-ftp-endpoint-tp26541447p26541877.html > Sent from the Camel - Users mailing list archive at Nabble.com. > > -- Claus Ibsen Apache Camel Committer Author of Camel in Action: http://www.manning.com/ibsen/ Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus
