Hi,
there was a blog post recently addressing just that by Josh Reagan:
http://joshdreagan.github.io/2017/01/05/faster_file_consumption_with_camel/
see if that helps,
zoran
On Mon, Jan 9, 2017 at 12:55 PM, ravishankarhassain
<ravishankarhass...@gmail.com> wrote:
> I am trying to create a file polling route which will poll the source SFTP
> server and transfer the file to the destination SFTP server.
> The file count is huge and the source can generate upto 270 - 300 files per
> second where as the file size is in KB.
>
> I am planing to have 3 instance (say A, B, C) of the same route in active
> mode where each route is capable of handling and transfering 100 files per
> each poll.
> Once a file is picked for processing by say intance A then the same file
> should not be picked for processing by the rest 2 active instance Intance B
> & C.
>
> I have created the below route and this route is transfering files ar the
> rate of 5-6 files per second.
>
>
> from("sftp://user:passw...@source.sftp.server.com/input";
> + "?readLock=changed"
> + "&readLockMinAge=1m"
> + "&readLockTimeout=70000"
> + "&readLockCheckInterval=5000"
> + "&delay=1000"
> + "&preMove=processing"
> + "&maxMessagesPerPoll=100"
> + "&move=../archive"
> + "&localWorkDirectory=Invoice_ProgressESB"
> + "&stepwise=false"
> + "&include=.*(txt)$")
> .threads(30, 35)
> .log(LoggingLevel.INFO, "downloading files
> from Source SFTP Server")
>
> .to("sftp://user:passw...@destination.sftp.server.com/output";)
> .end()
>
> Is there any other configuration that needs to be applied to achieve the
> processing speed of 100 files per poll and to process the file between
> instance in an
> intelligent and efficient manner.
>
> Any help suggestion or pointers is much appreciated .
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/How-to-increase-the-performance-for-SFTP-file-polling-consumer-tp5792312.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Zoran Regvart
