I am trying to create a file polling route which will poll the source SFTP
server and transfer the file to the destination SFTP server.
The file count is huge and the source can generate upto 270 - 300 files per
second where as the file size is in KB.
I am planing to have 3 instance (say A, B, C) of the same route in active
mode where each route is capable of handling and transfering 100 files per
each poll.
Once a file is picked for processing by say intance A then the same file
should not be picked for processing by the rest 2 active instance Intance B
& C.
I have created the below route and this route is transfering files ar the
rate of 5-6 files per second.
from("sftp://user:passw...@source.sftp.server.com/input";
+ "?readLock=changed"
+ "&readLockMinAge=1m"
+ "&readLockTimeout=70000"
+ "&readLockCheckInterval=5000"
+ "&delay=1000"
+ "&preMove=processing"
+ "&maxMessagesPerPoll=100"
+ "&move=../archive"
+ "&localWorkDirectory=Invoice_ProgressESB"
+ "&stepwise=false"
+ "&include=.*(txt)$")
.threads(30, 35)
.log(LoggingLevel.INFO, "downloading files from
Source SFTP Server")
.to("sftp://user:passw...@destination.sftp.server.com/output";)
.end()
Is there any other configuration that needs to be applied to achieve the
processing speed of 100 files per poll and to process the file between
instance in an
intelligent and efficient manner.
Any help suggestion or pointers is much appreciated .
--
View this message in context:
http://camel.465427.n5.nabble.com/How-to-increase-the-performance-for-SFTP-file-polling-consumer-tp5792312.html
Sent from the Camel - Users mailing list archive at Nabble.com.
