Hey everybody,
I made wrote a little piece of camel to consume a ftp server.
But after it was running for some time, it throws an exception, keeps
running but doesn't consume anything any more. Also when I start it again
and there are a larger number of file waiting to be consumed it will crash
again. I already added an exception handler but it seems like is doesn't
catch the exceptions.
This is the exception I receive:
Caused by:
[org.apache.camel.component.file.GenericFileOperationFailedException - File
operation failed: 150 Opening ASCII mode data connection for
2386442.XML(3895 bytes).
Accept timed out. Code: 150]
org.apache.camel.component.file.GenericFileOperationFailedException: File
operation failed: 150 Opening ASCII mode data connection for
2386442.XML(3895 bytes).
Accept timed out. Code: 150
at
org.apache.camel.component.file.remote.FtpOperations.retrieveFileToStreamInBody(FtpOperations.java:336)
at
org.apache.camel.component.file.remote.FtpOperations.retrieveFile(FtpOperations.java:297)
at
org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:333)
at
org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:94)
at
org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:175)
at
org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:136)
at
org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:140)
at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:92)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.SocketTimeoutException: Accept timed out
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
at java.net.ServerSocket.implAccept(ServerSocket.java:462)
at java.net.ServerSocket.accept(ServerSocket.java:430)
at
org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java:560)
at
org.apache.commons.net.ftp.FTPClient.retrieveFile(FTPClient.java:1442)
at
org.apache.camel.component.file.remote.FtpOperations.retrieveFileToStreamInBody(FtpOperations.java:328)
... 16 more
Caused by:
[org.apache.camel.component.file.GenericFileOperationFailedException -
Cannot retrieve file: GenericFile[2386448.XML] from:
Endpoint[ftp://1.1.1.1?delay=15000&delete=true&disconnect=true&exclude=((?i).*pdf$)&password=******&username=user]
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
retrieve file: GenericFile[2386448.XML] from:
Endpoint[ftp://1.1.1.1?delay=15000&delete=true&disconnect=true&exclude=((?i).*pdf$)&password=******&username=user]
at
org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:338)
at
org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:94)
at
org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:175)
at
org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:136)
at
org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:140)
at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:92)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
And this is the route I made using the Java DSL:
// XML Predicate
// only allows names without spaces
Predicate xmlPredicate = header(RssUtils.CAMEL_FILE_NAME).regex(
"([\\S]+(\\.(?i)(xml))$)");
// Images Predicate
// only allows names without spaces
Predicate imgPredicate = header(RssUtils.CAMEL_FILE_NAME).regex(
"([\\S]+(\\.(?i)(jpg|png|gif))$)");
onException(SchemaValidationException.class).to(
"file://" + props.getProperty(RssUtils.ROOT_DIR)
+
"/errors/SchemaValidationException");
onException(GenericFileOperationFailedException.class).to(
"file://" + props.getProperty(RssUtils.ROOT_DIR)
+
"/errors/GenericFileExceptions");
from(
"ftp://";
+
props.getProperty(RssUtils.FTP_URL)
+ "?username="
+
props.getProperty(RssUtils.FTP_USER)
+ "&password="
+
props.getProperty(RssUtils.FTP_PWD)
+
"&disconnect=true&delete=true&exclude=((?i).*pdf$)&delay="
+
props.getProperty(RssUtils.FTP_DELAY))
.choice()
.when(xmlPredicate)
.to("jms:xmlQueue")
.to("jms:archiveQueue")
.when(imgPredicate)
.to("file://" +
props.getProperty(RssUtils.ROOT_DIR) + "/img")
.otherwise()
.to("file://" +
props.getProperty(RssUtils.ROOT_DIR)
+ "/errors/other");
from("jms:xmlQueue").to("validator:FtpXmlValidator.xsd")
.to("xslt://XmlToRssConverter.xsl")
.process(rssFeedProcessor)
.to("file://" +
props.getProperty(RssUtils.ROOT_DIR) + "/rss/");
from("jms:archiveQueue")
.to("file://" + props.getProperty(RssUtils.ROOT_DIR) + "/archive/");
Is there anything I can do to avoid this kind af behavior? It is really
difficult to test so I'm hoping somebody spots a flaw in my code. I have
searching for quite some time now but I don't find anything solid.
There maybe a few things that I found somebody could give his tought on:
- use handled(true) when using the onException
- can I set the max batch size of the consumer? (can I use throttle for this
?)
- use explicit try catch finally because I'm using the Java DSL
Don't shoot me if I'm saying anything wrong here, I just learning Camel.
So if anybody has suggestions on the code above I would appreciate it!
Thanks a lot in advance!
--
View this message in context:
http://camel.465427.n5.nabble.com/Failing-FTP-Component-tp5715550.html
Sent from the Camel - Users mailing list archive at Nabble.com.
