Camel ftp read next file after close connection
-----------------------------------------------
Key: CAMEL-4605
URL: https://issues.apache.org/jira/browse/CAMEL-4605
Project: Camel
Issue Type: Bug
Components: camel-ftp
Affects Versions: 2.8.1
Reporter: Aleksey Sushko
There is an error with ftp server.
Many files on the server. All files have no time to be processed.
When the processing of the next file, the process stops.
Error - The request is a file after closing the connection.
Camel config route:
{code}
<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0";
xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0";
xmlns:camel="http://camel.apache.org/schema/blueprint";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
default-activation="lazy">
<cm:property-placeholder persistent-id="ccwe.exchange.oos">
<cm:default-properties>
<cm:property name="ftp.doc.url"
value="ftp://mail01.zakupki.gov.ru/auto/organization/all/"/>
<cm:property name="ftp.doc.add_params" value=""/>
</cm:default-properties>
</cm:property-placeholder>
<bean id="readProcess" class="mypackage.docread.ReadProcess"/>
<bean id="routePolicy"
class="org.apache.camel.routepolicy.quartz.CronScheduledRoutePolicy">
<property name="routeStartTime" value="* 0/10 * * * ?"/>
<property name="routeStopTime" value="* 4/10 * * * ?"/>
</bean>
<camelContext id="exchange-read"
xmlns="http://camel.apache.org/schema/blueprint"; autoStartup="false">
<threadPoolProfile id="fooProfile"
poolSize="2" maxPoolSize="2" maxQueueSize="-1"/>
<route id="readRoute"
routePolicyRef="routePolicy"
shutdownRunningTask= "CompleteAllTasks">
<from
uri="{{ftp.doc.url}}?noop=true&binary=true{{ftp.doc.add_params}}"/>
<to uri="bean:readProcess"/>
</route>
</camelContext>
</blueprint>
{code}
We have two streams
bq. <threadPoolProfile id="fooProfile" poolSize="2" maxPoolSize="2"
maxQueueSize="-1"/>
The first thread (thread#0) reading file from ftp server
{code}
2011-11-02 09:13:57,376 | TRACE | Camel (exchange-read) thread #0 -
ftp://mail01.zakupki.gov.ru/auto/organization/all/ | FtpConsumer
| mponent.file.GenericFileConsumer 261 | 94 -
org.apache.camel.camel-core - 2.8.1 |
Processing file: GenericFile[organization_all_20111023_010003_33.xml.zip]
2011-11-02 09:13:57,376 | TRACE | Camel (exchange-read) thread #0 -
ftp://mail01.zakupki.gov.ru/auto/organization/all/ | FtpConsumer
| mponent.file.GenericFileConsumer 293 | 94 -
org.apache.camel.camel-core - 2.8.1 |
Retrieving file:
auto/organization/all/organization_all_20111023_010003_33.xml.zip from:
Endpoint[ftp://mail01.zakupki.gov.ru/auto/organization/all/?
binary=true&connectTimeout=10000&delay=600000&disconnect=true&localWorkDirectory=%2Fhome%2Fuser%2Ftemp&noop=true&passiveMode=true&soTimeout=10000&timeout=15000]
{code}
The second stream (thread#1) is stopped
{code}
2011-11-02 09:14:00,004 | DEBUG | Camel (exchange-read) thread #1 -
ShutdownTask | ScheduledPollConsumer |
camel.impl.ScheduledPollConsumer 265 | 94 - org.apache.camel.camel-core -
2.8.1 |
This consumer is stopping, so cancelling scheduled task:
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@2bf8d0a4
2011-11-02 09:14:00,005 | DEBUG | Camel (exchange-read) thread #1 -
ShutdownTask | FtpConsumer |
pache.camel.impl.DefaultConsumer 83 | 94 - org.apache.camel.camel-core -
2.8.1 |
Stopping consumer:
FtpConsumer[ftp://mail01.zakupki.gov.ru/auto/organization/all/?binary=true&connectTimeout=10000&delay=600000&disconnect=true&localWorkDirectory=%2Fhome%2Fuser%2Ftemp&noop=true&passiveMode=true&soTimeout=10000&timeout=15000]
2011-11-02 09:14:00,007 | DEBUG | Camel (exchange-read) thread #1 -
ShutdownTask | ProcessorEndpoint$1 |
pache.camel.impl.DefaultProducer 76 | 94 - org.apache.camel.camel-core -
2.8.1 |
Stopping producer: Producer[bean://readProcess]
2011-11-02 09:14:00,010 | DEBUG | Camel (exchange-read) thread #1 -
ShutdownTask | FtpConsumer |
t.file.remote.RemoteFileConsumer 102 | 94 - org.apache.camel.camel-core -
2.8.1 |
Disconnecting from: ftp://anonym...@mail01.zakupki.gov.ru:21
{code}
Warning!!!! ftp close connection. Completed method FTPClient.disconnect().
Parament _controlInput_ = null.
{code}
2011-11-02 09:14:03,182 | INFO | Camel (exchange-read) thread #1 -
ShutdownTask | DefaultShutdownStrategy |
ultShutdownStrategy$ShutdownTask 460 | 94 - org.apache.camel.camel-core -
2.8.1 |
Route: readRoute shutdown complete, was consuming from: Endpoint[...]
2011-11-02 09:14:03,182 | INFO | DefaultQuartzScheduler-exchange-read_Worker-1
| DefaultShutdownStrategy | mel.impl.DefaultShutdownStrategy 158 | 94
- org.apache.camel.camel-core - 2.8.1 |
Graceful shutdown of 1 routes completed in 3 seconds
{code}
The first thread (thread#0) continues reading the file
[organization_all_20111023_010003_33.xml.zip].
{code}
2011-11-02 09:14:04,172 | ERROR | Camel (exchange-read) thread #0 -
ftp://mail01.zakupki.gov.ru/auto/organization/all/ | FtpConsumer
| ache.camel.processor.CamelLogger 232 | 94 -
org.apache.camel.camel-core - 2.8.1 | Caused by:
[java.lang.NullPointerException - null]
java.lang.NullPointerException
at
org.apache.commons.net.ftp.FTP.__getReply(FTP.java:295)[97:org.apache.commons.net:2.2]
at
org.apache.commons.net.ftp.FTP.getReply(FTP.java:622)[97:org.apache.commons.net:2.2]
at
org.apache.commons.net.ftp.FTPClient.completePendingCommand(FTPClient.java:1408)[97:org.apache.commons.net:2.2]
at
org.apache.commons.net.ftp.FTPClient.retrieveFile(FTPClient.java:1467)[97:org.apache.commons.net:2.2]
at
org.apache.camel.component.file.remote.FtpOperations.retrieveFileToFileInLocalWorkDirectory(FtpOperations.java:406)[99:org.apache.camel.camel-ftp:2.8.1]
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
