[
https://issues.apache.org/jira/browse/CAMEL-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143110#comment-13143110
]
Aleksey Sushko commented on CAMEL-4605:
---------------------------------------
Redefine the creation of a method of FTPClient FTPClient createFtpClient ()
class FtpEndpoint.
Add to the method displays the message
{code}
LOGGER.info("create FTPClient {}", client);
{code}
{code}
public class FTPClientTest extends FTPClient {
private static final Logger LOGGER =
LoggerFactory.getLogger(FTPClientTest.class);
public FTPClientTest() {
super();
}
@Override
public void disconnect() throws IOException {
LOGGER.info("FTPClient disconnect {}", this);
super.disconnect();
}
@Override
protected Socket _openDataConnection_(int command, String arg) throws
IOException {
LOGGER.info("FTPClient _openDataConnection_ {}", this);
return super._openDataConnection_(command, arg);
}
@Override
protected void _connectAction_() throws IOException {
LOGGER.info("FTPClient _connectAction_ {}", this);
super._connectAction_();
}
}
{code}
We observe the creation of two FTPClient
{code}
2011-11-03 15:22:00,862 | INFO | Blueprint Extender: 2 | FtpEndpointExt |
t.file.remote.ext.FtpEndpointExt 55 | 115 - org.apache.camel.c
amel-ftp - 2.8.1 | create FTPClient
org.apache.camel.component.file.remote.ext.FTPClientTest@2e81d81e
{code}
...
{code}
2011-11-03 15:22:01,213 | INFO | Blueprint Extender: 1 | FtpEndpointExt |
t.file.remote.ext.FtpEndpointExt 55 | 115 - org.apache.camel.c
amel-ftp - 2.8.1 | create FTPClient
org.apache.camel.component.file.remote.ext.FTPClientTest@738e7a09
{code}
Last post by FTPClient
{code}
2011-11-03 15:28:01,543 | INFO | Camel (exchange-read) thread #0 -
ftp://mail01.zakupki.gov.ru/auto/organization/all/ | FTPClientTest|
nt.file.remote.ext.FTPClientTest 25 | 115 - org.apache.camel.camel-ftp -
2.8.1 |
FTPClient _openDataConnection_
org.apache.camel.component.file.remote.ext.FTPClientTest@2e81d81e
{code}
Note the number of object *FTPClientTest@2e81d81e*.
Begins to stop the second process (thread # 1):
{code}
2011-11-03 15:28:09,010 | INFO | Camel (exchange-read) thread #1 -
ShutdownTask | DefaultShutdownStrategy |
ultShutdownStrategy$ShutdownTask 425 | 94 - org.apache.camel.camel-core -
2.8.1 |
Waiting as there are still 56 inflight and pending exchanges to complete,
timeout in 291 seco
nds.
2011-11-03 15:28:10,004 | WARN | DefaultQuartzScheduler-exchange-read_Worker-1
| DefaultShutdownStrategy | mel.impl.DefaultShutdownStrategy 143 | 94
- org.apache.camel.camel-core - 2.8.1 |
Timeout occurred. Now forcing the routes to be shutdown now.
2011-11-03 15:28:10,004 | WARN | Camel (exchange-read) thread #1 -
ShutdownTask | DefaultShutdownStrategy | ultShutdownStrategy$ShutdownTask 433
| 94 - org.apache.camel.camel-core - 2.8.1 |
Interrupted while waiting during graceful shutdown, will force shutdown now.
{code}
Completing the process (thread # 0):
{code}
2011-11-03 15:28:10,011 | TRACE | DefaultQuartzScheduler-exchange-read_Worker-1
| ServiceHelper | .apache.camel.util.ServiceHelper 103 | 94 -
org.apache.camel.camel-core - 2.8.1 |
Service already stopped:
org.apache.camel.impl.SharedProducerServicePool@50ec2522
2011-11-03 15:28:10,011 | TRACE | DefaultQuartzScheduler-exchange-read_Worker-1
| ServiceHelper | .apache.camel.util.ServiceHelper 108 | 94 -
org.apache.camel.camel-core - 2.8.1 |
Stopping service
DefaultErrorHandler[TraceInterceptor[To[bean:dictStoreProcess]]]
2011-11-03 15:28:10,011 | DEBUG | DefaultQuartzScheduler-exchange-read_Worker-1
| FtpConsumer | t.file.remote.RemoteFileConsumer 102 | 94 -
org.apache.camel.camel-core - 2.8.1 |
Disconnecting from: ftp://[email protected]:21
2011-11-03 15:28:12,908 | INFO | DefaultQuartzScheduler-exchange-read_Worker-1
| FTPClientTest | nt.file.remote.ext.FTPClientTest 19 | 115 -
org.apache.camel.camel-ftp - 2.8.1 |
FTPClient disconnect
org.apache.camel.component.file.remote.ext.FTPClientTest@2e81d81e
{code}
As you can see, the system is stopped *FTPClientTest@2e81d81e*. This object
belongs to the first thread (thread # 0).
> Camel ftp read file after close connection
> ------------------------------------------
>
> Key: CAMEL-4605
> URL: https://issues.apache.org/jira/browse/CAMEL-4605
> Project: Camel
> Issue Type: Bug
> Components: camel-ftp
> Affects Versions: 2.8.1
> Reporter: Aleksey Sushko
>
> There is an error with ftp server.
> Many files on the server. All files have no time to be processed.
> When the processing of the next file, the process stops.
> Error - The request is a file after closing the connection.
> Camel config route:
> {code}
> <?xml version="1.0" encoding="UTF-8"?>
> <blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0";
> xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0";
> xmlns:camel="http://camel.apache.org/schema/blueprint";
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> default-activation="lazy">
> <cm:property-placeholder persistent-id="ccwe.exchange.oos">
> <cm:default-properties>
> <cm:property name="ftp.doc.url"
> value="ftp://mail01.zakupki.gov.ru/auto/organization/all/"/>
> <cm:property name="ftp.doc.add_params" value=""/>
> </cm:default-properties>
> </cm:property-placeholder>
>
> <bean id="readProcess" class="mypackage.docread.ReadProcess"/>
> <bean id="routePolicy"
> class="org.apache.camel.routepolicy.quartz.CronScheduledRoutePolicy">
> <property name="routeStartTime" value="* 0/10 * * * ?"/>
> <property name="routeStopTime" value="* 4/10 * * * ?"/>
> </bean>
> <camelContext id="exchange-read"
> xmlns="http://camel.apache.org/schema/blueprint"; autoStartup="false">
>
> <threadPoolProfile id="fooProfile"
> poolSize="2" maxPoolSize="2" maxQueueSize="-1"/>
>
> <route id="readRoute"
> routePolicyRef="routePolicy"
> shutdownRunningTask= "CompleteAllTasks">
>
> <from
> uri="{{ftp.doc.url}}?noop=true&binary=true{{ftp.doc.add_params}}"/>
> <to uri="bean:readProcess"/>
> </route>
> </camelContext>
> </blueprint>
> {code}
> We have two streams
> bq. <threadPoolProfile id="fooProfile" poolSize="2" maxPoolSize="2"
> maxQueueSize="-1"/>
> The first thread (thread#0) reading file from ftp server
> {code}
> 2011-11-02 09:13:57,376 | TRACE | Camel (exchange-read) thread #0 -
> ftp://mail01.zakupki.gov.ru/auto/organization/all/ | FtpConsumer
> | mponent.file.GenericFileConsumer 261 | 94 -
> org.apache.camel.camel-core - 2.8.1 |
> Processing file: GenericFile[organization_all_20111023_010003_33.xml.zip]
> 2011-11-02 09:13:57,376 | TRACE | Camel (exchange-read) thread #0 -
> ftp://mail01.zakupki.gov.ru/auto/organization/all/ | FtpConsumer
> | mponent.file.GenericFileConsumer 293 | 94 -
> org.apache.camel.camel-core - 2.8.1 |
> Retrieving file:
> auto/organization/all/organization_all_20111023_010003_33.xml.zip from:
> Endpoint[ftp://mail01.zakupki.gov.ru/auto/organization/all/?
> binary=true&connectTimeout=10000&delay=600000&disconnect=true&localWorkDirectory=%2Fhome%2Fuser%2Ftemp&noop=true&passiveMode=true&soTimeout=10000&timeout=15000]
> {code}
> The second stream (thread#1) is stopped
> {code}
> 2011-11-02 09:14:00,004 | DEBUG | Camel (exchange-read) thread #1 -
> ShutdownTask | ScheduledPollConsumer |
> camel.impl.ScheduledPollConsumer 265 | 94 - org.apache.camel.camel-core -
> 2.8.1 |
> This consumer is stopping, so cancelling scheduled task:
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@2bf8d0a4
> 2011-11-02 09:14:00,005 | DEBUG | Camel (exchange-read) thread #1 -
> ShutdownTask | FtpConsumer |
> pache.camel.impl.DefaultConsumer 83 | 94 - org.apache.camel.camel-core -
> 2.8.1 |
> Stopping consumer:
> FtpConsumer[ftp://mail01.zakupki.gov.ru/auto/organization/all/?binary=true&connectTimeout=10000&delay=600000&disconnect=true&localWorkDirectory=%2Fhome%2Fuser%2Ftemp&noop=true&passiveMode=true&soTimeout=10000&timeout=15000]
> 2011-11-02 09:14:00,007 | DEBUG | Camel (exchange-read) thread #1 -
> ShutdownTask | ProcessorEndpoint$1 |
> pache.camel.impl.DefaultProducer 76 | 94 - org.apache.camel.camel-core -
> 2.8.1 |
> Stopping producer: Producer[bean://readProcess]
> 2011-11-02 09:14:00,010 | DEBUG | Camel (exchange-read) thread #1 -
> ShutdownTask | FtpConsumer |
> t.file.remote.RemoteFileConsumer 102 | 94 - org.apache.camel.camel-core -
> 2.8.1 |
> Disconnecting from: ftp://[email protected]:21
> {code}
> Warning!!!! ftp close connection. Completed method FTPClient.disconnect().
> Parament _controlInput_ = null.
> {code}
> 2011-11-02 09:14:03,182 | INFO | Camel (exchange-read) thread #1 -
> ShutdownTask | DefaultShutdownStrategy |
> ultShutdownStrategy$ShutdownTask 460 | 94 - org.apache.camel.camel-core -
> 2.8.1 |
> Route: readRoute shutdown complete, was consuming from: Endpoint[...]
> 2011-11-02 09:14:03,182 | INFO |
> DefaultQuartzScheduler-exchange-read_Worker-1 | DefaultShutdownStrategy
> | mel.impl.DefaultShutdownStrategy 158 | 94 - org.apache.camel.camel-core
> - 2.8.1 |
> Graceful shutdown of 1 routes completed in 3 seconds
> {code}
> The first thread (thread#0) continues reading the file
> [organization_all_20111023_010003_33.xml.zip].
> {code}
> 2011-11-02 09:14:04,172 | ERROR | Camel (exchange-read) thread #0 -
> ftp://mail01.zakupki.gov.ru/auto/organization/all/ | FtpConsumer
> | ache.camel.processor.CamelLogger 232 | 94 -
> org.apache.camel.camel-core - 2.8.1 | Caused by:
> [java.lang.NullPointerException - null]
> java.lang.NullPointerException
> at
> org.apache.commons.net.ftp.FTP.__getReply(FTP.java:295)[97:org.apache.commons.net:2.2]
> at
> org.apache.commons.net.ftp.FTP.getReply(FTP.java:622)[97:org.apache.commons.net:2.2]
> at
> org.apache.commons.net.ftp.FTPClient.completePendingCommand(FTPClient.java:1408)[97:org.apache.commons.net:2.2]
> at
> org.apache.commons.net.ftp.FTPClient.retrieveFile(FTPClient.java:1467)[97:org.apache.commons.net:2.2]
> at
> org.apache.camel.component.file.remote.FtpOperations.retrieveFileToFileInLocalWorkDirectory(FtpOperations.java:406)[99:org.apache.camel.camel-ftp:2.8.1]
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
