Cannot connect to sftp betweenwhiles
Hi all,
i have a simple route that is uploading to ftp,
and sometimes i have the following error while trying to upload file to ftp,
the strange thing is that it does not fail always, approximately 1 of 5
trial fails.
Any help will be appreciated.
Route:
from("file://folder?move=done&delay=2000&moveFailed=error")
.idempotentConsumer(simple("${routeId} : ${in.header.CamelFileName} -
${in.header.CamelFileLastModified}"), idempotentRepository)
.to("sftp://u...@host.com:port?password=pw&disconnect=true&binary=true&connectTimeout=3&soTimeout=30";)
.end();
Info:
OS: windows server 2008
camelVersion: 2.15.2
Exception:
Cause:Cannot connect to sftp://u...@host.com:port
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
connect to sftp://u...@host.com:port
at
org.apache.camel.component.file.remote.SftpOperations.connect(SftpOperations.java:143)
at
org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:202)
at
org.apache.camel.component.file.remote.RemoteFileProducer.recoverableConnectIfNecessary(RemoteFileProducer.java:194)
at
org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:126)
at
org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:113)
at
org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:56)
at
org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
at
org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:157)
at
org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:304)
at
org.apache.camel.processor.SendProcessor.process(SendProcessor.java:152)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at
org.apache.camel.processor.idempotent.IdempotentConsumer.process(IdempotentConsumer.java:125)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at
org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:435)
at
org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:211)
at
org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:175)
at
org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.jcraft.jsch.JSchException: Session.connect:
java.net.SocketException: Software caused connection abort: recv failed
at com.jcraft.jsch.Session.connect(Session.java:558)
at
org.apache.camel.component.file.remote.SftpOperations.connect(SftpOperations.java:115)
... 31 common frames omitted
--
View this message in context:
http://camel.465427.n5.nabble.com/Cannot-connect-to-sftp-betweenwhiles-tp5771048.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: FTP Reconnect attempt problem
Hi, This is what i was looking for, Thanks Claus! "backoff" saved my life. -- View this message in context: http://camel.465427.n5.nabble.com/FTP-Reconnect-attempt-problem-tp5765466p5765523.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: FTP Reconnect attempt problem
Hi,
How it is meaningful to put that "Thread.sleep(reconnectDelay)" to
CustomPollStrategy instead of the one in SftpOperations class ?
public class CustomPollStrategy extends
RemoteFilePollingConsumerPollStrategy {
@Override
public synchronized boolean rollback(Consumer consumer, Endpoint
endpoint, int retryCounter, Exception cause) throws Exception {
//retry 3 times
if(retryCounter <= 2) {
//force disconnect
super.rollback(consumer, endpoint, retryCounter, cause);
return true;
}
else {
//give up retry and wait for the next connection attempt.
*try {
Thread.sleep(((RemoteFileEndpoint)
endpoint).getReconnectDelay());
} catch (Exception e) {
}*
return false;
}
}
}
Does it have a negative impact on another place of the camel?
Thanks in advance.
--
View this message in context:
http://camel.465427.n5.nabble.com/FTP-Reconnect-attempt-problem-tp5765466p5765513.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: FTP Reconnect attempt problem
I think i found the problem,
The code cannot reach to the sleep(reconnectDelay) because
getMaximumReconnectAttempts() is zero.
I need maximumReconnectAttempts=0 to work with customPollStrategy to caught
exceptions,
but need to set reconnectDelay either.
Any ideas will be appreciated.
//From SftpOperations class:
...
LOG.trace("Cannot connect due: {}", failed.getMessage());
attempt++;
if (attempt > *endpoint.getMaximumReconnectAttempts()*) {
throw failed;
}
if (*endpoint.getReconnectDelay()* > 0) {
try {
Thread.sleep(endpoint.getReconnectDelay());
} catch (InterruptedException ie) {
// we could potentially also be interrupted during
sleep
Thread.currentThread().interrupt();
throw new
GenericFileOperationFailedException("Interrupted during sleeping", ie);
}
}
...
--
View this message in context:
http://camel.465427.n5.nabble.com/FTP-Reconnect-attempt-problem-tp5765466p5765484.html
Sent from the Camel - Users mailing list archive at Nabble.com.
FTP Reconnect attempt problem
Hi,
I have a route as below. My problem is about reconnect attempts.
I am waiting 1 min for the next poll from ftp with the parameter
"consumer.delay=6" that is ok, but if the connection fails, expecting 2
min wait before next connection attempt but
below code is still using 1 min delay for the next connection attempt.
How can i achive this?
(camel version: 2.14.0)
from("sftp://user@host:port/?password=pw";
+ "&*consumer.delay=6*"
+ "&passiveMode=true"
+ "&consumer.bridgeErrorHandler=true"
+ "&throwExceptionOnConnectFailed=true"
+ "&maximumReconnectAttempts=0"
+ "&*reconnectDelay=12*" //reconnect
attempt every 2 min
+ "&pollStrategy=#customPollStrategy")
.log("consuming file from ftp : ${headers.CamelFileName}");
and my custom poll strategy is as foolws:
public class CustomPollStrategy extends
RemoteFilePollingConsumerPollStrategy {
@Override
public synchronized boolean rollback(Consumer consumer, Endpoint
endpoint, int retryCounter, Exception cause) throws Exception {
//retry 3 times
if(retryCounter <= 2) {
//force disconnect
super.rollback(consumer, endpoint, retryCounter, cause);
return true;
}
else {
//give up retry and wait for the next connection attempt.
return false;
}
}
}
--
View this message in context:
http://camel.465427.n5.nabble.com/FTP-Reconnect-attempt-problem-tp5765466.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: retryWhile predicate is not called on ftp connection failure
Hi Claus, I can handle the exception with the help of custom poll strategy for the ftp consumer(from), thanks for the suggestion. But i noticed that if i use custom pollStrategy i can delay reconnect attempt with the "delay=xxx" attribute instead of "reconnectDelay=xxx" attribute. is this normal? Regards. -- View this message in context: http://camel.465427.n5.nabble.com/retryWhile-predicate-is-not-called-on-ftp-connection-failure-tp5762601p5762647.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: retryWhile predicate is not called on ftp connection failure
Yes, GenericFileOperationFailedException is thrown. Trying with
Exception.class did not change the situation.
onException(Exception.class)
.handled(true)
.retryWhile(new Predicate() {
@Override
public boolean matches(Exchange exchange) {
System.out.println("in predicate..");
return false;
}
})
.process(new Processor() {
@Override
public void process(Exchange exchange) throws Exception {
System.out.println("in precess..");
}
});
CamelExceptionCaught=org.apache.camel.component.file.GenericFileOperationFailedException:
File operation failed: 530 Login incorrect.
. Code: 530
--
View this message in context:
http://camel.465427.n5.nabble.com/retryWhile-predicate-is-not-called-on-ftp-connection-failure-tp5762601p5762628.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: retryWhile predicate is not called on ftp connection failure
Hi Morgan,
Thanks for the quick reply.
I looked at the page you suggested and also tried with handler,
But the strange thing is:
I have another route in which i connect to ftp too. But that route works as
i expected and retryWhile predicate is checked.
Predicate is considered:
from("direct:start")
.to("ftp://user@host?password=***&disconnect=true&binary=true";)
Predicate is not considered:
from("ftp://user@host?password=***";
+ "&passiveMode=true&consumer.delay=1&fastExistsCheck=true"
+ "&consumer.bridgeErrorHandler=true"
+ "&throwExceptionOnConnectFailed=true" )
.log("");
--
View this message in context:
http://camel.465427.n5.nabble.com/retryWhile-predicate-is-not-called-on-ftp-connection-failure-tp5762601p5762607.html
Sent from the Camel - Users mailing list archive at Nabble.com.
retryWhile predicate is not called on ftp connection failure
Hi,
I have an FTP route for consuming files and working fine. When the FTP fails
to connect I want to catch the exception, try for some more and later give
up retrying connection attempt.
Here is my routes, when i debug, process in onException is working and
printing "in process", but predicate in retryWhile is not working and not
printing "in predicate".
I use camel version 2.14.0.
onException(GenericFileOperationFailedException.class)
.handled(true)
.retryWhile(new Predicate() {
@Override
public boolean matches(Exchange exchange) {
System.out.println("in predicate..");
return false;
}
})
.process(new Processor() {
@Override
public void process(Exchange exchange) throws Exception {
System.out.println("in precess..");
}
});
from("ftp://user@host?password=***";
+ "&passiveMode=true&consumer.delay=1&fastExistsCheck=true"
+ "&consumer.bridgeErrorHandler=true"
+ "&throwExceptionOnConnectFailed=true"
).log("");
--
View this message in context:
http://camel.465427.n5.nabble.com/retryWhile-predicate-is-not-called-on-ftp-connection-failure-tp5762601.html
Sent from the Camel - Users mailing list archive at Nabble.com.
