Hi Great that the relative works. This is also what I expected.
In terms of absolute then Camel changes the directory to / (since its absolute) and the SFTP library returns OK for this. But in reality it does not go to the root path, but most likely just goes to the home directory of the user. And hence why absolute does not work for you. I don't see any API or way to tell SFTP library it should change to root path and not home folder. A solution to be able to poll files from an absolute folder is to create a symlink in the user home directory which can be polled by the SFTP library. > nlike the polling stage where it says "doPollDirectory from absolutePath: > /tmp/activity_logs". The reason why you see this is because Camel logs the path configured from the endpoint, when the poller starts. Hence you see the leading /. On Tue, Oct 5, 2010 at 12:58 AM, Lorrin Nelson <lhn_git...@nerdylorrin.net> wrote: > Hi Claus, thanks for the continued support. With the latest code (revision > 1003927) relative URIs are working again for me. > > This the first time I'm using camel-ftp. The server I'm talking to is a > Fedora 11 box running OpenSSH_5.2p1. Regular FTP is an unlikely option for > us. Relative URLs are an acceptable work-around. > > Below is a trace log of an absolute URL. > > On the first attempt, note that ~/tmp does exist. Therefore when it tries to > go to *relative* path tmp (which is the wrong thing to do), it succeeds. Then > when it tries to go one level deeper into pitch_activity_logs it fails. The > second attempt is really interesting. Note there that the current working > directory has remained at /home/tomcat/tmp. So when it fails to go to > *relative* path tmp, it fails right away. BTW, note that it says "Retrieving > file: tmp/activity_logs/activity_log.1285909200" with no leading slash, > unlike the polling stage where it says "doPollDirectory from absolutePath: > /tmp/activity_logs". > > Cheers > -Lorrin > > > SftpConsumer 2010-10-04 15:45:07,445 -- INFO -- Connected and logged in to: > sftp://tom...@host:22 > SftpConsumer 2010-10-04 15:45:07,494 -- TRACE -- doPollDirectory from > absolutePath: /tmp/activity_logs, dirName: null > SftpOperations 2010-10-04 15:45:07,494 -- TRACE -- Changing directory: / > SftpOperations 2010-10-04 15:45:07,608 -- TRACE -- Changing directory: tmp > SftpOperations 2010-10-04 15:45:07,697 -- TRACE -- Changing directory: > activity_logs > SftpConsumer 2010-10-04 15:45:07,788 -- TRACE -- Polling directory: > /tmp/activity_logs > SftpConsumer 2010-10-04 15:45:08,011 -- TRACE -- Found 4 in directory: > /tmp/activity_logs > SftpOperations 2010-10-04 15:45:08,012 -- TRACE -- Changing directory: / > SftpOperations 2010-10-04 15:45:08,099 -- TRACE -- Changing directory: home > SftpOperations 2010-10-04 15:45:08,189 -- TRACE -- Changing directory: tomcat > SftpConsumer 2010-10-04 15:45:08,274 -- DEBUG -- Took 0.829 seconds to poll: > /tmp/activity_logs/ > SftpConsumer 2010-10-04 15:45:08,275 -- DEBUG -- Total 1 files to consume > SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Processing file: > GenericFile[activity_log.1285909200] > SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Retrieving file: > tmp/activity_logs/activity_log.1285909200 from: > Endpoint[sftp://tom...@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******] > SftpOperations 2010-10-04 15:45:08,277 -- TRACE -- Changing directory: tmp > SftpOperations 2010-10-04 15:45:08,367 -- TRACE -- Changing directory: > activity_logs > SftpConsumer 2010-10-04 15:45:08,414 -- ERROR -- Caused by: > [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot > change directory to: activity_logs] > org.apache.camel.component.file.GenericFileOperationFailedException: Cannot > change directory to: activity_logs > at > org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383) > at > org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368) > at > org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451) > at > org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430) > at > org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299) > at > org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) > at > org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) > at > org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:637) > Caused by: 2: No such file > at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289) > at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822) > at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268) > at > org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381) > ... 16 more > ScheduledPollConsumer 2010-10-04 15:45:08,426 -- TRACE -- Finished polling: > Endpoint[sftp://tom...@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******] > > > > ScheduledPollConsumer 2010-10-04 15:45:21,413 -- TRACE -- Starting to poll: > Endpoint[sftp://tom...@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******] > SftpConsumer 2010-10-04 15:45:21,413 -- TRACE -- doPollDirectory from > absolutePath: /tmp/activity_logs, dirName: null > SftpOperations 2010-10-04 15:45:21,413 -- TRACE -- Changing directory: / > SftpOperations 2010-10-04 15:45:21,496 -- TRACE -- Changing directory: tmp > SftpOperations 2010-10-04 15:45:21,585 -- TRACE -- Changing directory: > activity_logs > SftpConsumer 2010-10-04 15:45:21,675 -- TRACE -- Polling directory: > /tmp/activity_logs > SftpConsumer 2010-10-04 15:45:21,907 -- TRACE -- Found 4 in directory: > /tmp/activity_logs > SftpOperations 2010-10-04 15:45:21,907 -- TRACE -- Changing directory: / > SftpOperations 2010-10-04 15:45:21,997 -- TRACE -- Changing directory: home > SftpOperations 2010-10-04 15:45:22,084 -- TRACE -- Changing directory: tomcat > SftpOperations 2010-10-04 15:45:22,177 -- TRACE -- Changing directory: tmp > SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Took 0.848 seconds to poll: > /tmp/activity_logs/ > SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Total 1 files to consume > SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Processing file: > GenericFile[activity_log.1285909200] > SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Retrieving file: > tmp/activity_logs/activity_log.1285909200 from: > Endpoint[sftp://tom...@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******] > SftpOperations 2010-10-04 15:45:22,262 -- TRACE -- Changing directory: tmp > SftpConsumer 2010-10-04 15:45:22,305 -- ERROR -- Caused by: > [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot > change directory to: tmp] > org.apache.camel.component.file.GenericFileOperationFailedException: Cannot > change directory to: tmp > at > org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383) > at > org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368) > at > org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451) > at > org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430) > at > org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299) > at > org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) > at > org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) > at > org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:637) > Caused by: 2: No such file > at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289) > at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822) > at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268) > at > org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381) > ... 16 more > ScheduledPollConsumer 2010-10-04 15:45:22,311 -- TRACE -- Finished polling: > Endpoint[sftp://tom...@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******] > > > > > On Oct 2, 2010, at 12:14 AM, Claus Ibsen wrote: > >> Hi Loririn >> >> I think you/we have to step back. We have other users in the past >> running with SFTP with no problems at all. Also fetching files >> recursively. >> So I wonder if we should revert back and keep the old logic! >> >> 1) >> Is this the first time you use camel-ftp? >> >> 2) >> What OS and FTP server are you using? >> >> 3) >> Can you use regular FTP instead? >> >> 4) >> Use relative urls over absolute. They where the first supported. >> >> 5) >> Can you enable TRACE logging on org.apache.camel.component.file and >> setup a simple example so it makes it easier to follow and understand >> what its supposed to do, and what happens in reality so we can see and >> understand the difference. >> >> >> >> >> On Fri, Oct 1, 2010 at 10:45 PM, Lorrin Nelson >> <lhn_git...@nerdylorrin.net> wrote: >>> Hi Claus. Ack, sorry about that. I'm pulling through github rather than SVN >>> directly and then running via a Maven project pointing at my locally built >>> Camel, so it's sometimes confusing how up-to-date I am. >>> >>> So I think I have current code, including checkin "CAMEL-3174: Changing dir >>> with ftp must do one dir at a time." and I think SFTP polling is now broken >>> for both absolute and relative URLs. See debug walk-throughs and stack >>> traces below. >>> >>> Absolute URI with double leading /: >>> sftp://u...@host//home/user/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=****** >>> >>> SftpOperations.changeCurrentDirectory("home/user/subdir") //note there's no >>> leading slash on the supposedly absolute path! >>> //splits into chunks successfully, starts looping through chunks >>> doChangeDirectory("home") >>> ChannelSftp.cd("home") >>> //path=remoteAbsolutePath(path); >>> //yields "/home/user/home". Doh! >>> >>> >>> Relative URI with single leading /: >>> sftp://u...@host/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=****** >>> SftpOperations.changeCurrentDirectory("subdir") >>> //splits into single chunk, starts looping through chunks >>> doChangeDirectory("subdir") >>> ChannelSftp.cd("home") >>> //path=remoteAbsolutePath(path); >>> //yields "/home/user/subdir". yay! >>> >>> ..but then, back in SftpOperations.retrieveFileToStreamInBody, it hits the >>> "change back to current directory code" >>> changeCurrentDirectory("/home/tomcat"); //now we have a leading slash, >>> unlike before >>> //splits info chunks "", "home", and "tomcat". Doh! >>> doChangeDirectory("") >>> ChannelSftp.cd("") >>> //path=remoteAbsolutePath(path) throws a >>> StringIndexOutOfBoundsException when it receives empty string. >>> >>> >>> Stack trace when attempting to supply absolute path: >>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot >>> change directory to: home >>> at >>> org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372) >>> at >>> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361) >>> at >>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:428) >>> at >>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407) >>> at >>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299) >>> at >>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) >>> at >>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) >>> at >>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>> at >>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>> at java.lang.Thread.run(Thread.java:637) >>> Caused by: 2: No such file >>> at >>> com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289) >>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822) >>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268) >>> at >>> org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370) >>> ... 16 more >>> >>> >>> Stack trace when relative URL tries to restore current working directory: >>> SftpConsumer 2010-10-01 13:41:58,529 -- ERROR -- Caused by: >>> [org.apache.camel.component.file.GenericFileOperationFailedException - >>> Cannot change directory to: ] >>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot >>> change directory to: >>> at >>> org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372) >>> at >>> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361) >>> at >>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:435) >>> at >>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407) >>> at >>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299) >>> at >>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) >>> at >>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) >>> at >>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>> at >>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>> at java.lang.Thread.run(Thread.java:637) >>> Caused by: 4: >>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:285) >>> at >>> org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370) >>> ... 16 more >>> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of >>> range: 0 >>> at java.lang.String.charAt(String.java:686) >>> at >>> com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2359) >>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:264) >>> ... 17 more >>> >>> Cheers >>> -Lorrin >>> >>> On Sep 29, 2010, at 9:37 PM, Claus Ibsen wrote: >>> >>>> Hi >>>> >>>> You have not used the latest source code. The stacktrace points to >>>> empty lines etc. >>>> Make sure you build camel-core and camel-ftp using latest source code. >>>> >>>> Revision: 1002946 >>>> Node Kind: directory >>>> Schedule: normal >>>> Last Changed Author: davsclaus >>>> Last Changed Rev: 1002821 >>>> Last Changed Date: 2010-09-29 21:31:23 +0200 (Wed, 29 Sep 2010) >>>> >>>> >>>> On Wed, Sep 29, 2010 at 11:55 PM, Lorrin Nelson >>>> <lhn_git...@nerdylorrin.net> wrote: >>>>> I'm pretty sure I was already at revision 1002541 before. Now I >>>>> definitely am. Same Exception. I think the problem is the lack of leading >>>>> /. >>>>> >>>>> SftpOperations.retrieveFileToStreamInBody computes path as "tmp/mult", >>>>> calls changeCurrentDirectory(path), which in turn calls channel.cd(path), >>>>> which (now we're in com.jcraft.jsch.ChannelSftp) calls >>>>> path=remoteAbsolutePath(path); and computes path: "/home/test/tmp/mult", >>>>> which does not exist. >>>>> >>>>> SftpConsumer 2010-09-29 14:51:50,487 -- ERROR -- Caused by: >>>>> [org.apache.camel.component.file.GenericFileOperationFailedException - >>>>> Cannot change current directory to: tmp/mult] >>>>> org.apache.camel.component.file.GenericFileOperationFailedException: >>>>> Cannot change current directory to: tmp/mult >>>>> at >>>>> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352) >>>>> at >>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408) >>>>> at >>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387) >>>>> at >>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299) >>>>> at >>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) >>>>> at >>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) >>>>> at >>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) >>>>> at >>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>>>> at >>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>>>> at >>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>>>> at >>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) >>>>> at >>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>>>> at java.lang.Thread.run(Thread.java:637) >>>>> Caused by: 2: No such file >>>>> at >>>>> com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289) >>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822) >>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268) >>>>> at >>>>> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350) >>>>> ... 15 more >>>>> >>>>> >>>>> Cheers! >>>>> -Lorrin >>>>> >>>>> On Sep 29, 2010, at 12:32 PM, Claus Ibsen wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> Can you try again with latest source code from trunk. I have changed >>>>>> the logic to change dir one folder at a time. >>>>>> >>>>>> >>>>>> On Wed, Sep 29, 2010 at 7:47 PM, Lorrin Nelson >>>>>> <lhn_git...@nerdylorrin.net> wrote: >>>>>>> Thanks for the quick response. I'm seeing a different failure now >>>>>>> ("Cannot change current directory to: tmp/mult" rather than "Cannot >>>>>>> retrieve file: tmp/mult/dummy-5.log"). Is it suspicious that in either >>>>>>> case there is no leading /? >>>>>>> >>>>>>> I tried uris >>>>>>> sftp://t...@host//tmp/mult?include=.*\.log >>>>>>> and >>>>>>> sftp://t...@host///tmp/mult?include=.*\.log >>>>>>> >>>>>>> SftpConsumer 2010-09-29 10:31:48,401 -- ERROR -- Caused by: >>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException - >>>>>>> Cannot change current directory to: tmp/mult] >>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: >>>>>>> Cannot change current directory to: tmp/mult >>>>>>> at >>>>>>> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352) >>>>>>> at >>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408) >>>>>>> at >>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387) >>>>>>> at >>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299) >>>>>>> at >>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) >>>>>>> at >>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) >>>>>>> at >>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) >>>>>>> at >>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>>>>>> at >>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>>>>>> at >>>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>>>>>> at >>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>>>>>> at >>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) >>>>>>> at >>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>>>>>> at java.lang.Thread.run(Thread.java:637) >>>>>>> Caused by: 2: No such file >>>>>>> at >>>>>>> com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289) >>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822) >>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268) >>>>>>> at >>>>>>> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350) >>>>>>> ... 15 more >>>>>>> >>>>>>> On a tangentially related note, there's some file name processing >>>>>>> during the move step that throws a NullPointerException when the URI >>>>>>> contains a trailing slash: >>>>>>> >>>>>>> sftp://t...@host/tmp/mult/?include=.*\.log >>>>>>> >>>>>>> The problem is that GenericFile.changeFileName calls >>>>>>> newFileName = ObjectHelper.after(newFileName, endpointPath + >>>>>>> getFileSeparator()); >>>>>>> ...without first checking if endPointPath contains a trailing separator. >>>>>>> >>>>>>> E.g. >>>>>>> newFileName: "tmp/mult/.done/dummy-2.log" >>>>>>> endPointPath: "tmp/mult/" >>>>>>> >>>>>>> ObjectHelper.after(): >>>>>>> text: "tmp/mult/.done/dummy-2.log" >>>>>>> after: "tmp/mult//" >>>>>>> >>>>>>> text doesn't contain after (because of double //), returns null, >>>>>>> changeFileName() doesn't expect null, throws NPE. >>>>>>> >>>>>>> GenericFileOnCompletion 2010-09-29 10:37:37,888 -- ERROR -- Caused by: >>>>>>> [java.lang.NullPointerException - null] >>>>>>> java.lang.NullPointerException >>>>>>> at java.io.File.<init>(File.java:222) >>>>>>> at >>>>>>> org.apache.camel.component.file.GenericFile.changeFileName(GenericFile.java:169) >>>>>>> at >>>>>>> org.apache.camel.component.file.strategy.GenericFileExpressionRenamer.renameFile(GenericFileExpressionRenamer.java:41) >>>>>>> at >>>>>>> org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:82) >>>>>>> at >>>>>>> org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121) >>>>>>> at >>>>>>> org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83) >>>>>>> at >>>>>>> org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52) >>>>>>> at >>>>>>> org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:55) >>>>>>> at >>>>>>> org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:173) >>>>>>> at >>>>>>> org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121) >>>>>>> at >>>>>>> org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36) >>>>>>> at >>>>>>> org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106) >>>>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:130) >>>>>>> at >>>>>>> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99) >>>>>>> at >>>>>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) >>>>>>> at >>>>>>> org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) >>>>>>> at >>>>>>> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) >>>>>>> at >>>>>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68) >>>>>>> at >>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322) >>>>>>> at >>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) >>>>>>> at >>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) >>>>>>> at >>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) >>>>>>> at >>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>>>>>> at >>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>>>>>> at >>>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>>>>>> at >>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>>>>>> at >>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) >>>>>>> at >>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>>>>>> at java.lang.Thread.run(Thread.java:637) >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Sep 29, 2010, at 2:46 AM, Claus Ibsen wrote: >>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> I have committed a fix to the SFTP component. Can you try with latest >>>>>>>> source code from trunk? >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Sep 28, 2010 at 7:36 PM, Lorrin <lhn_git...@nerdylorrin.net> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> I just bumped into the same problem. I'm running 2.5-SNAPSHOT. >>>>>>>>> >>>>>>>>> SftpConsumer 2010-09-28 10:31:28,094 -- ERROR -- Caused by: >>>>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException - >>>>>>>>> Cannot retrieve file: tmp/mult/dummy-5.log] >>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: >>>>>>>>> Cannot >>>>>>>>> retrieve file: tmp/mult/dummy-5.log >>>>>>>>> at >>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:403) >>>>>>>>> at >>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387) >>>>>>>>> at >>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299) >>>>>>>>> at >>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) >>>>>>>>> at >>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) >>>>>>>>> at >>>>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) >>>>>>>>> at >>>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>>>>>>>> at >>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>>>>>>>> at >>>>>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>>>>>>>> at >>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>>>>>>>> at >>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) >>>>>>>>> at >>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) >>>>>>>>> at >>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>>>>>>>> at >>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>>>>>>>> at java.lang.Thread.run(Thread.java:637) >>>>>>>>> Caused by: 2: No such file >>>>>>>>> at >>>>>>>>> com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289) >>>>>>>>> at com.jcraft.jsch.ChannelSftp._get(ChannelSftp.java:901) >>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:870) >>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:849) >>>>>>>>> at >>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:400) >>>>>>>>> ... 14 more >>>>>>>>> >>>>>>>>> -- >>>>>>>>> View this message in context: >>>>>>>>> http://camel.465427.n5.nabble.com/How-to-change-directory-while-using-sftp-component-tp2806817p2857187.html >>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com. >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Claus Ibsen >>>>>>>> Apache Camel Committer >>>>>>>> >>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/ >>>>>>>> Open Source Integration: http://fusesource.com >>>>>>>> Blog: http://davsclaus.blogspot.com/ >>>>>>>> Twitter: http://twitter.com/davsclaus >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Claus Ibsen >>>>>> Apache Camel Committer >>>>>> >>>>>> Author of Camel in Action: http://www.manning.com/ibsen/ >>>>>> Open Source Integration: http://fusesource.com >>>>>> Blog: http://davsclaus.blogspot.com/ >>>>>> Twitter: http://twitter.com/davsclaus >>>>>> >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Claus Ibsen >>>> Apache Camel Committer >>>> >>>> Author of Camel in Action: http://www.manning.com/ibsen/ >>>> Open Source Integration: http://fusesource.com >>>> Blog: http://davsclaus.blogspot.com/ >>>> Twitter: http://twitter.com/davsclaus >>>> >>> >>> >> >> >> >> -- >> Claus Ibsen >> Apache Camel Committer >> >> Author of Camel in Action: http://www.manning.com/ibsen/ >> Open Source Integration: http://fusesource.com >> Blog: http://davsclaus.blogspot.com/ >> Twitter: http://twitter.com/davsclaus >> > > -- Claus Ibsen Apache Camel Committer Author of Camel in Action: http://www.manning.com/ibsen/ Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus
