Hmm. How come when polling the absolute path it says this before listing the files in the directory (which works)? >> SftpOperations 2010-10-04 15:45:07,494 -- TRACE -- Changing directory: / >> SftpOperations 2010-10-04 15:45:07,608 -- TRACE -- Changing directory: tmp >> SftpOperations 2010-10-04 15:45:07,697 -- TRACE -- Changing directory: >> activity_logs
...but then when it tries to retrieve the file (which fails) it just says: >> SftpOperations 2010-10-04 15:45:08,277 -- TRACE -- Changing directory: tmp >> SftpOperations 2010-10-04 15:45:08,367 -- TRACE -- Changing directory: >> activity_logs That looks awfully like Camel isn't even asking the SFTP library to go to / first when it comes time to retrieve the file. Cheers! -Lorrin On Oct 5, 2010, at 12:02 AM, Claus Ibsen wrote: > Hi > > Great that the relative works. This is also what I expected. > > In terms of absolute then Camel changes the directory to / (since its > absolute) and the SFTP library returns OK for this. > But in reality it does not go to the root path, but most likely just > goes to the home directory of the user. > And hence why absolute does not work for you. > > I don't see any API or way to tell SFTP library it should change to > root path and not home folder. > > A solution to be able to poll files from an absolute folder is to > create a symlink in the user home directory which can be polled by the > SFTP library. > >> nlike the polling stage where it says "doPollDirectory from absolutePath: >> /tmp/activity_logs". > The reason why you see this is because Camel logs the path configured > from the endpoint, when the poller starts. Hence you see the leading > /. > > > > On Tue, Oct 5, 2010 at 12:58 AM, Lorrin Nelson > <lhn_git...@nerdylorrin.net> wrote: >> Hi Claus, thanks for the continued support. With the latest code (revision >> 1003927) relative URIs are working again for me. >> >> This the first time I'm using camel-ftp. The server I'm talking to is a >> Fedora 11 box running OpenSSH_5.2p1. Regular FTP is an unlikely option for >> us. Relative URLs are an acceptable work-around. >> >> Below is a trace log of an absolute URL. >> >> On the first attempt, note that ~/tmp does exist. Therefore when it tries to >> go to *relative* path tmp (which is the wrong thing to do), it succeeds. >> Then when it tries to go one level deeper into pitch_activity_logs it fails. >> The second attempt is really interesting. Note there that the current >> working directory has remained at /home/tomcat/tmp. So when it fails to go >> to *relative* path tmp, it fails right away. BTW, note that it says >> "Retrieving file: tmp/activity_logs/activity_log.1285909200" with no leading >> slash, unlike the polling stage where it says "doPollDirectory from >> absolutePath: /tmp/activity_logs". >> >> Cheers >> -Lorrin >> >> >> SftpConsumer 2010-10-04 15:45:07,445 -- INFO -- Connected and logged in to: >> sftp://tom...@host:22 >> SftpConsumer 2010-10-04 15:45:07,494 -- TRACE -- doPollDirectory from >> absolutePath: /tmp/activity_logs, dirName: null >> SftpOperations 2010-10-04 15:45:07,494 -- TRACE -- Changing directory: / >> SftpOperations 2010-10-04 15:45:07,608 -- TRACE -- Changing directory: tmp >> SftpOperations 2010-10-04 15:45:07,697 -- TRACE -- Changing directory: >> activity_logs >> SftpConsumer 2010-10-04 15:45:07,788 -- TRACE -- Polling directory: >> /tmp/activity_logs >> SftpConsumer 2010-10-04 15:45:08,011 -- TRACE -- Found 4 in directory: >> /tmp/activity_logs >> SftpOperations 2010-10-04 15:45:08,012 -- TRACE -- Changing directory: / >> SftpOperations 2010-10-04 15:45:08,099 -- TRACE -- Changing directory: home >> SftpOperations 2010-10-04 15:45:08,189 -- TRACE -- Changing directory: tomcat >> SftpConsumer 2010-10-04 15:45:08,274 -- DEBUG -- Took 0.829 seconds to poll: >> /tmp/activity_logs/ >> SftpConsumer 2010-10-04 15:45:08,275 -- DEBUG -- Total 1 files to consume >> SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Processing file: >> GenericFile[activity_log.1285909200] >> SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Retrieving file: >> tmp/activity_logs/activity_log.1285909200 from: >> Endpoint[sftp://tom...@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******] >> SftpOperations 2010-10-04 15:45:08,277 -- TRACE -- Changing directory: tmp >> SftpOperations 2010-10-04 15:45:08,367 -- TRACE -- Changing directory: >> activity_logs >> SftpConsumer 2010-10-04 15:45:08,414 -- ERROR -- Caused by: >> [org.apache.camel.component.file.GenericFileOperationFailedException - >> Cannot change directory to: activity_logs] >> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot >> change directory to: activity_logs >> at >> org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383) >> at >> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368) >> at >> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451) >> at >> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430) >> at >> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299) >> at >> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) >> at >> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) >> at >> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >> at >> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >> at java.lang.Thread.run(Thread.java:637) >> Caused by: 2: No such file >> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289) >> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822) >> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268) >> at >> org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381) >> ... 16 more >> ScheduledPollConsumer 2010-10-04 15:45:08,426 -- TRACE -- Finished polling: >> Endpoint[sftp://tom...@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******] >> >> >> >> ScheduledPollConsumer 2010-10-04 15:45:21,413 -- TRACE -- Starting to poll: >> Endpoint[sftp://tom...@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******] >> SftpConsumer 2010-10-04 15:45:21,413 -- TRACE -- doPollDirectory from >> absolutePath: /tmp/activity_logs, dirName: null >> SftpOperations 2010-10-04 15:45:21,413 -- TRACE -- Changing directory: / >> SftpOperations 2010-10-04 15:45:21,496 -- TRACE -- Changing directory: tmp >> SftpOperations 2010-10-04 15:45:21,585 -- TRACE -- Changing directory: >> activity_logs >> SftpConsumer 2010-10-04 15:45:21,675 -- TRACE -- Polling directory: >> /tmp/activity_logs >> SftpConsumer 2010-10-04 15:45:21,907 -- TRACE -- Found 4 in directory: >> /tmp/activity_logs >> SftpOperations 2010-10-04 15:45:21,907 -- TRACE -- Changing directory: / >> SftpOperations 2010-10-04 15:45:21,997 -- TRACE -- Changing directory: home >> SftpOperations 2010-10-04 15:45:22,084 -- TRACE -- Changing directory: tomcat >> SftpOperations 2010-10-04 15:45:22,177 -- TRACE -- Changing directory: tmp >> SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Took 0.848 seconds to poll: >> /tmp/activity_logs/ >> SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Total 1 files to consume >> SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Processing file: >> GenericFile[activity_log.1285909200] >> SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Retrieving file: >> tmp/activity_logs/activity_log.1285909200 from: >> Endpoint[sftp://tom...@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******] >> SftpOperations 2010-10-04 15:45:22,262 -- TRACE -- Changing directory: tmp >> SftpConsumer 2010-10-04 15:45:22,305 -- ERROR -- Caused by: >> [org.apache.camel.component.file.GenericFileOperationFailedException - >> Cannot change directory to: tmp] >> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot >> change directory to: tmp >> at >> org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383) >> at >> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368) >> at >> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451) >> at >> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430) >> at >> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299) >> at >> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) >> at >> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) >> at >> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >> at >> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >> at java.lang.Thread.run(Thread.java:637) >> Caused by: 2: No such file >> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289) >> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822) >> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268) >> at >> org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381) >> ... 16 more >> ScheduledPollConsumer 2010-10-04 15:45:22,311 -- TRACE -- Finished polling: >> Endpoint[sftp://tom...@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******] >> >> >> >> >> On Oct 2, 2010, at 12:14 AM, Claus Ibsen wrote: >> >>> Hi Loririn >>> >>> I think you/we have to step back. We have other users in the past >>> running with SFTP with no problems at all. Also fetching files >>> recursively. >>> So I wonder if we should revert back and keep the old logic! >>> >>> 1) >>> Is this the first time you use camel-ftp? >>> >>> 2) >>> What OS and FTP server are you using? >>> >>> 3) >>> Can you use regular FTP instead? >>> >>> 4) >>> Use relative urls over absolute. They where the first supported. >>> >>> 5) >>> Can you enable TRACE logging on org.apache.camel.component.file and >>> setup a simple example so it makes it easier to follow and understand >>> what its supposed to do, and what happens in reality so we can see and >>> understand the difference. >>> >>> >>> >>> >>> On Fri, Oct 1, 2010 at 10:45 PM, Lorrin Nelson >>> <lhn_git...@nerdylorrin.net> wrote: >>>> Hi Claus. Ack, sorry about that. I'm pulling through github rather than >>>> SVN directly and then running via a Maven project pointing at my locally >>>> built Camel, so it's sometimes confusing how up-to-date I am. >>>> >>>> So I think I have current code, including checkin "CAMEL-3174: Changing >>>> dir with ftp must do one dir at a time." and I think SFTP polling is now >>>> broken for both absolute and relative URLs. See debug walk-throughs and >>>> stack traces below. >>>> >>>> Absolute URI with double leading /: >>>> sftp://u...@host//home/user/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=****** >>>> >>>> SftpOperations.changeCurrentDirectory("home/user/subdir") //note there's >>>> no leading slash on the supposedly absolute path! >>>> //splits into chunks successfully, starts looping through chunks >>>> doChangeDirectory("home") >>>> ChannelSftp.cd("home") >>>> //path=remoteAbsolutePath(path); >>>> //yields "/home/user/home". Doh! >>>> >>>> >>>> Relative URI with single leading /: >>>> sftp://u...@host/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=****** >>>> SftpOperations.changeCurrentDirectory("subdir") >>>> //splits into single chunk, starts looping through chunks >>>> doChangeDirectory("subdir") >>>> ChannelSftp.cd("home") >>>> //path=remoteAbsolutePath(path); >>>> //yields "/home/user/subdir". yay! >>>> >>>> ..but then, back in SftpOperations.retrieveFileToStreamInBody, it hits the >>>> "change back to current directory code" >>>> changeCurrentDirectory("/home/tomcat"); //now we have a leading slash, >>>> unlike before >>>> //splits info chunks "", "home", and "tomcat". Doh! >>>> doChangeDirectory("") >>>> ChannelSftp.cd("") >>>> //path=remoteAbsolutePath(path) throws a >>>> StringIndexOutOfBoundsException when it receives empty string. >>>> >>>> >>>> Stack trace when attempting to supply absolute path: >>>> org.apache.camel.component.file.GenericFileOperationFailedException: >>>> Cannot change directory to: home >>>> at >>>> org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372) >>>> at >>>> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361) >>>> at >>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:428) >>>> at >>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407) >>>> at >>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299) >>>> at >>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) >>>> at >>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) >>>> at >>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>>> at >>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>>> at java.lang.Thread.run(Thread.java:637) >>>> Caused by: 2: No such file >>>> at >>>> com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289) >>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822) >>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268) >>>> at >>>> org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370) >>>> ... 16 more >>>> >>>> >>>> Stack trace when relative URL tries to restore current working directory: >>>> SftpConsumer 2010-10-01 13:41:58,529 -- ERROR -- Caused by: >>>> [org.apache.camel.component.file.GenericFileOperationFailedException - >>>> Cannot change directory to: ] >>>> org.apache.camel.component.file.GenericFileOperationFailedException: >>>> Cannot change directory to: >>>> at >>>> org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372) >>>> at >>>> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361) >>>> at >>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:435) >>>> at >>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407) >>>> at >>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299) >>>> at >>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) >>>> at >>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) >>>> at >>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>>> at >>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>>> at java.lang.Thread.run(Thread.java:637) >>>> Caused by: 4: >>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:285) >>>> at >>>> org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370) >>>> ... 16 more >>>> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of >>>> range: 0 >>>> at java.lang.String.charAt(String.java:686) >>>> at >>>> com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2359) >>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:264) >>>> ... 17 more >>>> >>>> Cheers >>>> -Lorrin >>>> >>>> On Sep 29, 2010, at 9:37 PM, Claus Ibsen wrote: >>>> >>>>> Hi >>>>> >>>>> You have not used the latest source code. The stacktrace points to >>>>> empty lines etc. >>>>> Make sure you build camel-core and camel-ftp using latest source code. >>>>> >>>>> Revision: 1002946 >>>>> Node Kind: directory >>>>> Schedule: normal >>>>> Last Changed Author: davsclaus >>>>> Last Changed Rev: 1002821 >>>>> Last Changed Date: 2010-09-29 21:31:23 +0200 (Wed, 29 Sep 2010) >>>>> >>>>> >>>>> On Wed, Sep 29, 2010 at 11:55 PM, Lorrin Nelson >>>>> <lhn_git...@nerdylorrin.net> wrote: >>>>>> I'm pretty sure I was already at revision 1002541 before. Now I >>>>>> definitely am. Same Exception. I think the problem is the lack of >>>>>> leading /. >>>>>> >>>>>> SftpOperations.retrieveFileToStreamInBody computes path as "tmp/mult", >>>>>> calls changeCurrentDirectory(path), which in turn calls >>>>>> channel.cd(path), which (now we're in com.jcraft.jsch.ChannelSftp) calls >>>>>> path=remoteAbsolutePath(path); and computes path: "/home/test/tmp/mult", >>>>>> which does not exist. >>>>>> >>>>>> SftpConsumer 2010-09-29 14:51:50,487 -- ERROR -- Caused by: >>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException - >>>>>> Cannot change current directory to: tmp/mult] >>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: >>>>>> Cannot change current directory to: tmp/mult >>>>>> at >>>>>> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352) >>>>>> at >>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408) >>>>>> at >>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387) >>>>>> at >>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299) >>>>>> at >>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) >>>>>> at >>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) >>>>>> at >>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) >>>>>> at >>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>>>>> at >>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>>>>> at >>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>>>>> at >>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>>>>> at >>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) >>>>>> at >>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>>>>> at java.lang.Thread.run(Thread.java:637) >>>>>> Caused by: 2: No such file >>>>>> at >>>>>> com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289) >>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822) >>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268) >>>>>> at >>>>>> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350) >>>>>> ... 15 more >>>>>> >>>>>> >>>>>> Cheers! >>>>>> -Lorrin >>>>>> >>>>>> On Sep 29, 2010, at 12:32 PM, Claus Ibsen wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> Can you try again with latest source code from trunk. I have changed >>>>>>> the logic to change dir one folder at a time. >>>>>>> >>>>>>> >>>>>>> On Wed, Sep 29, 2010 at 7:47 PM, Lorrin Nelson >>>>>>> <lhn_git...@nerdylorrin.net> wrote: >>>>>>>> Thanks for the quick response. I'm seeing a different failure now >>>>>>>> ("Cannot change current directory to: tmp/mult" rather than "Cannot >>>>>>>> retrieve file: tmp/mult/dummy-5.log"). Is it suspicious that in either >>>>>>>> case there is no leading /? >>>>>>>> >>>>>>>> I tried uris >>>>>>>> sftp://t...@host//tmp/mult?include=.*\.log >>>>>>>> and >>>>>>>> sftp://t...@host///tmp/mult?include=.*\.log >>>>>>>> >>>>>>>> SftpConsumer 2010-09-29 10:31:48,401 -- ERROR -- Caused by: >>>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException - >>>>>>>> Cannot change current directory to: tmp/mult] >>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: >>>>>>>> Cannot change current directory to: tmp/mult >>>>>>>> at >>>>>>>> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) >>>>>>>> at >>>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) >>>>>>>> at >>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>>>>>>> at >>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>>>>>>> at >>>>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>>>>>>> at >>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>>>>>>> at >>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) >>>>>>>> at >>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>>>>>>> at java.lang.Thread.run(Thread.java:637) >>>>>>>> Caused by: 2: No such file >>>>>>>> at >>>>>>>> com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289) >>>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822) >>>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350) >>>>>>>> ... 15 more >>>>>>>> >>>>>>>> On a tangentially related note, there's some file name processing >>>>>>>> during the move step that throws a NullPointerException when the URI >>>>>>>> contains a trailing slash: >>>>>>>> >>>>>>>> sftp://t...@host/tmp/mult/?include=.*\.log >>>>>>>> >>>>>>>> The problem is that GenericFile.changeFileName calls >>>>>>>> newFileName = ObjectHelper.after(newFileName, endpointPath + >>>>>>>> getFileSeparator()); >>>>>>>> ...without first checking if endPointPath contains a trailing >>>>>>>> separator. >>>>>>>> >>>>>>>> E.g. >>>>>>>> newFileName: "tmp/mult/.done/dummy-2.log" >>>>>>>> endPointPath: "tmp/mult/" >>>>>>>> >>>>>>>> ObjectHelper.after(): >>>>>>>> text: "tmp/mult/.done/dummy-2.log" >>>>>>>> after: "tmp/mult//" >>>>>>>> >>>>>>>> text doesn't contain after (because of double //), returns null, >>>>>>>> changeFileName() doesn't expect null, throws NPE. >>>>>>>> >>>>>>>> GenericFileOnCompletion 2010-09-29 10:37:37,888 -- ERROR -- Caused by: >>>>>>>> [java.lang.NullPointerException - null] >>>>>>>> java.lang.NullPointerException >>>>>>>> at java.io.File.<init>(File.java:222) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.GenericFile.changeFileName(GenericFile.java:169) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.strategy.GenericFileExpressionRenamer.renameFile(GenericFileExpressionRenamer.java:41) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:82) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52) >>>>>>>> at >>>>>>>> org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:55) >>>>>>>> at >>>>>>>> org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:173) >>>>>>>> at >>>>>>>> org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121) >>>>>>>> at >>>>>>>> org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36) >>>>>>>> at >>>>>>>> org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106) >>>>>>>> at >>>>>>>> org.apache.camel.processor.Pipeline.process(Pipeline.java:130) >>>>>>>> at >>>>>>>> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99) >>>>>>>> at >>>>>>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70) >>>>>>>> at >>>>>>>> org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98) >>>>>>>> at >>>>>>>> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89) >>>>>>>> at >>>>>>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) >>>>>>>> at >>>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) >>>>>>>> at >>>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) >>>>>>>> at >>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>>>>>>> at >>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>>>>>>> at >>>>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>>>>>>> at >>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>>>>>>> at >>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) >>>>>>>> at >>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>>>>>>> at java.lang.Thread.run(Thread.java:637) >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Sep 29, 2010, at 2:46 AM, Claus Ibsen wrote: >>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> I have committed a fix to the SFTP component. Can you try with latest >>>>>>>>> source code from trunk? >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Sep 28, 2010 at 7:36 PM, Lorrin <lhn_git...@nerdylorrin.net> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> I just bumped into the same problem. I'm running 2.5-SNAPSHOT. >>>>>>>>>> >>>>>>>>>> SftpConsumer 2010-09-28 10:31:28,094 -- ERROR -- Caused by: >>>>>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException >>>>>>>>>> - >>>>>>>>>> Cannot retrieve file: tmp/mult/dummy-5.log] >>>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException: >>>>>>>>>> Cannot >>>>>>>>>> retrieve file: tmp/mult/dummy-5.log >>>>>>>>>> at >>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:403) >>>>>>>>>> at >>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387) >>>>>>>>>> at >>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299) >>>>>>>>>> at >>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155) >>>>>>>>>> at >>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121) >>>>>>>>>> at >>>>>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>>>>>>>>> at java.lang.Thread.run(Thread.java:637) >>>>>>>>>> Caused by: 2: No such file >>>>>>>>>> at >>>>>>>>>> com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289) >>>>>>>>>> at com.jcraft.jsch.ChannelSftp._get(ChannelSftp.java:901) >>>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:870) >>>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:849) >>>>>>>>>> at >>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:400) >>>>>>>>>> ... 14 more >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> View this message in context: >>>>>>>>>> http://camel.465427.n5.nabble.com/How-to-change-directory-while-using-sftp-component-tp2806817p2857187.html >>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com. >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Claus Ibsen >>>>>>>>> Apache Camel Committer >>>>>>>>> >>>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/ >>>>>>>>> Open Source Integration: http://fusesource.com >>>>>>>>> Blog: http://davsclaus.blogspot.com/ >>>>>>>>> Twitter: http://twitter.com/davsclaus >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Claus Ibsen >>>>>>> Apache Camel Committer >>>>>>> >>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/ >>>>>>> Open Source Integration: http://fusesource.com >>>>>>> Blog: http://davsclaus.blogspot.com/ >>>>>>> Twitter: http://twitter.com/davsclaus >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Claus Ibsen >>>>> Apache Camel Committer >>>>> >>>>> Author of Camel in Action: http://www.manning.com/ibsen/ >>>>> Open Source Integration: http://fusesource.com >>>>> Blog: http://davsclaus.blogspot.com/ >>>>> Twitter: http://twitter.com/davsclaus >>>>> >>>> >>>> >>> >>> >>> >>> -- >>> Claus Ibsen >>> Apache Camel Committer >>> >>> Author of Camel in Action: http://www.manning.com/ibsen/ >>> Open Source Integration: http://fusesource.com >>> Blog: http://davsclaus.blogspot.com/ >>> Twitter: http://twitter.com/davsclaus >>> >> >> > > > > -- > Claus Ibsen > Apache Camel Committer > > Author of Camel in Action: http://www.manning.com/ibsen/ > Open Source Integration: http://fusesource.com > Blog: http://davsclaus.blogspot.com/ > Twitter: http://twitter.com/davsclaus >
