Thanks Mark and David. I will watch the Issue at Jira to be updated.
Thanks again! Regards. El jue, 3 feb 2022 a las 18:28, David Handermann (< exceptionfact...@apache.org>) escribió: > Guille, > > Thanks for confirming the same behavior on ListSFTP, and thanks Mark for > reproducing the issue. > > Apparently the problem is in the NiFi SFTPTransfer class, which is not > distinguishing between a symbolic link for a file and for a directory. In > this case, SFTPTransfer is attempting to process the symlinked file as a > directory, causing the error. I have assigned NIFI-6699 and will plan on > submitting a pull request to resolve the problem soon. > > Thanks again for reporting this issue and helping track down the problem! > > Regards, > David Handermann > > On Thu, Feb 3, 2022 at 10:58 AM Mark Payne <marka...@hotmail.com> wrote: > >> Guille, >> >> Thanks for the extra details. >> >> I just tried again. In my case, all worked as expected when I had a >> symlink to a directory. But when I had a symlink to a file, I got the same >> error and stack trace as you. So looks like we are handling the case >> properly for symlinked directories but not symlinked files. >> >> Thanks >> -Mark >> >> >> >> On Feb 3, 2022, at 11:43 AM, Guillermo Muñoz < >> guillermo.munoz.salg...@gmail.com> wrote: >> >> Hi, David. >> >> Sorry for the misunderstanding, my fault. Firstly, we tried using >> ListSFTP and FetchSFTP, and when it didn't work, we tried another option >> (GetSFTP), and I pasted the wrong stack trace. So, i've done the following >> tests: >> >> - ListSFTP + FetchSFTP: Error in ListSFTP [1] >> - GetSFTP: Error [2] >> - Generate flowfile + FetchSFTP with the name of the symlink in the * >> Remote File* property: OK, the file is downloaded. >> >> So, it seems the issue is in ListSFTP and GetSFTP, but FetchSFTP works >> fine. >> >> Thanks. Regards >> >> -- >> Guille >> >> [1] >> 2022-02-03 17:36:45,466 ERROR [Timer-Driven Process Thread-8] >> o.a.nifi.processors.standard.ListSFTP >> ListSFTP[id=64443154-ac76-1736-9e49-f2ca388dfbdf] Unable to get listing >> from *****.gz; skipping: java.io.FileNotFoundException: Could not >> perform listing on *****.gz because could not find the file on the >> remote server >> java.io.FileNotFoundException: Could not perform listing on *****.gz >> because could not find the file on the remote server >> at >> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:350) >> at >> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:365) >> at >> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:262) >> at >> org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:120) >> at >> org.apache.nifi.processors.standard.ListSFTP.performListing(ListSFTP.java:150) >> at >> org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:112) >> at >> org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingTimestamps(AbstractListProcessor.java:750) >> at >> org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:525) >> at >> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) >> at >> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1273) >> at >> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214) >> at >> org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:63) >> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >> at java.lang.Thread.run(Thread.java:748) >> >> >> [2] >> 2022-02-03 17:39:14,714 ERROR [Timer-Driven Process Thread-27] >> o.a.nifi.processors.standard.GetSFTP >> GetSFTP[id=c0300c77-017e-1000-ffff-fffffa9c1f31] Unable to get listing from >> *****.gz; skipping: java.io.FileNotFoundException: Could not perform >> listing on *****.gz because could not find the file on the remote server >> java.io.FileNotFoundException: Could not perform listing on *****.gz >> because could not find the file on the remote server >> at >> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:350) >> at >> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:365) >> at >> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:262) >> at >> org.apache.nifi.processors.standard.GetFileTransfer.fetchListing(GetFileTransfer.java:299) >> at >> org.apache.nifi.processors.standard.GetFileTransfer.onTrigger(GetFileTransfer.java:126) >> at >> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) >> at >> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1273) >> at >> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214) >> at >> org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:63) >> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >> at java.lang.Thread.run(Thread.java:748) >> >> El jue, 3 feb 2022 a las 17:09, David Handermann (< >> exceptionfact...@apache.org>) escribió: >> >>> Hi Guille, >>> >>> Thanks for raising this issue and providing a stack trace. You >>> mentioned using ListSFTP and FetchSFTP, but the stack references >>> GetFileTransfer, which corresponds to GetSFTP. >>> >>> Can you confirm the same error using FetchSFTP? If you can confirm the >>> same issue with FetchSFTP, it would be very helpful to add those details on >>> the newer Jira issue NIFI-6699. >>> >>> NiFi SFTP processors switched to a different SSH library after the >>> resolution of NIFI-5560, so it is possible that some changes may be >>> necessary. However, it would be helpful to confirm whether this is an >>> issue with FetchSFTP, GetSFTP, or both processors. >>> >>> Regards, >>> David Handermann >>> >>> On Thu, Feb 3, 2022 at 9:57 AM Guillermo Muñoz Salgado <mun...@gmail.com> >>> wrote: >>> >>>> Hi all, >>>> >>>> We are developing a use case in which we have to get some files from a >>>> server. We have implemented it by the listSFTP + FetchSFTP way in a 3 nodes >>>> cluster running nifi 1.15.3. But we are having some issues when what >>>> we want to get are symlinks instead of files. We have set true the >>>> property *"Follow >>>> symlink" *but we get the same results. Are we doing something wrong? >>>> Or is it a bug or a known issue? We have found this issue [1] but it is old >>>> and resolved and this other one [2], that is older and unresolved. We're >>>> not sure if they are related to this behaviour or not. >>>> >>>> I paste our error log: >>>> >>>> 2022-02-03 16:27:41,002 ERROR [Timer-Driven Process Thread-18] >>>> o.a.nifi.processors.standard.GetSFTP >>>> GetSFTP[id=c0300c77-017e-1000-ffff-fff-ffa9c1f31] Unable to get listing >>>> from testfile.gz; skipping: java.io.FileNotFoundException: Could not >>>> perform listing on testfile.gz because could not find the file on the >>>> remote server >>>> java.io.FileNotFoundException: Could not perform listing on testfile.gz >>>> because could not find the file on the remote server >>>> at >>>> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:350) >>>> at >>>> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:365) >>>> at >>>> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:262) >>>> at >>>> org.apache.nifi.processors.standard.GetFileTransfer.fetchListing(GetFileTransfer.java:299) >>>> at >>>> org.apache.nifi.processors.standard.GetFileTransfer.onTrigger(GetFileTransfer.java:126) >>>> at >>>> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) >>>> at >>>> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1273) >>>> at >>>> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214) >>>> at >>>> org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:63) >>>> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>>> at java.lang.Thread.run(Thread.java:748) >>>> >>>> Thanks in advance >>>> -- >>>> Guille >>>> >>>> [1] https://issues.apache.org/jira/browse/NIFI-5560 >>>> [2] https://issues.apache.org/jira/browse/NIFI-6699 >>>> >>> >> >> -- >> Guille >> >> >> -- Guille
