[ https://issues.apache.org/jira/browse/VFS-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16239069#comment-16239069 ]
Bernd Eckenfels edited comment on VFS-647 at 11/4/17 3:58 PM: -------------------------------------------------------------- The filesystems instance is the same when the parameter (incl options and base URL) are the same. This is part of how the caching works. (VFS.getManager() is a Singleton, but I removed that part of the comment as your code does not close it) was (Author: b.eckenfels): As I Said the filesystems instance is the same when the parameter (+options) are the same. > calling findFiles() causes copyFrom() to fail with a partially downloaded > file. > ------------------------------------------------------------------------------- > > Key: VFS-647 > URL: https://issues.apache.org/jira/browse/VFS-647 > Project: Commons VFS > Issue Type: Bug > Affects Versions: 2.0, 2.2 > Environment: Windows 7 and Linux Red Hat. > Reporter: Kenji > Priority: Minor > > Edit: The problem isn't specific to findFiles. All I need to do is have 1 > thread downloads the file, and the other thread just log in, have a while > loop for x seconds where x < download time (using Thread.sleep(x) causes > interruption to happen immediately). Once x seconds passed, the download will > get interupted with pipe closed. Using the same hostname but different > username and password does not cause interruption. > Original description: > When using FileObject.copyFrom(remote, new AllFileSelector()) to download > file from remote to local directory. If SftpFileObject.findFiles(new > FileDepthSelector(1,1)) is called and finished, then the copyFrom will get > interrupted with error pipe closed. > Below are test codes and stack trace error. In real scenario, this all > happens within 1 or 2 second time frame. However, with test scenario, I > wasn't able to reproduce it easily therefore I had to choose a file that > takes around 10 seconds to download and have Thread.sleep(5000) after > findFiles() call. Only tested jsch-0.1.52, jsch-0.1.54, commons-vfs2-2.0 and > commons-vfs2-2.2. > {code:java} > public class FtpClient { > > public static void main(String[] args) { > if (args.length < 5) { > throw new RuntimeException("args: host user pass local > remote"); > } > String hostname = args[0]; > String username = args[1]; > String password = args[2]; > int port = 22; > String local = args[3]; > String remote = args[4]; > final String remoteDir = remote.substring(0, Math.max(0, > remote.lastIndexOf("/"))); > Thread t0 = new Thread() { > public void run() { > try (Ftp ftp = new Ftp(hostname, port)) { > ftp.login(username, password); > ftp.list(remoteDir); > System.out.println("findFiles() > completed."); > } catch (Exception e) { > e.printStackTrace(); > } > } > }; > > Thread ti = new Thread() { > public void run() { > try (Ftp ftp = new Ftp(hostname, port)) { > ftp.login(username, password); > ftp.download(local, remote); > } catch (Exception e) { > e.printStackTrace(); > } > } > }; > t0.start(); > ti.start(); > } > } > public class Ftp implements AutoCloseable { > private int timeout = 0; > private StaticUserAuthenticator userAuth; > private FileSystemOptions fileSysOpts; > private FileObject scr = null; > private FileSystemManager fsm = null; > private String hostName; > private int port = 0; > public Ftp(String remoteHost, int controlPort) throws > FileSystemException { > hostName = remoteHost; > port = controlPort; > fsm = VFS.getManager(); > } > //login into a server with a valid account > public void login(String user, String password) throws IOException { > userAuth = new StaticUserAuthenticator(null, user, password); > fileSysOpts = new FileSystemOptions(); > > DefaultFileSystemConfigBuilder.getInstance().setUserAuthenticator(fileSysOpts, > userAuth); > > SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(fileSysOpts, > "no"); > > SftpFileSystemConfigBuilder.getInstance().setTimeout(fileSysOpts, timeout); > > SftpFileSystemConfigBuilder.getInstance().setUserDirIsRoot(fileSysOpts, > false); > scr = (FileObject)fsm.resolveFile("sftp://" + hostName, > fileSysOpts); > > System.out.println("login successfully."); > } > public void list(String dirName) throws Exception { > SftpFileObject RemoteFo = > (SftpFileObject)fsm.resolveFile("sftp://" + hostName + dirName, fileSysOpts); > > FileObject[] afo = RemoteFo.findFiles(new > FileDepthSelector(1,1)); > Thread.sleep(5000); //key. this must be here (or some other > processing that takes time) for error to reproduce consistently. > } > public void download(String localPath, String remoteFile) throws > FileSystemException { > > SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(fileSysOpts, > "no"); > SftpFileObject RemoteFo = > (SftpFileObject)fsm.resolveFile("sftp://" + hostName + remoteFile > ,fileSysOpts); > scr = fsm.resolveFile("file:" + localPath); > scr.copyFrom(RemoteFo, new AllFileSelector()); > } > > @Override > public void close() { > try{ > FileSystem fs = null; > if(this.scr!=null){ > this.scr.close(); > fs = this.scr.getFileSystem(); > this.fsm.closeFileSystem(fs); > } > }catch(Exception e){ > System.out.println("unable to release ftp connection."); > e.printStackTrace(); > } > } > } > {code} > StackTrace: > {noformat} > Oct 18, 2017 12:42:51 PM org.apache.commons.vfs2.VfsLog info > INFO: Using "C:\Users\kenji\AppData\Local\Temp\vfs_cache" as temporary files > stor > e. > login successfully. > login successfully. > findFiles() completed. > org.apache.commons.vfs2.FileSystemException: Could not copy > "sftp://hostname/path/to/file/FILE1" to "file:///C:/test/ftp/FILE1". > at > org.apache.commons.vfs2.provider.AbstractFileObject.copyFrom(Abstract > FileObject.java:1062) > at Ftp.download(Ftp.java:59) > at FtpClient$2.run(FtpClient.java:36) > Caused by: org.apache.commons.vfs2.FileSystemException: Could not close the > inpu > t stream for file "sftp://hostname/path/to/file/FILE1". > at > org.apache.commons.vfs2.provider.DefaultFileContent$FileContentInputS > tream.close(DefaultFileContent.java:611) > at org.apache.commons.vfs2.FileUtil.writeContent(FileUtil.java:95) > at org.apache.commons.vfs2.FileUtil.copyContent(FileUtil.java:114) > at > org.apache.commons.vfs2.provider.AbstractFileObject.copyFrom(Abstract > FileObject.java:1053) > ... 2 more > Caused by: java.io.IOException: Pipe closed > at java.io.PipedInputStream.read(PipedInputStream.java:307) > at java.io.PipedInputStream.read(PipedInputStream.java:377) > at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2882) > at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2908) > at com.jcraft.jsch.ChannelSftp.access$500(ChannelSftp.java:36) > at > com.jcraft.jsch.ChannelSftp$RequestQueue.cancel(ChannelSftp.java:1238 > ) > at com.jcraft.jsch.ChannelSftp$2.close(ChannelSftp.java:1503) > at java.io.BufferedInputStream.close(BufferedInputStream.java:483) > at > org.apache.commons.vfs2.util.MonitorInputStream.close(MonitorInputStr > eam.java:129) > at java.io.BufferedInputStream.close(BufferedInputStream.java:483) > at > org.apache.commons.vfs2.util.MonitorInputStream.close(MonitorInputStr > eam.java:129) > at > org.apache.commons.vfs2.provider.DefaultFileContent$FileContentInputS > tream.close(DefaultFileContent.java:607) > ... 5 more > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)