[jira] [Commented] (VFS-647) calling findFiles() causes copyFrom() to fail with a partially downloaded file.
[
https://issues.apache.org/jira/browse/VFS-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16238470#comment-16238470
]
Bernd Eckenfels commented on VFS-647:
-
It looks like your two threads both close the filesystems and the fsm
concurrently. You need to wait for all filesystems operations to be finished
before using close.
> calling findFiles() causes copyFrom() to fail with a partially downloaded
> file.
> ---
>
> Key: VFS-647
> URL: https://issues.apache.org/jira/browse/VFS-647
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.0, 2.2
> Environment: Windows 7 and Linux Red Hat.
>Reporter: Kenji
>Priority: Minor
>
> Edit: The problem isn't specific to findFiles. All I need to do is have 1
> thread downloads the file, and the other thread just log in, have a while
> loop for x seconds where x < download time (using Thread.sleep(x) causes
> interruption to happen immediately). Once x seconds passed, the download will
> get interupted with pipe closed. Using the same hostname but different
> username and password does not cause interruption.
> Original description:
> When using FileObject.copyFrom(remote, new AllFileSelector()) to download
> file from remote to local directory. If SftpFileObject.findFiles(new
> FileDepthSelector(1,1)) is called and finished, then the copyFrom will get
> interrupted with error pipe closed.
> Below are test codes and stack trace error. In real scenario, this all
> happens within 1 or 2 second time frame. However, with test scenario, I
> wasn't able to reproduce it easily therefore I had to choose a file that
> takes around 10 seconds to download and have Thread.sleep(5000) after
> findFiles() call. Only tested jsch-0.1.52, jsch-0.1.54, commons-vfs2-2.0 and
> commons-vfs2-2.2.
> {code:java}
> public class FtpClient {
>
> public static void main(String[] args) {
> if (args.length < 5) {
> throw new RuntimeException("args: host user pass local
> remote");
> }
> String hostname = args[0];
> String username = args[1];
> String password = args[2];
> int port = 22;
> String local = args[3];
> String remote = args[4];
> final String remoteDir = remote.substring(0, Math.max(0,
> remote.lastIndexOf("/")));
> Thread t0 = new Thread() {
> public void run() {
> try (Ftp ftp = new Ftp(hostname, port)) {
> ftp.login(username, password);
> ftp.list(remoteDir);
> System.out.println("findFiles()
> completed.");
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> };
>
> Thread ti = new Thread() {
> public void run() {
> try (Ftp ftp = new Ftp(hostname, port)) {
> ftp.login(username, password);
> ftp.download(local, remote);
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> };
> t0.start();
> ti.start();
> }
> }
> public class Ftp implements AutoCloseable {
> private int timeout = 0;
> private StaticUserAuthenticator userAuth;
> private FileSystemOptions fileSysOpts;
> private FileObject scr = null;
> private FileSystemManager fsm = null;
> private String hostName;
> private int port = 0;
> public Ftp(String remoteHost, int controlPort) throws
> FileSystemException {
> hostName = remoteHost;
> port = controlPort;
> fsm = VFS.getManager();
> }
> //login into a server with a valid account
> public void login(String user, String password) throws IOException {
> userAuth = new StaticUserAuthenticator(null, user, password);
> fileSysOpts = new FileSystemOptions();
>
> DefaultFileSystemConfigBuilder.getInstance().setUserAuthenticator(fileSysOpts,
> userAuth);
>
> SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(fileSysOpts,
> "no");
>
> SftpFileSystemConfigBuilder.getInstance().setTimeout(fileSysOpts, timeout);
>
> SftpFileSystemConfigBuilder.getInstance().setUserDirIsRoot(fileSysOpts,
> false);
>
[jira] [Commented] (VFS-647) calling findFiles() causes copyFrom() to fail with a partially downloaded file.
[
https://issues.apache.org/jira/browse/VFS-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16238710#comment-16238710
]
Kenji commented on VFS-647:
---
If it's a singleton, how is it I can close an instance of Ftp object safely
without affecting other instances if they're different users or different
hosts? Does it save the object and return the same object base on host username
combination?
If so, I guess can do the same thing and have each instance referenced by
hostname and username and close it only when all instances with the same user
and host are done. Judging by the error I'm getting, I guess that's it...
> calling findFiles() causes copyFrom() to fail with a partially downloaded
> file.
> ---
>
> Key: VFS-647
> URL: https://issues.apache.org/jira/browse/VFS-647
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.0, 2.2
> Environment: Windows 7 and Linux Red Hat.
>Reporter: Kenji
>Priority: Minor
>
> Edit: The problem isn't specific to findFiles. All I need to do is have 1
> thread downloads the file, and the other thread just log in, have a while
> loop for x seconds where x < download time (using Thread.sleep(x) causes
> interruption to happen immediately). Once x seconds passed, the download will
> get interupted with pipe closed. Using the same hostname but different
> username and password does not cause interruption.
> Original description:
> When using FileObject.copyFrom(remote, new AllFileSelector()) to download
> file from remote to local directory. If SftpFileObject.findFiles(new
> FileDepthSelector(1,1)) is called and finished, then the copyFrom will get
> interrupted with error pipe closed.
> Below are test codes and stack trace error. In real scenario, this all
> happens within 1 or 2 second time frame. However, with test scenario, I
> wasn't able to reproduce it easily therefore I had to choose a file that
> takes around 10 seconds to download and have Thread.sleep(5000) after
> findFiles() call. Only tested jsch-0.1.52, jsch-0.1.54, commons-vfs2-2.0 and
> commons-vfs2-2.2.
> {code:java}
> public class FtpClient {
>
> public static void main(String[] args) {
> if (args.length < 5) {
> throw new RuntimeException("args: host user pass local
> remote");
> }
> String hostname = args[0];
> String username = args[1];
> String password = args[2];
> int port = 22;
> String local = args[3];
> String remote = args[4];
> final String remoteDir = remote.substring(0, Math.max(0,
> remote.lastIndexOf("/")));
> Thread t0 = new Thread() {
> public void run() {
> try (Ftp ftp = new Ftp(hostname, port)) {
> ftp.login(username, password);
> ftp.list(remoteDir);
> System.out.println("findFiles()
> completed.");
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> };
>
> Thread ti = new Thread() {
> public void run() {
> try (Ftp ftp = new Ftp(hostname, port)) {
> ftp.login(username, password);
> ftp.download(local, remote);
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> };
> t0.start();
> ti.start();
> }
> }
> public class Ftp implements AutoCloseable {
> private int timeout = 0;
> private StaticUserAuthenticator userAuth;
> private FileSystemOptions fileSysOpts;
> private FileObject scr = null;
> private FileSystemManager fsm = null;
> private String hostName;
> private int port = 0;
> public Ftp(String remoteHost, int controlPort) throws
> FileSystemException {
> hostName = remoteHost;
> port = controlPort;
> fsm = VFS.getManager();
> }
> //login into a server with a valid account
> public void login(String user, String password) throws IOException {
> userAuth = new StaticUserAuthenticator(null, user, password);
> fileSysOpts = new FileSystemOptions();
>
> DefaultFileSystemConfigBuilder.getInstance().setUserAuthenticator(fileSysOpts,
> userAuth);
>
> SftpFi
[jira] [Commented] (VFS-647) calling findFiles() causes copyFrom() to fail with a partially downloaded file.
[
https://issues.apache.org/jira/browse/VFS-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16239069#comment-16239069
]
Bernd Eckenfels commented on VFS-647:
-
As I Said the filesystems instance is the same when the parameter (+options)
are the same.
> calling findFiles() causes copyFrom() to fail with a partially downloaded
> file.
> ---
>
> Key: VFS-647
> URL: https://issues.apache.org/jira/browse/VFS-647
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.0, 2.2
> Environment: Windows 7 and Linux Red Hat.
>Reporter: Kenji
>Priority: Minor
>
> Edit: The problem isn't specific to findFiles. All I need to do is have 1
> thread downloads the file, and the other thread just log in, have a while
> loop for x seconds where x < download time (using Thread.sleep(x) causes
> interruption to happen immediately). Once x seconds passed, the download will
> get interupted with pipe closed. Using the same hostname but different
> username and password does not cause interruption.
> Original description:
> When using FileObject.copyFrom(remote, new AllFileSelector()) to download
> file from remote to local directory. If SftpFileObject.findFiles(new
> FileDepthSelector(1,1)) is called and finished, then the copyFrom will get
> interrupted with error pipe closed.
> Below are test codes and stack trace error. In real scenario, this all
> happens within 1 or 2 second time frame. However, with test scenario, I
> wasn't able to reproduce it easily therefore I had to choose a file that
> takes around 10 seconds to download and have Thread.sleep(5000) after
> findFiles() call. Only tested jsch-0.1.52, jsch-0.1.54, commons-vfs2-2.0 and
> commons-vfs2-2.2.
> {code:java}
> public class FtpClient {
>
> public static void main(String[] args) {
> if (args.length < 5) {
> throw new RuntimeException("args: host user pass local
> remote");
> }
> String hostname = args[0];
> String username = args[1];
> String password = args[2];
> int port = 22;
> String local = args[3];
> String remote = args[4];
> final String remoteDir = remote.substring(0, Math.max(0,
> remote.lastIndexOf("/")));
> Thread t0 = new Thread() {
> public void run() {
> try (Ftp ftp = new Ftp(hostname, port)) {
> ftp.login(username, password);
> ftp.list(remoteDir);
> System.out.println("findFiles()
> completed.");
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> };
>
> Thread ti = new Thread() {
> public void run() {
> try (Ftp ftp = new Ftp(hostname, port)) {
> ftp.login(username, password);
> ftp.download(local, remote);
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> };
> t0.start();
> ti.start();
> }
> }
> public class Ftp implements AutoCloseable {
> private int timeout = 0;
> private StaticUserAuthenticator userAuth;
> private FileSystemOptions fileSysOpts;
> private FileObject scr = null;
> private FileSystemManager fsm = null;
> private String hostName;
> private int port = 0;
> public Ftp(String remoteHost, int controlPort) throws
> FileSystemException {
> hostName = remoteHost;
> port = controlPort;
> fsm = VFS.getManager();
> }
> //login into a server with a valid account
> public void login(String user, String password) throws IOException {
> userAuth = new StaticUserAuthenticator(null, user, password);
> fileSysOpts = new FileSystemOptions();
>
> DefaultFileSystemConfigBuilder.getInstance().setUserAuthenticator(fileSysOpts,
> userAuth);
>
> SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(fileSysOpts,
> "no");
>
> SftpFileSystemConfigBuilder.getInstance().setTimeout(fileSysOpts, timeout);
>
> SftpFileSystemConfigBuilder.getInstance().setUserDirIsRoot(fileSysOpts,
> false);
> scr = (FileObject)fsm.resolveFile("sftp://"; + hostName,
> fileSy
