[ https://issues.apache.org/jira/browse/HDFS-12325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135633#comment-16135633 ]
Hudson commented on HDFS-12325: ------------------------------- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12219 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12219/]) HDFS-12325. SFTPFileSystem operations should restore cwd. Contributed by (arp: rev 736ceab2f58fb9ab5907c5b5110bd44384038e6b) * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/sftp/SFTPFileSystem.java > SFTPFileSystem operations should restore cwd > -------------------------------------------- > > Key: HDFS-12325 > URL: https://issues.apache.org/jira/browse/HDFS-12325 > Project: Hadoop HDFS > Issue Type: Bug > Reporter: Namit Maheshwari > Assignee: Chen Liang > Fix For: 2.9.0, 3.0.0-beta1 > > Attachments: HDFS-12325.001.patch > > > We've seen a case where writing to {{SFTPFileSystem}} led to unexpected > behaviour: > Given a directory ./data with more than one files in it, the steps it took to > get this error was simply: > {code} > hdfs dfs -fs sftp://x.y.z -mkdir dir0 > hdfs dfs -fs sftp://x.y.z -copyFromLocal data dir0 > hdfs dfs -fs sftp://x.y.z -ls -R dir0 > {code} > But not all files show up as in the ls output, in fact more often just one > single file shows up in that path... > Digging deeper, we found that rename, mkdirs and create operations in > {{SFTPFileSystem}} are changing the current working directory during it's > execution. For example in create there are: > {code} > client.cd(parent.toUri().getPath()); > os = client.put(f.getName()); > {code} > The issue here is {{SFTPConnectionPool}} is caching SFTP sessions (in > {{idleConnections}}), which contains their current working directory. So > after these operations, the sessions will be put back to cache with a changed > working directory. This accumulates in each call and ends up causing > unexpected weird behaviour. Basically this error happens when processing > multiple file system objects in one operation, and relative path is being > used. > The fix here is to restore the current working directory of the SFTP sessions. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org