[ https://issues.apache.org/jira/browse/HDFS-12325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135574#comment-16135574 ]
Íñigo Goiri commented on HDFS-12325: ------------------------------------ Shouldn't the restore be done in a finally block? Something like: {code} boolean renamed = true; final String previousCwd = channel.pwd(); channel.cd("/"); try { channel.rename(src.toUri().getPath(), dst.toUri().getPath()); } catch (SftpException e) { renamed = false; } finally { channel.cd(previousCwd); } {code} With proper handling of the {{cd}} exception. > SFTPFileSystem operations should restore cwd > -------------------------------------------- > > Key: HDFS-12325 > URL: https://issues.apache.org/jira/browse/HDFS-12325 > Project: Hadoop HDFS > Issue Type: Bug > Reporter: Chen Liang > Assignee: Chen Liang > Fix For: 2.9.0, 3.0.0-beta1 > > Attachments: HDFS-12325.001.patch > > > We've seen a case where writing to {{SFTPFileSystem}} led to unexpected > behaviour: > Given a directory ./data with more than one files in it, the steps it took to > get this error was simply: > {code} > hdfs dfs -fs sftp://x.y.z -mkdir dir0 > hdfs dfs -fs sftp://x.y.z -copyFromLocal data dir0 > hdfs dfs -fs sftp://x.y.z -ls -R dir0 > {code} > But not all files show up as in the ls output, in fact more often just one > single file shows up in that path... > Digging deeper, we found that rename, mkdirs and create operations in > {{SFTPFileSystem}} are changing the current working directory during it's > execution. For example in create there are: > {code} > client.cd(parent.toUri().getPath()); > os = client.put(f.getName()); > {code} > The issue here is {{SFTPConnectionPool}} is caching SFTP sessions (in > {{idleConnections}}), which contains their current working directory. So > after these operations, the sessions will be put back to cache with a changed > working directory. This accumulates in each call and ends up causing > unexpected weird behaviour. Basically this error happens when processing > multiple file system objects in one operation, and relative path is being > used. > The fix here is to restore the current working directory of the SFTP sessions. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org