Dear Wiki user, You have subscribed to a wiki page or wiki category on "Commons Wiki" for change notification.
The following page has been changed by KenTanaka: http://wiki.apache.org/jakarta-commons/SimpleSftpFileDownload The comment on the change is: Added code and output ------------------------------------------------------------------------------ = Key Concepts = Access to a remote system using SFTP uses the SSH secure shell protocols. Although the behavior is similar to FTP, it is ''not'' FTP run over a secure connection. So there are some differences between FTP and SFTP that should be noted. One of these it the lack of the FTP binary/ASCII transfer mode, in SFTP all transfers are binary as if they were executed with an "scp" (secure copy) command. - This example code uses a regular expression to match files on the remote system, so that not all of the files in the source directory are transferred. + This example code uses a regular expression to match files on the remote system, so that not all of the files in the source directory are transferred. The {{{filePatternString}}} is set to ".*/smoke\\d{8}_wkt\\.txt". This has the regular expression components of: + * ".*/" matches any path that precedes the filename. The "." is wildcard character, "*" specifies that 0 or more of these may be present. The "/" matches the directory separator. + * "\\d{8}" matches 8 digits (4 for the year, 2 for the month number, and 2 for the day of the month). The doubled backslash is to escape a single backslash for Java string literals. "\d" indicates any digit, and "{8}" specifies exactly 8 of the preceding character or pattern. + * "\\." is an escaped "\." which means a literal period instead of a wild card interpretation of "." + + It is important to call the [http://commons.apache.org/vfs/apidocs/org/apache/commons/vfs/impl/DefaultFileSystemManager.html#close() close()] method of the !DefaultFileSystemManager to clean up any temporary files and close all providers. Otherwise the program will appear to hang after downloading files. = Source Code = + The code provided below is for a Maven 2 project. + == pom.xml Project File == - This example uses Maven2. There is a '''{{{pom.xml}}}''' to define the project + The '''{{{pom.xml}}}''' file defines how the project is built for maven 2: {{{ + <project xmlns="http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd";> + <modelVersion>4.0.0</modelVersion> + <groupId>gov.noaa.eds.byExample</groupId> + <artifactId>trySimpleVfsSftp</artifactId> + <packaging>jar</packaging> + <version>1.0-SNAPSHOT</version> + <name>trySimpleVfsSftp</name> + <url>http://maven.apache.org</url> + <build> + <extensions> + <extension> + <groupId>org.apache.maven.wagon</groupId> + <artifactId>wagon-ssh-external</artifactId> + <version>1.0-beta-2</version> + </extension> + </extensions> + <plugins> + <plugin> + <artifactId>maven-compiler-plugin</artifactId> + <configuration> + <source>1.5</source> + <target>1.5</target> + </configuration> + </plugin> + <plugin> + <artifactId>maven-assembly-plugin</artifactId> + <configuration> + <descriptorRefs> + <descriptorRef>jar-with-dependencies</descriptorRef> + </descriptorRefs> + <archive> + <manifest> + <mainClass>gov.noaa.eds.byExample.trySimpleVfsSftp.App</mainClass> + </manifest> + </archive> + </configuration> + </plugin> + <plugin> + <artifactId>maven-antrun-plugin</artifactId> + <configuration> + <tasks> + <java classname="gov.noaa.eds.byExample.trySimpleVfsSftp.App" classpathref="maven.runtime.classpath"> + </java> + </tasks> + </configuration> + </plugin> + </plugins> + </build> + <dependencies> + <!-- Supports VFS SFTP --> + <dependency> + <groupId>com.jcraft</groupId> + <artifactId>jsch</artifactId> + <version>0.1.23</version> + <optional>true</optional> + </dependency> + <dependency> + <groupId>commons-vfs</groupId> + <artifactId>commons-vfs</artifactId> + <version>1.0</version> + </dependency> + <dependency> + <groupId>junit</groupId> + <artifactId>junit</artifactId> + <version>3.8.1</version> + <scope>test</scope> + </dependency> + </dependencies> + </project> }}} @@ -32, +109 @@ /* * App.java */ + package gov.noaa.eds.byExample.trySimpleVfsSftp; + + import java.io.File; + import java.util.regex.Pattern; + import org.apache.commons.vfs.AllFileSelector; + import org.apache.commons.vfs.FileObject; + import org.apache.commons.vfs.FileSystemException; + import org.apache.commons.vfs.FileSystemManager; + import org.apache.commons.vfs.FileSystemOptions; + import org.apache.commons.vfs.FileType; + import org.apache.commons.vfs.UserAuthenticator; + import org.apache.commons.vfs.VFS; + import org.apache.commons.vfs.auth.StaticUserAuthenticator; + import org.apache.commons.vfs.impl.DefaultFileSystemConfigBuilder; + import org.apache.commons.vfs.impl.DefaultFileSystemManager; + import org.apache.commons.vfs.impl.StandardFileSystemManager; + import org.apache.commons.vfs.provider.local.LocalFile; + + + /** + * Example use of VFS sftp + * + */ + public class App { + + // Set these variables for your testing environment: + private String host = "sftpremote.example.com"; // Remote SFTP hostname + private String user = "smokey"; // Remote system login name + private String password = "bear"; // Remote system password + private String remoteDir = "/data/source/fires/smoke"; + // Look for a file path like "smoke20070128_wkt.txt" + private String filePatternString = ".*/smoke\\d{8}_wkt\\.txt"; + // Local directory to receive file + private String localDir = "/extra/data/fires/smoke"; + + + private File localDirFile; + private Pattern filePattern; + private FileSystemManager fsManager = null; + private FileSystemOptions opts = null; + private FileObject sftpFile; + + + public static void main(String[] args) { + System.out.println("SFTP download"); + App app = new App(); + + app.initialize(); + + app.process(); + + app.release(); + + } // main( String[] args ) + + + /** + * Creates the download directory localDir if it + * does not exist and makes a connection to the remote SFTP server. + * + */ + public void initialize() { + if (localDirFile == null) { + localDirFile = new File(localDir); + } + if (!this.localDirFile.exists()) { + localDirFile.mkdirs(); + } + + try { + this.fsManager = VFS.getManager(); + } catch (FileSystemException ex) { + throw new RuntimeException("failed to get fsManager from VFS", ex); + } + + UserAuthenticator auth = new StaticUserAuthenticator(null, this.user, + this.password); + this.opts = new FileSystemOptions(); + try { + DefaultFileSystemConfigBuilder.getInstance().setUserAuthenticator(opts, + auth); + } catch (FileSystemException ex) { + throw new RuntimeException("setUserAuthenticator failed", ex); + } + + this.filePattern = Pattern.compile(filePatternString); + } // initialize() + + + /** + * Retrieves files that match the specified FileSpec from the SFTP server + * and stores them in the local directory. + */ + public void process() { + + String startPath = "sftp://"; + this.host + this.remoteDir; + FileObject[] children; + + // Set starting path on remote SFTP server. + try { + this.sftpFile = this.fsManager.resolveFile(startPath, opts); + + System.out.println("SFTP connection successfully established to " + + startPath); + } catch (FileSystemException ex) { + throw new RuntimeException("SFTP error parsing path " + + this.remoteDir, + ex); + } + + + // Get a directory listing + try { + children = this.sftpFile.getChildren(); + } catch (FileSystemException ex) { + throw new RuntimeException("Error collecting directory listing of " + + startPath, ex); + } + + search: + for (FileObject f : children) { + try { + String relativePath = + File.separatorChar + f.getName().getBaseName(); + + if (f.getType() == FileType.FILE) { + System.out.println("Examining remote file " + f.getName()); + + if (!this.filePattern.matcher(f.getName().getPath()).matches()) { + System.out.println(" Filename does not match, skipping file ." + + relativePath); + continue search; + } + + String localUrl = "file://" + this.localDir + relativePath; + String standardPath = this.localDir + relativePath; + System.out.println(" Standard local path is " + standardPath); + LocalFile localFile = + (LocalFile) this.fsManager.resolveFile(localUrl); + System.out.println(" Resolved local file name: " + + localFile.getName()); + + if (!localFile.getParent().exists()) { + localFile.getParent().createFolder(); + } + + System.out.println(" ### Retrieving file ###"); + localFile.copyFrom(f, + new AllFileSelector()); + } else { + System.out.println("Ignoring non-file " + f.getName()); + } + } catch (FileSystemException ex) { + throw new RuntimeException("Error getting file type for " + + f.getName(), ex); + } + } // for (FileObject f : children) + } // process(Object obj) + + + public void release() { + ((DefaultFileSystemManager) this.fsManager).close(); + } // release() + } // class App }}} = Compiling = @@ -48, +289 @@ == Sample Output == {{{ - + SFTP download + Mar 25, 2008 1:00:44 PM org.apache.commons.vfs.VfsLog info + INFO: Using "/tmp/vfs_cache" as temporary files store. + SFTP connection successfully established to sftp://sftpremote.example.com/data/source/fires/smoke + Examining remote file sftp://sftpremote.example.com/data/source/fires/smoke/README.txt + Filename does not match, skipping file ./README.txt + Examining remote file sftp://sftpremote.example.com/data/source/fires/smoke/smoke20070328_wkt.txt + Standard local path is /extra/data/fires/smoke/smoke20070328_wkt.txt + Resolved local file name: file:///extra/data/fires/smoke/smoke20070328_wkt.txt + ### Retrieving file ### + Examining remote file sftp://sftpremote.example.com/data/source/fires/smoke/smoke20070329.txt + Filename does not match, skipping file ./smoke20070329.txt }}} There should now be files matching the {{{filePatternString}}} in the local machine directory "/extra/data/fires/smoke". --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
