On Mon, Oct 4, 2010 at 7:14 PM, coolgold <grok...@edmunds.com> wrote: > > I'm considering using camel and activemq for moving gzipped (~1gb) files form > ftp server to hdfs. I'm aware of existence of ftp and hdfs components, but > I'm not sure about the support for splitting for gzipped files or streaming > method with FTP entry point. What is the most straightforward/scalable > approach? > --
On the FTP endpoint you can configure a local work path which tells Camel to stream the remote FTP file directly to a temporary file in the work path. Then you wont read the file into memory. We dont have any Camel component for reading compressed files such as gzip. The Apache Commons VFS project do have such features but the project is dying/died. There may be other libraries out there which makes it easy to work with compressed files. And we love contributions. > View this message in context: > http://camel.465427.n5.nabble.com/FTP-to-HDFS-large-gzipped-files-tp3192431p3192431.html > Sent from the Camel - Users mailing list archive at Nabble.com. > -- Claus Ibsen Apache Camel Committer Author of Camel in Action: http://www.manning.com/ibsen/ Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus
