Re: Streaming data into Hadoop
Note also that cat foo | bin/hadoop fs -put - some/hdfs/path will use stdin. - Aaron On Mon, Dec 8, 2008 at 5:56 PM, Ryan LeCompte [EMAIL PROTECTED] wrote: Just what I need -- thanks! On Mon, Dec 8, 2008 at 7:31 PM, Alex Loddengaard [EMAIL PROTECTED] wrote: This should answer your questions: http://wiki.apache.org/hadoop/MountableHDFS Alex On Mon, Dec 8, 2008 at 2:19 PM, Ryan LeCompte [EMAIL PROTECTED] wrote: Hello all, I normally upload files into hadoop via bin/hadoop fs -put file dest. However, is there a way to somehow stream data into Hadoop? For example, I'd love to do something like this: zcat xxx HADOOP_HDFS_DESTINATION This would save me a ton of time since I don't have to first unpack the .tgz file and upload the raw file into HDFS. Is this possible with Hadoop 0.19? Thanks, Ryan
Re: Streaming data into Hadoop
Even better! I'll try this out tomorrow. Thanks, Ryan On Dec 9, 2008, at 10:36 PM, Aaron Kimball [EMAIL PROTECTED] wrote: Note also that cat foo | bin/hadoop fs -put - some/hdfs/path will use stdin. - Aaron On Mon, Dec 8, 2008 at 5:56 PM, Ryan LeCompte [EMAIL PROTECTED] wrote: Just what I need -- thanks! On Mon, Dec 8, 2008 at 7:31 PM, Alex Loddengaard [EMAIL PROTECTED] wrote: This should answer your questions: http://wiki.apache.org/hadoop/MountableHDFS Alex On Mon, Dec 8, 2008 at 2:19 PM, Ryan LeCompte [EMAIL PROTECTED] wrote: Hello all, I normally upload files into hadoop via bin/hadoop fs -put file dest. However, is there a way to somehow stream data into Hadoop? For example, I'd love to do something like this: zcat xxx HADOOP_HDFS_DESTINATION This would save me a ton of time since I don't have to first unpack the .tgz file and upload the raw file into HDFS. Is this possible with Hadoop 0.19? Thanks, Ryan
Streaming data into Hadoop
Hello all, I normally upload files into hadoop via bin/hadoop fs -put file dest. However, is there a way to somehow stream data into Hadoop? For example, I'd love to do something like this: zcat xxx HADOOP_HDFS_DESTINATION This would save me a ton of time since I don't have to first unpack the .tgz file and upload the raw file into HDFS. Is this possible with Hadoop 0.19? Thanks, Ryan
Re: Streaming data into Hadoop
This should answer your questions: http://wiki.apache.org/hadoop/MountableHDFS Alex On Mon, Dec 8, 2008 at 2:19 PM, Ryan LeCompte [EMAIL PROTECTED] wrote: Hello all, I normally upload files into hadoop via bin/hadoop fs -put file dest. However, is there a way to somehow stream data into Hadoop? For example, I'd love to do something like this: zcat xxx HADOOP_HDFS_DESTINATION This would save me a ton of time since I don't have to first unpack the .tgz file and upload the raw file into HDFS. Is this possible with Hadoop 0.19? Thanks, Ryan
Re: Streaming data into Hadoop
Just what I need -- thanks! On Mon, Dec 8, 2008 at 7:31 PM, Alex Loddengaard [EMAIL PROTECTED] wrote: This should answer your questions: http://wiki.apache.org/hadoop/MountableHDFS Alex On Mon, Dec 8, 2008 at 2:19 PM, Ryan LeCompte [EMAIL PROTECTED] wrote: Hello all, I normally upload files into hadoop via bin/hadoop fs -put file dest. However, is there a way to somehow stream data into Hadoop? For example, I'd love to do something like this: zcat xxx HADOOP_HDFS_DESTINATION This would save me a ton of time since I don't have to first unpack the .tgz file and upload the raw file into HDFS. Is this possible with Hadoop 0.19? Thanks, Ryan
