Re: Streaming data into Hadoop

2008-12-09 Thread Aaron Kimball
Note also that cat foo | bin/hadoop fs -put - some/hdfs/path will use stdin. - Aaron On Mon, Dec 8, 2008 at 5:56 PM, Ryan LeCompte [EMAIL PROTECTED] wrote: Just what I need -- thanks! On Mon, Dec 8, 2008 at 7:31 PM, Alex Loddengaard [EMAIL PROTECTED] wrote: This should answer your

Re: Streaming data into Hadoop

2008-12-09 Thread Ryan LeCompte
Even better! I'll try this out tomorrow. Thanks, Ryan On Dec 9, 2008, at 10:36 PM, Aaron Kimball [EMAIL PROTECTED] wrote: Note also that cat foo | bin/hadoop fs -put - some/hdfs/path will use stdin. - Aaron On Mon, Dec 8, 2008 at 5:56 PM, Ryan LeCompte [EMAIL PROTECTED] wrote: Just

Streaming data into Hadoop

2008-12-08 Thread Ryan LeCompte
Hello all, I normally upload files into hadoop via bin/hadoop fs -put file dest. However, is there a way to somehow stream data into Hadoop? For example, I'd love to do something like this: zcat xxx HADOOP_HDFS_DESTINATION This would save me a ton of time since I don't have to first unpack

Re: Streaming data into Hadoop

2008-12-08 Thread Alex Loddengaard
This should answer your questions: http://wiki.apache.org/hadoop/MountableHDFS Alex On Mon, Dec 8, 2008 at 2:19 PM, Ryan LeCompte [EMAIL PROTECTED] wrote: Hello all, I normally upload files into hadoop via bin/hadoop fs -put file dest. However, is there a way to somehow stream data into

Re: Streaming data into Hadoop

2008-12-08 Thread Ryan LeCompte
Just what I need -- thanks! On Mon, Dec 8, 2008 at 7:31 PM, Alex Loddengaard [EMAIL PROTECTED] wrote: This should answer your questions: http://wiki.apache.org/hadoop/MountableHDFS Alex On Mon, Dec 8, 2008 at 2:19 PM, Ryan LeCompte [EMAIL PROTECTED] wrote: Hello all, I normally upload