1. Run map-only (no reducer) job. Output goes directly to hdfs. or 2. echo abc | hadoop dfs put - /user/knoguchi/somefile would write to hdfs reading from stdin.
Koji On 1/12/11 2:07 AM, "Mapred Learn" <mapred.le...@gmail.com> wrote: > Hi, > I found out that : > https://github.com/cloudera/hue/blob/master/desktop/libs/hadoop/src/hadoop/fs/ > hadoopfs.py > <https://github.com/cloudera/hue/blob/master/desktop/libs/hadoop/src/hadoop/fs > /hadoopfs.py> > > can be used to write data directly to HDFS without writing to a local > filesystem but I am not able to understand how. > > Could somebody tell me how can I do this ? Or is there some other best way to > do it ? > > Basiclly my use case is to pipe standard out from a program directly > to HDFS so that it does not have to go through the disk write. > > > Thanks in advance ! > >