Re: Using different file systems for Map Reduce job input and output
Thanks ! Naama On Mon, Oct 6, 2008 at 10:27 AM, Amareshwari Sriramadasu < [EMAIL PROTECTED]> wrote: > Hi Naama, > > Yes. It is possible to specify using the apis > > FileInputFormat#setInputPaths(), FileOutputFormat#setOutputPath(). > You can specify the FileSystem uri for the path. > > Thanks, > Amareshwari > > Naama Kraus wrote: > >> Hi, >> >> I wanted to know if it is possible to use different file systems for Map >> Reduce job input and output. >> I.e. have a M/R job input reside on one file system and the M/R output be >> written to another file system (e.g. input on HDFS, output on KFS. Input >> on >> HDFS output on local file system, or anything else ...). >> >> Is it possible to somehow specify that through >> FileInputFormat#setInputPaths(), FileOutputFormat#setOutputPath() ? >> Or by any other mechanism ? >> >> Thanks, Naama >> >> >> > > -- oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo "If you want your children to be intelligent, read them fairy tales. If you want them to be more intelligent, read them more fairy tales." (Albert Einstein)
Re: Using different file systems for Map Reduce job input and output
Hi Naama, Yes. It is possible to specify using the apis FileInputFormat#setInputPaths(), FileOutputFormat#setOutputPath(). You can specify the FileSystem uri for the path. Thanks, Amareshwari Naama Kraus wrote: Hi, I wanted to know if it is possible to use different file systems for Map Reduce job input and output. I.e. have a M/R job input reside on one file system and the M/R output be written to another file system (e.g. input on HDFS, output on KFS. Input on HDFS output on local file system, or anything else ...). Is it possible to somehow specify that through FileInputFormat#setInputPaths(), FileOutputFormat#setOutputPath() ? Or by any other mechanism ? Thanks, Naama
Using different file systems for Map Reduce job input and output
Hi, I wanted to know if it is possible to use different file systems for Map Reduce job input and output. I.e. have a M/R job input reside on one file system and the M/R output be written to another file system (e.g. input on HDFS, output on KFS. Input on HDFS output on local file system, or anything else ...). Is it possible to somehow specify that through FileInputFormat#setInputPaths(), FileOutputFormat#setOutputPath() ? Or by any other mechanism ? Thanks, Naama -- oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo 00 oo "If you want your children to be intelligent, read them fairy tales. If you want them to be more intelligent, read them more fairy tales." (Albert Einstein)
