Hi Sadak
You really don't need to fire a map reduce job to copy files from
a local file system to hdfs. You can do it in two easy ways
*Using linux CLI* - if you are going in with a shell script. The most
convenient option and handy.
hadoop fs -copyFromLocal <file/dir in lfs> <destination dir in hdfs>
*Using JAVA API*
//load the hadoop configuration
Configuration hadoopConf=new Configuration();
//get the default associated file system
FileSystem fileSystem=FileSystem.get(hadoopConf) ;
//copy from lfs to hdfs
fileSystem.copyFromLocalFile(new Path("source file/dir in lfs"), new
Path("Destn dir in hdfs"));
Please read the API before implementation. There are variants of the method
copyFromLocalFile as well lot of other methods which you'd find useful if
you are choosing java API path.
You can do the reversal operation as
hadoop fs -copyToLocal
fileSystem.copyToLocalFile(new Path("dir/file dir in hdfs"), new Path("Destn
dir in lfs"));
Hope it helps and gives you a kick start into hadoop.
Thanks and Regards
Bejoy.K.S
On Wed, Oct 5, 2011 at 12:23 AM, visioner sadak <visioner.sa...@gmail.com>wrote:
> Hello guys,
>
> I would like to know how to do file uploads in HDFS using
> java,is it to be done using map reduce what if i have a large number of
> small files should i use sequence file along with map reduce???,It will be
> great if you can provide some sort of information...
