Hi,
I am running hadoop 0.15.3 on 2 EC2 instances from a public ami (
ami-381df851) . Our input files are on S3.
When I try to do a distcp for an Input file from S3 onto hdfs on EC2, the
copy fails with an error that the file does not exist. However, if I run
copyToLocal from S3 onto local machine, the copy succeeds, confirming that the
input file does exist on S3 bucket.
Furthermore, I can see the Input file on S3 bucket from Mozilla S3 Firefox
organizer as well.
I am left with no explanation as to why I am getting the File Not Found error
on S3 when I have every confirmation that the file does exist over there.
Just to make sure that I am specifying the parameters correctly :
> bin/hadoop distcp s3//<id>:<secret>@<bucket>/fileone.txt
/somefolder_on_hdfs/fileone.txt : Fails - Input source doesnt exist.
> bin/hadoop fs -copyToLocal s3://<id>:<secret>@bucket/fileone.txt
/somefolder_on_local/fileone.txt : succeeds
Any correction?
Thanks.
---------------------------------
You rock. That's why Blockbuster's offering you one month of Blockbuster Total
Access, No Cost.
