Yes Imran, I would try to define the fs.defaultFS for the S3 in core-site.xml and see if it will help Sqoop to accept the S3 path.
Jarcec On Tue, Feb 04, 2014 at 08:08:17AM -0800, Imran Akbar wrote: > thanks Jarek, > How would I do that? Do I need to set fs.defaultFS in core-site.xml, or > is it something else? Is there a document somewhere which describes this? > > yours, > imran > > > On Mon, Feb 3, 2014 at 9:31 PM, Jarek Jarcec Cecho <[email protected]>wrote: > > > Would you mind trying to set the S3 filesystem as the default one for > > Sqoop? > > > > Jarcec > > > > On Mon, Feb 03, 2014 at 10:25:50AM -0800, Imran Akbar wrote: > > > Hi, > > > I've been able to sqoop from MySQL into HDFS, but I was wondering if > > it > > > was possible to send the data directly to S3 instead. I've read some > > posts > > > on this forum and others that indicate that it's not possible to do this > > - > > > could someone confirm? > > > > > > I tried to get it to work by setting: > > > --warehouse-dir s3n://MYS3APIKEY:MYS3SECRETKEY@bucketname/folder/ > > > or > > > --target-dir s3n://MYS3APIKEY:MYS3SECRETKEY@bucketname/folder/ > > > > > > options but I get the error: > > > ERROR tool.ImportTool: Imported Failed: This file system object (hdfs:// > > > 10.168.22.133:9000) does not support access to the request path > > > 's3n://****:****@iakbar.emr/new-hive-output/_logs' You possibly called > > > FileSystem.get(conf) when you should have called FileSystem.get(uri, > > conf) > > > to obtain a file system supporting your path > > > > > > If it's not possible to do this, should I just import to HDFS and then > > > output to S3? Is there an easy way to do this without having to specify > > > the schema of the whole table again? > > > > > > thanks, > > > imran > >
signature.asc
Description: Digital signature
