[
https://issues.apache.org/jira/browse/SQOOP-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rodrigo Matihara updated SQOOP-1306:
------------------------------------
Attachment: scenarios.png
Jarek, I understand your point, but I think that distcp is not what I need.
I created a diagram (attached file) with 2 scenarios, that I think that you
will understand what I'm seeking for.
The first scenario is working perfect today, I can import data from a MySQL to
HDFS/S3 creating temp files on the target-dir.
The second scenario is what I need, import data from MySQL to HDFS/S3, creating
temp files locally and write final files to HDFS/S3.
> Allow Sqoop to move files from different FileSystems on incremental import
> --------------------------------------------------------------------------
>
> Key: SQOOP-1306
> URL: https://issues.apache.org/jira/browse/SQOOP-1306
> Project: Sqoop
> Issue Type: Improvement
> Affects Versions: 1.4.4
> Reporter: Rodrigo Matihara
> Attachments: scenarios.png
>
>
> Sqoop doesn't allow to move files from different FileSystems executing an
> import (--append or --incremental).
> Trying to import using a local temp-dir and a S3 target-dir, the operation is
> complete, however the file is not created in S3 bucket and this warning is
> raised:
> WARN - Cannot append files to target dir; no such directory:
> _sqoop/15151724000000436_31417_localhost.localdomain_<tablename>
> Looking into on source, I found that AppendUtils.java considers that tempDir
> and targetDir are at the same Filesystem.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
