----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56770/ -----------------------------------------------------------
Review request for Sqoop, Jarek Cecho and Venkat Ranganathan. Repository: sqoop-trunk Description ------- Currently Sqoop assumes default file system when it comes to IO operations. It makes it hard to use other FileSystem implementations as source or destination. https://issues.apache.org/jira/browse/SQOOP-3136 Diffs ----- src/java/com/cloudera/sqoop/io/LobReaderCache.java 3394296 src/java/org/apache/sqoop/hive/HiveImport.java 4828375 src/java/org/apache/sqoop/hive/TableDefWriter.java c9962e9 src/java/org/apache/sqoop/io/LobReaderCache.java bd75374 src/java/org/apache/sqoop/io/SplittingOutputStream.java 5f98192 src/java/org/apache/sqoop/lib/LargeObjectLoader.java 70c0f4e src/java/org/apache/sqoop/manager/oracle/OraOopUtilities.java e81588c src/java/org/apache/sqoop/mapreduce/CombineFileInputFormat.java e08f997 src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java 260bc29 src/java/org/apache/sqoop/mapreduce/ExportJobBase.java 27f84da src/java/org/apache/sqoop/mapreduce/HBaseBulkImportJob.java b32cdd1 src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 626119b src/java/org/apache/sqoop/mapreduce/JdbcUpdateExportJob.java f911280 src/java/org/apache/sqoop/mapreduce/MergeJob.java 5b6c4df src/java/org/apache/sqoop/tool/ImportTool.java 258ef79 src/java/org/apache/sqoop/util/FileSystemUtil.java PRE-CREATION src/java/org/apache/sqoop/util/FileUploader.java 155cffc src/test/org/apache/sqoop/util/TestFileSystemUtil.java PRE-CREATION Diff: https://reviews.apache.org/r/56770/diff/ Testing ------- ** Build: ant clean package ... BUILD SUCCESSFUL Total time: 51 seconds ** Test: ant test ... BUILD SUCCESSFUL Total time: 7 minutes 21 seconds * On Hadoop Cluster: ** original version of sqoop: sqoop import --connect <JDBC URL> --table table1 --driver <JDBC DRIVER> --username root --password **** --delete-target-dir --target-dir s3a://some-bucket/tmp/sqoop ... 17/02/15 19:16:59 ERROR tool.ImportTool: Imported Failed: Wrong FS: s3a://some-bucket/tmp/sqoop, expected: hdfs://<DNS>:8020 ** updated version of sqoop: sqoop import --connect <JDBC URL> --table table1 --driver <JDBC DRIVER> --username root --password **** --delete-target-dir --target-dir s3a://some-bucket/tmp/sqoop ... 17/02/15 22:24:42 INFO mapreduce.Job: Running job: job_1487183144282_0004 17/02/15 22:24:52 INFO mapreduce.Job: Job job_1487183144282_0004 running in uber mode : false 17/02/15 22:24:52 INFO mapreduce.Job: map 0% reduce 0% 17/02/15 22:25:04 INFO mapreduce.Job: map 25% reduce 0% 17/02/15 22:25:06 INFO mapreduce.Job: map 50% reduce 0% 17/02/15 22:25:07 INFO mapreduce.Job: map 75% reduce 0% 17/02/15 22:25:08 INFO mapreduce.Job: map 100% reduce 0% 17/02/15 22:25:08 INFO mapreduce.Job: Job job_1487183144282_0004 completed successfully 17/02/15 22:25:08 INFO mapreduce.Job: Counters: 36 ... 17/02/15 22:25:08 INFO mapreduce.ImportJobBase: Retrieved 4993 records. Thanks, Illya Yalovyy