----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3126/#review3829 -----------------------------------------------------------
Hi Eric, thank you very much for your time and effort that you've spent on this issue. It looks like you did a lot of changes across entire sqoop code base. The patch seems to me pretty heavy for just wrapping one method that is used only twice in sqoop code base. Instead of wrapping Configuration class into own object and then doing all the changes, I would propose to create our own method getInstances and put it somewhere (ConfigurationHelper seems as a good candidate) as a static method with following definition getInstances(Configuration conf, String name, Class<U> xface). This method would firstly test configuration object whether it defines method getInstances using JAVA reflection and if so, it would call the default implementation that is present in hadoop 0.21+ or CHD3. If not we would execute our own code that would simulate the default behavior for hadoop. What do you think about that? Jarcec - Jarek On 2011-12-09 21:41:48, Eric Wadsworth wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/3126/ > ----------------------------------------------------------- > > (Updated 2011-12-09 21:41:48) > > > Review request for Sqoop and Jarek Cecho. > > > Summary > ------- > > I added a ConfigurationHolder class, to try and make sqoop more compatible > with hadoop 0.20.x. > > > This addresses bug https://issues.apache.org/jira/browse/SQOOP-384. > > https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/SQOOP-384 > > > Diffs > ----- > > /src/java/com/cloudera/sqoop/ConnFactory.java 1212018 > /src/java/com/cloudera/sqoop/Sqoop.java 1212018 > /src/java/com/cloudera/sqoop/SqoopOptions.java 1212018 > /src/java/com/cloudera/sqoop/config/ConfigurationHelper.java 1212018 > /src/java/com/cloudera/sqoop/hive/HiveImport.java 1212018 > /src/java/com/cloudera/sqoop/hive/TableDefWriter.java 1212018 > /src/java/com/cloudera/sqoop/io/CodecMap.java 1212018 > /src/java/com/cloudera/sqoop/io/LobFile.java 1212018 > /src/java/com/cloudera/sqoop/io/LobReaderCache.java 1212018 > /src/java/com/cloudera/sqoop/io/SplittingOutputStream.java 1212018 > /src/java/com/cloudera/sqoop/lib/LargeObjectLoader.java 1212018 > /src/java/com/cloudera/sqoop/manager/MySQLUtils.java 1212018 > /src/java/com/cloudera/sqoop/mapreduce/AvroJob.java 1212018 > /src/java/com/cloudera/sqoop/mapreduce/ExportJobBase.java 1212018 > /src/java/com/cloudera/sqoop/mapreduce/MySQLDumpMapper.java 1212018 > /src/java/com/cloudera/sqoop/mapreduce/db/DBConfiguration.java 1212018 > /src/java/com/cloudera/sqoop/mapreduce/db/DBRecordReader.java 1212018 > /src/java/com/cloudera/sqoop/mapreduce/db/DataDrivenDBInputFormat.java > 1212018 > /src/java/com/cloudera/sqoop/mapreduce/db/DataDrivenDBRecordReader.java > 1212018 > /src/java/com/cloudera/sqoop/mapreduce/db/OracleDBRecordReader.java 1212018 > > /src/java/com/cloudera/sqoop/mapreduce/db/OracleDataDrivenDBRecordReader.java > 1212018 > /src/java/com/cloudera/sqoop/metastore/JobStorageFactory.java 1212018 > /src/java/com/cloudera/sqoop/metastore/hsqldb/HsqldbMetaStore.java 1212018 > /src/java/com/cloudera/sqoop/util/DirectImportUtils.java 1212018 > /src/java/com/cloudera/sqoop/util/TaskId.java 1212018 > /src/java/org/apache/sqoop/ConnFactory.java 1212018 > /src/java/org/apache/sqoop/Sqoop.java 1212018 > /src/java/org/apache/sqoop/SqoopOptions.java 1212018 > /src/java/org/apache/sqoop/config/ConfigurationHelper.java 1212018 > /src/java/org/apache/sqoop/config/ConfigurationHolder.java PRE-CREATION > /src/java/org/apache/sqoop/hbase/HBasePutProcessor.java 1212018 > /src/java/org/apache/sqoop/hive/HiveImport.java 1212018 > /src/java/org/apache/sqoop/hive/TableDefWriter.java 1212018 > /src/java/org/apache/sqoop/io/CodecMap.java 1212018 > /src/java/org/apache/sqoop/io/LobFile.java 1212018 > /src/java/org/apache/sqoop/io/LobReaderCache.java 1212018 > /src/java/org/apache/sqoop/io/SplittingOutputStream.java 1212018 > /src/java/org/apache/sqoop/lib/LargeObjectLoader.java 1212018 > /src/java/org/apache/sqoop/lib/LobRef.java 1212018 > /src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java 1212018 > /src/java/org/apache/sqoop/manager/MySQLUtils.java 1212018 > /src/java/org/apache/sqoop/mapreduce/AsyncSqlRecordWriter.java 1212018 > /src/java/org/apache/sqoop/mapreduce/AutoProgressMapper.java 1212018 > /src/java/org/apache/sqoop/mapreduce/AvroExportMapper.java 1212018 > /src/java/org/apache/sqoop/mapreduce/AvroImportMapper.java 1212018 > /src/java/org/apache/sqoop/mapreduce/AvroJob.java 1212018 > /src/java/org/apache/sqoop/mapreduce/AvroOutputFormat.java 1212018 > /src/java/org/apache/sqoop/mapreduce/AvroRecordReader.java 1212018 > /src/java/org/apache/sqoop/mapreduce/CombineShimRecordReader.java 1212018 > /src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java 1212018 > /src/java/org/apache/sqoop/mapreduce/DelegatingOutputFormat.java 1212018 > /src/java/org/apache/sqoop/mapreduce/ExportJobBase.java 1212018 > /src/java/org/apache/sqoop/mapreduce/ExportOutputFormat.java 1212018 > /src/java/org/apache/sqoop/mapreduce/HBaseImportJob.java 1212018 > /src/java/org/apache/sqoop/mapreduce/ImportJobBase.java 1212018 > /src/java/org/apache/sqoop/mapreduce/JobBase.java 1212018 > /src/java/org/apache/sqoop/mapreduce/MergeJob.java 1212018 > /src/java/org/apache/sqoop/mapreduce/MergeMapperBase.java 1212018 > /src/java/org/apache/sqoop/mapreduce/MergeRecord.java 1212018 > /src/java/org/apache/sqoop/mapreduce/MergeTextMapper.java 1212018 > /src/java/org/apache/sqoop/mapreduce/MySQLDumpImportJob.java 1212018 > /src/java/org/apache/sqoop/mapreduce/MySQLDumpMapper.java 1212018 > /src/java/org/apache/sqoop/mapreduce/MySQLExportJob.java 1212018 > /src/java/org/apache/sqoop/mapreduce/MySQLExportMapper.java 1212018 > /src/java/org/apache/sqoop/mapreduce/RawKeyTextOutputFormat.java 1212018 > /src/java/org/apache/sqoop/mapreduce/TextExportMapper.java 1212018 > /src/java/org/apache/sqoop/mapreduce/UpdateOutputFormat.java 1212018 > /src/java/org/apache/sqoop/mapreduce/db/BigDecimalSplitter.java 1212018 > /src/java/org/apache/sqoop/mapreduce/db/BooleanSplitter.java 1212018 > /src/java/org/apache/sqoop/mapreduce/db/DBConfiguration.java 1212018 > /src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java 1212018 > /src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java 1212018 > /src/java/org/apache/sqoop/mapreduce/db/DBSplitter.java 1212018 > /src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java > 1212018 > /src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java > 1212018 > /src/java/org/apache/sqoop/mapreduce/db/DateSplitter.java 1212018 > /src/java/org/apache/sqoop/mapreduce/db/FloatSplitter.java 1212018 > /src/java/org/apache/sqoop/mapreduce/db/IntegerSplitter.java 1212018 > /src/java/org/apache/sqoop/mapreduce/db/OracleDBRecordReader.java 1212018 > /src/java/org/apache/sqoop/mapreduce/db/OracleDataDrivenDBInputFormat.java > 1212018 > /src/java/org/apache/sqoop/mapreduce/db/OracleDataDrivenDBRecordReader.java > 1212018 > /src/java/org/apache/sqoop/mapreduce/db/TextSplitter.java 1212018 > /src/java/org/apache/sqoop/metastore/JobStorageFactory.java 1212018 > /src/java/org/apache/sqoop/metastore/hsqldb/AutoHsqldbStorage.java 1212018 > /src/java/org/apache/sqoop/metastore/hsqldb/HsqldbJobStorage.java 1212018 > /src/java/org/apache/sqoop/metastore/hsqldb/HsqldbMetaStore.java 1212018 > /src/java/org/apache/sqoop/tool/BaseSqoopTool.java 1212018 > /src/java/org/apache/sqoop/tool/CodeGenTool.java 1212018 > /src/java/org/apache/sqoop/tool/CreateHiveTableTool.java 1212018 > /src/java/org/apache/sqoop/tool/ImportAllTablesTool.java 1212018 > /src/java/org/apache/sqoop/tool/ImportTool.java 1212018 > /src/java/org/apache/sqoop/tool/JobTool.java 1212018 > /src/java/org/apache/sqoop/tool/MetastoreTool.java 1212018 > /src/java/org/apache/sqoop/tool/SqoopTool.java 1212018 > /src/java/org/apache/sqoop/util/AppendUtils.java 1212018 > /src/java/org/apache/sqoop/util/DirectImportUtils.java 1212018 > /src/java/org/apache/sqoop/util/TaskId.java 1212018 > /src/test/com/cloudera/sqoop/TestIncrementalImport.java 1212018 > /src/test/com/cloudera/sqoop/TestMerge.java 1212018 > /src/test/com/cloudera/sqoop/lib/TestBlobRef.java 1212018 > /src/test/com/cloudera/sqoop/lib/TestClobRef.java 1212018 > /src/test/com/cloudera/sqoop/lib/TestLargeObjectLoader.java 1212018 > /src/test/com/cloudera/sqoop/mapreduce/TestImportJob.java 1212018 > /src/test/com/cloudera/sqoop/testutil/ImportJobTestCase.java 1212018 > /src/test/com/cloudera/sqoop/testutil/InjectableConnManager.java 1212018 > /src/test/com/cloudera/sqoop/tool/TestToolPlugin.java 1212018 > > Diff: https://reviews.apache.org/r/3126/diff > > > Testing > ------- > > "ant test" passes. I haven't had much success running sqoop at all, due to > the bug this patch is trying to fix (SQOOP-384). > > > Thanks, > > Eric > >
