> On Sept. 8, 2015, 6:35 a.m., Dian Fu wrote: > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java, > > line 55 > > <https://reviews.apache.org/r/37870/diff/1/?file=1057165#file1057165line55> > > > > Seems that it should be "boolean incremental = jobConfig.incremental != > > null && jobConfig.incremental.incrementalType == > > IncrementalType.NEW_FILES;".
(jobConfig.incremental == null) should never be the case, I'll add assert for that. > On Sept. 8, 2015, 6:35 a.m., Dian Fu wrote: > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java, > > line 56 > > <https://reviews.apache.org/r/37870/diff/1/?file=1057165#file1057165line56> > > > > This log should be wrapped under if (incremental) ... Lol, good catch, my bad :) > On Sept. 8, 2015, 6:35 a.m., Dian Fu wrote: > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java, > > line 127 > > <https://reviews.apache.org/r/37870/diff/1/?file=1057166#file1057166line127> > > > > Seems that it should be "fromJobConfig.incremental != null ? > > fromJobConfig.incremental.lastImportedDate : 0;". The "lastImportedDate" actually can be null. I'll add an assert to ensure that fromJobConfig.incremental is not null (this shouldn't happen, ever) and leave this one unchanged. - Jarek ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37870/#review97970 ----------------------------------------------------------- On Aug. 28, 2015, 3:40 a.m., Jarek Cecho wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/37870/ > ----------------------------------------------------------- > > (Updated Aug. 28, 2015, 3:40 a.m.) > > > Review request for Sqoop. > > > Bugs: SQOOP-2525 > https://issues.apache.org/jira/browse/SQOOP-2525 > > > Repository: sqoop-sqoop2 > > > Description > ------- > > Implemented incremental import as suggested in the JIRA. > > > Diffs > ----- > > common/src/main/java/org/apache/sqoop/error/code/HdfsConnectorError.java > 6cd66cc > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConstants.java > 9d20a79 > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java > 6d79db7 > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java > f5d9e1f > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java > 119955d > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/FromJobConfiguration.java > 618366e > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/IncrementalRead.java > PRE-CREATION > > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/IncrementalType.java > PRE-CREATION > > connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties > db23a95 > > connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestFromDestroyer.java > PRE-CREATION > > connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestFromInitializer.java > 5215901 > > test/src/test/java/org/apache/sqoop/integration/connector/hdfs/HdfsIncrementalReadTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/37870/diff/ > > > Testing > ------- > > Added new integration test validating the end-to-end use case. > > > Thanks, > > Jarek Cecho > >
