> On Oct. 1, 2013, 7:49 p.m., Jarek Cecho wrote: > > Thank you Alexandre for taking up this ticket! > > > > Would you mind adding automated tests to ensure that the functionality is > > indeed working? > > Alexandre Normand wrote: > I'd like to be able to integration test bulk load but this is still not a > possibility in non-distributed mode. We currently have the same problems in > some of our map/reduce jobs that use bulk loading. The way we've approached > it with those is that we have two modes for each job: bulk load enabled/bulk > load disabled. The integration tests always run with bulk load disabled. The > "bulk load mode" is tested on a real cluster with automated tests that run > from jenkins. > > For what it's worth, I've done the manual testing on our cluster for this > patch. > > Jarek Cecho wrote: > Thank you for the feedback Alexandre! Do you happen to know if there is > already a JIRA for adding the bulk load mode into the HBase MiniCluster? I > tried to search for now, but didn't found it.
Yes, I think this captures the current blocker (although it's possible that other things in local mode would still need work once that's fixed): https://issues.apache.org/jira/browse/HBASE-7744. - Alexandre ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14240/#review26576 ----------------------------------------------------------- On Oct. 4, 2013, 5:45 p.m., Alexandre Normand wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/14240/ > ----------------------------------------------------------- > > (Updated Oct. 4, 2013, 5:45 p.m.) > > > Review request for Sqoop, Jarek Cecho, Jean-Marc Spaggiari, and Vasanth kumar > RJ. > > > Repository: sqoop-trunk > > > Description > ------- > > This is the patch to address some of the comments on Zhancheng Deng's review > request (https://reviews.apache.org/r/13052/) which was basically the change > to add the --bulk-load-dir option to support the HBase doBulkLoad function. > > > Diffs > ----- > > src/java/org/apache/sqoop/SqoopOptions.java 01805f9 > src/java/org/apache/sqoop/hbase/HBasePutProcessor.java 9ceb5bd > src/java/org/apache/sqoop/hbase/ToStringPutTransformer.java 5ccf311 > src/java/org/apache/sqoop/manager/SqlManager.java 2a4992d > src/java/org/apache/sqoop/mapreduce/HBaseBulkImportJob.java PRE-CREATION > src/java/org/apache/sqoop/mapreduce/HBaseBulkImportMapper.java PRE-CREATION > src/java/org/apache/sqoop/mapreduce/ImportJobBase.java ab7f21e > src/java/org/apache/sqoop/tool/BaseSqoopTool.java 0eca991 > src/test/com/cloudera/sqoop/TestSqoopOptions.java 03e2504 > > Diff: https://reviews.apache.org/r/14240/diff/ > > > Testing > ------- > > > Thanks, > > Alexandre Normand > >
