Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12715 )
Change subject: [java] Make the KuduScanner iterable ...................................................................... [java] Make the KuduScanner iterable This patch makes the KuduScanner iterable to simplify the API for processing the RowResults. It also pushes the keep alive calls down into the KuduScannerIterator so that any user of the KuduClient can automatically leverage the API the same way the Spark integration does. The existing RowResultIterator implementation reuses the same RowResult object for all rows in the batch. This can cause unexpected behavior when storing RowResults from the scanner. This patch changes the RowResultIterator to instead create a RowResult object for every row, while still sharing the underlying data. I have replaced as many places where the scanner is iterated as I can find with this new implementation. The Spark implementations were replaced as well, but a special hasNext method with a callback was required. Change-Id: I3e4ac59e30d0562c0a381d5e304af1dcfdcf5a1a Reviewed-on: http://gerrit.cloudera.org:8080/12715 Reviewed-by: Adar Dembo <[email protected]> Tested-by: Kudu Jenkins --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala M java/kudu-client/src/main/java/org/apache/kudu/client/AbstractKuduScannerBuilder.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanToken.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java A java/kudu-client/src/main/java/org/apache/kudu/client/KuduScannerIterator.java M java/kudu-client/src/main/java/org/apache/kudu/client/RowResult.java M java/kudu-client/src/main/java/org/apache/kudu/client/RowResultIterator.java M java/kudu-client/src/main/java/org/apache/kudu/util/Slice.java M java/kudu-client/src/test/java/org/apache/kudu/client/ITClient.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestFlexiblePartitioning.java A java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java M java/kudu-mapreduce/src/main/java/org/apache/kudu/mapreduce/KuduTableInputFormat.java M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduReadOptions.scala M java/kudu-test-utils/src/main/java/org/apache/kudu/test/ClientTestUtil.java M src/kudu/client/client.proto 19 files changed, 413 insertions(+), 157 deletions(-) Approvals: Adar Dembo: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/12715 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I3e4ac59e30d0562c0a381d5e304af1dcfdcf5a1a Gerrit-Change-Number: 12715 Gerrit-PatchSet: 7 Gerrit-Owner: Grant Henke <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Grant Henke <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mike Percy <[email protected]> Gerrit-Reviewer: Todd Lipcon <[email protected]> Gerrit-Reviewer: Will Berkeley <[email protected]>
