Hello Will Berkeley, Mike Percy, Kudu Jenkins, Adar Dembo, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12715
to look at the new patch set (#6).
Change subject: [java] Make the KuduScanner iterable
......................................................................
[java] Make the KuduScanner iterable
This patch makes the KuduScanner iterable to simplify
the API for processing the RowResults. It also pushes
the keep alive calls down into the KuduScannerIterator
so that any user of the KuduClient can automatically
leverage the API the same way the Spark integration does.
The existing RowResultIterator implementation reuses
the same RowResult object for all rows in the batch.
This can cause unexpected behavior when storing
RowResults from the scanner. This patch changes
the RowResultIterator to instead create a RowResult
object for every row, while still sharing the underlying
data.
I have replaced as many places where the scanner
is iterated as I can find with this new implementation.
The Spark implementations were replaced as well, but
a special hasNext method with a callback was required.
Change-Id: I3e4ac59e30d0562c0a381d5e304af1dcfdcf5a1a
---
M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupOptions.scala
M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduBackupRDD.scala
M
java/kudu-client/src/main/java/org/apache/kudu/client/AbstractKuduScannerBuilder.java
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java
M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanToken.java
M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java
A java/kudu-client/src/main/java/org/apache/kudu/client/KuduScannerIterator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RowResult.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RowResultIterator.java
M java/kudu-client/src/main/java/org/apache/kudu/util/Slice.java
M java/kudu-client/src/test/java/org/apache/kudu/client/ITClient.java
M
java/kudu-client/src/test/java/org/apache/kudu/client/TestFlexiblePartitioning.java
A java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java
M
java/kudu-mapreduce/src/main/java/org/apache/kudu/mapreduce/KuduTableInputFormat.java
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala
M
java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduReadOptions.scala
M java/kudu-test-utils/src/main/java/org/apache/kudu/test/ClientTestUtil.java
M src/kudu/client/client.proto
19 files changed, 413 insertions(+), 157 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/15/12715/6
--
To view, visit http://gerrit.cloudera.org:8080/12715
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3e4ac59e30d0562c0a381d5e304af1dcfdcf5a1a
Gerrit-Change-Number: 12715
Gerrit-PatchSet: 6
Gerrit-Owner: Grant Henke <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
Gerrit-Reviewer: Grant Henke <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <[email protected]>
Gerrit-Reviewer: Todd Lipcon <[email protected]>
Gerrit-Reviewer: Will Berkeley <[email protected]>
