Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11571
Change subject: KUDU-2563: [spark] Use the scanner keepAlive API ...................................................................... KUDU-2563: [spark] Use the scanner keepAlive API Adds scheduled keepAlive calls to the scanner in the KuduRDD RowIterator. The period in which the calls are made is configurable via keepAlivePeriodMs and has a default of 15 seconds (which is 1/4 the default scanner ttl). This implementation is similar to the Impala integration, except that it uses a background thread instead of performing the calls in the processing loop. I think sending a keepAlive at a fixed rate regardless of iterator progress will make for a more stable understandable implementation. There will be no spark job that will prevent the keepAlive calls from happening. Change-Id: Ia7f26d6ab8deb24982055d247938a11e188c35db --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduReadOptions.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduRDDTest.scala 6 files changed, 114 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/11571/1 -- To view, visit http://gerrit.cloudera.org:8080/11571 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia7f26d6ab8deb24982055d247938a11e188c35db Gerrit-Change-Number: 11571 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke <granthe...@apache.org>
