Dan Burkert has submitted this change and it was merged. Change subject: Fix Spark shutdown ......................................................................
Fix Spark shutdown The Kudu client was creating Netty timers with non-daemon threads, which prevented the Spark shell JVM from beginning the shutdown process. This commit also adds a shutdown hook to the Spark connector so that when the shell process exits it will properly shut down the Kudu client which prevents batched but unflushed writes from being lost. See http://apache-spark-developers-list.1001551.n3.nabble.com/graceful-shutdown-in-external-data-sources-td16684.html for more discussion. Change-Id: I30a64ec5eb30d70361204646523c9947d88c251f Reviewed-on: http://gerrit.cloudera.org:8080/2571 Reviewed-by: Adar Dembo <[email protected]> Tested-by: Kudu Jenkins --- M java/kudu-client/src/main/java/org/kududb/client/AsyncKuduClient.java M java/kudu-spark/src/main/scala/org/kududb/spark/KuduContext.scala 2 files changed, 38 insertions(+), 5 deletions(-) Approvals: Adar Dembo: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/2571 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I30a64ec5eb30d70361204646523c9947d88c251f Gerrit-PatchSet: 5 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Dan Burkert <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Binglin Chang <[email protected]> Gerrit-Reviewer: Dan Burkert <[email protected]> Gerrit-Reviewer: Jean-Daniel Cryans Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon <[email protected]>
