Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/3871
to look at the new patch set (#3).
Change subject: KUDU-1533 Spark Kudu Rdd/Dataframe upsert
......................................................................
KUDU-1533 Spark Kudu Rdd/Dataframe upsert
This patch improves the Kudu SparkSQL integration in two ways:
1) Removed trait creatableRelationProvider from DefaultSource. This
is an improvement because this trait cannot be correctly implemented
for Kudu without support for table truncation and because some modes
require auto-table creation but there's no satisfactory mechanism to
specify things like the partition schema in that case.
2) Added {insert, update, upsert, delete}Rows methods to KuduContext.
This replaces and expands on the removed functionality of
DefaultSource.
The change is backwards-incompatible. Syntax like
df.write.options(newOptions).mode("append").kudu
no longer works, and now needs to be written like
kuduContext.insertRows(df, tableName)
Change-Id: Ib8e0d50fb74dc2ce5e757e8a56fc1e863f699822
---
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/package.scala
M
java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala
4 files changed, 160 insertions(+), 92 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/3871/3
--
To view, visit http://gerrit.cloudera.org:8080/3871
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib8e0d50fb74dc2ce5e757e8a56fc1e863f699822
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wdberke...@gmail.com>
Gerrit-Reviewer: Chris George <chris.geo...@rms.com>
Gerrit-Reviewer: Dan Burkert <d...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wdberke...@gmail.com>
