GitHub user rdblue opened a pull request:
https://github.com/apache/spark/pull/21888
[SPARK-24253][SQL][WIP] Implement DeleteFrom for v2 tables
## What changes were proposed in this pull request?
This adds support for DELETE FROM in SQL using the new DeleteFrom logical
plan and v2 DeleteSupport mix-in.
To identify the v2 table to delete data from, this uses the TableCatalog
API introduced in #21306.
TableIdentifier has been updated with a new superclass,
CatalogTableIdentifier. Code paths that don't support identifiers with a
catalog continue to use TableIdentifier, allowing a smooth transition even
though some code assumes there is no support for multiple catalogs.
UnresovledRelation now supports CatalogTableIdentifier, but resolution will
only happen when the catalog is not defined or through a new rule with support
for v2 TableCatalog. Existing uses of UnresolvedRelation access the catalog
using tableIdentifier, which asserts that the catalog is not defined before
returning to ensure catalog identifiers don't leak to code without catalog
support.
WIP: This is based on #21306, #21305, and #21877 and includes the changes
from those PRs.
## How was this patch tested?
WIP, will add tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rdblue/spark SPARK-24253-add-delete-from
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21888.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21888
----
commit dc977d26acb29731f62addaec67ea6dbe077e670
Author: Ryan Blue <blue@...>
Date: 2018-05-05T01:13:01Z
SPARK-24252: Add v2 data source mix-in for catalog support.
commit 7d75da5e6abc2bf90b8cf7f530412d6b01ee6938
Author: Ryan Blue <blue@...>
Date: 2018-05-11T21:27:47Z
SPARK-24252: Add copyright headers.
commit 3be3b7c320c7f9b68270112c9d3f7fa0a1ff377c
Author: Ryan Blue <blue@...>
Date: 2018-07-04T17:02:52Z
SPARK-24252: Update for review comments.
* Rename CatalogSupport to TableSupport
* Rename DataSourceCatalog to TableCatalog
* Remove name and database from Table
commit 7ad0c8e8fe45f1b85011fc91dd162dba3951593e
Author: Ryan Blue <blue@...>
Date: 2018-07-04T17:19:45Z
SPARK-24252: Add TableChange example to Javadocs.
commit 3de2cfa3d069aa905c2c5076acc8d59b5ee0c63e
Author: Ryan Blue <blue@...>
Date: 2018-07-25T18:11:22Z
SPARK-24252: Update catalog API in org.apache.spark.sql.catalog.v2.
commit ae61a797f584fba8ca2601b76f80774d22c3845c
Author: Ryan Blue <blue@...>
Date: 2018-07-25T19:25:32Z
SPARK-24252: Add tests and clarify javadoc.
commit 8b135607512fda53ff5adc34ac8b965b04f3a3d9
Author: Ryan Blue <blue@...>
Date: 2018-07-25T20:00:23Z
SPARK-24252: Fix Map.Entry reference causing doc failures.
commit b4c505758a5137a311ac098965ace19af33bd198
Author: Ryan Blue <blue@...>
Date: 2018-07-26T05:07:52Z
SPARK-24252: Move catalog.v2 classes into catalyst.
This is needed for catalyst plans to use catalogs.
commit 85a49d729e0ae75cf6e1725216829c7d5e79a8a1
Author: Ryan Blue <blue@...>
Date: 2018-05-07T15:54:37Z
SPARK-24251: Add AppendData logical plan.
This adds a new logical plan, AppendData, that was proposed in
SPARK-23521. This also adds an analyzer rule to validate data written
with AppendData against the target table. DataFrameWriter is also
updated so that v2 writes use the new AppendData logical plan.
commit 770a31cf43903736b10455a82b4743987dd27db6
Author: Ryan Blue <blue@...>
Date: 2018-05-11T22:04:15Z
SPARK-24253: Add DeleteSupport mix-in for DataSourceV2.
commit f769341547cdc63f77456c1c2677070a4d3d363c
Author: Ryan Blue <blue@...>
Date: 2018-05-11T22:09:22Z
SPARK-24253: Add interface description to DeleteSupport.
commit af16c4238fd486a82e86dda02ad21279104afe21
Author: Ryan Blue <blue@...>
Date: 2018-06-13T23:08:21Z
Add DeleteFrom logical plan, add to parser.
commit 88c12a47cbc3eea04ed9d90e91c62285de594b7e
Author: Ryan Blue <blue@...>
Date: 2018-07-25T18:11:45Z
Add CTAS and RTAS support.
This uses the catalog API introduced in SPARK-24252 to implement CTAS
and RTAS plans.
commit 19b83bd796406ccc69ed8a138ff7652bcb6049d1
Author: Ryan Blue <blue@...>
Date: 2018-07-26T22:02:25Z
SPARK-24253: Implement DELETE FROM for v2 tables.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
