[
https://issues.apache.org/jira/browse/SPARK-35801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated SPARK-35801:
------------------------------
Fix Version/s: 3.5.0
> SPIP: Row-level operations in Data Source V2
> --------------------------------------------
>
> Key: SPARK-35801
> URL: https://issues.apache.org/jira/browse/SPARK-35801
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: Anton Okolnychyi
> Priority: Major
> Labels: SPIP
> Fix For: 3.5.0
>
>
> Row-level operations such as UPDATE, DELETE, MERGE are becoming more and more
> important for modern Big Data workflows. Use cases include but are not
> limited to deleting a set of records for regulatory compliance, updating a
> set of records to fix an issue in the ingestion pipeline, applying changes in
> a transaction log to a fact table. Row-level operations allow users to easily
> express their use cases that would otherwise require much more SQL. Common
> patterns for updating partitions are to read, union, and overwrite or read,
> diff, and append. Using commands like MERGE, these operations are easier to
> express and can be more efficient to run.
> Hive supports [MERGE|https://blog.cloudera.com/update-hive-tables-easy-way/]
> and Spark should implement similar support.
> SPIP:
> https://docs.google.com/document/d/12Ywmc47j3l2WF4anG5vL4qlrhT2OKigb7_EbIKhxg60
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
