Hi all, I think mostly we are in favor for the SPIP as I've seen.
If not more comments or discussion on the SPIP doc, I will raise a vote soon. Thanks. On Tue, Nov 2, 2021 at 9:58 AM L. C. Hsieh <vii...@gmail.com> wrote: > > +1 for the idea to commit the work earlier. > > I think we will raise the voting soon. Once it is passed, we can submit the > PRs. > > What do you think? Anton. > > On Mon, Nov 1, 2021 at 7:59 AM Wenchen Fan <cloud0...@gmail.com> wrote: > > > > The general idea looks great. This is indeed a complicated API and we > > probably need more time to evaluate the API design. It's better to commit > > this work earlier so that we have more time to verify it before the 3.3 > > release. Maybe we can commit the group-based API first, then the > > delta-based one, as the delta-based API is significantly more convoluted. > > > > On Thu, Oct 28, 2021 at 12:53 AM L. C. Hsieh <vii...@apache.org> wrote: > >> > >> > >> Thanks for the initial feedback. > >> > >> I think previously the community is busy on the works related to Spark 3.2 > >> release. > >> As 3.2 release was done, I'd like to bring this up to the surface again > >> and seek for more discussion and feedback. > >> > >> Thanks. > >> > >> On 2021/06/25 15:49:49, huaxin gao <huaxin.ga...@gmail.com> wrote: > >> > I took a quick look at the PR and it looks like a great feature to have. > >> > It > >> > provides unified APIs for data sources to perform the commonly used > >> > operations easily and efficiently, so users don't have to implement > >> > customer extensions on their own. Thanks Anton for the work! > >> > > >> > On Thu, Jun 24, 2021 at 9:42 PM L. C. Hsieh <vii...@apache.org> wrote: > >> > > >> > > Thanks Anton. I'm voluntarily to be the shepherd of the SPIP. This is > >> > > also > >> > > my first time to shepherd a SPIP, so please let me know if anything I > >> > > can > >> > > improve. > >> > > > >> > > This looks great features and the rationale claimed by the proposal > >> > > makes > >> > > sense. These operations are getting more common and more important in > >> > > big > >> > > data workloads. Instead of building custom extensions by individual > >> > > data > >> > > sources, it makes more sense to support the API from Spark. > >> > > > >> > > Please provide your thoughts about the proposal and the design. > >> > > Appreciate > >> > > your feedback. Thank you! > >> > > > >> > > On 2021/06/24 23:53:32, Anton Okolnychyi <aokolnyc...@gmail.com> wrote: > >> > > > Hey everyone, > >> > > > > >> > > > I'd like to start a discussion on adding support for executing > >> > > > row-level > >> > > > operations such as DELETE, UPDATE, MERGE for v2 tables > >> > > > (SPARK-35801). The > >> > > > execution should be the same across data sources and the best way to > >> > > > do > >> > > > that is to implement it in Spark. > >> > > > > >> > > > Right now, Spark can only parse and to some extent analyze DELETE, > >> > > UPDATE, > >> > > > MERGE commands. Data sources that support row-level changes have to > >> > > > build > >> > > > custom Spark extensions to execute such statements. The goal of this > >> > > effort > >> > > > is to come up with a flexible and easy-to-use API that will work > >> > > > across > >> > > > data sources. > >> > > > > >> > > > Design doc: > >> > > > > >> > > https://docs.google.com/document/d/12Ywmc47j3l2WF4anG5vL4qlrhT2OKigb7_EbIKhxg60/ > >> > > > > >> > > > PR for handling DELETE statements: > >> > > > https://github.com/apache/spark/pull/33008 > >> > > > > >> > > > Any feedback is more than welcome. > >> > > > > >> > > > Liang-Chi was kind enough to shepherd this effort. Thanks! > >> > > > > >> > > > - Anton > >> > > > > >> > > > >> > > --------------------------------------------------------------------- > >> > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >> > > > >> > > > >> > > >> > >> --------------------------------------------------------------------- > >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >> --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
