Hi all,
I would love to not introduce new constructs like "timestamp", "snapshots.
Hudi already has a clear notion of commit times, that can unlock this.
Can we just use this as an opportunity to standardize the incremental
query's schema?
In fact, don't we already have change feed with our incremental query - we
need to emit delete records, and the old images of records.
Those are the only gaps I see.
+1 for an RFC. I would be happy to jam on the design!
Thanks
Vinoth
On Mon, Feb 14, 2022 at 6:25 AM Sivabalan <n.siv...@gmail.com> wrote:
> +1 for the feature. I see a lot of benefits like clustering, index
> building etc.
>
> On Sun, 13 Feb 2022 at 22:21, leesf <leesf0...@gmail.com> wrote:
> >
> > +1 for the feature.
> >
> > vino yang <yanghua1...@gmail.com> 于2022年2月12日周六 22:14写道：
> >
> > > +1 for this feature, looking forward to share more details or design
> doc.
> > >
> > > Best,
> > > Vino
> > >
> > > Xianghu Wang <wangxian...@apache.org> 于2022年2月12日周六 17:06写道：
> > >
> > > > this is definitely a great feature
> > > > +1
> > > >
> > > > On 2022/02/12 02:32:32 Forward Xu wrote:
> > > > > Hi All,
> > > > >
> > > > > I want to support change data feed for to spark sql, This feature
> can
> > > be
> > > > > achieved in two ways.
> > > > >
> > > > > 1. Call Procedure Command
> > > > > sql syntax
> > > > > CALL system.table_changes('tableName', start_timestamp,
> end_timestamp)
> > > > > example:
> > > > > CALL system.table_changes('tableName', TIMESTAMP '2021-01-23
> 04:30:45',
> > > > > TIMESTAMP '2021-02-23 6:00:00')
> > > > >
> > > > > 2. Support querying MOR(CDC) table as of a savepoint
> > > > > SELECT * FROM A.B TIMESTAMP AS OF 1643119574;
> > > > > SELECT * FROM A.B TIMESTAMP AS OF '2019-01-29 00:37:58' ;
> > > > >
> > > > > SELECT * FROM A.B TIMESTAMP AS OF '2019-01-29 00:37:58' AND
> > > '2021-02-23
> > > > > 6:00:00' ;
> > > > > SELECT * FROM A.B VERSION AS OF 'Snapshot123456789';
> > > > >
> > > > > Any feedback is welcome!
> > > > >
> > > > > Thank you.
> > > > >
> > > > > Regards,
> > > > > Forward Xu
> > > > >
> > > > > Related Links:
> > > > > [1] Call Procedure Command <
> > > > https://issues.apache.org/jira/browse/HUDI-3161>
> > > > > [2] Support querying a table as of a savepoint
> > > > > <https://issues.apache.org/jira/browse/HUDI-3221>
> > > > > [3] Change data feed
> > > > > <
> > > >
> > >
> https://docs.databricks.com/delta/delta-change-data-feed.html#language-sql
> > > > >
> > > > >
> > > >
> > >
>
>
>
> --
> Regards,
> -Sivabalan
>
