[DISCUSS] Metadata based bloom index

2021-11-05 Thread Manoj Govindassamy
Hi Hudi Community, Hudi has several indices to help lookup records. The most commonly used one is the BloomFilter based index. This index today works by loading the bloom filter from all the data files of interested partitions. This is a time consuming operation. Better would be if can leverage th

Re: [DISCUSS] RFC for Synchronous Metadata table for File listing

2021-11-13 Thread Manoj Govindassamy
+1 for the synchronous metadata updates. Looking forward to the RFC. On Fri, Nov 12, 2021 at 4:46 PM Vinoth Chandar wrote: > +1 on this. > > On Fri, Nov 5, 2021 at 9:17 AM Sivabalan wrote: > > > RFC-15 > > < > > > https://cwiki.apache.org/confluence/display/HUDI/RFC+-+15%3A+HUDI+File+Listing+I

Re: [DISCUSS] Propose to implement a deltastreamer source for Debezium

2021-11-19 Thread Manoj Govindassamy
+1 On Fri, Nov 19, 2021 at 1:42 PM Rajesh Mahindra wrote: > Hi Community, > > We intend to implement a source for ingesting Debezium Change Data Capture > (CDC) logs into Deltastreamer/ Hudi. With this capability, we can > continuously capture row-level changes that insert, update and delete > r

Re: [DISCUSS] Hudi 0.10.0 Release

2021-11-19 Thread Manoj Govindassamy
table record payload (Owner: Manoj Govindassamy) - [HUDI-2475] Rolling Upgrade downgrade story for 0.10 & enabling metadata (Owner: Manoj Govindassamy) *Pending:* - [HUDI-2590] Validate Diff key gen w/ and w/o glob path with and w/o metadata enabled *Completed:* - [HUDI-2716] Fix InLi

Re: [DISCUSS] Hudi 0.10.0 Release

2021-11-26 Thread Manoj Govindassamy
; > > > > > >- [HUDI-2191] Bump flink version to 1.13.1 > > > > > > > >- [HUDI-2285] Metadata Table Synchronous Design > > > > > > > >- [HUDI-2316] Support Flink batch upsert > > > > > > > >- [

Re: [DISCUSS] Hudi 0.10.0 Release

2021-11-26 Thread Manoj Govindassamy
Hi Danny, All the planned tickets have landed in master and we are good for cutting 0.10 RC. Please let us know if you see any CI issues with the latest master and we can jump in to do the needful. Thanks for your patience. thanks, Manoj On Fri, Nov 26, 2021 at 8:07 PM Manoj Govindassamy

Re: [VOTE] Release 0.10.0, release candidate #1

2021-11-27 Thread Manoj Govindassamy
+1 On Sat, Nov 27, 2021 at 4:49 AM Danny Chan wrote: > Hi everyone, > > Please review and vote on the release candidate #1 for the version 0.10.0, > as follows: > > [ ] +1, Approve the release > > [ ] -1, Do not approve the release (please provide specific comments) > > The complete staging area

Re: [VOTE] Release 0.10.0, release candidate #3

2021-12-06 Thread Manoj Govindassamy
+1 (non-binding) - Release validation script - passed - Spark quick start guide using Spark 2.4.4 - passed - Hudi table write and other operations from spark data source - passed thanks, Manoj On Mon, Dec 6, 2021 at 8:23 PM sagar sumit wrote: > +1 (non-binding) > > - Builds for Spark2/3 [OK]