That would be of great help Allen. Much appreciated. On Wed, Aug 19, 2020 at 9:30 AM Allen Underwood <[email protected]> wrote:
> Thanks Sivabalan, > > That's definitely the issue I had to resolve when introducing Joda time. > I'll have a look back at my code to see how I got around it. If I remember > correctly, in the original code, the DateTime object was being saved as an > instance variable on the object and I had to do things differently in the > Joda time approach - but it turned out not to be a big deal because the > instance variable wasn't needed. > > I'll see if I can get a PR in to help fix this. > > The worst part is no unit tests catch this - it's a runtime issue I assume > due to how Spark serializes state. > > Thanks, > > Allen > > On Wed, Aug 19, 2020 at 9:23 AM Sivabalan <[email protected]> wrote: > >> I am not sure if all findings so far have been documented here. but this >> is >> >> >> the ticket AFSIK: https://issues.apache.org/jira/browse/HUDI-1177 >> >> >> >> >> >> On Wed, Aug 19, 2020 at 9:15 AM Allen Underwood >> >> >> <[email protected]> wrote: >> >> >> >> >> >> > Just out of curiosity - what's the blocker - you have an issue? I had >> >> >> > originally done the code to make that work. >> >> >> > >> >> >> > On Wed, Aug 19, 2020 at 12:51 AM Vinoth Chandar <[email protected]> >> wrote: >> >> >> > >> >> >> >> We still have 1 blocker issue from TimestampKeyGenerator issue with >> joda >> >> >> >> DateTimeFormatter. Sudha (RM) and Pratyaksh are going to look into >> this. >> >> >> >> >> >> >> >> In the meantime, here's the progress, plans around testing so far. If >> >> >> >> folks >> >> >> >> in the community can help test the release branch in the next couple of >> >> >> >> days, it would be of great help! >> >> >> >> >> >> >> >> >> >> >> >> - Testing plan here: Stability tested by balaji running spark >> >> >> >> streaming, >> >> >> >> Correctness (Nishith testing on release branch, Udit to help with >> EMR >> >> >> >> setup) >> >> >> >> - Testing plan continued: Spark datasource MOR, .. Marker based >> >> >> >> rollback, upgrade-downgrade (vinoth is on this) >> >> >> >> - Testing plan : bootstrap (balaji tested for hive), udit has it >> >> >> >> working >> >> >> >> for Presto in EMR. >> >> >> >> - Testing plan: bulk_insert v2 (vinoth tested for correctness, file >> >> >> >> sizes. ), performance (tested on a cluster, microbenchmarks) >> >> >> >> >> >> >> >> >> >> >> >> There will also be some blogs/docs to explain new stuff. >> >> >> >> >> >> >> >> - Blogs to explain new stuff: sort modes (vinoth), simple index >> >> >> >> (vinoth), bootstrap (balaji), multi delta-streamer (pratyaksh), >> Spark >> >> >> >> Streaming/DeltaStreamer continuous mode blog (balaji) >> >> >> >> - Docs: All missing docs will be added by Sudha. >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> On Tue, Aug 18, 2020 at 1:58 AM Bhavani Sudha <[email protected] >> > >> >> >> >> wrote: >> >> >> >> >> >> >> >> > Quick update on the RC. >> >> >> >> > >> >> >> >> > Found a build issue when building scala 2.12 and sent a PR for that - >> >> >> >> > https://github.com/apache/hudi/pull/1976 . Working on resolving >> this in >> >> >> >> > the >> >> >> >> > release branch and updating RC. Will update soon. >> >> >> >> > >> >> >> >> > Thanks, >> >> >> >> > Sudha >> >> >> >> > >> >> >> >> > On Fri, Aug 14, 2020 at 5:56 PM Vinoth Chandar <[email protected]> >> >> >> >> wrote: >> >> >> >> > >> >> >> >> > > Thanks Sudha! This is means master is now open for regular PRs. >> Thanks >> >> >> >> > for >> >> >> >> > > your patience, everyone. >> >> >> >> > > >> >> >> >> > > On Fri, Aug 14, 2020 at 3:51 PM Bhavani Sudha < >> >> >> >> [email protected]> >> >> >> >> > > wrote: >> >> >> >> > > >> >> >> >> > > > Hello all, >> >> >> >> > > > >> >> >> >> > > > We have cut the release branch - >> >> >> >> > > > https://github.com/apache/hudi/tree/release-0.6.0 . Since it is >> >> >> >> > already >> >> >> >> > > > Friday, we will be sending the release candidate early next week >> >> >> >> (after >> >> >> >> > > > some testing). >> >> >> >> > > > >> >> >> >> > > > Happy Friday! >> >> >> >> > > > >> >> >> >> > > > Thanks, >> >> >> >> > > > Sudha >> >> >> >> > > > >> >> >> >> > > > On Wed, Aug 12, 2020 at 3:56 PM [email protected] < >> >> >> >> [email protected] >> >> >> >> > > >> >> >> >> > > > wrote: >> >> >> >> > > > >> >> >> >> > > > > >> >> >> >> > > > > Hi Folks, >> >> >> >> > > > > We are continuing to work on CI stabilization and will cut the >> >> >> >> > release >> >> >> >> > > > > once we stabilize the builds hopefully tonight/tomorrow. >> >> >> >> > > > > Thanks,Balaji.V >> >> >> >> > > > > On Tuesday, August 11, 2020, 09:15:05 PM PDT, Vinoth >> Chandar < >> >> >> >> > > > > [email protected]> wrote: >> >> >> >> > > > > >> >> >> >> > > > > Hello all, >> >> >> >> > > > > >> >> >> >> > > > > Update on this. We have landed most of the blockers for the >> 0.6.0 >> >> >> >> > > release >> >> >> >> > > > > and I am currently working on the last major blocker, >> HUDI-1013. >> >> >> >> > > > > We are working through some unexpected CI flakiness. We hope to >> >> >> >> > > stabilize >> >> >> >> > > > > master, cut the RC, and then open up master for regular PR >> merges. >> >> >> >> > > > > ETA for this is tomorrow night PST (Aug 12, PST). >> >> >> >> > > > > >> >> >> >> > > > > We will keep this thread posted! >> >> >> >> > > > > >> >> >> >> > > > > Thanks >> >> >> >> > > > > Vinoth >> >> >> >> > > > > >> >> >> >> > > > > On Tue, Aug 4, 2020 at 9:47 PM Vinoth Chandar < >> [email protected]> >> >> >> >> > > wrote: >> >> >> >> > > > > >> >> >> >> > > > > > Small correction: >> >> >> >> > > > > > >> >> >> >> > > > > > >> Vinoth working on code review, tests for PR 1876, >> >> >> >> > > > > > This is landed! >> >> >> >> > > > > > >> >> >> >> > > > > > >> >> >> >> > > > > > On Tue, Aug 4, 2020 at 9:44 PM Bhavani Sudha < >> >> >> >> > > [email protected]> >> >> >> >> > > > > > wrote: >> >> >> >> > > > > > >> >> >> >> > > > > >> Hello all, >> >> >> >> > > > > >> >> >> >> >> > > > > >> We are targeting the end of this week to cut RC. Here is an >> >> >> >> update >> >> >> >> > > of >> >> >> >> > > > > >> where >> >> >> >> > > > > >> we are at release blockers. >> >> >> >> > > > > >> >> >> >> >> > > > > >> 0.6.0 Release blocker status (board >> >> >> >> > > > > >> < >> >> >> >> > > > > >> >> >> >> >> > > > > >> >> >> >> > > > >> >> >> >> > > >> >> >> >> > >> >> >> >> >> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=397&projectKey=HUDI&view=detail&selectedIssue=HUDI-69 >> >> >> >> > > > > >> >) >> >> >> >> > > > > >> , >> >> >> >> > > > > >> >> >> >> >> > > > > >> - Spark Datasource/MOR >> >> >> >> > https://github.com/apache/hudi/pull/1848 >> >> >> >> > > > > needs >> >> >> >> > > > > >> to >> >> >> >> > > > > >> be tested by gary/balaji (About to land) >> >> >> >> > > > > >> - Hive Sync restructuring (Review done, about to land) >> >> >> >> > > > > >> - Bootstrap >> >> >> >> > > > > >> - Vinoth working on code review, tests for PR 1876, >> >> >> >> > > > > >> - then udit will rework PR 1702 (In Code review) >> >> >> >> > > > > >> - then we will review, land PR 1870, 1869 >> >> >> >> > > > > >> - Bulk insert V2 PR 1834, lower risk, independent PR, >> well >> >> >> >> > tested >> >> >> >> > > > > >> already >> >> >> >> > > > > >> - Dependent PR 1149 to be landed, >> >> >> >> > > > > >> - and modes to be respected in V2 impl as well (At >> risk) >> >> >> >> > > > > >> - Upgrade Downgrade Hooks, PR 1858 : (In Code review) >> >> >> >> > > > > >> - HUDI-1054- Marker list perf improvement, Udit has a PR >> out >> >> >> >> > > > > >> - HUDI-115 : Overwrite with... ordering issue, Sudha has >> a >> >> >> >> PR >> >> >> >> > > > nearing >> >> >> >> > > > > >> landing >> >> >> >> > > > > >> - HUDI-1098 : Marker file issue with non-existent files. >> (In >> >> >> >> > Code >> >> >> >> > > > > >> review) >> >> >> >> > > > > >> - Spark Streaming + Async Compaction , test complete, >> code >> >> >> >> > review >> >> >> >> > > > > >> comments and land PR 1752 (About to land) >> >> >> >> > > > > >> - Spark DataSource/Hive MOR Incremental Query HUDI-920 >> (At >> >> >> >> > risk) >> >> >> >> > > > > >> - Flink/Multi Engine refactor, will need a large rebase >> and >> >> >> >> > > rework, >> >> >> >> > > > > >> review, land (At risk for 0.6.0) >> >> >> >> > > > > >> - BloomIndex V2 - Global index implementation. (At risk) >> >> >> >> > > > > >> - HUDI-845 : Parallel writing i.e allow multiple writers >> >> >> >> > (Pushed >> >> >> >> > > > out >> >> >> >> > > > > of >> >> >> >> > > > > >> 0.6.0) >> >> >> >> > > > > >> - HUDI-860 : Small File Handling without memory caching >> >> >> >> (Pushed >> >> >> >> > > out >> >> >> >> > > > > of >> >> >> >> > > > > >> 0.6.0) >> >> >> >> > > > > >> >> >> >> >> > > > > >> >> >> >> >> > > > > >> Thanks, >> >> >> >> > > > > >> Sudha >> >> >> >> > > > > >> >> >> >> >> > > > > >> On Mon, Aug 3, 2020 at 3:41 PM Vinoth Chandar < >> >> >> >> [email protected]> >> >> >> >> > > > > wrote: >> >> >> >> > > > > >> >> >> >> >> > > > > >> > +1 (we need to formalize this well) >> >> >> >> > > > > >> > But having just blockers land first, would help not just >> with >> >> >> >> > > > > rebasing, >> >> >> >> > > > > >> but >> >> >> >> > > > > >> > also wind down towards cutting an RC by end of week. >> >> >> >> > > > > >> > >> >> >> >> > > > > >> > >> >> >> >> > > > > >> > On Mon, Aug 3, 2020 at 2:53 PM Bhavani Sudha < >> >> >> >> > > > [email protected] >> >> >> >> > > > > > >> >> >> >> > > > > >> > wrote: >> >> >> >> > > > > >> > >> >> >> >> > > > > >> > > Hello all, >> >> >> >> > > > > >> > > >> >> >> >> > > > > >> > > As we are all hustling towards getting the blockers in, >> I >> >> >> >> > wanted >> >> >> >> > > > to >> >> >> >> > > > > >> > propose >> >> >> >> > > > > >> > > a code/merge freeze until we cut a release for 0.6.0 >> and >> >> >> >> > > restrict >> >> >> >> > > > > it >> >> >> >> > > > > >> to >> >> >> >> > > > > >> > > only merging blockers identified for this release. It >> would >> >> >> >> > > reduce >> >> >> >> > > > > >> > rebasing >> >> >> >> > > > > >> > > time for blockers in progress. If we feel some issue is >> a >> >> >> >> > > serious >> >> >> >> > > > > >> blocker >> >> >> >> > > > > >> > > we can discuss it here and bump it's priority. >> >> >> >> > > > > >> > > >> >> >> >> > > > > >> > > Please share your thoughts or concerns. >> >> >> >> > > > > >> > > >> >> >> >> > > > > >> > > Thanks, >> >> >> >> > > > > >> > > Sudha >> >> >> >> > > > > >> > > >> >> >> >> > > > > >> > > >> >> >> >> > > > > >> > > On Mon, Aug 3, 2020 at 8:19 AM Vinoth Chandar < >> >> >> >> > > [email protected]> >> >> >> >> > > > > >> wrote: >> >> >> >> > > > > >> > > >> >> >> >> > > > > >> > > > Given enough time has passed, Sudha can be our RM for >> >> >> >> 0.6.0. >> >> >> >> > > > > >> > > > >> >> >> >> > > > > >> > > > On the release blocker progress, we landed few >> blockers >> >> >> >> over >> >> >> >> > > the >> >> >> >> > > > > >> > weekend, >> >> >> >> > > > > >> > > > with some almost ready for landing >> >> >> >> > > > > >> > > > >> >> >> >> > > > > >> > > > Will send out a status update again tomorrow night >> PST! >> >> >> >> > > > > >> > > > >> >> >> >> > > > > >> > > > On Mon, Aug 3, 2020 at 8:17 AM Vinoth Chandar < >> >> >> >> > > > [email protected]> >> >> >> >> > > > > >> > wrote: >> >> >> >> > > > > >> > > > >> >> >> >> > > > > >> > > > > Hi anton. >> >> >> >> > > > > >> > > > > >> >> >> >> > > > > >> > > > > We were hoping to cut a release by last weekend. New >> >> >> >> > target >> >> >> >> > > is >> >> >> >> > > > > >> this >> >> >> >> > > > > >> > > > > weekend! >> >> >> >> > > > > >> > > > > (tbh we were thrown off a bit due to COVID in Q2, >> >> >> >> given a >> >> >> >> > > lot >> >> >> >> > > > of >> >> >> >> > > > > >> > > > > PMC/Committers had additional kid care duties. Now >> we >> >> >> >> are >> >> >> >> > > back >> >> >> >> > > > > to >> >> >> >> > > > > >> > > normal >> >> >> >> > > > > >> > > > > cadence) >> >> >> >> > > > > >> > > > > >> >> >> >> > > > > >> > > > > Going forward, I plan to start a discussion around >> >> >> >> > planning, >> >> >> >> > > > > >> > > prioritizing >> >> >> >> > > > > >> > > > > and other release processes after 0.6.0. Would be >> >> >> >> great to >> >> >> >> > > > have >> >> >> >> > > > > >> the >> >> >> >> > > > > >> > > > > community weigh in even more in these things >> upfront. >> >> >> >> > > > > >> > > > > >> >> >> >> > > > > >> > > > > Thanks >> >> >> >> > > > > >> > > > > Vinoth >> >> >> >> > > > > >> > > > > >> >> >> >> > > > > >> > > > > On Fri, Jul 31, 2020 at 6:49 PM Anton Zuyeu < >> >> >> >> > > > > >> [email protected]> >> >> >> >> > > > > >> > > > wrote: >> >> >> >> > > > > >> > > > > >> >> >> >> > > > > >> > > > >> Hi All, >> >> >> >> > > > > >> > > > >> >> >> >> >> > > > > >> > > > >> I apologize for possibly dumb question but when was >> >> >> >> 0.6.0 >> >> >> >> > > > > >> planned to >> >> >> >> > > > > >> > > be >> >> >> >> > > > > >> > > > >> released? Can't find any dates on Hudi related >> pages. >> >> >> >> > > > > >> > > > >> >> >> >> >> > > > > >> > > > >> On Thu, Jul 30, 2020 at 10:36 AM Vinoth Chandar < >> >> >> >> > > > > >> [email protected]> >> >> >> >> > > > > >> > > > >> wrote: >> >> >> >> > > > > >> > > > >> >> >> >> >> > > > > >> > > > >> > Is anyone able to help with the at risk items? :) >> >> >> >> > > > > >> > > > >> > >> >> >> >> > > > > >> > > > >> > On Thu, Jul 30, 2020 at 7:07 AM leesf < >> >> >> >> > > [email protected] >> >> >> >> > > > > >> >> >> >> > > > > >> > wrote: >> >> >> >> > > > > >> > > > >> > >> >> >> >> > > > > >> > > > >> > > @Vinoth Chandar <[email protected]> Thanks for >> >> >> >> the >> >> >> >> > > > > reminder, >> >> >> >> > > > > >> > > marked >> >> >> >> > > > > >> > > > >> to >> >> >> >> > > > > >> > > > >> > > blocker, and next week would be ok to me. >> >> >> >> > > > > >> > > > >> > > >> >> >> >> > > > > >> > > > >> > > Vinoth Chandar <[email protected]> >> 于2020年7月30日周四 >> >> >> >> > > > > 上午11:35写道： >> >> >> >> > > > > >> > > > >> > > >> >> >> >> > > > > >> > > > >> > > > @leesf <[email protected]> can we please >> >> >> >> mark >> >> >> >> > the >> >> >> >> > > > > >> relevant >> >> >> >> > > > > >> > > > >> > ticket(s) >> >> >> >> > > > > >> > > > >> > > > with blocker priority, so it's easier to >> track? >> >> >> >> > > > > >> > > > >> > > > >> >> >> >> > > > > >> > > > >> > > > Looks like we are nearing a choice for RM. >> >> >> >> > > > > >> > > > >> > > > Any more thoughts on timelines? Looks like >> >> >> >> everyone >> >> >> >> > > so >> >> >> >> > > > > far >> >> >> >> > > > > >> is >> >> >> >> > > > > >> > > > >> leaning >> >> >> >> > > > > >> > > > >> > > > towards completeness of the release over >> doing >> >> >> >> it >> >> >> >> > > > sooner? >> >> >> >> > > > > >> > > > >> > > > >> >> >> >> > > > > >> > > > >> > > > On Wed, Jul 29, 2020 at 6:36 PM vino yang < >> >> >> >> > > > > >> > > [email protected]> >> >> >> >> > > > > >> > > > >> > wrote: >> >> >> >> > > > > >> > > > >> > > > >> >> >> >> > > > > >> > > > >> > > > > +1 on Sudha being RM for the release. And >> >> >> >> looking >> >> >> >> > > > > >> forward to >> >> >> >> > > > > >> > > > >> 0.6.0. >> >> >> >> > > > > >> > > > >> > > > > >> >> >> >> > > > > >> > > > >> > > > > Best, >> >> >> >> > > > > >> > > > >> > > > > Vino >> >> >> >> > > > > >> > > > >> > > > > >> >> >> >> > > > > >> > > > >> > > > > leesf <[email protected]> 于2020年7月30日周四 >> >> >> >> > > 上午9:15写道： >> >> >> >> > > > > >> > > > >> > > > > >> >> >> >> > > > > >> > > > >> > > > > > +1 on Sudha on being RM, and PR#1810 >> >> >> >> > > > > >> > > > >> > > > > > https://github.com/apache/hudi/pull/1810 >> >> >> >> > > (abstract >> >> >> >> > > > > >> hive >> >> >> >> > > > > >> > > sync >> >> >> >> > > > > >> > > > >> > module) >> >> >> >> > > > > >> > > > >> > > > > would >> >> >> >> > > > > >> > > > >> > > > > > also goes to this release. >> >> >> >> > > > > >> > > > >> > > > > > >> >> >> >> > > > > >> > > > >> > > > > > Sivabalan <[email protected]> >> >> >> >> 于2020年7月30日周四 >> >> >> >> > > > > 上午2:18写道： >> >> >> >> > > > > >> > > > >> > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > +1 on Sudha being RM for the release. >> >> >> >> Makes >> >> >> >> > > sense >> >> >> >> > > > > to >> >> >> >> > > > > >> > push >> >> >> >> > > > > >> > > > the >> >> >> >> > > > > >> > > > >> > > release >> >> >> >> > > > > >> > > > >> > > > > by >> >> >> >> > > > > >> > > > >> > > > > > a >> >> >> >> > > > > >> > > > >> > > > > > > week. >> >> >> >> > > > > >> > > > >> > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > On Wed, Jul 29, 2020 at 1:35 AM >> >> >> >> > > > [email protected] >> >> >> >> > > > > < >> >> >> >> > > > > >> > > > >> > > > [email protected] >> >> >> >> > > > > >> > > > >> > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > wrote: >> >> >> >> > > > > >> > > > >> > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > > +1 on Sudha on being RM for this >> >> >> >> release. >> >> >> >> > > Also >> >> >> >> > > > > >> agree >> >> >> >> > > > > >> > on >> >> >> >> > > > > >> > > > >> > pushing >> >> >> >> > > > > >> > > > >> > > > the >> >> >> >> > > > > >> > > > >> > > > > > > > release date by a week. >> >> >> >> > > > > >> > > > >> > > > > > > > Balaji.V >> >> >> >> > > > > >> > > > >> > > > > > > > On Tuesday, July 28, 2020, >> 10:08:41 >> >> >> >> PM >> >> >> >> > > PDT, >> >> >> >> > > > > >> > Bhavani >> >> >> >> > > > > >> > > > >> Sudha < >> >> >> >> > > > > >> > > > >> > > > > > > > [email protected]> wrote: >> >> >> >> > > > > >> > > > >> > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > > Thanks Vinoth for the update. I can >> >> >> >> > > volunteer >> >> >> >> > > > to >> >> >> >> > > > > >> RM >> >> >> >> > > > > >> > > this >> >> >> >> > > > > >> > > > >> > > release. >> >> >> >> > > > > >> > > > >> > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > > Understand 0.6.0 release is delayed >> than >> >> >> >> > what >> >> >> >> > > > we >> >> >> >> > > > > >> > > > originally >> >> >> >> > > > > >> > > > >> > > > > discussed. >> >> >> >> > > > > >> > > > >> > > > > > Q2 >> >> >> >> > > > > >> > > > >> > > > > > > > has been really hard with COVID and >> >> >> >> > > everything >> >> >> >> > > > > >> going >> >> >> >> > > > > >> > on. >> >> >> >> > > > > >> > > > >> Given >> >> >> >> > > > > >> > > > >> > > that >> >> >> >> > > > > >> > > > >> > > > > we >> >> >> >> > > > > >> > > > >> > > > > > > are >> >> >> >> > > > > >> > > > >> > > > > > > > at this point, I feel by delaying >> the RC >> >> >> >> > by a >> >> >> >> > > > > week >> >> >> >> > > > > >> or >> >> >> >> > > > > >> > so >> >> >> >> > > > > >> > > > >> more >> >> >> >> > > > > >> > > > >> > if >> >> >> >> > > > > >> > > > >> > > we >> >> >> >> > > > > >> > > > >> > > > > can >> >> >> >> > > > > >> > > > >> > > > > > > get >> >> >> >> > > > > >> > > > >> > > > > > > > some of the 'At risk' items in, I >> would >> >> >> >> > vote >> >> >> >> > > > for >> >> >> >> > > > > >> that. >> >> >> >> > > > > >> > > > That >> >> >> >> > > > > >> > > > >> is >> >> >> >> > > > > >> > > > >> > > just >> >> >> >> > > > > >> > > > >> > > > > my >> >> >> >> > > > > >> > > > >> > > > > > > > personal opinion. I ll let others >> chime >> >> >> >> in. >> >> >> >> > > > > >> > > > >> > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > > Thanks, >> >> >> >> > > > > >> > > > >> > > > > > > > Sudha >> >> >> >> > > > > >> > > > >> > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > > On Tue, Jul 28, 2020 at 9:48 PM >> Vinoth >> >> >> >> > > Chandar >> >> >> >> > > > < >> >> >> >> > > > > >> > > > >> > > [email protected]> >> >> >> >> > > > > >> > > > >> > > > > > > wrote: >> >> >> >> > > > > >> > > > >> > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > > > Hello all, >> >> >> >> > > > > >> > > > >> > > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > > > Just wanted to kickstart a thread >> to >> >> >> >> firm >> >> >> >> > > up >> >> >> >> > > > > the >> >> >> >> > > > > >> RC >> >> >> >> > > > > >> > > cut >> >> >> >> > > > > >> > > > >> date >> >> >> >> > > > > >> > > > >> > > for >> >> >> >> > > > > >> > > > >> > > > > > 0.6.0 >> >> >> >> > > > > >> > > > >> > > > > > > > and >> >> >> >> > > > > >> > > > >> > > > > > > > > pick a RM. (any volunteers?, if >> not I >> >> >> >> > self >> >> >> >> > > > > >> nominate >> >> >> >> > > > > >> > > > >> myself) >> >> >> >> > > > > >> > > > >> > > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > > > Here's an update on where we are at >> >> >> >> with >> >> >> >> > > the >> >> >> >> > > > > >> > remaining >> >> >> >> > > > > >> > > > >> > release >> >> >> >> > > > > >> > > > >> > > > > > > blockers. >> >> >> >> > > > > >> > > > >> > > > > > > > I >> >> >> >> > > > > >> > > > >> > > > > > > > > have marked items as "At risk" >> >> >> >> assuming >> >> >> >> > we >> >> >> >> > > > cut >> >> >> >> > > > > RC >> >> >> >> > > > > >> > > > sometime >> >> >> >> > > > > >> > > > >> > next >> >> >> >> > > > > >> > > > >> > > > > week. >> >> >> >> > > > > >> > > > >> > > > > > > > > Please chime in with your thoughts. >> >> >> >> > > Ideally, >> >> >> >> > > > we >> >> >> >> > > > > >> > don't >> >> >> >> > > > > >> > > > take >> >> >> >> > > > > >> > > > >> > any >> >> >> >> > > > > >> > > > >> > > > more >> >> >> >> > > > > >> > > > >> > > > > > > > > blockers. If we also want to knock >> off >> >> >> >> > the >> >> >> >> > > at >> >> >> >> > > > > >> risk >> >> >> >> > > > > >> > > > items, >> >> >> >> > > > > >> > > > >> > then >> >> >> >> > > > > >> > > > >> > > we >> >> >> >> > > > > >> > > > >> > > > > > would >> >> >> >> > > > > >> > > > >> > > > > > > > > at-least push dates by another week >> >> >> >> (my >> >> >> >> > > > guess). >> >> >> >> > > > > >> > > > >> > > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > > > 0.6.0 Release blocker status (board >> >> >> >> > > > > >> > > > >> > > > > > > > > < >> >> >> >> > > > > >> > > > >> > > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > >> >> >> >> > > > > >> > > > >> > > > > >> >> >> >> > > > > >> > > > >> > > > >> >> >> >> > > > > >> > > > >> > > >> >> >> >> > > > > >> > > > >> > >> >> >> >> > > > > >> > > > >> >> >> >> >> > > > > >> > > > >> >> >> >> > > > > >> > > >> >> >> >> > > > > >> > >> >> >> >> > > > > >> >> >> >> >> > > > > >> >> >> >> > > > >> >> >> >> > > >> >> >> >> > >> >> >> >> >> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=397&projectKey=HUDI&view=detail&selectedIssue=HUDI-69 >> >> >> >> > > > > >> > > > >> > > > > > > > > >) >> >> >> >> > > > > >> > > > >> > > > > > > > > , >> >> >> >> > > > > >> > > > >> > > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > > > - Spark Datasource/MOR >> >> >> >> > > > > >> > > > >> > > > https://github.com/apache/hudi/pull/1848 >> >> >> >> > > > > >> > > > >> > > > > > > needs >> >> >> >> > > > > >> > > > >> > > > > > > > > to >> >> >> >> > > > > >> > > > >> > > > > > > > > be tested by gary/balaji >> >> >> >> > > > > >> > > > >> > > > > > > > > - Bootstrap >> >> >> >> > > > > >> > > > >> > > > > > > > > - Vinoth working on code >> review, >> >> >> >> > tests >> >> >> >> > > > for >> >> >> >> > > > > >> PR >> >> >> >> > > > > >> > > 1876, >> >> >> >> > > > > >> > > > >> > > > > > > > > - then udit will rework PR >> 1702 >> >> >> >> > > > > >> > > > >> > > > > > > > > - then we will review, land PR >> >> >> >> 1870, >> >> >> >> > > > 1869 >> >> >> >> > > > > >> > > > >> > > > > > > > > - Also need to fix HUDI-999, >> >> >> >> > HUDI-1021 >> >> >> >> > > > > >> > > > >> > > > > > > > > - Bulk insert V2 PR 1834, lower >> >> >> >> risk, >> >> >> >> > > > > >> independent >> >> >> >> > > > > >> > > PR, >> >> >> >> > > > > >> > > > >> well >> >> >> >> > > > > >> > > > >> > > > > tested >> >> >> >> > > > > >> > > > >> > > > > > > > > already >> >> >> >> > > > > >> > > > >> > > > > > > > > - Dependent PR 1149 to be >> landed, >> >> >> >> > > > > >> > > > >> > > > > > > > > - and modes to be respected >> in V2 >> >> >> >> > impl >> >> >> >> > > > as >> >> >> >> > > > > >> well >> >> >> >> > > > > >> > > (At >> >> >> >> > > > > >> > > > >> risk) >> >> >> >> > > > > >> > > > >> > > > > > > > > - Upgrade Downgrade Hooks, PR >> 1858 >> >> >> >> : >> >> >> >> > > Siva >> >> >> >> > > > > has >> >> >> >> > > > > >> a >> >> >> >> > > > > >> > PR >> >> >> >> > > > > >> > > > out, >> >> >> >> > > > > >> > > > >> > code >> >> >> >> > > > > >> > > > >> > > > > > > > completing >> >> >> >> > > > > >> > > > >> > > > > > > > > this week >> >> >> >> > > > > >> > > > >> > > > > > > > > - HUDI-1054- Marker list perf >> >> >> >> > > improvement, >> >> >> >> > > > > >> Udit >> >> >> >> > > > > >> > > has a >> >> >> >> > > > > >> > > > >> PR >> >> >> >> > > > > >> > > > >> > out >> >> >> >> > > > > >> > > > >> > > > > > > > > - HUDI-115 : Overwrite with... >> >> >> >> > ordering >> >> >> >> > > > > issue, >> >> >> >> > > > > >> > > Sudha >> >> >> >> > > > > >> > > > >> has a >> >> >> >> > > > > >> > > > >> > > PR >> >> >> >> > > > > >> > > > >> > > > > > > nearing >> >> >> >> > > > > >> > > > >> > > > > > > > > landing >> >> >> >> > > > > >> > > > >> > > > > > > > > - HUDI-1098 : Marker file issue >> >> >> >> with >> >> >> >> > > > > >> non-existent >> >> >> >> > > > > >> > > > >> files. >> >> >> >> > > > > >> > > > >> > > Siva >> >> >> >> > > > > >> > > > >> > > > to >> >> >> >> > > > > >> > > > >> > > > > > > begin >> >> >> >> > > > > >> > > > >> > > > > > > > > impl >> >> >> >> > > > > >> > > > >> > > > > > > > > - Spark Streaming + Async >> >> >> >> Compaction , >> >> >> >> > > > test >> >> >> >> > > > > >> > > complete, >> >> >> >> > > > > >> > > > >> code >> >> >> >> > > > > >> > > > >> > > > > review >> >> >> >> > > > > >> > > > >> > > > > > > > > comments and land PR 1752 >> >> >> >> > > > > >> > > > >> > > > > > > > > - Spark DataSource/Hive MOR >> >> >> >> > Incremental >> >> >> >> > > > > Query >> >> >> >> > > > > >> > > > HUDI-920 >> >> >> >> > > > > >> > > > >> (At >> >> >> >> > > > > >> > > > >> > > > risk) >> >> >> >> > > > > >> > > > >> > > > > > > > > - Flink/Multi Engine refactor, >> will >> >> >> >> > > need a >> >> >> >> > > > > >> large >> >> >> >> > > > > >> > > > rebase >> >> >> >> > > > > >> > > > >> > and >> >> >> >> > > > > >> > > > >> > > > > > rework, >> >> >> >> > > > > >> > > > >> > > > > > > > > review, land (At risk for 0.6.0, >> >> >> >> high >> >> >> >> > > > scope, >> >> >> >> > > > > >> may >> >> >> >> > > > > >> > > not >> >> >> >> > > > > >> > > > >> have >> >> >> >> > > > > >> > > > >> > > > enough >> >> >> >> > > > > >> > > > >> > > > > > > time) >> >> >> >> > > > > >> > > > >> > > > > > > > > - BloomIndex V2 - Global index >> >> >> >> > > > > implementation. >> >> >> >> > > > > >> > (At >> >> >> >> > > > > >> > > > >> risk) >> >> >> >> > > > > >> > > > >> > > > > > > > > - HUDI-845 : Parallel writing >> i.e >> >> >> >> > allow >> >> >> >> > > > > >> multiple >> >> >> >> > > > > >> > > > >> writers >> >> >> >> > > > > >> > > > >> > (At >> >> >> >> > > > > >> > > > >> > > > > risk) >> >> >> >> > > > > >> > > > >> > > > > > > > > - HUDI-860 : Small File Handling >> >> >> >> > without >> >> >> >> > > > > >> memory >> >> >> >> > > > > >> > > > caching >> >> >> >> > > > > >> > > > >> > (At >> >> >> >> > > > > >> > > > >> > > > > risk) >> >> >> >> > > > > >> > > > >> > > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > > > Thanks >> >> >> >> > > > > >> > > > >> > > > > > > > > Vinoth >> >> >> >> > > > > >> > > > >> > > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > > -- >> >> >> >> > > > > >> > > > >> > > > > > > Regards, >> >> >> >> > > > > >> > > > >> > > > > > > -Sivabalan >> >> >> >> > > > > >> > > > >> > > > > > > >> >> >> >> > > > > >> > > > >> > > > > > >> >> >> >> > > > > >> > > > >> > > > > >> >> >> >> > > > > >> > > > >> > > > >> >> >> >> > > > > >> > > > >> > > >> >> >> >> > > > > >> > > > >> > >> >> >> >> > > > > >> > > > >> >> >> >> >> > > > > >> > > > > >> >> >> >> > > > > >> > > > >> >> >> >> > > > > >> > > >> >> >> >> > > > > >> > >> >> >> >> > > > > >> >> >> >> >> > > > > > >> >> >> >> > > > >> >> >> >> > > >> >> >> >> > >> >> >> >> >> >> >> > >> >> >> > >> >> >> > -- >> > >> >> > *Allen Underwood* >> >> >> > Principal Software Engineer >> >> >> > Broadcom | Symantec Enterprise Division >> >> >> > *Mobile*: 404.808.5926 >> >> >> > >> >> >> >> >> >> >> >> >> -- >> >> >> Regards, >> >> >> -Sivabalan >> >> >> > > -- > *Allen Underwood* > Principal Software Engineer > Broadcom | Symantec Enterprise Division > *Mobile*: 404.808.5926 > > > -- Regards, -Sivabalan
