Thanks Vinoth, also we would consider support full schema evolution(such as drop some fields) of hudi in 0.7.0, since right now hudi follows avro schema compatibility
tanu dua <[email protected]> 于2020年9月23日周三 下午12:38写道： > Thanks Vinoth. These are really exciting items and hats off to you and team > in pushing the releases swiftly and improving the framework all the time. I > hope someday I will start contributing once I will get free from my major > deliverables and have more understanding the nitty gritty details of Hudi. > > You have mentioned Spark3.0 support in next release. We were actually > thinking of moving to Spark 3.0 but thought it’s too early with 0.6 > release. Is 0.6 not fully tested with Spark 3.0 ? > > > On Wed, 23 Sep 2020 at 8:25 AM, Vinoth Chandar <[email protected]> wrote: > > > Hello all, > > > > > > > > Pursuant to our conversation around release planning, I am happy to share > > > > the initial set of proposals for the next minor/major releases (minor > > > > release ofc can go out based on time) > > > > > > > > *Next Minor version 0.6.1 (with stuff that did not make it to 0.6.0..) * > > > > Flink/Writer common refactoring for Flink > > > > Small file handling support w/o caching > > > > Spark3 Support > > > > Remaining bootstrap items > > > > Completing bulk_insertV2 (sort mode, de-dup etc) > > > > Full list here : > > > > https://issues.apache.org/jira/projects/HUDI/versions/12348168 > > > > <https://issues.apache.org/jira/projects/HUDI/versions/12348168> > > > > > > > > *0.7.0 with major new features * > > > > RFC-15: metadata, range index (w/ spark support), bloom index (eliminate > > > > file listing, query pruning, improve bloom index perf) > > > > RFC-08: Record Index (to solve global index scalability/perf) > > > > RFC-18/19: Clustering/Insert overwrite > > > > Spark 3 based datasource rewrite (structured streaming sink/source, > > > > DELETE/MERGE) > > > > Incremental Query on logs (Hive, Spark) > > > > Parallel writing support > > > > Redesign of marker files for S3 > > > > Stretch: ORC, PrestoSQL Support > > > > > > > > Full list here : > > > > https://issues.apache.org/jira/projects/HUDI/versions/12348721 > > > > > > > > Please chime in with your thoughts. If you would like to commit to > > > > contributing a feature towards a release, please do so by marking *`Fix > > > > Version/s`* field with that release number. > > > > > > > > Thanks > > > > Vinoth > > > > >
