Hi Pratyaksh, Actually we voted on the mailing list here, to move over to 2.4.. :) With 3.0 on the horizon, it may make sense to actually upgrade? Can you file an issue with how exactly 0.5.1 is failing with 2.3? are you hitting issues with parquet-avro mismatches? I am wondering if we can do a custom build overriding few versions of parquet/avro... (spark-avro 2.4.4 should interplay with spark 2.3?)
Thanks VInoth On Sun, Feb 23, 2020 at 2:02 PM Pratyaksh Sharma <[email protected]> wrote: > Hi, > > As discussed in last to last week's weekly sync, I want to put forward this > point on our mailing list also. Since with 0.5.1 release, we have upgraded > spark to 2.4 in our master branch, we are facing difficulties after > rebasing our codebase with master. At our organisation we are using spark > 2.3.2 cluster in production and spark 2.3 to 2.4 upgrade has breaking > changes. > > So I wanted to know if any workaround is possible to bring back support for > spark 2.3? I am sure there are other guys/organisations using Hudi who are > not yet on spark 2.4. > > We really do not want to miss out on any of the cool features being > developed by the community. >
