Hi Ryan, Thanks, I absolutely agree with you that we should keep support for Hive2 as well. I've created a draft PR, which highlights the changes: https://github.com/apache/iceberg/pull/1455 We can continue the discussion there.
Thanks, Marton On Fri, 11 Sep 2020 at 22:20, Ryan Blue <[email protected]> wrote: > Hi Marton, could you share a link to your branch with the changes? It > would be great to see what needs to be done. A quick summary would help as > well. > > Knowing what changes between Hive 2 and 3 in our iceberg-hive-metastore > project is important because we would ideally use whatever is available at > runtime. I'm all for upgrading to enable people on the newer versions, but > I think we want to make sure we maintain support for people that haven't > migrated yet as well. > > rb > > On Fri, Sep 11, 2020 at 3:28 AM Marton Bod <[email protected]> wrote: > >> Hi Team, >> >> We would like to start a discussion on upgrading Iceberg components to >> use Hive 3 and Hadoop 3. We have a fork where we have bumped up the hive >> and hadoop dependency versions and made the necessary changes to get all >> tests to pass. >> >> As some components cannot (e.g. spark2) or might not want to upgrade yet, >> our solution was to create a separate iceberg-hive2-metastore module, which >> would keep on using Hive2 and Hadoop2. This would give an option for each >> component to do the upgrade at their own pace or not at all. >> >> At this point, our primary goal is to upgrade iceberg-mr to Hive3. >> Upgrading iceberg-flink posed no major issues either, but of course it's up >> to the Flink iceberg community to make this call. As for spark2/spark3, we >> have left them for now to use Hive2. >> >> Any thoughts from the community on this upgrade? >> >> Thank you, >> >> Marton >> > > > -- > Ryan Blue > Software Engineer > Netflix >
