Hi Marton, could you share a link to your branch with the changes? It would be great to see what needs to be done. A quick summary would help as well.
Knowing what changes between Hive 2 and 3 in our iceberg-hive-metastore project is important because we would ideally use whatever is available at runtime. I'm all for upgrading to enable people on the newer versions, but I think we want to make sure we maintain support for people that haven't migrated yet as well. rb On Fri, Sep 11, 2020 at 3:28 AM Marton Bod <[email protected]> wrote: > Hi Team, > > We would like to start a discussion on upgrading Iceberg components to use > Hive 3 and Hadoop 3. We have a fork where we have bumped up the hive and > hadoop dependency versions and made the necessary changes to get all tests > to pass. > > As some components cannot (e.g. spark2) or might not want to upgrade yet, > our solution was to create a separate iceberg-hive2-metastore module, which > would keep on using Hive2 and Hadoop2. This would give an option for each > component to do the upgrade at their own pace or not at all. > > At this point, our primary goal is to upgrade iceberg-mr to Hive3. > Upgrading iceberg-flink posed no major issues either, but of course it's up > to the Flink iceberg community to make this call. As for spark2/spark3, we > have left them for now to use Hive2. > > Any thoughts from the community on this upgrade? > > Thank you, > > Marton > -- Ryan Blue Software Engineer Netflix
