Thank you, Okumin, for taking on the responsibility of the next Hive release.
I'd also like to discuss an issue here. Since Hive 4.0.1, Apache Spark has been unable to query Iceberg tables in HMS. The specific issue discussion can be found at [1]. This problem occurred because we removed the `get_table` method in [2]. Spark relies on Hive 2.3.10, which depends on the `get_table` method to access Iceberg tables in HMS. The compatibility issue emerged after HIVE-26537. To resolve this, the ideal solution would be for Apache Spark to upgrade its built-in Hive dependency from version 2.3.10 to 4.0.1 or 4.1.0. I've noticed that there is an upgrade ticket [3] in the Spark community for this. However, this upgrade appears to involve significant changes and may not be resolved for quite some time. Given that Apache Spark is one of the most widely used engines with HMS and its adoption would help promote the use of Hive 4, is it possible for us to reintroduce the removed `get_table` method? I understand that restoring deprecated code is generally not a reasonable approach, so I'd like to ask if there are any more elegant solutions on the HMS side to address this issue? Additionally, I have a personal thought: when removing or deprecating public methods in Hive, we should carefully consider the dependencies of downstream components on these methods to avoid long-term upgrade difficulties for them. For example, when working on federated catalog [4] related tasks, I needed to add catalog parameters to many methods in `Hive.java`. However, since Spark code depends on `Hive.java` to access Hive, I had to introduce many new methods with catalog parameters, which may lead to significant code duplication. But there was no alternative, as modifying existing methods would cause compatibility issues for downstream components with newer Hive versions. Currently, this incompatibility risk relies on individual prior knowledge, but I hope we can develop better methods in the future to proactively identify such compatibility issues. [1]https://github.com/apache/iceberg/issues/12878 [2]https://issues.apache.org/jira/browse/HIVE-26537 [3]https://issues.apache.org/jira/browse/SPARK-52408 [4]https://issues.apache.org/jira/browse/HIVE-29177 Thanks, Butao Zhang On 2025/10/24 09:03:28 Shohei Okumiya wrote: > Hi Stamatis, > > Thanks for initiating this discussion. I'd love to have the great > release cadence! > > 1. I don't have any objections about the timeline. We might want to > release 4.1.1 in the future. I feel it should not block us from > shipping the master branch anyway. > 2. I'm willing to try being a release manager someday, but I have > frequent hospital visits nowadays. I'm probably not the right person, > given my unstable availability. > 3. Except for issues that have already been discussed in mailing > lists, I am not aware of critical problems. > > Best, > Okumin > > On Thu, Oct 23, 2025 at 11:50 AM Butao Zhang <[email protected]> wrote: > > > > Should we release version 4.1.1? If we decide to proceed with 4.1.1, we may > > need to identify valuable patches from recent commits for the 4.1 branch, > > such as fixes for correctness issues, performance optimizations, and > > security problems. > > > > If we skip 4.1.1 and instead release 4.2.0, would this imply that we've > > reached a default agreement to only publish major version releases based on > > the master branch going forward? > > > > Additionally, I have two tickets that may need to be addressed in the next > > new version: one is HIVE-29145 (Iceberg Rest server cannot work if launched > > by non-standalone HMS), and the other is HIVE-29213 (HS2 reports 'Failed to > > get primary keys' when using old beeline client). > > > > Thanks, > > Butao Zhang > > > > > > On 2025/10/22 08:48:19 Denys Kuzmenko wrote: > > > Hi Stamatis, > > > > > > Thanks for bringing this up. > > > > > > We already have enough items ready to ship, including JDK 21. > > > I'm not sure what’s causing the delay. I thought we already had a release > > > manager, but looks like we don’t. > > > > > > Regards, > > > Denys > > > >
