As I advocated in previous threads I don't think we have the necessary (human & CI) resources to maintain multiple release lines so from my perspective when 4.2.0 goes out 4.1.0 should go automatically EOL; in other words there won't be a 4.1.1 release. We can continue iterating on this discussion but it's important to converge before we publish the 4.2.0 so that there is a clear communication to our users.
Regarding HIVE-29145 and HIVE-29213, I posted some thoughts under the respective tickets and I would encourage others to do that as well. Regarding the addition of the get_table method, I don't fully understand how it would help Spark if they don't upgrade to a supported version but feel free to log a jira ticket and we can iterate there to find a solution. Deprecation and removal is a very important discussion but slightly off scope in the current thread. Both contributors and reviewers should carefully evaluate these aspects when raising/accepting PRs. Any concrete ideas/initiatives to improve on this aspect are always welcome so feel free to start/submit threads/docs/tickets/tests on this topic. Best, Stamatis On Sat, Oct 25, 2025 at 5:55 AM Butao Zhang <[email protected]> wrote: > > Thank you, Okumin, for taking on the responsibility of the next Hive release. > > I'd also like to discuss an issue here. Since Hive 4.0.1, Apache Spark has > been unable to query Iceberg tables in HMS. The specific issue discussion can > be found at [1]. This problem occurred because we removed the `get_table` > method in [2]. Spark relies on Hive 2.3.10, which depends on the `get_table` > method to access Iceberg tables in HMS. The compatibility issue emerged after > HIVE-26537. > > To resolve this, the ideal solution would be for Apache Spark to upgrade its > built-in Hive dependency from version 2.3.10 to 4.0.1 or 4.1.0. I've noticed > that there is an upgrade ticket [3] in the Spark community for this. However, > this upgrade appears to involve significant changes and may not be resolved > for quite some time. > > Given that Apache Spark is one of the most widely used engines with HMS and > its adoption would help promote the use of Hive 4, is it possible for us to > reintroduce the removed `get_table` method? I understand that restoring > deprecated code is generally not a reasonable approach, so I'd like to ask if > there are any more elegant solutions on the HMS side to address this issue? > > Additionally, I have a personal thought: when removing or deprecating public > methods in Hive, we should carefully consider the dependencies of downstream > components on these methods to avoid long-term upgrade difficulties for them. > For example, when working on federated catalog [4] related tasks, I needed to > add catalog parameters to many methods in `Hive.java`. However, since Spark > code depends on `Hive.java` to access Hive, I had to introduce many new > methods with catalog parameters, which may lead to significant code > duplication. But there was no alternative, as modifying existing methods > would cause compatibility issues for downstream components with newer Hive > versions. Currently, this incompatibility risk relies on individual prior > knowledge, but I hope we can develop better methods in the future to > proactively identify such compatibility issues. > > [1]https://github.com/apache/iceberg/issues/12878 > [2]https://issues.apache.org/jira/browse/HIVE-26537 > [3]https://issues.apache.org/jira/browse/SPARK-52408 > [4]https://issues.apache.org/jira/browse/HIVE-29177 > > Thanks, > Butao Zhang > > On 2025/10/24 09:03:28 Shohei Okumiya wrote: > > Hi Stamatis, > > > > Thanks for initiating this discussion. I'd love to have the great > > release cadence! > > > > 1. I don't have any objections about the timeline. We might want to > > release 4.1.1 in the future. I feel it should not block us from > > shipping the master branch anyway. > > 2. I'm willing to try being a release manager someday, but I have > > frequent hospital visits nowadays. I'm probably not the right person, > > given my unstable availability. > > 3. Except for issues that have already been discussed in mailing > > lists, I am not aware of critical problems. > > > > Best, > > Okumin > > > > On Thu, Oct 23, 2025 at 11:50 AM Butao Zhang <[email protected]> wrote: > > > > > > Should we release version 4.1.1? If we decide to proceed with 4.1.1, we > > > may need to identify valuable patches from recent commits for the 4.1 > > > branch, such as fixes for correctness issues, performance optimizations, > > > and security problems. > > > > > > If we skip 4.1.1 and instead release 4.2.0, would this imply that we've > > > reached a default agreement to only publish major version releases based > > > on the master branch going forward? > > > > > > Additionally, I have two tickets that may need to be addressed in the > > > next new version: one is HIVE-29145 (Iceberg Rest server cannot work if > > > launched by non-standalone HMS), and the other is HIVE-29213 (HS2 reports > > > 'Failed to get primary keys' when using old beeline client). > > > > > > Thanks, > > > Butao Zhang > > > > > > > > > On 2025/10/22 08:48:19 Denys Kuzmenko wrote: > > > > Hi Stamatis, > > > > > > > > Thanks for bringing this up. > > > > > > > > We already have enough items ready to ship, including JDK 21. > > > > I'm not sure what’s causing the delay. I thought we already had a > > > > release manager, but looks like we don’t. > > > > > > > > Regards, > > > > Denys > > > > > >
