[VOTE] Apache Hive 2.3.10 Release Candidate 1
+1 (non-binding) Pass integration test with Apache Spark[1] and Apache Kyuubi[2]. [1] https://github.com/apache/spark/pull/45372 [2] https://github.com/apache/kyuubi/pull/6328 Thanks, Cheng Pan
Re: [VOTE] Apache Hive 2.3.10 Release Candidate 0
Hi Stamatis, HIVE-26882 got in 2.3.10, while a correctness issue was identified recently, HIVE-28121 fixes that. I highly suggest including such a correctness patch in the final release of 2.3 serials. Thanks, Cheng Pan
Re: [VOTE] Apache Hive 2.3.10 Release Candidate 0
I made integration tests with Apache Spark[1] and Apache Kyuubi[2], and everything looks good so far. We may need to wait for HIVE-28121(affects HMS)[3] and prepare for the next RC. Thanks to Chao for your efforts in making this release. [1] https://github.com/apache/spark/pull/45372 [2] https://github.com/apache/kyuubi/pull/6328 [3] https://github.com/apache/hive/pull/5204 Thanks, Cheng Pan
Re: Issue with joda-time library bundled in hive-exec:4.0.0
Simhadri, thank you for sharing the information. Thanks, Cheng Pan
Re: [DISCUSS] End of life for Hive 1.x, 2.x, 3.x
Hi Chao, The Spark community is starting to discuss the 4.0 release[1], can we make the Hive 2.3.10 release happen soon? [1] https://lists.apache.org/thread/nxmvz2j7kp96otzlnl3kd277knlb6qgb Thanks, Cheng Pan On 2024/01/17 17:50:37 Chao Sun wrote: > Hi Ayush, > > I'm working on the last few commits to backport to the branch. > Hopefully within 1-2 months I can start the release process. Our goal > is to upgrade Hive 2.x before the Spark 4.0 release coming up mid this > year. > > Chao > > On Tue, Jan 16, 2024 at 10:17 PM Ayush Saxena wrote: > > > > Thanx everyone for the feedback, I have started a formal thread to mark 1.x > > EOL. We can have one last release for 2.x as Chao mentioned, with some > > required changes + our CVE's & get the release line marked as EOL then. > > > > @Chao Sun Do let us know if you have a proposed > > timeline for that. > > > > -Ayush > > > > On Wed, 17 Jan 2024 at 08:23, vihang karajgaonkar > > wrote: > > > > > I was confused about the subject line since it says 3.x as well along with > > > 1.x and 2.x. Does this discussion include all 1.x, 2.x and 3.x or just 1.x > > > and 2.x? > > > > > > I think it makes sense to EOL 1.x. Looks like 2.x is still being > > > maintained > > > by Chao and I think we were backporting PRs to the 3.x line pretty > > > recently > > > so I believe we should wait out for a release on Hive 3.x. > > > > > > Thanks, > > > Vihang > > > > > > On Tue, Jan 16, 2024 at 3:40 PM Attila Turoczy > > > wrote: > > > > > > > Dear PMC's, > > > > > > > > Do we have a verdict / decision about this? > > > > > > > > -Attila > > > > > > > > On Wed, Jan 10, 2024 at 5:45 PM Chao Sun wrote: > > > > > > > > > On Hive 2.x, I'm still preparing for another release 2.3.10 (Hive 2.3 > > > > > branch is being actively maintained so far). Hopefully this will be > > > > > the last release in the branch-2 line. > > > > > > > > > > +1 on making Hive 1 EOL for the time being. > > > > > > > > > > Chao > > > > > > > > > > On Wed, Jan 10, 2024 at 8:10 AM Sankar Hariappan > > > > > wrote: > > > > > > > > > > > > +1 for making both Hive 1&2 EOL > > > > > > > > > > > > -Sankar > > > > > > -Original Message- > > > > > > From: Attila Turoczy > > > > > > Sent: Wednesday, January 10, 2024 7:37 PM > > > > > > To: dev@hive.apache.org > > > > > > Subject: [EXTERNAL] Re: [DISCUSS] End of life for Hive 1.x, 2.x, 3.x > > > > > > > > > > > > [You don't often get email from aturo...@cloudera.com.invalid. Learn > > > > > why this is important at https://aka.ms/LearnAboutSenderIdentification > > > ] > > > > > > > > > > > > +1 for making it EOL for Hive 1 and Hive 2. I do not think these 2 > > > > > product > > > > > > branches are relevant in 2023. > > > > > > > > > > > > -Attila > > > > > > > > > > > > On Wed, Jan 10, 2024 at 12:59 PM Denys Kuzmenko < > > > dkuzme...@apache.org> > > > > > > wrote: > > > > > > > > > > > > > +1 for marking Hive 1.x EOL > > > > > > > > > > > > > > Assuming no volunteers willing to take ownership of branch-2 > > > > > maintenance, > > > > > > > +1 to declare it EOL as well. > > > > > > > > > > > > > > Regards, > > > > > > > Denys > > > > > > > > > > > > > > > > > > > >
Re: Issue with joda-time library bundled in hive-exec:4.0.0
Hi Ayush, > Hive is already in discussion of marking Hive-2.x EOL, so at very best we > would have one release and immediately after that we will announce it EOL Does the discussion happen in public? Is there an ETA for the final release of branch-2.3? Thanks, Cheng Pan > On Apr 17, 2024, at 18:03, Ayush Saxena wrote: > > Thanx Cheng Pan for sharing the pointers, Do you have any list of issues or > pointers on what are the challenges for Spark to move to a higher Hive > version? I know upgrading libraries is quite challenging but it is inevitable. > > Hive is already in discussion of marking Hive-2.x EOL, so at very best we > would have one release and immediately after that we will announce it EOL, > maintaining a release line is quite an effort for us at Hive & doing it > because other projects doesn't want to upgrade isn't a convincing reason for > most of us. The best we can do is or are trying is to address issues for > Spark whatever we can do as part of Hive code & would definitely need > help/support from Spark side as well, since the move is from 2.x to 4.x, it > would be a big change and would offer resistance on both sides. > > So, it would be great help if any pointers can be shared from Spark side for > the move, if there is no help/interest from Spark then we can't do anything & > there is no need for Hive-2.x either in that case :-) > > -Ayush > > On Wed, 17 Apr 2024 at 15:00, Cheng Pan wrote: > > … we are exploring ways to get Spark move from 2.3.9 to 4.0, Our initial > > hunch is that it would be quite challenging without a hive-exec slim jar … > > It should be challenging to upgrade Spark’s built-in Hive version. Actually, > we already did lots of work on branch-2.3 which focuses on CVE reduction, for > example, allowing Spark to upgrade Guava to modern versions to get rid of > Guava 14, it was tested with the latest Spark master branch[1], maybe we need > a release for 2.3.10 now. > > [1] https://github.com/apache/spark/pull/45372 > > Thanks, > Cheng Pan > >
Re: Issue with joda-time library bundled in hive-exec:4.0.0
> … we are exploring ways to get Spark move from 2.3.9 to 4.0, Our initial > hunch is that it would be quite challenging without a hive-exec slim jar … It should be challenging to upgrade Spark’s built-in Hive version. Actually, we already did lots of work on branch-2.3 which focuses on CVE reduction, for example, allowing Spark to upgrade Guava to modern versions to get rid of Guava 14, it was tested with the latest Spark master branch[1], maybe we need a release for 2.3.10 now. [1] https://github.com/apache/spark/pull/45372 Thanks, Cheng Pan
Re: Issue with joda-time library bundled in hive-exec:4.0.0
There is a JIRA ticket[1] that tracks "upgrading built-in Hive to 3+” BTW, regarding HMS API used by Spark, the Hive 2.3.9 client is compatible with HMS from 2.0 to 4.0, while the upcoming Hive 2.3.10 client should be compatible with HMS from 1.2 to 4.0, if we decide to upgrade the built-in Hive, it’s better to keep such compatibility. [1] https://issues.apache.org/jira/browse/SPARK-44114 Thanks, Cheng Pan > On Apr 17, 2024, at 18:03, Ayush Saxena wrote: > > Thanx Cheng Pan for sharing the pointers, Do you have any list of issues or > pointers on what are the challenges for Spark to move to a higher Hive > version? I know upgrading libraries is quite challenging but it is inevitable. > > Hive is already in discussion of marking Hive-2.x EOL, so at very best we > would have one release and immediately after that we will announce it EOL, > maintaining a release line is quite an effort for us at Hive & doing it > because other projects doesn't want to upgrade isn't a convincing reason for > most of us. The best we can do is or are trying is to address issues for > Spark whatever we can do as part of Hive code & would definitely need > help/support from Spark side as well, since the move is from 2.x to 4.x, it > would be a big change and would offer resistance on both sides. > > So, it would be great help if any pointers can be shared from Spark side for > the move, if there is no help/interest from Spark then we can't do anything & > there is no need for Hive-2.x either in that case :-) > > -Ayush > > On Wed, 17 Apr 2024 at 15:00, Cheng Pan wrote: > > … we are exploring ways to get Spark move from 2.3.9 to 4.0, Our initial > > hunch is that it would be quite challenging without a hive-exec slim jar … > > It should be challenging to upgrade Spark’s built-in Hive version. Actually, > we already did lots of work on branch-2.3 which focuses on CVE reduction, for > example, allowing Spark to upgrade Guava to modern versions to get rid of > Guava 14, it was tested with the latest Spark master branch[1], maybe we need > a release for 2.3.10 now. > > [1] https://github.com/apache/spark/pull/45372 > > Thanks, > Cheng Pan > >
Re: Hive 2.3.10 release?
+1 Please consider including Thrift upgrading for security purpose. On 2023/07/12 04:09:19 Chao Sun wrote: > Hi all, > > It's been quite a while since the last 2.3.9 release, and there are > several commits accumulated in the branch-2.3, including a few > critical bug fixes. Since Hive 2.3.x is still actively being used by > projects such as Apache Spark, I'm thinking about initiating a new > release process, if there's no objections. Please let me know your > thoughts. Thanks > > Best, > Chao >
RE: Branch-3 backports and build stability
+1 for removing Hive on Spark from 3.2. Hive on Spark has been out of maintenance for a long time, the related tests are broken in branch-2.3, and branch-3.1. The known Hadoop releases including CDP[1] HDP[2] MR3[3] removed the HoS in Hive3 too. Thus, I suppose HoS is deprecated since Hive 2.3 in fact, users won't be surprised if we remove it in 3.2. [1] https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/upgrade-cdh/topics/cdp-data-migration-hive-on-spark.html [2] https://community.cloudera.com/t5/Support-Questions/Does-Hive-on-Spark-supported-in-HDP-3-x/m-p/280189 [3] https://github.com/mr3project/hive-mr3/commit/7f03e754953fbf47bfca141a19ce6cf257c70110 On 2023/02/07 11:08:35 Stamatis Zampetakis wrote: Hi all, The build in branch-3 is not yet green; there are ~25 test failures. It is a common practice that we shouldn't push changes on top of a broken build unless they are addressing test failures. Some people (mainly Aman Raj, Chris Nauroth, and Laszlo Bodor) are working hard to stabilize the build for quite some time now. If you want to help out then start by reviewing, merging, and fixing things around test failures. It's not yet the time to bring new features, upgrades, bugs, etc., in branch-3. I would encourage committers to not approve such changes till we get back to a stable branch. Best, Stamatis Thanks, Cheng Pan
[jira] [Created] (HIVE-26434) HMS thrift method recv_get_table_objects_by_name exception list is broken
Cheng Pan created HIVE-26434: Summary: HMS thrift method recv_get_table_objects_by_name exception list is broken Key: HIVE-26434 URL: https://issues.apache.org/jira/browse/HIVE-26434 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 2.3.9 Reporter: Cheng Pan HIVE-15062 accidentally removed the HMS thrift method recv_get_table_objects_by_name exception list, after HIVE-24608, some IT consistently failed because of it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-26336) Hive JDBC Driver should respect JDBC DriverManager#loginTimeout
Cheng Pan created HIVE-26336: Summary: Hive JDBC Driver should respect JDBC DriverManager#loginTimeout Key: HIVE-26336 URL: https://issues.apache.org/jira/browse/HIVE-26336 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 4.0.0-alpha-1 Reporter: Cheng Pan Before HIVE-12371, the Hive JDBC Driver uses DriverManager#loginTimeout as both connectTimeout and socketTimeout, which usually cause socket timeout exceptions for users who use Hive JDBC Driver in Spring Boot project, because Spring Boot will setLoginTimeout to 30s (default values). HIVE-12371 introduced a new parameter socketTimeout, and does not care about DriverManager#loginTimeout anymore, I think it's not a correct solution. I think the for loginTimeout, prefer to use loginTimeout (in milliseconds) from jdbc connection url, and fallback to use DriverManger#getLoginTimeout (in seconds). For socketTimeout, use socketTimeout (in milliseconds) from jdbc connection url if present. -- This message was sent by Atlassian Jira (v8.20.7#820007)
