Re: Poor Hive 3.1.0 performance

[gwenael le barzic]

Hey guys.

Analyzing the jstack, I found that the ALTER DATABASE SET LOCATION performs
a check on all the folders inside the database folder in HDFS.
And I have some databases with some tables with a lt of partitions ;-).
So nothing abnormal here.

Best regards.


On Tue, Nov 23, 2021 at 10:26 AM gwenael le barzic 
wrote:

> Hello.
>
> I send you this mail because I migrated my cluster recently to HDP 3.1.4.
> (hive 3.1.0)
>
> I experience very slow queries when I perform ALTER DATABASE  SET
> LOCATION ''.
>
> When I check the logs of hiveserver2, I can see that the compiling step of
> the query takes a looot of time (around 8 minuts).
>
> I checked my database hive (postgresql) to see if there is any problem
> there.
>
> I was wondering if anyone has already seen this kind of slowness in hive
> 3.1.0 ?
>
> Best regards.
>
>

RE: Patches to Hive 3.1.2,

[Battula, Brahma Reddy]

Thanks, Sungwoo Park!!.
Looks 3.1.2 released on 26 August 2019, any plans to 3.1.3..?

How about cherry-pick following critical issues to branch-3.1 and release..?

From: Sungwoo Park 
Sent: Thursday, August 12, 2021 9:21 PM
To: user@hive.apache.org
Subject: Patches to Hive 3.1.2,

Hello Hive users,

We have updated the repository that backports patches to Hive 3.1.2. Now it 
backports about 350 patches from the master branch to branch-3.1 of November 
2020. You can ignore the last two commits which add MR3 backend and remove Hive 
on Spark.

https://github.com/mr3project/hive-mr3

The focus is mainly on fixing bugs in Hive 3.1.2 and stabilizing the 
performance when using AWS S3. We will keep backporting more patches, so if you 
think important patches are missing, please feel free to create issues.

Hope you find it useful!

--- Sungwoo

Poor Hive 3.1.0 performance

[gwenael le barzic]

Hello.

I send you this mail because I migrated my cluster recently to HDP 3.1.4.
(hive 3.1.0)

I experience very slow queries when I perform ALTER DATABASE  SET
LOCATION ''.

When I check the logs of hiveserver2, I can see that the compiling step of
the query takes a looot of time (around 8 minuts).

I checked my database hive (postgresql) to see if there is any problem
there.

I was wondering if anyone has already seen this kind of slowness in hive
3.1.0 ?

Best regards.