che.org
Subject: Re: Hive +Tez+LLAP does not have obvious performance improvement than
HIVE + Tez
Hi,
If you've got the 1st starvation fixed (with Hadoop 2.8 patch), all these
configs + enable log4j2 async logging, you should definitely see a performance
improvement.
Here's the
Hi,
If you've got the 1st starvation fixed (with Hadoop 2.8 patch), all these
configs + enable log4j2 async logging, you should definitely see a performance
improvement.
Here's the log patches, which need a corresponding LLAP config (& have to be
disabled in HS2, for the progress bar to work)
hive.apache.org
Subject: Re: Hive +Tez+LLAP does not have obvious performance improvement than
HIVE + Tez
Hi,
> With these configurations, the cpu utilization of llap is very low.
Low CPU usage has been observed with LLAP due to RPC starvation.
I'm going to assume that the build you
--
From: Gopal Vijayaraghavan [mailto:gop...@apache.org]
Sent: Saturday, November 25, 2017 3:39 AM
To: user@hive.apache.org
Subject: Re: Hive +Tez+LLAP does not have obvious performance improvement than
HIVE + Tez
Hi,
> In our test, we found the shuffle stage of LLAP is very slow. Whether n
Hi,
> In our test, we found the shuffle stage of LLAP is very slow. Whether need to
> configure some related shuffle value or not?
Shuffle is the one hit by the 2nd, 3rd and 4th resource starvation issues
listed earlier (FDs, somaxconn & DNS UDP packet loss).
> And we get the following log f
uggestions? Whether the "all" mode can gain the
>best performance or not? And how the "auto" and "only" mode work?
Regards,
Jia Ke
-Original Message-
From: Gopal Vijayaraghavan [mailto:gop...@apache.org]
Sent: Thursday, November 23, 2017 4:03 AM
To:
Hi,
> With these configurations, the cpu utilization of llap is very low.
Low CPU usage has been observed with LLAP due to RPC starvation.
I'm going to assume that the build you're testing is a raw Hadoop 2.7.3 with no
additional patches?
Hadoop-RPC is single-threaded & has a single mutex loc
p is very low.
Regards,
Jia Ke
-Original Message-
From: Gopal Vijayaraghavan [mailto:gop...@apache.org]
Sent: Wednesday, November 22, 2017 11:20 AM
To: user@hive.apache.org
Subject: Re: Hive +Tez+LLAP does not have obvious performance improvement than
HIVE + Tez
Hi,
> Please help us
Hi,
> Please help us find whether we use the wrong configuration. Thanks for your
> help.
Since there are no details, I'm not sure what configuration you are discussing
here.
A first step would be to check if LLAP cache is actually being used (the LLAP
IO in the explain), vectorization is be
Hi all,
Now, we are running the benchmark of Hive +Tez+LLAP and Apache Tez in TPC-DS
with 3TB orc data. But the result of Hive +Tez+LLAP is almost similar with
hive+Tez and some queries may be poorer than hive+tez .
The following is our cluster and llap configuration,
Cluster: 1 master + 7 slave
10 matches
Mail list logo