over in alphabetic
order, the server is up.
This is not the case in Hive 2.0.1. Is there a setting we are missing?
--
Regards,
Premal Shah.
rker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> INFO cli.LlapServiceDriver: LLAP service driver finished
>
>
>
> Thanks
>
> Rajesh
>
--
Regards,
Premal Shah.
ot;:"true","COLUMN_STATS":{"id":"true","col2":"true","*
*col3**":"true","**col4**":"true"}}, numFiles=6}*
Does this mean some stats are stored?
Any help is appreciated.
Thanx.
--
Regards,
Premal Shah.
t; (i.e retries are more expensive, happy path is faster)
>
> select count(distinct id) from ip_table;
>
> Java's hashCode() implementation is pretty horrible (& Hive defaults to
> using it). If you're seeing a high collision count, I think I might know
> what's happening here.
>
> Cheers,
> Gopal
>
>
>
--
Regards,
Premal Shah.
> by collisions desc limit 10;
>
> And, if those show many collisions
>
> set tez.runtime.io.sort.mb=640;
> set hive.map.aggr=false;
> set tez.runtime.pipelined.shuffle=true; // this reduces failure tolerance
> (i.e retries are more expensive, happy path is faster)
>
> select count(distinct ip) from ip_table;
>
> Cheers,
> Gopal
>
>
>
>
--
Regards,
Premal Shah.
>
> set hive.optimize.distinct.rewrite=true;
>
> or try a rewrite
>
> select id from accounts group by id having count(1) > 1;
>
> Both approaches enable full-speed vectorization for the query.
>
> Cheers,
> Gopal
>
>
>
--
Regards,
Premal Shah.
What can be done to get the hive query to run faster in hive?
--
Regards,
Premal Shah.
My bad. Looks like the thrift server is cycling through various AMs it
started when the thrift server was started. I think this is different from
either Hive 2.0.1 or LLAP.
On Mon, Mar 27, 2017 at 11:38 PM, Premal Shah <premal.j.s...@gmail.com>
wrote:
> Hi,
> I have a thrift s
APSED TIME: 8.49 s
--
OK
Query ID = hadoop_20170328053153_8677d9d6-e748-4eb7-bfeb-1f1abdbb367c
Total jobs = 1
Launching Job 1 out of 1
--
Regards,
Premal Shah.
e submitted vertex?
>
> set hive.tez.container.size=?
>
> Cheers,
> Gopal
>
>
>
--
Regards,
Premal Shah.
]
When I switched the execution engine to mr, the query finished in 30 mins.
Are there any knobs we have to tweak?
--
Regards,
Premal Shah.
ggested, the '$f0' is probably the auto-generated name for the count(0).
>
> Naming that column explicitly on both branches of the UNION ALL, might get
> CBO back up.
>
> Cheers,
> Gopal
>
>
>
--
Regards,
Premal Shah.
s not happen to all CTAS queries.
>
> Not sure if that's related to Tez at all.
>
> Can try running it with
>
> set hive.cbo.enable=false;
>
> Cheers,
> Gopal
>
>
>
--
Regards,
Premal Shah.
)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
--
Regards,
Premal Shah.
Sorry,
here's the link -
https://gist.github.com/premal/d054d4cc0ed00efdf60351ca2517db3d
On Wed, Nov 2, 2016 at 8:11 PM, Premal Shah <premal.j.s...@gmail.com> wrote:
> Hi Prasanth,
> Here's a link to the hive log4js properties file.
>
> We are on Hive 2.0.1.
>
> We canno
to https://issues.apache.org/jira/browse/HIVE-11751
>
> The debug strings gets generated but gets filtered. Can you share your
> log4j2 properties file?
>
> What version of hive are you using?
>
> Thanks
> Prasanth
>
> On Nov 2, 2016, at 5:06 PM, Premal Shah <premal.j.s...@g
> You can check the /tmp/$USER/hive.log and see what's happening in detail.
>
>
>
> Cheers,
>
> Gopal
>
--
Regards,
Premal Shah.
4cbb0
>
> If you are more adventurous and want to run LLAP on an unsupported
> platform, I maintain scripts which will configure and install it
>
> https://github.com/t3rmin4t0r/tez-autobuild/blob/llap/README.md
>
> Cheers,
> Gopal
>
>
>
--
Regards,
Premal Shah.
My guess is this happens only in
> DEBUG log level.
>
> Thanks
> Prasanth
>
>
>
>
> On Fri, Oct 28, 2016 at 9:40 PM -0700, "Premal Shah" <
> premal.j.s...@gmail.com> wrote:
>
> Hive 2.0.1
> Hadoop 2.7.2
> Tex 0.8.4
>
> We have a UDF in h
the query on the cluster.
The hive shell starts with an Xmx of 4G.
If I set hive.execution.engine = mr, then the query works, because it runs
on the hadoop cluster.
What should we change to avoid this problem?
Thanx
--
Regards,
Premal Shah.
to partition the tables so that the joins are faster?
--
Regards,
Premal Shah.
(orc.compress.size=8192);
On Thu, May 15, 2014 at 8:11 PM, Premal Shah premal.j.s...@gmail.comwrote:
I have a table in hive stored as text file with 3283 columns. All columns
are of string data type.
I'm trying to convert that table into an orc file table using this command
*create table orc_table
(83008K), 0.0041840 secs]
1.371: [GC 18505K-2249K(83008K), 0.0097240 secs]
34.779: [GC 28384K(4177280K), 0.0014050 secs]
Anything I can tweak to make it work?
--
Regards,
Premal Shah.
(83008K), 0.0041840 secs]
1.371: [GC 18505K-2249K(83008K), 0.0097240 secs]
34.779: [GC 28384K(4177280K), 0.0014050 secs]
Anything I can tweak to make it work?
--
Regards,
Premal Shah.
Sorry for the double post. I did not show up for a while and then I could
not get to the archives page, so I thought I'd needed to resend.
On Fri, May 16, 2014 at 12:54 AM, Premal Shah premal.j.s...@gmail.comwrote:
I have a table in hive stored as text file with 3283 columns. All columns
25 matches
Mail list logo
