Re: RE: Re: For Apache Hive HS2 , what is the largest heap size setting that works well?

>> -- 已转发邮件 ------ >> From: eric wong >> To: user@hive.apache.org >> Cc: >> Bcc: >> Date: Thu, 30 Nov 2017 00:07:47 +0800 >> Subject: Re: For Apache Hive HS2 , what is the largest heap size setting >> that works well? >> >> 1

Re: RE: Re: For Apache Hive HS2 , what is the largest heap size setting that works well?

AM PST, eric wong < > win19...@gmail.com> wrote: > > > > > > -- 已转发邮件 -- > From: eric wong > To: user@hive.apache.org > Cc: > Bcc: > Date: Thu, 30 Nov 2017 00:07:47 +0800 > Subject: Re: For Apache Hive HS2 , what is the largest heap size

Re: RE: Re: For Apache Hive HS2 , what is the largest heap size setting that works well?

+0800 Subject: Re: For Apache Hive HS2 , what is the largest heap size setting that works well? 1）Long GC pause should has detailed reson, like InitialMark/Remark、Full GC due to Concurrent Mode failure/Promotion failure。Please check it 2）CMS is nice and steady for our production. Please

RE: Re: For Apache Hive HS2 , what is the largest heap size setting that works well?

> > > -- 已转发邮件 -- > From: eric wong > To: user@hive.apache.org > Cc: > Bcc: > Date: Thu, 30 Nov 2017 00:07:47 +0800 > Subject: Re: For Apache Hive HS2 , what is the largest heap size setting > that works well? > > 1）Long GC pause should has det

Re: For Apache Hive HS2 , what is the largest heap size setting that works well?

1）Long GC pause should has detailed reson, like InitialMark/Remark、Full GC due to Concurrent Mode failure/Promotion failure。Please check it 2）CMS is nice and steady for our production. Please troubleshooting case by case. G1 is just difficult to perceive. 3）You shouldupdate to JDK8 ，for one case，

Re: For Apache Hive HS2 , what is the largest heap size setting that works well?

I also recommend it you will have also performance improvements with JDK8 in general (use the latest version). Keep also in mind that more and more big data libraries etc will drop JDK7 support soon (Aside that JDK7 is anyway not maintained anymore). > On 29. Nov 2017, at 01:31, Johannes Albert

Re: For Apache Hive HS2 , what is the largest heap size setting that works well?

Yes, I would recommend to go to Java 8 and give it a shot with G1 and report back :) Sent from my iPhone > On Nov 28, 2017, at 3:30 PM, Sharanya Santhanam > wrote: > > HI Johannes , > > We are running on Java version jdk1.7.0_67 . We are using > ConcurrentMarkAndSweep. Would you recommend

Re: For Apache Hive HS2 , what is the largest heap size setting that works well?

HI Johannes , We are running on Java version jdk1.7.0_67 . We are using ConcurrentMarkAndSweep. Would you recommend using G1GC ? These are our current settings -XX:NewRatio=8 -XX:+UseParNewGC -XX:-UseGCOverheadLimit -XX:PermSize=256m -Xloggc:<> -XX:HeapDumpPath=oom -XX:+PrintGCDetails -XX:+Pri

Re: For Apache Hive HS2 , what is the largest heap size setting that works well?

Hi Sharanya, Can you share your current GC settings and Java version. Are you using Java 8/9 w/ G1 already? Regards, Johannes Sent from my iPhone > On Nov 28, 2017, at 12:57 PM, Sharanya Santhanam > wrote: > > Hello , > > I am currently trying to upgrade hive version on our prod clusters

For Apache Hive HS2 , what is the largest heap size setting that works well?

Hello , I am currently trying to upgrade hive version on our prod clusters form V1.2 to v2.1 We also want to adopt HS2 on the new upgraded cluster. Earlier all queries were submitted via Hive cli. Would like to understand how large a single HS2 Heap size can be ? And is there any formula to figur