Re: Hive optimizer

Its both. Some of the optimizations are rule based and some are cost based. John From: Ashok Kumar > Reply-To: "user@hive.apache.org" >, Ashok Kumar

Re: VolcanoPlanner vs HepPlanner

Yes, we used to do VolcanPlanner before but we switched in Hive 14 time frame (2014) due to the performance limitations. I am not sure why you are after VolcanoPlanner; but yes you can get back to VolcanoPlanner and probably get it to work. I know this because I did most of those changes back

Re: VolcanoPlanner vs HepPlanner

This would be a broad change. Hep Planner does enumerate different join orders through "LoptOptimizeJoinRule". Volcano planner is not used as it has scalability issues. Thanks John From: Raajay > Reply-To:

Re: [ANNOUNCE] New Hive PMC Chair - Ashutosh Chauhan

Congrats Ashutosh! From: Vaibhav Gumashta > Reply-To: "user@hive.apache.org" > Date: Wednesday, September 16, 2015 at 1:01 PM To:

Re: CBO - get cost of the plan

is accurate ? Also, what is the definition of cumulative cost ? Thanks for the help, Raajay On Mon, Aug 24, 2015 at 8:51 PM, John Pullokkaran jpullokka...@hortonworks.commailto:jpullokka...@hortonworks.com wrote: From the text below it seems like you are not collecting table level stats. You can

Re: CBO - get cost of the plan

Raajay, You don't have col stats hence it assumes 1 for row count. What version of Hive are you on? Thanks John From: Raajay raaja...@gmail.commailto:raaja...@gmail.com Reply-To: user@hive.apache.orgmailto:user@hive.apache.org user@hive.apache.orgmailto:user@hive.apache.org Date: Monday,

Re: CBO - get cost of the plan

In addition to col stats you also need table stats. From: John Pullokkaran jpullokka...@hortonworks.commailto:jpullokka...@hortonworks.com Reply-To: user@hive.apache.orgmailto:user@hive.apache.org user@hive.apache.orgmailto:user@hive.apache.org Date: Monday, August 24, 2015 at 5:23 PM To: user

Re: CBO - get cost of the plan

= c.c_alternate; On Mon, Aug 24, 2015 at 7:25 PM, John Pullokkaran jpullokka...@hortonworks.com wrote: In addition to col stats you also need table stats. From: John Pullokkaran jpullokka...@hortonworks.com Reply-To: user@hive.apache.org user@hive.apache.org Date: Monday, August 24, 2015 at 5

Re: Hive CBO - Calcite Interface

Hi Raajay, #1 No, there is no API for this. #2 If you enable Logging (BaseSemanticAnalyzer) then CalcitePlanner will print out the plan with cost. John From: Raajay raaja...@gmail.commailto:raaja...@gmail.com Reply-To: user@hive.apache.orgmailto:user@hive.apache.org

Re: Hive indexing optimization

Index doesn’t seems to be kicking in this case. Please file a bug for this. Thanks John From: Bennie Leo tben...@hotmail.commailto:tben...@hotmail.com Reply-To: user@hive.apache.orgmailto:user@hive.apache.org user@hive.apache.orgmailto:user@hive.apache.org Date: Monday, June 29, 2015 at 5:25 PM

Re: Hive indexing optimization

Could you post explain extended output? From: Bennie Leo tben...@hotmail.commailto:tben...@hotmail.com Reply-To: user@hive.apache.orgmailto:user@hive.apache.org user@hive.apache.orgmailto:user@hive.apache.org Date: Monday, June 29, 2015 at 10:35 AM To:

Re: Hive indexing optimization

SELECT StartIp, EndIp, Country FROM ipv4geotable” should have been rewritten as a scan against index table. BitMap Indexes seems to support inequalities (=, , =). Post the explain plan. On 6/26/15, 8:56 PM, Gopal Vijayaraghavan gop...@apache.org wrote: Hi, Hive indexes won¹t really help you

Re: Cost based optimization

, that are primarily responsible for Query Planning. Thanks, Raajay On Thu, Jun 25, 2015 at 4:45 PM, John Pullokkaran jpullokka...@hortonworks.commailto:jpullokka...@hortonworks.com wrote: Hive does look in to alternate join orders and pick the best plan that minimizes cost. It uses a greedy

Re: Cost based optimization

Hive does look in to alternate join orders and pick the best plan that minimizes cost. It uses a greedy algorithm to enumerate plan space. Thanks John From: Raajay raaja...@gmail.commailto:raaja...@gmail.com Reply-To: user@hive.apache.orgmailto:user@hive.apache.org

Re: Hive indexing optimization

Set hive.optimize.index.filter=true; Thanks John From: Bennie Leo tben...@hotmail.commailto:tben...@hotmail.com Reply-To: user@hive.apache.orgmailto:user@hive.apache.org user@hive.apache.orgmailto:user@hive.apache.org Date: Thursday, June 25, 2015 at 5:48 PM To:

Re: Orc file and Hive Optimiser

ORC format is transparent to CBO. Currently we are working on a new cost model which might reflect ORC's performance advantages in optimization decisions. Thanks John From: Mich Talebzadeh m...@peridale.co.ukmailto:m...@peridale.co.uk Reply-To: user@hive.apache.orgmailto:user@hive.apache.org

Re: Orc file and Hive Optimiser

subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility. From: John Pullokkaran [mailto:jpullokka...@hortonworks.com

Re: [ANNOUNCE] New Hive Committer - John Pullokkaran

: The Apache Hive PMC has voted to make John Pullokkaran a committer on the Apache Hive Project. Please join me in congratulating John! Thanks. - Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed

Re: [ANNOUNCE] New Hive Committer - Gunther Hagleitner

Congrats Gunther! On Mon, Jul 22, 2013 at 8:24 AM, Vikram Dixit vik...@hortonworks.comwrote: Congrats Gunther! Great news! On Jul 21, 2013, at 8:52 PM, Vaibhav Gumashta wrote: Congratulations Gunther! On Sun, Jul 21, 2013 at 8:36 PM, David Schorow dscho...@hortonworks.com wrote: