Re: Review Request 43115: HIVE-12924 CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver groupby_ppr_multi_distinct.q failure

2016-02-02 Thread John Pullokkaran
/translator/HiveGBOpConvUtil.java (line 730) <https://reviews.apache.org/r/43115/#comment178784> Why is this being changed on MapSide RS. I thought the issue was with distincts on ReduceSide GB when map side is turned off. - John Pullokkaran On Feb. 2, 2016, 10:16 p.m., Hari Sankar Si

Re: Review Request 43115: HIVE-12924 CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver groupby_ppr_multi_distinct.q failure

2016-02-02 Thread John Pullokkaran
/translator/HiveGBOpConvUtil.java (line 225) <https://reviews.apache.org/r/43115/#comment178783> Instead of walking the expression you could keep a map of inputref to exprnode. - John Pullokkaran On Feb. 2, 2016, 10:16 p.m., Hari Sankar Sivarama Subramaniyan

Re: Review Request 42960: HIVE-11752

2016-02-02 Thread John Pullokkaran
ul/non successful execution) - John Pullokkaran On Feb. 1, 2016, 4:36 p.m., Jesús Camacho Rodríguez wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.

Re: Review Request 42487: Use bit vector to track NDV

2016-01-27 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42487/#review116706 --- Ship it! Ship It! - John Pullokkaran On Jan. 26, 2016, 8:10

Re: Review Request 41807: further improve user level explain

2016-01-27 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41807/#review116704 --- Ship it! Ship It! - John Pullokkaran On Jan. 21, 2016, 7:46

Re: Review Request 41807: further improve user level explain

2016-01-26 Thread John Pullokkaran
) <https://reviews.apache.org/r/41807/#comment177571> Could we write keys as expression i.e instead of keys RS_9:_col0=RS_11:d_date_sk - John Pullokkaran On Jan. 21, 2016, 7:46 a.m., pengcheng xiong wrote: > > ---

Re: Review Request 41807: further improve user level explain

2016-01-26 Thread John Pullokkaran
) <https://reviews.apache.org/r/41807/#comment177570> We are missing info on join type: inner vs outer - John Pullokkaran On Jan. 21, 2016, 7:46 a.m., pengcheng xiong wrote: > > --- > This is an automatically generated e

Re: Review Request 42190: HIVE-12478

2016-01-25 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42190/#review116291 --- Ship it! Ship It! - John Pullokkaran On Jan. 25, 2016, 7:20

Re: Review Request 42190: HIVE-12478

2016-01-22 Thread John Pullokkaran
er operators? HiveFilter, HiveProject . ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRulesRegistry.java (line 34) <https://reviews.apache.org/r/42190/#comment176996> Does this needs to be multimap? - John Pullokkaran On Jan. 21, 2016, 10:45 p.m., Jesús Camac

Re: Review Request 42487: Use bit vector to track NDV

2016-01-22 Thread John Pullokkaran
e/ql/stats/StatsUtils.java (line 1563) <https://reviews.apache.org/r/42487/#comment176980> Why is this needed? Seems like its not used. - John Pullokkaran On Jan. 22, 2016, 7:09 p.m., pengcheng xiong wrote: > > --

Re: Review Request 42521: HIVE-12867 : Semantic Exception Error Msg should be with in the range of "10000 to 19999"

2016-01-21 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42521/#review115755 --- Ship it! Ship It! - John Pullokkaran On Jan. 21, 2016, 11:08

Re: Review Request 42521: HIVE-12867 : Semantic Exception Error Msg should be with in the range of "10000 to 19999"

2016-01-19 Thread John Pullokkaran
) <https://reviews.apache.org/r/42521/#comment176140> May be change this to "Semantic error" Also why do we need a generic error - John Pullokkaran On Jan. 19, 2016, 9:03 p.m., Hari Sankar Sivarama Subramaniyan wrote: > >

Re: Review Request 42461: HIVE-12879

2016-01-18 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42461/#review115093 --- Ship it! Ship It! - John Pullokkaran On Jan. 18, 2016, 2:40

Re: Review Request 42281: Upgrade Hive to Calcite 1.6

2016-01-15 Thread John Pullokkaran
/HiveRelMdDistribution.java <https://reviews.apache.org/r/42281/#comment175650> Why is this removed? - John Pullokkaran On Jan. 15, 2016, 7:10 a.m., pengcheng xiong wrote: > > --- > This is an automatically generated e

Re: Review Request 42281: Upgrade Hive to Calcite 1.6

2016-01-15 Thread John Pullokkaran
281/#comment175623> Add a TODO: to handle CorrelationId - John Pullokkaran On Jan. 15, 2016, 7:10 a.m., pengcheng xiong wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > h

[jira] [Created] (HIVE-12867) Semantic Exception Error Msg should be with in the range of "10000 to 19999"

2016-01-13 Thread Laljo John Pullokkaran (JIRA)
Laljo John Pullokkaran created HIVE-12867: - Summary: Semantic Exception Error Msg should be with in the range of "1 to 1" Key: HIVE-12867 URL: https://issues.apache.org/jira/browse/

Re: Review Request 41297: HIVE-12590

2016-01-07 Thread John Pullokkaran
245) <https://reviews.apache.org/r/41297/#comment173867> May be use a local var to store the str value; this way we could avoid duplicating logic at 2 places ? - John Pullokkaran On Jan. 7, 2016, 9:17 p.m., Ashutosh Chauhan

Re: Review Request 41297: HIVE-12590

2016-01-07 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41297/#review113337 --- Ship it! Ship It! - John Pullokkaran On Jan. 7, 2016, 9:17 p.m

Re: Review Request 41297: HIVE-12590

2015-12-22 Thread John Pullokkaran
267) <https://reviews.apache.org/r/41297/#comment171861> It may be better to remove toStringTree and replace it with toLowerStringTree. Since in Hive all col names, udf/udaf are case insensitive, we might be ok if we make only literals case sensitive. - John Pullokkaran On D

Re: Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

2015-12-21 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/40865/#review111537 --- Ship it! Ship It! - John Pullokkaran On Dec. 21, 2015, 5:44

Re: Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

2015-12-21 Thread John Pullokkaran
/HiveReduceExpressionsRule.java (line 320) <https://reviews.apache.org/r/40865/#comment171729> Remove this code; seems like not used. - John Pullokkaran On Dec. 21, 2015, 5:44 p.m., pengcheng xiong wrote: > > ---

Re: Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

2015-12-18 Thread John Pullokkaran
/HiveReduceExpressionsRule.java (line 320) <https://reviews.apache.org/r/40865/#comment171507> Rename class? - John Pullokkaran On Dec. 19, 2015, 12:28 a.m., pengcheng xiong wrote: > > --- > This is an automatically gener

Re: Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

2015-12-18 Thread John Pullokkaran
ious hepPlan and check for null ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java (line 1267) <https://reviews.apache.org/r/40865/#comment171505> Could we add executor to previous hepPlan and check for null - John Pullokkaran On Dec. 19, 2015, 12:28 a.m., pen

Re: Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

2015-12-18 Thread John Pullokkaran
/translator/RexNodeConverter.java (line 334) <https://reviews.apache.org/r/40865/#comment171500> Hive's null literal has void type. This is handled down in void switch. So we might want to switch calcite null literal to void. - John Pullokkaran On Dec. 19, 2015, 12:28 a.m.

Re: Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

2015-12-18 Thread John Pullokkaran
he.org/r/40865/#comment171494> Why add milli seconds? - John Pullokkaran On Dec. 19, 2015, 12:28 a.m., pengcheng xiong wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https

Re: Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

2015-12-18 Thread John Pullokkaran
<https://reviews.apache.org/r/40865/#comment171490> Why are we looking at operands arguments? - John Pullokkaran On Dec. 19, 2015, 12:28 a.m., pengcheng xiong wrote: > > --- > This is an automatically generated e-mail. To

Re: Review Request 40589: Implement limit push down through union all in CBO

2015-12-18 Thread John Pullokkaran
if union's input is synthetic limit) then i think we should remove it. May be we should enhance HiveSortRemoveRule to look for pattern SortRel-union-SortRel and remove synthetic SortRel below union if it has sort in it. Otherwise we end up paying double sorting cost. - John P

Re: Review Request 41457: HIVE-11865

2015-12-18 Thread John Pullokkaran
167) <https://reviews.apache.org/r/41457/#comment171356> This is fragile. State will have to be maintained; i.e unset if a filter merge happened. - John Pullokkaran On Dec. 18, 2015, 5:28 p.m., Jesús Camacho Rodríguez

Re: Review Request 41457: HIVE-11865

2015-12-18 Thread John Pullokkaran
(line 491) <https://reviews.apache.org/r/41457/#comment171355> This is fragile; i.e reusing the flag whether to run CBO or not for whether CBO was actually run successfully. We already keep state in parsecontext about this. - John Pullokkaran On Dec. 18, 2015, 5:28 p.m.,

Re: Review Request 41457: HIVE-11865

2015-12-18 Thread John Pullokkaran
izer/calcite/rules/HiveSortProjectTransposeRule.java (line 10) <https://reviews.apache.org/r/41457/#comment171350> These seems cosmetic, do we need these? - John Pullokkaran On Dec. 18, 2015, 5:28 p.m., Jesús Camacho Rodríguez wrote: > > --

Re: Review Request 41457: HIVE-11865

2015-12-18 Thread John Pullokkaran
(line 110) <https://reviews.apache.org/r/41457/#comment171344> In the CBO Success path the additional const prop seems unnecessary. All we have done between two const prop is is synthetic join predicates (not null predicates) && Simple predicate push down. - John Pullokkara

Re: Review Request 40589: Implement limit push down through union all in CBO

2015-12-17 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/40589/#review111083 --- Ship it! Ship It! - John Pullokkaran On Dec. 17, 2015, 10:47

Re: Review Request 40589: Implement limit push down through union all in CBO

2015-12-17 Thread John Pullokkaran
ld really be if ((RexLiteral.intValue(sort.offset) + RexLiteral.intValue(sort.fetch)) < RelMetadataQuery.getRowCount(input)) { } - John Pullokkaran On Dec. 17, 2015, 10:09 p.m., pengcheng xiong wrote: > > --- >

Re: Review Request 40589: Implement limit push down through union all in CBO

2015-12-17 Thread John Pullokkaran
he flag that signifies its use ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortUnionReduceRule.java (line 106) <https://reviews.apache.org/r/40589/#comment171019> Shouldn't this be < instead of >=. - John Pullokkaran On Dec. 16, 2015, 6:4

Re: Review Request 40589: Implement limit push down through union all in CBO

2015-12-17 Thread John Pullokkaran
e.java (line 94) <https://reviews.apache.org/r/40589/#comment171013> In case of pure oder by fetch will be null. Wouldn't "RexLiteral.intValue(sort.fetch)" result in an NPE? - John Pullokkaran On Dec. 16, 2015, 6:46

Re: Review Request 41457: HIVE-11865

2015-12-16 Thread John Pullokkaran
(line 104) <https://reviews.apache.org/r/41457/#comment170852> What about const propagation? - John Pullokkaran On Dec. 16, 2015, 7:14 p.m., Jesús Camacho Rodríguez wrote: > > --- > This is an automatically generated e

Re: Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

2015-12-16 Thread John Pullokkaran
stack trace? Log it instead ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveReduceExpressionsRule.java (line 235) <https://reviews.apache.org/r/40865/#comment170849> Nit Pick: space - John Pullokkaran On Dec. 17, 2015, 1:26 a.m., pen

Re: Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

2015-12-16 Thread John Pullokkaran
/HiveRelFactories.java (line 41) <https://reviews.apache.org/r/40865/#comment170845> Remove trailing space. - John Pullokkaran On Dec. 17, 2015, 1:26 a.m., pengcheng xiong wrote: > > --- > This is an automatically generated e

Re: Review Request 40865: Implement/Enable constant related optimization rules in Calcite: enable HiveReduceExpressionsRule to fold constants

2015-12-16 Thread John Pullokkaran
/HiveRelFactories.java (line 49) <https://reviews.apache.org/r/40865/#comment170843> What about rest of the factories? - John Pullokkaran On Dec. 17, 2015, 1:26 a.m., pengcheng xiong wrote: > > --- > This is an automatically g

Re: Review Request 41387: HIVE-12675: PerfLogger should log performance metrics at debug level

2015-12-16 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41387/#review110748 --- Ship it! Ship It! - John Pullokkaran On Dec. 15, 2015, 2:26

Re: Review Request 40589: Implement limit push down through union all in CBO

2015-12-14 Thread John Pullokkaran
(line 1071) <https://reviews.apache.org/r/40589/#comment170171> This should be added to previous hepplan invocation so that limit can be pushed as low as possible. Ex: Limit-Sel-Union-Join-Proj-TS1 TS2 - John Pullokkaran On Dec. 14, 2015, 6:

Re: Review Request 41051: HIVE-12570 : Incorrect error message Expression not in GROUP BY key thrown instead of Invalid function

2015-12-11 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41051/#review110061 --- Ship it! Ship It! - John Pullokkaran On Dec. 12, 2015, 12:53

Re: Review Request 41051: HIVE-12570 : Incorrect error message Expression not in GROUP BY key thrown instead of Invalid function

2015-12-11 Thread John Pullokkaran
(line 539) <https://reviews.apache.org/r/41051/#comment169827> Why don't we use functionName instead of expressionTree.getChild(0) ? - John Pullokkaran On Dec. 7, 2015, 9:11 p.m., Hari Sankar Sivarama Subrama

Re: Review Request 41051: HIVE-12570 : Incorrect error message Expression not in GROUP BY key thrown instead of Invalid function

2015-12-11 Thread John Pullokkaran
(line 537) <https://reviews.apache.org/r/41051/#comment169791> What about Window Functions that is not valid? Check gen plan if it handles it correctly. If not then that should also be covered. ex: select average(x) over() from t1; - John Pullokkaran On Dec. 7, 2015

[jira] [Created] (HIVE-12590) Repeated UDAFs with literals can produce incorrect result

2015-12-03 Thread Laljo John Pullokkaran (JIRA)
Laljo John Pullokkaran created HIVE-12590: - Summary: Repeated UDAFs with literals can produce incorrect result Key: HIVE-12590 URL: https://issues.apache.org/jira/browse/HIVE-12590 Project

Re: Review Request 40589: Implement limit push down through union all in CBO

2015-12-02 Thread John Pullokkaran
/HiveSortUnionReduceRule.java (line 110) <https://reviews.apache.org/r/40589/#comment168155> what if the previous limit got pushed past project/join due to limitjointranspose rules. Then this will fire in a loop. Limit-Union-Proj-Limit - John Pullokkaran On Nov. 23, 2

Re: Review Request 40589: Implement limit push down through union all in CBO

2015-12-02 Thread John Pullokkaran
ould then add this rule to where limit push through join is. - John Pullokkaran On Nov. 23, 2015, 5:59 a.m., pengcheng xiong wrote: > > --- > This is an automatically generated e-mail. To reply, visit: &

[jira] [Created] (HIVE-12559) Vectorization on MR produces different results

2015-12-01 Thread Laljo John Pullokkaran (JIRA)
Laljo John Pullokkaran created HIVE-12559: - Summary: Vectorization on MR produces different results Key: HIVE-12559 URL: https://issues.apache.org/jira/browse/HIVE-12559 Project: Hive

[jira] [Created] (HIVE-12524) SemiJoinProjectTranspose Rule in CBO is not firing

2015-11-25 Thread Laljo John Pullokkaran (JIRA)
Laljo John Pullokkaran created HIVE-12524: - Summary: SemiJoinProjectTranspose Rule in CBO is not firing Key: HIVE-12524 URL: https://issues.apache.org/jira/browse/HIVE-12524 Project: Hive

Re: [ANNOUNCE] New PMC Member : John Pullokkaran

2015-11-25 Thread John Pullokkaran
>Congrats John! > > >On Tue, Nov 24, 2015 at 4:41 PM, Pengcheng Xiong >wrote: >> Congrats! John! >> >> On Wed, Nov 25, 2015 at 6:59 AM, Ashutosh Chauhan >> wrote: >> >>> On behalf of the Hive PMC I am delighted to announce John Pullokka

Re: Review Request 38211: HIVE-11684

2015-11-20 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38211/#review107412 --- Ship it! Ship It! - John Pullokkaran On Nov. 18, 2015, 10:40

Re: Review Request 40359: HIVE-11110 Cost Based Optimizer improvements

2015-11-20 Thread John Pullokkaran
This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/40359/#review106759 ------- On Nov. 16, 2015, 6:54 p.m., John Pullokkaran wrote: > > -

[jira] [Created] (HIVE-12478) Improve Hive/Calcite Trasitive Predicate inference

2015-11-20 Thread Laljo John Pullokkaran (JIRA)
Laljo John Pullokkaran created HIVE-12478: - Summary: Improve Hive/Calcite Trasitive Predicate inference Key: HIVE-12478 URL: https://issues.apache.org/jira/browse/HIVE-12478 Project: Hive

Re: Review Request 40359: HIVE-11110 Cost Based Optimizer improvements

2015-11-19 Thread John Pullokkaran
ply, visit: https://reviews.apache.org/r/40359/#review106759 --- On Nov. 16, 2015, 6:54 p.m., John Pullokkaran wrote: > > --- > This is an automatically generated e-mail

Re: Review Request 40359: HIVE-11110 Cost Based Optimizer improvements

2015-11-16 Thread John Pullokkaran
- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/40359/#review106736 --- On Nov. 16, 2015, 6:54 p.m., John Pullokkaran wrote: > > -

Re: Review Request 40359: HIVE-11110 Cost Based Optimizer improvements

2015-11-16 Thread John Pullokkaran
an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/40359/#review106736 --- On Nov. 16, 2015, 6:54 p.m., John Pullokkaran wrote: > > ---

[jira] [Created] (HIVE-12423) Remove Union Operator in Physical Compiler

2015-11-16 Thread Laljo John Pullokkaran (JIRA)
Laljo John Pullokkaran created HIVE-12423: - Summary: Remove Union Operator in Physical Compiler Key: HIVE-12423 URL: https://issues.apache.org/jira/browse/HIVE-12423 Project: Hive

Review Request 40359: HIVE-11110 Cost Based Optimizer improvements

2015-11-16 Thread John Pullokkaran
/vectorized_shufflejoin.q.out f32cc38 Diff: https://reviews.apache.org/r/40359/diff/ Testing --- HIVE QA Thanks, John Pullokkaran

Re: Review Request 38211: HIVE-11684

2015-11-12 Thread John Pullokkaran
e the trait from project input. Practically Once sort is introduced only ordering trait can survive. - John Pullokkaran On Nov. 12, 2015, 10:47 a.m., Jesús Camacho Rodríguez wrote: > > --- > This is an automatically generated e-m

Re: Review Request 38211: HIVE-11684

2015-11-12 Thread John Pullokkaran
/HiveSortMergeRule.java (line 56) <https://reviews.apache.org/r/38211/#comment165064> Don't you need to merge the collation from top? Otherwise you will top OB. - John Pullokkaran On Nov. 12, 2015, 10:47 a.m., Jesús Camacho Rodr

[jira] [Created] (HIVE-12384) Union Operator may produce incorrect result on TEZ

2015-11-11 Thread Laljo John Pullokkaran (JIRA)
Laljo John Pullokkaran created HIVE-12384: - Summary: Union Operator may produce incorrect result on TEZ Key: HIVE-12384 URL: https://issues.apache.org/jira/browse/HIVE-12384 Project: Hive

Re: Review Request 39425: Invalid column ref error in order by when using column alias in select clause and using having

2015-11-10 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/39425/#review106005 --- Ship it! Ship It! - John Pullokkaran On Oct. 18, 2015, 9:47

[jira] [Created] (HIVE-12361) Hive 0.14: IndexOutOfBoundsException thrown when accessing a union all subquery and filtering on a column which does not exist in all underlying tables

2015-11-06 Thread Laljo John Pullokkaran (JIRA)
Laljo John Pullokkaran created HIVE-12361: - Summary: Hive 0.14: IndexOutOfBoundsException thrown when accessing a union all subquery and filtering on a column which does not exist in all underlying tables Key: HIVE-12361

[jira] [Created] (HIVE-12356) Capture if a rule mutated the plan.

2015-11-05 Thread Laljo John Pullokkaran (JIRA)
Laljo John Pullokkaran created HIVE-12356: - Summary: Capture if a rule mutated the plan. Key: HIVE-12356 URL: https://issues.apache.org/jira/browse/HIVE-12356 Project: Hive Issue

[jira] [Created] (HIVE-12355) Keep Obj Inspectors in Sync with RowSchema

2015-11-05 Thread Laljo John Pullokkaran (JIRA)
Laljo John Pullokkaran created HIVE-12355: - Summary: Keep Obj Inspectors in Sync with RowSchema Key: HIVE-12355 URL: https://issues.apache.org/jira/browse/HIVE-12355 Project: Hive

Re: Review Request 39872: expanding view doesn't quote reserved keyword

2015-11-04 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/39872/#review105120 --- Ship it! Ship It! - John Pullokkaran On Nov. 2, 2015, 7:25 p.m

Re: Review Request 37778: HIVE-11634

2015-11-02 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37778/#review104741 --- Ship it! - John Pullokkaran On Oct. 29, 2015, 11:32 p.m., Hari

Re: Review Request 37778: HIVE-11634

2015-11-02 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37778/#review104740 --- Ship it! Ship It! - John Pullokkaran On Oct. 29, 2015, 11:32

Re: Review Request 37778: HIVE-11634

2015-10-29 Thread John Pullokkaran
ake care of generating appropriate struct col names.. - John Pullokkaran On Oct. 29, 2015, 11:32 p.m., Hari Sankar Sivarama Subramaniyan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://re

Re: Review Request 39425: Invalid column ref error in order by when using column alias in select clause and using having

2015-10-26 Thread John Pullokkaran
y key, value having c1 like '%abc%' order by c0; With this patch it will let this query through which is obviously wrong. - John Pullokkaran On Oct. 18, 2015, 9:47 p.m., pengcheng xiong wrote: > > --- > This is an

Re: Review Request 38211: HIVE-11684

2015-10-22 Thread John Pullokkaran
(line 962) <https://reviews.apache.org/r/38211/#comment161786> Pushing just limit without pushing orderby is not sufficent. Limit almost always will have preceeding order by. Order by pushdown also nneds to be handled. - John Pullokkaran On Oct. 23, 2015, 3:21 a.m.,

Re: Review Request 38211: HIVE-11684

2015-10-22 Thread John Pullokkaran
1307) <https://reviews.apache.org/r/38211/#comment161785> May be all the config should be changed to LimitPushDown instead of using join; since limit can be pushed down below outer joins, projects, union, Map Side GB... - John Pullokkaran On Oct. 23, 2015, 3:21 a.m., Jesús C

Re: Review Request 38211: HIVE-11684

2015-10-22 Thread John Pullokkaran
/HiveRelMdRowCount.java (line 39) <https://reviews.apache.org/r/38211/#comment161734> We should enhance Join cardinality estimation for outer joins. Outer Join cardinality should be: max(outer side cardinality, inner join cardinality) - John Pullokkaran On Oct. 11, 2015, 12:

Re: Review Request 38211: HIVE-11684

2015-10-22 Thread John Pullokkaran
(line 997) <https://reviews.apache.org/r/38211/#comment161733> Redundant Limit removal should happen bottom up. - John Pullokkaran On Oct. 11, 2015, 12:57 p.m., Jesús Camacho Rodríguez wrote: > > --- > This is a

Re: Review Request 39199: HIVE-12084 : Hive queries with ORDER BY and large LIMIT fails with OutOfMemoryError Java heap space

2015-10-22 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/39199/#review103649 --- Ship it! Ship It! - John Pullokkaran On Oct. 22, 2015, 12:41

Re: Review Request 39199: HIVE-12084 : Hive queries with ORDER BY and large LIMIT fails with OutOfMemoryError Java heap space

2015-10-14 Thread John Pullokkaran
tps://reviews.apache.org/r/39199/#comment160467> We may be better of by flushing the hash table if we can not fit in memory; this logic can be very similar to map side GB. - John Pullokkaran On Oct. 14, 2015, 6:10 p.m., Hari Sankar Sivarama Subramaniyan

Re: Review Request 39199: HIVE-12084 : Hive queries with ORDER BY and large LIMIT fails with OutOfMemoryError Java heap space

2015-10-14 Thread John Pullokkaran
ng to store TopN. If col stats is available then we should use it. Also we need to handle composite keys. If stats are not available then may be a configurable heuristics may be an option; i.e lets say NDV is 10% of cardinality. - John Pullokkaran On Oct. 14, 2015, 6:10 p.m., Hari

Re: Review Request 39199: HIVE-12084 : Hive queries with ORDER BY and large LIMIT fails with OutOfMemoryError Java heap space

2015-10-14 Thread John Pullokkaran
/ReduceSinkOperator.java (line 525) <https://reviews.apache.org/r/39199/#comment160464> This should be in ReduceSinkDesc since thats the one that has knowledge of reducer function - John Pullokkaran On Oct. 14, 2015, 6:10 p.m., Hari Sankar Sivarama Subramaniyan

Re: Review Request 39199: HIVE-12084 : Hive queries with ORDER BY and large LIMIT fails with OutOfMemoryError Java heap space

2015-10-13 Thread John Pullokkaran
n TopNHash use memory required from #1 - John Pullokkaran On Oct. 13, 2015, 7:37 p.m., Hari Sankar Sivarama Subramaniyan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://

Re: Review Request 38211: HIVE-11684

2015-10-08 Thread John Pullokkaran
t reduction criteria. Also the reduction criteria probably needs to specify the minimun no of tuples to reduce for limit to be cost effective (this should go away once we have execution cost model). - John Pullokkaran On Sept. 28, 2015, 11:40 a.m., Jesús Camacho Rodr

Re: Review Request 38211: HIVE-11684

2015-10-07 Thread John Pullokkaran
/HiveSortJoinReduceRule.java (line 80) <https://reviews.apache.org/r/38211/#comment159364> "reduction" as it is set is not a %. Shouldn't you multiply by ((100-reduction)/100) guarding for reduction to be <= 100. - John Pullokkaran On Sept. 28, 2015, 1

Re: Review Request 37778: HIVE-11634

2015-10-07 Thread John Pullokkaran
/PcrExprProcFactory.java (line 392) <https://reviews.apache.org/r/37778/#comment159360> This too should be unknown right? - John Pullokkaran On Oct. 7, 2015, 10:57 p.m., Hari Sankar Sivarama Subramaniyan wrote: > > --- > This is a

Re: Review Request 37778: HIVE-11634

2015-10-07 Thread John Pullokkaran
/PcrExprProcFactory.java (line 385) <https://reviews.apache.org/r/37778/#comment159310> Add unit test cases for: 1. DNF(CNF) expr1 or expr2 (where expr2 is conjuctine tree) 2. In clause involving part/non-part col that are combined by disjunction - John Pullokkaran On Oct. 7, 2015, 7:

Re: Review Request 37778: HIVE-11634

2015-10-07 Thread John Pullokkaran
/PcrExprProcFactory.java (line 384) <https://reviews.apache.org/r/37778/#comment159305> WALKERSTATE should be UNKNOWN. - John Pullokkaran On Oct. 7, 2015, 7:47 p.m., Hari Sankar Sivarama Subramaniyan wrote: > > --- > This is a

Re: Review Request 37778: HIVE-11634

2015-10-07 Thread John Pullokkaran
Add a test case involving conditional expressions (case, If..) Ex: select key from t1 where (case when key=1 then 10 else 20 end) in (10,11); - John Pullokkaran On Oct. 1, 2015, 11:42 p.m., Hari Sankar Sivarama Subramaniyan wrote: > > --

Re: Review Request 38940: unable to create temporary table using CTAS if regular table with that name already exists

2015-10-05 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38940/#review101559 --- Ship it! Ship It! - John Pullokkaran On Oct. 1, 2015, 10:50

Review Request 39004: HIVE-11919: Union type during Union OP inits may choose type different than decided during plan gen.

2015-10-05 Thread John Pullokkaran
/clientpositive/union36.q.out PRE-CREATION Diff: https://reviews.apache.org/r/39004/diff/ Testing --- QA Run Added unit tests Thanks, John Pullokkaran

Re: Review Request 37778: HIVE-11634

2015-10-01 Thread John Pullokkaran
/PcrExprProcFactory.java (line 372) <https://reviews.apache.org/r/37778/#comment158671> 1. The check "children.size() < 2" is either unnecessary or we have a bug some where else. 2. We should also remove single col in clause where col is partition col - John Pullokkaran On Oct.

Re: Review Request 38503: Support special characters in quoted table names

2015-09-30 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38503/#review101162 --- Ship it! Ship It! - John Pullokkaran On Sept. 30, 2015, 9:57

Re: Review Request 38503: Support special characters in quoted table names

2015-09-29 Thread John Pullokkaran
/MetaStoreUtils.java (line 1737) <https://reviews.apache.org/r/38503/#comment158353> Make a note that unicode is not supported in table names. And we have explicit checks for it. - John Pullokkaran On Sept. 22, 2015, 6:12 a.m., pengcheng xiong

Re: Review Request 38503: Support special characters in quoted table names

2015-09-29 Thread John Pullokkaran
2177) <https://reviews.apache.org/r/38503/#comment158352> We should list what special chars are supported; may be by accessing MetaStoreUtils field. - John Pullokkaran On Sept. 22, 2015, 6:12 a.m., pengcheng xiong

Re: Review Request 38768: HIVE-11964: RelOptHiveTable.hiveColStatsMap might contain mismatched column stats

2015-09-29 Thread John Pullokkaran
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38768/#review101012 --- Ship it! Ship It! - John Pullokkaran On Sept. 25, 2015, 6:37

Re: Review Request 38768: HIVE-11964: RelOptHiveTable.hiveColStatsMap might contain mismatched column stats

2015-09-25 Thread John Pullokkaran
/RelOptHiveTable.java (line 315) <https://reviews.apache.org/r/38768/#comment157948> yes. - John Pullokkaran On Sept. 25, 2015, 6:37 p.m., Chaoyu Tang wrote: > > --- > This is an automatically generated e-mail. To reply,

Re: Review Request 38768: HIVE-11964: RelOptHiveTable.hiveColStatsMap might contain mismatched column stats

2015-09-25 Thread John Pullokkaran
> On Sept. 25, 2015, 6:55 p.m., John Pullokkaran wrote: > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java, > > line 316 > > <https://reviews.apache.org/r/38768/diff/1/?file=1084857#file1084857line316> > > > > Also

Re: Review Request 38768: HIVE-11964: RelOptHiveTable.hiveColStatsMap might contain mismatched column stats

2015-09-25 Thread John Pullokkaran
> On Sept. 25, 2015, 6:52 p.m., John Pullokkaran wrote: > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java, > > line 315 > > <https://reviews.apache.org/r/38768/diff/1/?file=1084857#file1084857line315> > > > > Shouldn&#

Re: Review Request 38768: HIVE-11964: RelOptHiveTable.hiveColStatsMap might contain mismatched column stats

2015-09-25 Thread John Pullokkaran
ter to fix stats API to guarantee the order. - John Pullokkaran On Sept. 25, 2015, 6:37 p.m., Chaoyu Tang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://revi

Re: Review Request 38768: HIVE-11964: RelOptHiveTable.hiveColStatsMap might contain mismatched column stats

2015-09-25 Thread John Pullokkaran
/RelOptHiveTable.java (line 315) <https://reviews.apache.org/r/38768/#comment157883> Shouldn't we fix the API to provide the stats in the same order as it was given? In this case cols are given in the same order as it appears in the table. - John Pullokkaran On Sept. 25, 201

Re: Review Request 38211: HIVE-11684

2015-09-24 Thread John Pullokkaran
/HiveSortJoinReduceRule.java (line 172) <https://reviews.apache.org/r/38211/#comment157655> We should seperate Limit push through Project from Limit Push through Join. Looking for Limit/Proj/Join is a hard coupling. - John Pullokkaran On Sept. 17, 2015, 1:55 p.m., Jesús Camacho Rod

Re: Review Request 38211: HIVE-11684

2015-09-24 Thread John Pullokkaran
/HiveSortJoinReduceRule.java (line 73) <https://reviews.apache.org/r/38211/#comment157654> May be reuse HiveCalciteUtil.orderRelNode - John Pullokkaran On Sept. 17, 2015, 1:55 p.m., Jesús Camacho Rodríguez wrote: > > ---

[jira] [Created] (HIVE-11919) Hive Union Type Mismatch

2015-09-22 Thread Laljo John Pullokkaran (JIRA)
Laljo John Pullokkaran created HIVE-11919: - Summary: Hive Union Type Mismatch Key: HIVE-11919 URL: https://issues.apache.org/jira/browse/HIVE-11919 Project: Hive Issue Type: Bug

  1   2   3   4   5   6   7   8   9   10   >