[jira] [Updated] (HIVE-7310) Turning CBO on results in NPE on some queries

2014-06-27 Thread Gunther Hagleitner (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-7310:
-

Labels: cbo  (was: )

 Turning CBO on results in NPE on some queries
 -

 Key: HIVE-7310
 URL: https://issues.apache.org/jira/browse/HIVE-7310
 Project: Hive
  Issue Type: Sub-task
Reporter: Gunther Hagleitner
Assignee: Laljo John Pullokkaran
  Labels: cbo

 On the CBO branch if I do the following:
 hive set hive.cbo.enable=true;
 hive select  i_item_id,   
  s_state, 
  GROUPING__ID,
  avg(ss_quantity) agg1,
  avg(ss_list_price) agg2,
  avg(ss_coupon_amt) agg3,
  avg(ss_sales_price) agg4
  from store_sales ss
  join customer_demographics cd on (ss.ss_cdemo_sk = cd.cd_demo_sk)
  join date_dim d on (ss.ss_sold_date_sk = d.d_date_sk)
  join store s on (ss.ss_store_sk = s.s_store_sk)
  join item i on (ss.ss_item_sk = i.i_item_sk)
  where cd_gender = 'M' and
 cd_marital_status = 'S' and
 cd_education_status = 'Secondary' and
 d_year = 2002 and
 s_state in ('OH','SD', 'LA', 'MO', 'WA', 'MN')
  group by i_item_id, s_state with rollup
  order by i_item_id
   ,s_state
  limit 100
  ;
 I get an NPE. The stack trace is:
 java.lang.RuntimeException: java.lang.RuntimeException: 
 java.lang.RuntimeException: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9555)
   at 
 org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:328)
   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:412)
   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
   at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:962)
   at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1027)
   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:898)
   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:888)
   at 
 org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:277)
   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:229)
   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:439)
   at 
 org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:812)
   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:706)
   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 Caused by: java.lang.RuntimeException: java.lang.RuntimeException: 
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getOptimizedAST(SemanticAnalyzer.java:11732)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.access$200(SemanticAnalyzer.java:11711)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9532)
   ... 18 more
 Caused by: java.lang.RuntimeException: java.lang.NullPointerException
   at 
 net.hydromatic.optiq.tools.Frameworks.withPrepare(Frameworks.java:170)
   at 
 net.hydromatic.optiq.tools.Frameworks.withPlanner(Frameworks.java:142)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getOptimizedAST(SemanticAnalyzer.java:11727)
   ... 20 more
 Caused by: java.lang.NullPointerException
   at 
 org.eigenbase.reltype.RelDataTypeImpl.getField(RelDataTypeImpl.java:79)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.convertAgg(SemanticAnalyzer.java:12129)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genGBRelNode(SemanticAnalyzer.java:12184)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genGBLogicalPlan(SemanticAnalyzer.java:12324)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genLogicalPlan(SemanticAnalyzer.java:12749)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.apply(SemanticAnalyzer.java:11758)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.apply(SemanticAnalyzer.java:11711)
   at net.hydromatic.optiq.tools.Frameworks$1.apply(Frameworks.java:146)
   at 
 

[jira] [Updated] (HIVE-7310) Turning CBO on results in NPE on some queries

2014-06-27 Thread Laljo John Pullokkaran (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-7310:
-

Attachment: HIVE-7310.patch

 Turning CBO on results in NPE on some queries
 -

 Key: HIVE-7310
 URL: https://issues.apache.org/jira/browse/HIVE-7310
 Project: Hive
  Issue Type: Sub-task
Reporter: Gunther Hagleitner
Assignee: Laljo John Pullokkaran
  Labels: cbo
 Attachments: HIVE-7310.patch


 On the CBO branch if I do the following:
 hive set hive.cbo.enable=true;
 hive select  i_item_id,   
  s_state, 
  GROUPING__ID,
  avg(ss_quantity) agg1,
  avg(ss_list_price) agg2,
  avg(ss_coupon_amt) agg3,
  avg(ss_sales_price) agg4
  from store_sales ss
  join customer_demographics cd on (ss.ss_cdemo_sk = cd.cd_demo_sk)
  join date_dim d on (ss.ss_sold_date_sk = d.d_date_sk)
  join store s on (ss.ss_store_sk = s.s_store_sk)
  join item i on (ss.ss_item_sk = i.i_item_sk)
  where cd_gender = 'M' and
 cd_marital_status = 'S' and
 cd_education_status = 'Secondary' and
 d_year = 2002 and
 s_state in ('OH','SD', 'LA', 'MO', 'WA', 'MN')
  group by i_item_id, s_state with rollup
  order by i_item_id
   ,s_state
  limit 100
  ;
 I get an NPE. The stack trace is:
 java.lang.RuntimeException: java.lang.RuntimeException: 
 java.lang.RuntimeException: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9555)
   at 
 org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:328)
   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:412)
   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
   at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:962)
   at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1027)
   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:898)
   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:888)
   at 
 org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:277)
   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:229)
   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:439)
   at 
 org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:812)
   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:706)
   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 Caused by: java.lang.RuntimeException: java.lang.RuntimeException: 
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getOptimizedAST(SemanticAnalyzer.java:11732)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.access$200(SemanticAnalyzer.java:11711)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9532)
   ... 18 more
 Caused by: java.lang.RuntimeException: java.lang.NullPointerException
   at 
 net.hydromatic.optiq.tools.Frameworks.withPrepare(Frameworks.java:170)
   at 
 net.hydromatic.optiq.tools.Frameworks.withPlanner(Frameworks.java:142)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getOptimizedAST(SemanticAnalyzer.java:11727)
   ... 20 more
 Caused by: java.lang.NullPointerException
   at 
 org.eigenbase.reltype.RelDataTypeImpl.getField(RelDataTypeImpl.java:79)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.convertAgg(SemanticAnalyzer.java:12129)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genGBRelNode(SemanticAnalyzer.java:12184)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genGBLogicalPlan(SemanticAnalyzer.java:12324)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genLogicalPlan(SemanticAnalyzer.java:12749)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.apply(SemanticAnalyzer.java:11758)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.apply(SemanticAnalyzer.java:11711)
   at net.hydromatic.optiq.tools.Frameworks$1.apply(Frameworks.java:146)
   

[jira] [Updated] (HIVE-7310) Turning CBO on results in NPE on some queries

2014-06-27 Thread Laljo John Pullokkaran (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-7310:
-

Status: Patch Available  (was: Open)

 Turning CBO on results in NPE on some queries
 -

 Key: HIVE-7310
 URL: https://issues.apache.org/jira/browse/HIVE-7310
 Project: Hive
  Issue Type: Sub-task
Reporter: Gunther Hagleitner
Assignee: Laljo John Pullokkaran
  Labels: cbo
 Attachments: HIVE-7310.patch


 On the CBO branch if I do the following:
 hive set hive.cbo.enable=true;
 hive select  i_item_id,   
  s_state, 
  GROUPING__ID,
  avg(ss_quantity) agg1,
  avg(ss_list_price) agg2,
  avg(ss_coupon_amt) agg3,
  avg(ss_sales_price) agg4
  from store_sales ss
  join customer_demographics cd on (ss.ss_cdemo_sk = cd.cd_demo_sk)
  join date_dim d on (ss.ss_sold_date_sk = d.d_date_sk)
  join store s on (ss.ss_store_sk = s.s_store_sk)
  join item i on (ss.ss_item_sk = i.i_item_sk)
  where cd_gender = 'M' and
 cd_marital_status = 'S' and
 cd_education_status = 'Secondary' and
 d_year = 2002 and
 s_state in ('OH','SD', 'LA', 'MO', 'WA', 'MN')
  group by i_item_id, s_state with rollup
  order by i_item_id
   ,s_state
  limit 100
  ;
 I get an NPE. The stack trace is:
 java.lang.RuntimeException: java.lang.RuntimeException: 
 java.lang.RuntimeException: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9555)
   at 
 org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:328)
   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:412)
   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
   at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:962)
   at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1027)
   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:898)
   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:888)
   at 
 org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:277)
   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:229)
   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:439)
   at 
 org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:812)
   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:706)
   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 Caused by: java.lang.RuntimeException: java.lang.RuntimeException: 
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getOptimizedAST(SemanticAnalyzer.java:11732)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.access$200(SemanticAnalyzer.java:11711)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9532)
   ... 18 more
 Caused by: java.lang.RuntimeException: java.lang.NullPointerException
   at 
 net.hydromatic.optiq.tools.Frameworks.withPrepare(Frameworks.java:170)
   at 
 net.hydromatic.optiq.tools.Frameworks.withPlanner(Frameworks.java:142)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getOptimizedAST(SemanticAnalyzer.java:11727)
   ... 20 more
 Caused by: java.lang.NullPointerException
   at 
 org.eigenbase.reltype.RelDataTypeImpl.getField(RelDataTypeImpl.java:79)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.convertAgg(SemanticAnalyzer.java:12129)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genGBRelNode(SemanticAnalyzer.java:12184)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genGBLogicalPlan(SemanticAnalyzer.java:12324)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genLogicalPlan(SemanticAnalyzer.java:12749)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.apply(SemanticAnalyzer.java:11758)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.apply(SemanticAnalyzer.java:11711)
   at 

[jira] [Updated] (HIVE-7310) Turning CBO on results in NPE on some queries

2014-06-27 Thread Gunther Hagleitner (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-7310:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to cbo branch. Thanks [~jpullokkaran]!

 Turning CBO on results in NPE on some queries
 -

 Key: HIVE-7310
 URL: https://issues.apache.org/jira/browse/HIVE-7310
 Project: Hive
  Issue Type: Sub-task
Reporter: Gunther Hagleitner
Assignee: Laljo John Pullokkaran
  Labels: cbo
 Attachments: HIVE-7310.patch


 On the CBO branch if I do the following:
 hive set hive.cbo.enable=true;
 hive select  i_item_id,   
  s_state, 
  GROUPING__ID,
  avg(ss_quantity) agg1,
  avg(ss_list_price) agg2,
  avg(ss_coupon_amt) agg3,
  avg(ss_sales_price) agg4
  from store_sales ss
  join customer_demographics cd on (ss.ss_cdemo_sk = cd.cd_demo_sk)
  join date_dim d on (ss.ss_sold_date_sk = d.d_date_sk)
  join store s on (ss.ss_store_sk = s.s_store_sk)
  join item i on (ss.ss_item_sk = i.i_item_sk)
  where cd_gender = 'M' and
 cd_marital_status = 'S' and
 cd_education_status = 'Secondary' and
 d_year = 2002 and
 s_state in ('OH','SD', 'LA', 'MO', 'WA', 'MN')
  group by i_item_id, s_state with rollup
  order by i_item_id
   ,s_state
  limit 100
  ;
 I get an NPE. The stack trace is:
 java.lang.RuntimeException: java.lang.RuntimeException: 
 java.lang.RuntimeException: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9555)
   at 
 org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:328)
   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:412)
   at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
   at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:962)
   at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1027)
   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:898)
   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:888)
   at 
 org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:277)
   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:229)
   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:439)
   at 
 org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:812)
   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:706)
   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 Caused by: java.lang.RuntimeException: java.lang.RuntimeException: 
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getOptimizedAST(SemanticAnalyzer.java:11732)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.access$200(SemanticAnalyzer.java:11711)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9532)
   ... 18 more
 Caused by: java.lang.RuntimeException: java.lang.NullPointerException
   at 
 net.hydromatic.optiq.tools.Frameworks.withPrepare(Frameworks.java:170)
   at 
 net.hydromatic.optiq.tools.Frameworks.withPlanner(Frameworks.java:142)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getOptimizedAST(SemanticAnalyzer.java:11727)
   ... 20 more
 Caused by: java.lang.NullPointerException
   at 
 org.eigenbase.reltype.RelDataTypeImpl.getField(RelDataTypeImpl.java:79)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.convertAgg(SemanticAnalyzer.java:12129)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genGBRelNode(SemanticAnalyzer.java:12184)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genGBLogicalPlan(SemanticAnalyzer.java:12324)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genLogicalPlan(SemanticAnalyzer.java:12749)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.apply(SemanticAnalyzer.java:11758)
   at 
 org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.apply(SemanticAnalyzer.java:11711)