[jira] [Updated] (HIVE-2848) Add validation to HiveConf ConfVars
[ https://issues.apache.org/jira/browse/HIVE-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2848: Status: Patch Available (was: Open) Add validation to HiveConf ConfVars --- Key: HIVE-2848 URL: https://issues.apache.org/jira/browse/HIVE-2848 Project: Hive Issue Type: Improvement Affects Versions: 0.9.0 Reporter: Kevin Wilfong Assignee: Navis Attachments: HIVE-2848.D2313.1.patch, HIVE-2848.D2313.2.patch, HIVE-2848.D2313.3.patch, HIVE-2848.D2313.4.patch, HIVE-2848.D2313.5.patch It would be good if we could add some validation to ConfVars in the HiveConf when they are set. At least if a variable is supposed to be boolean, numeric, or strict/nonstrict, it shouldn't be too difficult to check for these. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2958) GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger]
[
https://issues.apache.org/jira/browse/HIVE-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2958:
Status: Patch Available (was: Open)
Passed all tests
GROUP BY causing ClassCastException [LazyDioInteger cannot be cast
LazyInteger]
---
Key: HIVE-2958
URL: https://issues.apache.org/jira/browse/HIVE-2958
Project: Hive
Issue Type: Bug
Components: HBase Handler
Affects Versions: 0.9.0
Environment: HBase 0.90.4, Hive 0.90 snapshot (trunk) built today
Reporter: Tim Robertson
Assignee: Navis
Priority: Blocker
Attachments: HIVE-2958.D2871.1.patch
This relates to https://issues.apache.org/jira/browse/HIVE-1634.
The following work fine:
{code}
CREATE EXTERNAL TABLE tim_hbase_occurrence (
id int,
scientific_name string,
data_resource_id int
) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH
SERDEPROPERTIES (
hbase.columns.mapping = :key#b,v:scientific_name#s,v:data_resource_id#b
) TBLPROPERTIES(
hbase.table.name = mini_occurrences,
hbase.table.default.storage.type = binary
);
SELECT * FROM tim_hbase_occurrence LIMIT 3;
SELECT * FROM tim_hbase_occurrence WHERE data_resource_id=1081 LIMIT 3;
{code}
However, the following fails:
{code}
SELECT data_resource_id, count(*) FROM tim_hbase_occurrence GROUP BY
data_resource_id;
{code}
The error given:
{code}
0 TS
2012-04-17 16:58:45,693 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Initialization Done 7 MAP
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Processing alias tim_hbase_occurrence for file
hdfs://c1n2.gbif.org/user/hive/warehouse/tim_hbase_occurrence
2012-04-17 16:58:45,714 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 7
forwarding 1 rows
2012-04-17 16:58:45,714 INFO
org.apache.hadoop.hive.ql.exec.TableScanOperator: 0 forwarding 1 rows
2012-04-17 16:58:45,716 INFO org.apache.hadoop.hive.ql.exec.SelectOperator: 1
forwarding 1 rows
2012-04-17 16:58:45,723 FATAL ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {id:1444,scientific_name:null,data_resource_id:1081}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to
org.apache.hadoop.hive.serde2.lazy.LazyInteger
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:737)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
... 9 more
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazydio.LazyDioInteger cannot be cast to
org.apache.hadoop.hive.serde2.lazy.LazyInteger
at
org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyIntObjectInspector.copyObject(LazyIntObjectInspector.java:43)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:239)
at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:150)
at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.deepCopyElements(KeyWrapperFactory.java:142)
at
org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.copyKey(KeyWrapperFactory.java:119)
at
[jira] [Updated] (HIVE-2723) should throw Ambiguous column reference key Exception in particular join condition
[ https://issues.apache.org/jira/browse/HIVE-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2723: Affects Version/s: (was: 0.8.0) Status: Patch Available (was: Open) Passed all tests should throw Ambiguous column reference key Exception in particular join condition -- Key: HIVE-2723 URL: https://issues.apache.org/jira/browse/HIVE-2723 Project: Hive Issue Type: Bug Components: Query Processor Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux java version 1.6.0_25 hadoop-0.20.2-cdh3u0 hive-0.7.0-cdh3u0 Reporter: caofangkun Assignee: Navis Priority: Minor Labels: exception-handling, query, queryparser Fix For: 0.9.0 Attachments: HIVE-2723.D1275.1.patch, HIVE-2723.D1275.2.patch This Bug can be Repeated as following : create table test(key string, value string); create table test1(key string, value string); 1: Correct! select t.key from (select a.key, b.key from (select * from src ) a right outer join (select * from src1) b on (a.key = b.key)) t; FAILED: Error in semantic analysis: Ambiguous column reference key 2: Uncorrect!! Should throw Exception as above too! select t.key --Is this a.key or b.key ? It's ambiduous! from (select a.\*, b.\* from (select * from src ) a right outer join (select * from src1) b on (a.value = b.value)) t; Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks not specified. Defaulting to jobconf value of: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=number In order to limit the maximum number of reducers: set hive.exec.reducers.max=number In order to set a constant number of reducers: set mapred.reduce.tasks=number Starting Job = job_201201170959_0004, Tracking URL = http://zongren-VirtualBox:50030/jobdetails.jsp?jobid=job_201201170959_0004 Kill Command = /home/zongren/workspace/hadoop-adh/bin/hadoop job -Dmapred.job.tracker=zongren-VirtualBox:9001 -kill job_201201170959_0004 Hadoop job information for Stage-1: number of mappers: 2; number of reducers: 1 2012-01-17 11:02:47,507 Stage-1 map = 0%, reduce = 0% 2012-01-17 11:02:55,002 Stage-1 map = 100%, reduce = 0% 2012-01-17 11:03:04,240 Stage-1 map = 100%, reduce = 33% 2012-01-17 11:03:05,258 Stage-1 map = 100%, reduce = 100% Ended Job = job_201201170959_0004 MapReduce Jobs Launched: Job 0: Map: 2 Reduce: 1 HDFS Read: 669 HDFS Write: 216 SUCESS Total MapReduce CPU Time Spent: 0 msec OK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2956) [hive] Provide error message when using UDAF in the place of UDF instead of throwing NPE
[
https://issues.apache.org/jira/browse/HIVE-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2956:
Status: Patch Available (was: Open)
Passed all tests
[hive] Provide error message when using UDAF in the place of UDF instead of
throwing NPE
Key: HIVE-2956
URL: https://issues.apache.org/jira/browse/HIVE-2956
Project: Hive
Issue Type: Improvement
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-2956.D2823.1.patch
For example,
{code}
hive select distinct deptno, sum(deptno) from emp;
FAILED: Hive Internal Error: java.lang.NullPointerException(null)
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:214)
at
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:767)
at
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:888)
at
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:88)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:125)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:102)
at
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:165)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:7755)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:7713)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanMapGroupByOperator(SemanticAnalyzer.java:2793)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanMapAggr1MR(SemanticAnalyzer.java:3651)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:6125)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:6762)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7531)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:250)
{code}
Trivial.. but people always reports this confused by esoteric custom function
names.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2955) Queries consists of metadata-only-query returns always empty value
[
https://issues.apache.org/jira/browse/HIVE-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2955:
Status: Patch Available (was: Open)
passed all tests
Queries consists of metadata-only-query returns always empty value
--
Key: HIVE-2955
URL: https://issues.apache.org/jira/browse/HIVE-2955
Project: Hive
Issue Type: Bug
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-2955.D2817.1.patch
For partitioned table, simple query on partition column returns always null
or empty value, for example,
{code}
create table emppart(empno int, ename string) partitioned by (deptno int);
.. load partitions..
select distinct deptno from emppart; // empty
select min(deptno), max(deptno) from emppart; // NULL and NULL
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2937) TestHiveServerSessions hangs when executed directly
[
https://issues.apache.org/jira/browse/HIVE-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2937:
Description:
{code}
ant test -Doffline=true -Dtestcase=TestHiveServerSessions
{code}
Hangs infinitely.
I couldn't imagine exact cause of the problem, but found that by adding 'new
HiveServer.HiveServerHandler();' in setup(), test resulted to success.
was:
code}
ant test -Doffline=true -Dtestcase=TestHiveServerSessions
{code}
Hangs infinitely.
I couldn't imagine exact cause of the problem, but found that by adding 'new
HiveServer.HiveServerHandler();' in setup(), test resulted to success.
TestHiveServerSessions hangs when executed directly
---
Key: HIVE-2937
URL: https://issues.apache.org/jira/browse/HIVE-2937
Project: Hive
Issue Type: Test
Reporter: Navis
Priority: Trivial
{code}
ant test -Doffline=true -Dtestcase=TestHiveServerSessions
{code}
Hangs infinitely.
I couldn't imagine exact cause of the problem, but found that by adding 'new
HiveServer.HiveServerHandler();' in setup(), test resulted to success.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2937) TestHiveServerSessions hangs when executed directly
[
https://issues.apache.org/jira/browse/HIVE-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2937:
Status: Patch Available (was: Open)
TestHiveServerSessions hangs when executed directly
---
Key: HIVE-2937
URL: https://issues.apache.org/jira/browse/HIVE-2937
Project: Hive
Issue Type: Test
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-2937.D2697.1.patch
{code}
ant test -Doffline=true -Dtestcase=TestHiveServerSessions
{code}
Hangs infinitely.
I couldn't imagine exact cause of the problem, but found that by adding 'new
HiveServer.HiveServerHandler();' in setup(), test resulted to success.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2901) Hive union with NULL constant and string in same column returns all null
[ https://issues.apache.org/jira/browse/HIVE-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2901: Status: Patch Available (was: Open) Hive union with NULL constant and string in same column returns all null Key: HIVE-2901 URL: https://issues.apache.org/jira/browse/HIVE-2901 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Navis Priority: Critical Attachments: HIVE-2901.D2541.1.patch, HIVE-2901.D2541.2.patch, HIVE-2901.D2541.3.patch select x from (select value as x from src union all select NULL as x from src)a; This query produces all nulls, where value is a string column. Notably, select x from (select key as x from src union all select NULL as x from src)a; where key is a string, but can be cast to a double, the query returns correct results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2906) Support providing some table properties by user via SQL
[
https://issues.apache.org/jira/browse/HIVE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2906:
Status: Patch Available (was: Open)
Support providing some table properties by user via SQL
---
Key: HIVE-2906
URL: https://issues.apache.org/jira/browse/HIVE-2906
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Attachments: HIVE-2906.D2499.1.patch, HIVE-2906.D2499.2.patch,
HIVE-2906.D2499.3.patch, HIVE-2906.D2499.4.patch
Some properties are needed to be provided to StorageHandler by user in
runtime. It might be an address for remote resource or retry count for access
or maximum version count(for hbase), etc.
For example,
{code}
select emp.empno, emp.ename from hbase_emp ('max.version'='3') emp;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2906) Support providing some table properties by user via SQL
[
https://issues.apache.org/jira/browse/HIVE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2906:
Status: Patch Available (was: Open)
Support providing some table properties by user via SQL
---
Key: HIVE-2906
URL: https://issues.apache.org/jira/browse/HIVE-2906
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Attachments: HIVE-2906.D2499.1.patch, HIVE-2906.D2499.2.patch
Some properties are needed to be provided to StorageHandler by user in
runtime. It might be an address for remote resource or retry count for access
or maximum version count(for hbase), etc.
For example,
{code}
select emp.empno, emp.ename from hbase_emp ('max.version'='3') emp;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2906) Support providing some table properties by user via SQL
[
https://issues.apache.org/jira/browse/HIVE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2906:
Status: Open (was: Patch Available)
Support providing some table properties by user via SQL
---
Key: HIVE-2906
URL: https://issues.apache.org/jira/browse/HIVE-2906
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Attachments: HIVE-2906.D2499.1.patch, HIVE-2906.D2499.2.patch,
HIVE-2906.D2499.3.patch
Some properties are needed to be provided to StorageHandler by user in
runtime. It might be an address for remote resource or retry count for access
or maximum version count(for hbase), etc.
For example,
{code}
select emp.empno, emp.ename from hbase_emp ('max.version'='3') emp;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2848) Add validation to HiveConf ConfVars
[ https://issues.apache.org/jira/browse/HIVE-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2848: Affects Version/s: 0.9.0 Status: Patch Available (was: Open) Add validation to HiveConf ConfVars --- Key: HIVE-2848 URL: https://issues.apache.org/jira/browse/HIVE-2848 Project: Hive Issue Type: Improvement Affects Versions: 0.9.0 Reporter: Kevin Wilfong Assignee: Navis Attachments: HIVE-2848.D2313.1.patch, HIVE-2848.D2313.2.patch It would be good if we could add some validation to ConfVars in the HiveConf when they are set. At least if a variable is supposed to be boolean, numeric, or strict/nonstrict, it shouldn't be too difficult to check for these. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2903) Numeric binary type keys are not compared properly
[
https://issues.apache.org/jira/browse/HIVE-2903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2903:
Status: Patch Available (was: Open)
Passed all tests.
Numeric binary type keys are not compared properly
--
Key: HIVE-2903
URL: https://issues.apache.org/jira/browse/HIVE-2903
Project: Hive
Issue Type: Bug
Components: HBase Handler
Reporter: Navis
Assignee: Navis
Attachments: HIVE-2903.D2481.1.patch
In current binary format for numbers, minus values are always greater than
plus values, for example.
{code}
System.our.println(Bytes.compareTo(Bytes.toBytes(-100), Bytes.toBytes(100)));
// 255
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2906) Support providing some table properties by user via SQL
[
https://issues.apache.org/jira/browse/HIVE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2906:
Status: Patch Available (was: Open)
Passed all tests.
Support providing some table properties by user via SQL
---
Key: HIVE-2906
URL: https://issues.apache.org/jira/browse/HIVE-2906
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Attachments: HIVE-2906.D2499.1.patch
Some properties are needed to be provided to StorageHandler by user in
runtime. It might be an address for remote resource or retry count for access
or maximum version count(for hbase), etc.
For example,
{code}
select emp.empno, emp.ename from hbase_emp ('max.version'='3') emp;
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2881) Remove redundant key comparing in SMBMapJoinOperator
[ https://issues.apache.org/jira/browse/HIVE-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2881: Status: Patch Available (was: Open) Passed all tests. Remove redundant key comparing in SMBMapJoinOperator Key: HIVE-2881 URL: https://issues.apache.org/jira/browse/HIVE-2881 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-2881.D2379.1.patch Currently, SMBJoin compares keys twice in #findSmallestKey and #joinObject. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2863) Ambiguous table name or column reference message displays when table and column names are the same
[ https://issues.apache.org/jira/browse/HIVE-2863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2863: Assignee: Navis Status: Patch Available (was: Open) Passed all tests. Ambiguous table name or column reference message displays when table and column names are the same -- Key: HIVE-2863 URL: https://issues.apache.org/jira/browse/HIVE-2863 Project: Hive Issue Type: Bug Reporter: Mauro Cazzari Assignee: Navis Attachments: HIVE-2863.D2361.1.patch Given the following table: CREATE TABLE `Y` (`y` DOUBLE) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' STORED AS TEXTFILE; The following query fails: SELECT `Y`.`y` FROM `Y` WHERE ( `y` = 1 ) ERROR: java.sql.SQLException: Query returned non-zero code: 10, cause: FAILED: Error in semantic analysis: Line 1:36 Ambiguous table alias or column reference '`y`' ERROR: Unable to execute Hadoop query. ERROR: Prepare error. SQL statement: SELECT `Y`.`y` FROM `Y` WHERE ( `y` = 1 ). The problem goes away if the table and column names do not match. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2828) make timestamp accessible in the hbase KeyValue
[
https://issues.apache.org/jira/browse/HIVE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2828:
Status: Patch Available (was: Open)
make timestamp accessible in the hbase KeyValue
Key: HIVE-2828
URL: https://issues.apache.org/jira/browse/HIVE-2828
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-2828.D1989.1.patch, HIVE-2828.D1989.2.patch,
HIVE-2828.D1989.3.patch, HIVE-2828.D1989.4.patch
Originated from HIVE-2781 and not accepted, but I think this could be helpful
to someone.
By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user
might access timestamp value in hbase KeyValue.
{code}
CREATE TABLE hbase_table (key int, value string, time timestamp)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (hbase.columns.mapping = :key,cf:string,:timestamp)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2778) Fail on table sampling
[
https://issues.apache.org/jira/browse/HIVE-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2778:
Attachment: HIVE-2778.D1593.2.patch
Granted license to ASF.. Couldn't it be automated by phabricator?
Fail on table sampling
---
Key: HIVE-2778
URL: https://issues.apache.org/jira/browse/HIVE-2778
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Environment: Reproduced only on hadoop-0.20.2-CDH3u1, work fine on
hadoop-0.20.2
Reporter: Navis
Assignee: Navis
Fix For: 0.9.0
Attachments: HIVE-2778.D1593.1.patch, HIVE-2778.D1593.2.patch,
HIVE-2778.D1593.2.patch
Trying table sampling on any non-empty table throws NPE. This does not occur
by test on mini-MR.
{noformat}
select count(*) from emp tablesample (0.1 percent);
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=number
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=number
In order to set a constant number of reducers:
set mapred.reduce.tasks=number
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.sampleSplits(CombineHiveInputFormat.java:450)
at
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:403)
at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:971)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:963)
at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807)
at
org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:432)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Job Submission failed with exception 'java.lang.NullPointerException(null)'
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MapRedTask
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2861) Support eventual constant expression for filter pushdown for key ranges in hbase
[
https://issues.apache.org/jira/browse/HIVE-2861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2861:
Status: Patch Available (was: Open)
Support eventual constant expression for filter pushdown for key ranges in
hbase
Key: HIVE-2861
URL: https://issues.apache.org/jira/browse/HIVE-2861
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-2861.D2259.1.patch
Minor upgrade from HIVE-2771, which supports simple eventual constant
expression as a filter (especially 'cast').
For example,
{noformat}
select * from hbase_pushdown where key cast(20 + 30 as string);
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2854) Support between filter pushdown for key ranges in hbase
[ https://issues.apache.org/jira/browse/HIVE-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2854: Status: Patch Available (was: Open) Support between filter pushdown for key ranges in hbase --- Key: HIVE-2854 URL: https://issues.apache.org/jira/browse/HIVE-2854 Project: Hive Issue Type: Improvement Components: HBase Handler Environment: ubuntu 10.04 Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-2854.D2169.1.patch https://issues.apache.org/jira/browse/HIVE-2771 omitted 'between' operator. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2820) Invalid tag is used for MapJoinProcessor
[
https://issues.apache.org/jira/browse/HIVE-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2820:
Status: Patch Available (was: Open)
The problem above was caused by HIVE-2839. Patch seemed to be still valid.
Invalid tag is used for MapJoinProcessor
Key: HIVE-2820
URL: https://issues.apache.org/jira/browse/HIVE-2820
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Environment: ubuntu
Reporter: Navis
Assignee: Navis
Fix For: 0.9.0
Attachments: HIVE-2820.D1935.1.patch, HIVE-2820.D1935.2.patch
Testing HIVE-2810, I've found tag and alias are used in very confusing
manner. For example, query below fails..
{code}
hive set hive.auto.convert.join=true;
hive select /*+ STREAMTABLE(a) */ * from myinput1 a join myinput1 b on
a.key=b.key join myinput1 c on a.key=c.key;
Total MapReduce jobs = 4
Ended Job = 1667415037, job is filtered out (removed at runtime).
Ended Job = 1739566906, job is filtered out (removed at runtime).
Ended Job = 1113337780, job is filtered out (removed at runtime).
12/02/24 10:27:14 WARN conf.HiveConf: DEPRECATED: Ignoring hive-default.xml
found on the CLASSPATH at /home/navis/hive/conf/hive-default.xml
Execution log at:
/tmp/navis/navis_20120224102727_cafe0d8d-9b21-441d-bd4e-b83303b31cdc.log
2012-02-24 10:27:14 Starting to launch local task to process map join;
maximum memory = 932118528
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.HashTableSinkOperator.processOp(HashTableSinkOperator.java:312)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at
org.apache.hadoop.hive.ql.exec.MapredLocalTask.startForward(MapredLocalTask.java:325)
at
org.apache.hadoop.hive.ql.exec.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:272)
at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:685)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Execution failed with exit status: 2
Obtaining error information
{code}
Failed task has a plan which doesn't make sense.
{noformat}
Stage: Stage-8
Map Reduce Local Work
Alias - Map Local Tables:
b
Fetch Operator
limit: -1
c
Fetch Operator
limit: -1
Alias - Map Local Operator Tree:
b
TableScan
alias: b
HashTable Sink Operator
condition expressions:
0 {key} {value}
1 {key} {value}
2 {key} {value}
handleSkewJoin: false
keys:
0 [Column[key]]
1 [Column[key]]
2 [Column[key]]
Position of Big Table: 0
c
TableScan
alias: c
Map Join Operator
condition map:
Inner Join 0 to 1
Inner Join 0 to 2
condition expressions:
0 {key} {value}
1 {key} {value}
2 {key} {value}
handleSkewJoin: false
keys:
0 [Column[key]]
1 [Column[key]]
2 [Column[key]]
outputColumnNames: _col0, _col1, _col4, _col5, _col8, _col9
Position of Big Table: 0
Select Operator
expressions:
expr: _col0
type: int
expr: _col1
type: int
expr: _col4
type: int
expr: _col5
type: int
expr: _col8
type: int
expr: _col9
type: int
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
File Output Operator
compressed: false
GlobalTableId: 0
table:
input
[jira] [Updated] (HIVE-2839) Filters on outer join with mapjoin hint is not applied correctly
[
https://issues.apache.org/jira/browse/HIVE-2839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2839:
Status: Patch Available (was: Open)
Filters on outer join with mapjoin hint is not applied correctly
Key: HIVE-2839
URL: https://issues.apache.org/jira/browse/HIVE-2839
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Fix For: 0.9.0
Attachments: HIVE-2839.D2079.1.patch
Testing HIVE-2820, I've found some queries with mapjoin hint makes exceptions.
{code}
SELECT /*+ MAPJOIN(a) */ * FROM src a RIGHT OUTER JOIN src b on a.key=b.key
AND true limit 10;
FAILED: Hive Internal Error:
java.lang.ClassCastException(org.apache.hadoop.hive.ql.plan.ExprNodeConstantDesc
cannot be cast to org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
java.lang.ClassCastException:
org.apache.hadoop.hive.ql.plan.ExprNodeConstantDesc cannot be cast to
org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc
at
org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.convertMapJoin(MapJoinProcessor.java:363)
at
org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.generateMapJoinOperator(MapJoinProcessor.java:483)
at
org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor.transform(MapJoinProcessor.java:689)
at
org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:87)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:7519)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:250)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:431)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:336)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:891)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
{code}
and
{code}
SELECT /*+ MAPJOIN(a) */ * FROM src a RIGHT OUTER JOIN src b on a.key=b.key
AND b.key * 10 '1000' limit 10;
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:198)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.cleanUpInputFileChangedOp(MapJoinOperator.java:212)
at
org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1321)
at
org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1325)
at
org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1325)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:495)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
... 8 more
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2827) Implement nullsafe equi-join
[
https://issues.apache.org/jira/browse/HIVE-2827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2827:
Fix Version/s: 0.9.0
Status: Patch Available (was: Open)
Implement nullsafe equi-join
Key: HIVE-2827
URL: https://issues.apache.org/jira/browse/HIVE-2827
Project: Hive
Issue Type: Improvement
Components: Query Processor
Environment: ubuntu 10.04
Reporter: Navis
Assignee: Navis
Priority: Minor
Fix For: 0.9.0
Attachments: HIVE-2827.D1971.1.patch, HIVE-2827.D1971.2.patch,
HIVE-2827.D1971.3.patch, HIVE-2827.D1971.4.patch, HIVE-2827.D1971.5.patch,
HIVE-2827.D1971.6.patch, HIVE-2827.D1971.7.patch
was part of HIVE-2810, but separated because it affected more classes than
expected.
{noformat}
SELECT * FROM a JOIN b ON a.key = b.key
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2810) Implement NULL-safe equality operator =
[ https://issues.apache.org/jira/browse/HIVE-2810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2810: Affects Version/s: (was: 0.9.0) Status: Patch Available (was: Open) Implement NULL-safe equality operator = - Key: HIVE-2810 URL: https://issues.apache.org/jira/browse/HIVE-2810 Project: Hive Issue Type: New Feature Components: Query Processor, UDF Reporter: Carl Steinbach Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2810.D1791.1.patch, HIVE-2810.D1791.2.patch, HIVE-2810.D1791.3.patch, HIVE-2810.D1791.4.patch Ref: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_equal-to -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2820) Invalid tag is used for MapJoinProcessor
[
https://issues.apache.org/jira/browse/HIVE-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2820:
Status: Open (was: Patch Available)
Still cannot execute query like,
{code}
select /*+ MAPJOIN(a) */ * from src a right outer join src b on a.key=b.key AND
concat(b.key, '0') '1' limit 10;
{code}
I'll check it out.
Invalid tag is used for MapJoinProcessor
Key: HIVE-2820
URL: https://issues.apache.org/jira/browse/HIVE-2820
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Environment: ubuntu
Reporter: Navis
Assignee: Navis
Fix For: 0.9.0
Attachments: HIVE-2820.D1935.1.patch, HIVE-2820.D1935.2.patch
Testing HIVE-2810, I've found tag and alias are used in very confusing
manner. For example, query below fails..
{code}
hive set hive.auto.convert.join=true;
hive select /*+ STREAMTABLE(a) */ * from myinput1 a join myinput1 b on
a.key=b.key join myinput1 c on a.key=c.key;
Total MapReduce jobs = 4
Ended Job = 1667415037, job is filtered out (removed at runtime).
Ended Job = 1739566906, job is filtered out (removed at runtime).
Ended Job = 1113337780, job is filtered out (removed at runtime).
12/02/24 10:27:14 WARN conf.HiveConf: DEPRECATED: Ignoring hive-default.xml
found on the CLASSPATH at /home/navis/hive/conf/hive-default.xml
Execution log at:
/tmp/navis/navis_20120224102727_cafe0d8d-9b21-441d-bd4e-b83303b31cdc.log
2012-02-24 10:27:14 Starting to launch local task to process map join;
maximum memory = 932118528
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.HashTableSinkOperator.processOp(HashTableSinkOperator.java:312)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at
org.apache.hadoop.hive.ql.exec.MapredLocalTask.startForward(MapredLocalTask.java:325)
at
org.apache.hadoop.hive.ql.exec.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:272)
at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:685)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Execution failed with exit status: 2
Obtaining error information
{code}
Failed task has a plan which doesn't make sense.
{noformat}
Stage: Stage-8
Map Reduce Local Work
Alias - Map Local Tables:
b
Fetch Operator
limit: -1
c
Fetch Operator
limit: -1
Alias - Map Local Operator Tree:
b
TableScan
alias: b
HashTable Sink Operator
condition expressions:
0 {key} {value}
1 {key} {value}
2 {key} {value}
handleSkewJoin: false
keys:
0 [Column[key]]
1 [Column[key]]
2 [Column[key]]
Position of Big Table: 0
c
TableScan
alias: c
Map Join Operator
condition map:
Inner Join 0 to 1
Inner Join 0 to 2
condition expressions:
0 {key} {value}
1 {key} {value}
2 {key} {value}
handleSkewJoin: false
keys:
0 [Column[key]]
1 [Column[key]]
2 [Column[key]]
outputColumnNames: _col0, _col1, _col4, _col5, _col8, _col9
Position of Big Table: 0
Select Operator
expressions:
expr: _col0
type: int
expr: _col1
type: int
expr: _col4
type: int
expr: _col5
type: int
expr: _col8
type: int
expr: _col9
type: int
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
File Output Operator
[jira] [Updated] (HIVE-2810) Implement NULL-safe equality operator =
[ https://issues.apache.org/jira/browse/HIVE-2810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2810: Status: Patch Available (was: Open) Implement NULL-safe equality operator = - Key: HIVE-2810 URL: https://issues.apache.org/jira/browse/HIVE-2810 Project: Hive Issue Type: New Feature Components: Query Processor, UDF Affects Versions: 0.9.0 Reporter: Carl Steinbach Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2810.D1791.1.patch, HIVE-2810.D1791.2.patch, HIVE-2810.D1791.3.patch Ref: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_equal-to -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2827) Implement nullsafe equi-join
[
https://issues.apache.org/jira/browse/HIVE-2827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2827:
Status: Patch Available (was: Open)
Passed all tests.
Implement nullsafe equi-join
Key: HIVE-2827
URL: https://issues.apache.org/jira/browse/HIVE-2827
Project: Hive
Issue Type: Improvement
Components: Query Processor
Environment: ubuntu 10.04
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-2827.D1971.1.patch, HIVE-2827.D1971.2.patch,
HIVE-2827.D1971.3.patch, HIVE-2827.D1971.4.patch
was part of HIVE-2810, but separated because it affected more classes than
expected.
{noformat}
SELECT * FROM a JOIN b ON a.key = b.key
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2820) Invalid tag is used for MapJoinProcessor
[
https://issues.apache.org/jira/browse/HIVE-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2820:
Status: Patch Available (was: Open)
Confirmed that the patch have passed all the tests.
Invalid tag is used for MapJoinProcessor
Key: HIVE-2820
URL: https://issues.apache.org/jira/browse/HIVE-2820
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Environment: ubuntu
Reporter: Navis
Assignee: Navis
Fix For: 0.9.0
Attachments: HIVE-2820.D1935.1.patch, HIVE-2820.D1935.2.patch
Testing HIVE-2810, I've found tag and alias are used in very confusing
manner. For example, query below fails..
{code}
hive set hive.auto.convert.join=true;
hive select /*+ STREAMTABLE(a) */ * from myinput1 a join myinput1 b on
a.key=b.key join myinput1 c on a.key=c.key;
Total MapReduce jobs = 4
Ended Job = 1667415037, job is filtered out (removed at runtime).
Ended Job = 1739566906, job is filtered out (removed at runtime).
Ended Job = 1113337780, job is filtered out (removed at runtime).
12/02/24 10:27:14 WARN conf.HiveConf: DEPRECATED: Ignoring hive-default.xml
found on the CLASSPATH at /home/navis/hive/conf/hive-default.xml
Execution log at:
/tmp/navis/navis_20120224102727_cafe0d8d-9b21-441d-bd4e-b83303b31cdc.log
2012-02-24 10:27:14 Starting to launch local task to process map join;
maximum memory = 932118528
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.HashTableSinkOperator.processOp(HashTableSinkOperator.java:312)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at
org.apache.hadoop.hive.ql.exec.MapredLocalTask.startForward(MapredLocalTask.java:325)
at
org.apache.hadoop.hive.ql.exec.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:272)
at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:685)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Execution failed with exit status: 2
Obtaining error information
{code}
Failed task has a plan which doesn't make sense.
{noformat}
Stage: Stage-8
Map Reduce Local Work
Alias - Map Local Tables:
b
Fetch Operator
limit: -1
c
Fetch Operator
limit: -1
Alias - Map Local Operator Tree:
b
TableScan
alias: b
HashTable Sink Operator
condition expressions:
0 {key} {value}
1 {key} {value}
2 {key} {value}
handleSkewJoin: false
keys:
0 [Column[key]]
1 [Column[key]]
2 [Column[key]]
Position of Big Table: 0
c
TableScan
alias: c
Map Join Operator
condition map:
Inner Join 0 to 1
Inner Join 0 to 2
condition expressions:
0 {key} {value}
1 {key} {value}
2 {key} {value}
handleSkewJoin: false
keys:
0 [Column[key]]
1 [Column[key]]
2 [Column[key]]
outputColumnNames: _col0, _col1, _col4, _col5, _col8, _col9
Position of Big Table: 0
Select Operator
expressions:
expr: _col0
type: int
expr: _col1
type: int
expr: _col4
type: int
expr: _col5
type: int
expr: _col8
type: int
expr: _col9
type: int
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
File Output Operator
compressed: false
GlobalTableId: 0
table:
input format:
[jira] [Updated] (HIVE-2781) HBaseSerDe should allow users to specify the timestamp passed to Puts
[ https://issues.apache.org/jira/browse/HIVE-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2781: Fix Version/s: 0.9.0 Affects Version/s: 0.9.0 Status: Patch Available (was: Open) HBaseSerDe should allow users to specify the timestamp passed to Puts -- Key: HIVE-2781 URL: https://issues.apache.org/jira/browse/HIVE-2781 Project: Hive Issue Type: Improvement Affects Versions: 0.9.0 Reporter: Francis Liu Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2781.D1863.1.patch, HIVE-2781.D1863.2.patch Users may want to specify the timestamp used for Put requests to hbase. Thus enabling users to have the same timestamp for a single batch of writes. Which would be useful for a number of things. HCatalog's HBase storageHandler implementation makes use of this feature to provide users with snapshot isolation and write transactions. My proposal is to add the timestamp option as a final static member: public static final long HBASE_PUT_TIMESTAMP = hbase.put_timestamp And passing this value to all the Puts created by serialize() -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2792) SUBSTR(CAST(string AS BINARY)) produces unexpected results
[ https://issues.apache.org/jira/browse/HIVE-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2792: Attachment: HIVE-2792.D1797.2.patch SUBSTR(CAST(string AS BINARY)) produces unexpected results Key: HIVE-2792 URL: https://issues.apache.org/jira/browse/HIVE-2792 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.8.0, 0.8.1 Reporter: Carl Steinbach Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2792.D1797.1.patch, HIVE-2792.D1797.2.patch, HIVE-2792.D1797.2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2796) Support auto completion for hive configs in CliDriver
[ https://issues.apache.org/jira/browse/HIVE-2796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2796: Fix Version/s: 0.9.0 Affects Version/s: 0.9.0 Status: Patch Available (was: Open) Support auto completion for hive configs in CliDriver - Key: HIVE-2796 URL: https://issues.apache.org/jira/browse/HIVE-2796 Project: Hive Issue Type: Improvement Affects Versions: 0.9.0 Reporter: Navis Assignee: Navis Priority: Trivial Fix For: 0.9.0 Attachments: HIVE-2796.D1689.1.patch, HIVE-2796.D1689.2.patch It's very cumbersome to memorize hive conf vars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2261) Add cleanup stages for UDFs
[ https://issues.apache.org/jira/browse/HIVE-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2261: Fix Version/s: 0.9.0 Affects Version/s: 0.9.0 Status: Patch Available (was: Open) Add cleanup stages for UDFs --- Key: HIVE-2261 URL: https://issues.apache.org/jira/browse/HIVE-2261 Project: Hive Issue Type: Wish Components: Query Processor Affects Versions: 0.9.0 Reporter: Navis Assignee: Navis Priority: Trivial Fix For: 0.9.0 Attachments: HIVE-2261.D1329.1.patch, HIVE-2261.D1329.2.patch In some cases, we bind values at last stage of big SQL from other sources, especially from memcached. I made that kind of UDFs for internal-use. I found 'initialize' method of GenericUDF class is good place for making connections to memcached cluster, but failed to find somewhere to close/cleanup the connections. If there is cleaup method in GenericUDF class, things can be more neat. If initializing entity like map/reduce/fetch could be also providable to life-cycles(init/close), that makes perfect. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2503) HiveServer should provide per session configuration
[ https://issues.apache.org/jira/browse/HIVE-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2503: Status: Patch Available (was: Open) HiveServer should provide per session configuration --- Key: HIVE-2503 URL: https://issues.apache.org/jira/browse/HIVE-2503 Project: Hive Issue Type: Bug Components: CLI, Server Infrastructure Affects Versions: 0.9.0 Reporter: Navis Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2503.1.patch.txt Currently ThriftHiveProcessorFactory returns same HiveConf instance to HiveServerHandler, making impossible to use per sesssion configuration. Just wrapping 'conf' - 'new HiveConf(conf)' seemed to solve this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2778) Fail on table sampling
[
https://issues.apache.org/jira/browse/HIVE-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2778:
Environment: Reproduced only on hadoop-0.20.2-CDH3u1, work fine on
hadoop-0.20.2
Fail on table sampling
---
Key: HIVE-2778
URL: https://issues.apache.org/jira/browse/HIVE-2778
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Environment: Reproduced only on hadoop-0.20.2-CDH3u1, work fine on
hadoop-0.20.2
Reporter: Navis
Assignee: Navis
Fix For: 0.9.0
Attachments: HIVE-2778.D1593.1.patch
Trying table sampling on any non-empty table throws NPE. This does not occur
by test on mini-MR.
{noformat}
select count(*) from emp tablesample (0.1 percent);
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=number
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=number
In order to set a constant number of reducers:
set mapred.reduce.tasks=number
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.sampleSplits(CombineHiveInputFormat.java:450)
at
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:403)
at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:971)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:963)
at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807)
at
org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:432)
at
org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Job Submission failed with exception 'java.lang.NullPointerException(null)'
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MapRedTask
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2477) Use name of original expression for name of CAST output
[ https://issues.apache.org/jira/browse/HIVE-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2477: Attachment: HIVE-2477.1.patch.txt Simple patch. If a column is function with single param and the param is a column, use the column name as alias. Use name of original expression for name of CAST output --- Key: HIVE-2477 URL: https://issues.apache.org/jira/browse/HIVE-2477 Project: Hive Issue Type: Improvement Reporter: Adam Kramer Priority: Minor Attachments: HIVE-2477.1.patch.txt CAST(foo AS INT) should, by default, consider itself a column named foo if unspecified/unaliased. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2709) Semantic Analysis failed for GroupBy query with aliase.
[ https://issues.apache.org/jira/browse/HIVE-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2709: Attachment: HIVE-2709.1.patch.txt Hive does not search column properly if row resolver of previous operator is expressional (rs or gby). If some equivalent logic is added, this issue could be resolved. Attached patch is just a simple scratch. Semantic Analysis failed for GroupBy query with aliase. --- Key: HIVE-2709 URL: https://issues.apache.org/jira/browse/HIVE-2709 Project: Hive Issue Type: Bug Components: SQL Affects Versions: 0.8.0 Environment: SuSE11 SP1 Reporter: rohithsharma Priority: Minor Attachments: HIVE-2709.1.patch.txt In the SQL , select empno from test t group by t.empno type of query work fine where as in hive ,below exception org.apache.hadoop.hive.ql.parse.SemanticException: line 1:7 Expression Not In Group By Key ename. For the above, column_name with aliase work fine i.e select t.empno from test t group by t.empno is work fine. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2005) Implement BETWEEN operator
[ https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2005: Attachment: (was: HIVE-2005.1.patch.txt) Implement BETWEEN operator -- Key: HIVE-2005 URL: https://issues.apache.org/jira/browse/HIVE-2005 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.9.0 Reporter: Carl Steinbach Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2005.6.patch.txt Reference: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2005) Implement BETWEEN operator
[ https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2005: Attachment: HIVE-2005.6.patch.txt Implement BETWEEN operator -- Key: HIVE-2005 URL: https://issues.apache.org/jira/browse/HIVE-2005 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.9.0 Reporter: Carl Steinbach Assignee: Navis Fix For: 0.9.0 Attachments: HIVE-2005.6.patch.txt Reference: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2586) Float comparison doesn't work
[ https://issues.apache.org/jira/browse/HIVE-2586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2586: Attachment: HIVE-2586.1.patch.txt https://reviews.apache.org/r/3073/ Float comparison doesn't work - Key: HIVE-2586 URL: https://issues.apache.org/jira/browse/HIVE-2586 Project: Hive Issue Type: Bug Reporter: Robert Surówka Priority: Minor Fix For: 0.9.0 Attachments: HIVE-2586.1.patch.txt Create table with float column, insert to it e.g. 1410.1, and then do select * from that table where that column = 1410.1 . Nothing will be found. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2329) Not using map aggregation, fails to execute group-by after cluster-by with same key
[ https://issues.apache.org/jira/browse/HIVE-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2329: Attachment: (was: HIVE-2329.0.8.0.patch) Not using map aggregation, fails to execute group-by after cluster-by with same key --- Key: HIVE-2329 URL: https://issues.apache.org/jira/browse/HIVE-2329 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.8.0 Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-2329.1.patch.txt hive.map.aggr=false select Q1.key_int1, sum(Q1.key_int1), sum(distinct Q1.key_int1) from (select * from t1 cluster by key_int1) Q1 group by Q1.key_int1 resulted.. FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask from hadoop logs.. Caused by: java.lang.RuntimeException: cannot find field key from [] at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:321) at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:119) at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:82) at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:198) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433) I think the problem is caused by ReduceSinkDeDuplication, removing RS which was providing rs.key for GBY operation. If child of child RS is a GBY, we should bypass the optimization. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2329) Not using map aggregation, fails to execute group-by after cluster-by with same key
[ https://issues.apache.org/jira/browse/HIVE-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2329: Attachment: HIVE-2329.1.patch.txt rebased to trunk Not using map aggregation, fails to execute group-by after cluster-by with same key --- Key: HIVE-2329 URL: https://issues.apache.org/jira/browse/HIVE-2329 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.8.0 Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-2329.1.patch.txt hive.map.aggr=false select Q1.key_int1, sum(Q1.key_int1), sum(distinct Q1.key_int1) from (select * from t1 cluster by key_int1) Q1 group by Q1.key_int1 resulted.. FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask from hadoop logs.. Caused by: java.lang.RuntimeException: cannot find field key from [] at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:321) at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:119) at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:82) at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:198) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433) I think the problem is caused by ReduceSinkDeDuplication, removing RS which was providing rs.key for GBY operation. If child of child RS is a GBY, we should bypass the optimization. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2329) Not using map aggregation, fails to execute group-by after cluster-by with same key
[ https://issues.apache.org/jira/browse/HIVE-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2329: Attachment: (was: HIVE-2329.1.patch) Not using map aggregation, fails to execute group-by after cluster-by with same key --- Key: HIVE-2329 URL: https://issues.apache.org/jira/browse/HIVE-2329 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.8.0 Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-2329.1.patch.txt hive.map.aggr=false select Q1.key_int1, sum(Q1.key_int1), sum(distinct Q1.key_int1) from (select * from t1 cluster by key_int1) Q1 group by Q1.key_int1 resulted.. FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask from hadoop logs.. Caused by: java.lang.RuntimeException: cannot find field key from [] at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:321) at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:119) at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:82) at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:198) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433) I think the problem is caused by ReduceSinkDeDuplication, removing RS which was providing rs.key for GBY operation. If child of child RS is a GBY, we should bypass the optimization. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2329) Not using map aggregation, fails to execute group-by after cluster-by with same key
[ https://issues.apache.org/jira/browse/HIVE-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2329: Fix Version/s: 0.9.0 Affects Version/s: (was: 0.8.0) Status: Patch Available (was: Open) Not using map aggregation, fails to execute group-by after cluster-by with same key --- Key: HIVE-2329 URL: https://issues.apache.org/jira/browse/HIVE-2329 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Navis Assignee: Navis Priority: Minor Fix For: 0.9.0 Attachments: HIVE-2329.1.patch.txt hive.map.aggr=false select Q1.key_int1, sum(Q1.key_int1), sum(distinct Q1.key_int1) from (select * from t1 cluster by key_int1) Q1 group by Q1.key_int1 resulted.. FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask from hadoop logs.. Caused by: java.lang.RuntimeException: cannot find field key from [] at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:321) at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:119) at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:82) at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:198) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433) I think the problem is caused by ReduceSinkDeDuplication, removing RS which was providing rs.key for GBY operation. If child of child RS is a GBY, we should bypass the optimization. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2264) Hive server is SHUTTING DOWN when invalid queries beeing executed.
[
https://issues.apache.org/jira/browse/HIVE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2264:
Fix Version/s: 0.9.0
Affects Version/s: (was: 0.7.1)
(was: 0.7.0)
0.9.0
Status: Patch Available (was: Open)
https://reviews.apache.org/r/3025/
Hive server is SHUTTING DOWN when invalid queries beeing executed.
--
Key: HIVE-2264
URL: https://issues.apache.org/jira/browse/HIVE-2264
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Environment: SuSE-Linux-11
Reporter: rohithsharma
Priority: Critical
Fix For: 0.9.0
Attachments: HIVE-2264.1.patch.txt
When invalid query is beeing executed, Hive server is shutting down.
{noformat}
CREATE TABLE SAMPLETABLE(IP STRING , showtime BIGINT ) partitioned by (ds
string,ipz int) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\040'
ALTER TABLE SAMPLETABLE add Partition(ds='sf') location
'/user/hive/warehouse' Partition(ipz=100) location '/user/hive/warehouse'
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2264) Hive server is SHUTTING DOWN when invalid queries beeing executed.
[
https://issues.apache.org/jira/browse/HIVE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2264:
Attachment: HIVE-2264.1.patch.txt
Hive server is SHUTTING DOWN when invalid queries beeing executed.
--
Key: HIVE-2264
URL: https://issues.apache.org/jira/browse/HIVE-2264
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.9.0
Environment: SuSE-Linux-11
Reporter: rohithsharma
Priority: Critical
Fix For: 0.9.0
Attachments: HIVE-2264.1.patch.txt
When invalid query is beeing executed, Hive server is shutting down.
{noformat}
CREATE TABLE SAMPLETABLE(IP STRING , showtime BIGINT ) partitioned by (ds
string,ipz int) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\040'
ALTER TABLE SAMPLETABLE add Partition(ds='sf') location
'/user/hive/warehouse' Partition(ipz=100) location '/user/hive/warehouse'
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
[
https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2332:
Fix Version/s: 0.9.0
Status: Patch Available (was: Open)
If all of the parameters of distinct functions are exists in group by
columns, query fails in runtime
-
Key: HIVE-2332
URL: https://issues.apache.org/jira/browse/HIVE-2332
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Critical
Fix For: 0.9.0
Attachments: HIVE-2332.1.patch.txt
select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
fails with message..
{code}
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
{code}
hadoop says..
{code}
Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
{code}
I think the deficient number of key expression, compared to number of key
column, is the problem, which should be equal or more.
Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2005) Implement BETWEEN operator
[ https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2005: Attachment: (was: HIVE-2005.1.patch) Implement BETWEEN operator -- Key: HIVE-2005 URL: https://issues.apache.org/jira/browse/HIVE-2005 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Carl Steinbach Assignee: Carl Steinbach Reference: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2005) Implement BETWEEN operator
[ https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2005: Attachment: HIVE-2005.1.patch.txt Revised to use between UDF (added newly) to avoid evaluating target expression twice. Implement BETWEEN operator -- Key: HIVE-2005 URL: https://issues.apache.org/jira/browse/HIVE-2005 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Carl Steinbach Assignee: Carl Steinbach Attachments: HIVE-2005.1.patch.txt Reference: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2005) Implement BETWEEN operator
[ https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2005: Fix Version/s: 0.9.0 Affects Version/s: 0.9.0 Status: Patch Available (was: Open) https://reviews.apache.org/r/3008/ Implement BETWEEN operator -- Key: HIVE-2005 URL: https://issues.apache.org/jira/browse/HIVE-2005 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.9.0 Reporter: Carl Steinbach Assignee: Carl Steinbach Fix For: 0.9.0 Attachments: HIVE-2005.1.patch.txt Reference: http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2340) optimize orderby followed by a groupby
[ https://issues.apache.org/jira/browse/HIVE-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2340: Attachment: (was: HIVE-2340.2.patch) optimize orderby followed by a groupby -- Key: HIVE-2340 URL: https://issues.apache.org/jira/browse/HIVE-2340 Project: Hive Issue Type: Sub-task Components: Query Processor Reporter: Navis Priority: Minor Attachments: HIVE-2340.1.patch.txt Before implementing optimizer for JOIN-GBY, try to implement RS-GBY optimizer(cluster-by following group-by). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2340) optimize orderby followed by a groupby
[ https://issues.apache.org/jira/browse/HIVE-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2340: Status: Patch Available (was: Open) optimize orderby followed by a groupby -- Key: HIVE-2340 URL: https://issues.apache.org/jira/browse/HIVE-2340 Project: Hive Issue Type: Sub-task Components: Query Processor Reporter: Navis Priority: Minor Attachments: HIVE-2340.1.patch.txt Before implementing optimizer for JOIN-GBY, try to implement RS-GBY optimizer(cluster-by following group-by). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2503) HiveServer should provide per session configuration
[ https://issues.apache.org/jira/browse/HIVE-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2503: Fix Version/s: 0.9.0 Affects Version/s: 0.9.0 Status: Patch Available (was: Open) HiveServer should provide per session configuration --- Key: HIVE-2503 URL: https://issues.apache.org/jira/browse/HIVE-2503 Project: Hive Issue Type: Bug Components: CLI, Server Infrastructure Affects Versions: 0.9.0 Reporter: Navis Fix For: 0.9.0 Attachments: HIVE-2503.1.patch.txt Currently ThriftHiveProcessorFactory returns same HiveConf instance to HiveServerHandler, making impossible to use per sesssion configuration. Just wrapping 'conf' - 'new HiveConf(conf)' seemed to solve this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2253) Merge failing of join tree in exceptional case
[ https://issues.apache.org/jira/browse/HIVE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2253: Fix Version/s: 0.9.0 Affects Version/s: (was: 0.7.0) 0.9.0 Status: Patch Available (was: Open) Merge failing of join tree in exceptional case -- Key: HIVE-2253 URL: https://issues.apache.org/jira/browse/HIVE-2253 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.9.0 Environment: hadoop 0.20.2, hive 0.7.0 Reporter: Navis Assignee: Navis Priority: Minor Fix For: 0.9.0 Attachments: HIVE-2253-0.8.0.patch, HIVE-2253.1.patch.txt In some very exceptional cases, SemanticAnayzer fails to merge join tree. Example is below. create table a (val1 int, val2 int) create table b (val1 int, val2 int) create table c (val1 int, val2 int) create table d (val1 int, val2 int) create table e (val1 int, val2 int) 1. all same(single) join key -- one MR, good select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val1=e.val1 2. two join keys -- expected to have two MR, but resulted to three MR select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val2=e.val2 3. by changing the join order, we could attain two MR as first-expectation. select * from a join e on a.val2=e.val2 join c on a.val1=c.val1 join d on a.val1=d.val1 join b on a.val1=b.val1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
[
https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2332:
Attachment: HIVE-2332.1.patch.txt
If all of the parameters of distinct functions are exists in group by
columns, query fails in runtime
-
Key: HIVE-2332
URL: https://issues.apache.org/jira/browse/HIVE-2332
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Critical
Attachments: HIVE-2332.0.8.0.patch, HIVE-2332.1.patch.txt
select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
fails with message..
{code}
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
{code}
hadoop says..
{code}
Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
{code}
I think the deficient number of key expression, compared to number of key
column, is the problem, which should be equal or more.
Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
[
https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2332:
Attachment: (was: HIVE-2332.0.8.0.patch)
If all of the parameters of distinct functions are exists in group by
columns, query fails in runtime
-
Key: HIVE-2332
URL: https://issues.apache.org/jira/browse/HIVE-2332
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Critical
Attachments: HIVE-2332.1.patch.txt
select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
fails with message..
{code}
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
{code}
hadoop says..
{code}
Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
{code}
I think the deficient number of key expression, compared to number of key
column, is the problem, which should be equal or more.
Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2253) Merge failing of join tree in exceptional case
[ https://issues.apache.org/jira/browse/HIVE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2253: Attachment: HIVE-2253.1.patch.txt Merge failing of join tree in exceptional case -- Key: HIVE-2253 URL: https://issues.apache.org/jira/browse/HIVE-2253 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Environment: hadoop 0.20.2, hive 0.7.0 Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-2253-0.8.0.patch, HIVE-2253.1.patch.txt In some very exceptional cases, SemanticAnayzer fails to merge join tree. Example is below. create table a (val1 int, val2 int) create table b (val1 int, val2 int) create table c (val1 int, val2 int) create table d (val1 int, val2 int) create table e (val1 int, val2 int) 1. all same(single) join key -- one MR, good select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val1=e.val1 2. two join keys -- expected to have two MR, but resulted to three MR select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val2=e.val2 3. by changing the join order, we could attain two MR as first-expectation. select * from a join e on a.val2=e.val2 join c on a.val1=c.val1 join d on a.val1=d.val1 join b on a.val1=b.val1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2503) HiveServer should provide per session configuration
[ https://issues.apache.org/jira/browse/HIVE-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2503: Attachment: HIVE-2503.1.patch.txt Removed diff-prefix and renamed patch file as supposed to be. HiveServer should provide per session configuration --- Key: HIVE-2503 URL: https://issues.apache.org/jira/browse/HIVE-2503 Project: Hive Issue Type: Bug Components: CLI, Server Infrastructure Reporter: Navis Attachments: HIVE-2503.1.patch.txt Currently ThriftHiveProcessorFactory returns same HiveConf instance to HiveServerHandler, making impossible to use per sesssion configuration. Just wrapping 'conf' - 'new HiveConf(conf)' seemed to solve this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2573) Create per-session function registry
[
https://issues.apache.org/jira/browse/HIVE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2573:
Attachment: HIVE-2573.1.patch.txt
This patch induces sentences like this
{code}
CREATE [SESSION] TEMPORARY FUNCTION ...
DROP [SESSION] TEMPORARY FUNCTION ...
{code}
I've thought keyword 'TEMPORARY' could mean the 'session' thing but it seemed
to be not safe.
Create per-session function registry
-
Key: HIVE-2573
URL: https://issues.apache.org/jira/browse/HIVE-2573
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Navis
Priority: Minor
Attachments: HIVE-2573.1.patch.txt
Currently the function registry is shared resource and could be overrided by
other users when using HiveServer. If per-session function registry is
provided, this situation could be prevented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2573) Create per-session function registry
[ https://issues.apache.org/jira/browse/HIVE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2573: Attachment: HIVE-2573.2.patch.txt added test query result Create per-session function registry - Key: HIVE-2573 URL: https://issues.apache.org/jira/browse/HIVE-2573 Project: Hive Issue Type: Improvement Components: Server Infrastructure Reporter: Navis Priority: Minor Attachments: HIVE-2573.1.patch.txt, HIVE-2573.2.patch.txt Currently the function registry is shared resource and could be overrided by other users when using HiveServer. If per-session function registry is provided, this situation could be prevented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2573) Create per-session function registry
[ https://issues.apache.org/jira/browse/HIVE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2573: Attachment: HIVE-2573.3.patch.txt Create per-session function registry - Key: HIVE-2573 URL: https://issues.apache.org/jira/browse/HIVE-2573 Project: Hive Issue Type: Improvement Components: Server Infrastructure Reporter: Navis Priority: Minor Attachments: HIVE-2573.1.patch.txt, HIVE-2573.2.patch.txt, HIVE-2573.3.patch.txt Currently the function registry is shared resource and could be overrided by other users when using HiveServer. If per-session function registry is provided, this situation could be prevented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2503) HiveServer should provide per session configuration
[ https://issues.apache.org/jira/browse/HIVE-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2503: Attachment: (was: HIVE-2503-0.8.0.patch) HiveServer should provide per session configuration --- Key: HIVE-2503 URL: https://issues.apache.org/jira/browse/HIVE-2503 Project: Hive Issue Type: Bug Components: CLI, Server Infrastructure Reporter: Navis Currently ThriftHiveProcessorFactory returns same HiveConf instance to HiveServerHandler, making impossible to use per sesssion configuration. Just wrapping 'conf' - 'new HiveConf(conf)' seemed to solve this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2253) Merge failing of join tree in exceptional case
[ https://issues.apache.org/jira/browse/HIVE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2253: Attachment: (was: HIVE-2253.3.patch) Merge failing of join tree in exceptional case -- Key: HIVE-2253 URL: https://issues.apache.org/jira/browse/HIVE-2253 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Environment: hadoop 0.20.2, hive 0.7.0 Reporter: Navis Assignee: Navis Priority: Minor In some very exceptional cases, SemanticAnayzer fails to merge join tree. Example is below. create table a (val1 int, val2 int) create table b (val1 int, val2 int) create table c (val1 int, val2 int) create table d (val1 int, val2 int) create table e (val1 int, val2 int) 1. all same(single) join key -- one MR, good select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val1=e.val1 2. two join keys -- expected to have two MR, but resulted to three MR select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val2=e.val2 3. by changing the join order, we could attain two MR as first-expectation. select * from a join e on a.val2=e.val2 join c on a.val1=c.val1 join d on a.val1=d.val1 join b on a.val1=b.val1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2253) Merge failing of join tree in exceptional case
[ https://issues.apache.org/jira/browse/HIVE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2253: Attachment: (was: HIVE-2253.2.patch) Merge failing of join tree in exceptional case -- Key: HIVE-2253 URL: https://issues.apache.org/jira/browse/HIVE-2253 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Environment: hadoop 0.20.2, hive 0.7.0 Reporter: Navis Assignee: Navis Priority: Minor In some very exceptional cases, SemanticAnayzer fails to merge join tree. Example is below. create table a (val1 int, val2 int) create table b (val1 int, val2 int) create table c (val1 int, val2 int) create table d (val1 int, val2 int) create table e (val1 int, val2 int) 1. all same(single) join key -- one MR, good select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val1=e.val1 2. two join keys -- expected to have two MR, but resulted to three MR select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val2=e.val2 3. by changing the join order, we could attain two MR as first-expectation. select * from a join e on a.val2=e.val2 join c on a.val1=c.val1 join d on a.val1=d.val1 join b on a.val1=b.val1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2253) Merge failing of join tree in exceptional case
[ https://issues.apache.org/jira/browse/HIVE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2253: Attachment: (was: HIVE-2253.4.patch) Merge failing of join tree in exceptional case -- Key: HIVE-2253 URL: https://issues.apache.org/jira/browse/HIVE-2253 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Environment: hadoop 0.20.2, hive 0.7.0 Reporter: Navis Assignee: Navis Priority: Minor In some very exceptional cases, SemanticAnayzer fails to merge join tree. Example is below. create table a (val1 int, val2 int) create table b (val1 int, val2 int) create table c (val1 int, val2 int) create table d (val1 int, val2 int) create table e (val1 int, val2 int) 1. all same(single) join key -- one MR, good select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val1=e.val1 2. two join keys -- expected to have two MR, but resulted to three MR select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val2=e.val2 3. by changing the join order, we could attain two MR as first-expectation. select * from a join e on a.val2=e.val2 join c on a.val1=c.val1 join d on a.val1=d.val1 join b on a.val1=b.val1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2253) Merge failing of join tree in exceptional case
[ https://issues.apache.org/jira/browse/HIVE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2253: Attachment: (was: HIVE-2253.patch) Merge failing of join tree in exceptional case -- Key: HIVE-2253 URL: https://issues.apache.org/jira/browse/HIVE-2253 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Environment: hadoop 0.20.2, hive 0.7.0 Reporter: Navis Assignee: Navis Priority: Minor In some very exceptional cases, SemanticAnayzer fails to merge join tree. Example is below. create table a (val1 int, val2 int) create table b (val1 int, val2 int) create table c (val1 int, val2 int) create table d (val1 int, val2 int) create table e (val1 int, val2 int) 1. all same(single) join key -- one MR, good select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val1=e.val1 2. two join keys -- expected to have two MR, but resulted to three MR select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val2=e.val2 3. by changing the join order, we could attain two MR as first-expectation. select * from a join e on a.val2=e.val2 join c on a.val1=c.val1 join d on a.val1=d.val1 join b on a.val1=b.val1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2253) Merge failing of join tree in exceptional case
[ https://issues.apache.org/jira/browse/HIVE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2253: Attachment: HIVE-2253-0.8.0.patch merged to 0.8.0-SNAPSHOT Merge failing of join tree in exceptional case -- Key: HIVE-2253 URL: https://issues.apache.org/jira/browse/HIVE-2253 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.0 Environment: hadoop 0.20.2, hive 0.7.0 Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-2253-0.8.0.patch In some very exceptional cases, SemanticAnayzer fails to merge join tree. Example is below. create table a (val1 int, val2 int) create table b (val1 int, val2 int) create table c (val1 int, val2 int) create table d (val1 int, val2 int) create table e (val1 int, val2 int) 1. all same(single) join key -- one MR, good select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val1=e.val1 2. two join keys -- expected to have two MR, but resulted to three MR select * from a join b on a.val1=b.val1 join c on a.val1=c.val1 join d on a.val1=d.val1 join e on a.val2=e.val2 3. by changing the join order, we could attain two MR as first-expectation. select * from a join e on a.val2=e.val2 join c on a.val1=c.val1 join d on a.val1=d.val1 join b on a.val1=b.val1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2503) HiveServer should provide per session configuration
[ https://issues.apache.org/jira/browse/HIVE-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2503: Attachment: HIVE-2503-0.8.0.patch HiveServer should provide per session configuration --- Key: HIVE-2503 URL: https://issues.apache.org/jira/browse/HIVE-2503 Project: Hive Issue Type: Bug Components: CLI, Server Infrastructure Reporter: Navis Attachments: HIVE-2503-0.8.0.patch Currently ThriftHiveProcessorFactory returns same HiveConf instance to HiveServerHandler, making impossible to use per sesssion configuration. Just wrapping 'conf' - 'new HiveConf(conf)' seemed to solve this problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2329) Not using map aggregation, fails to execute group-by after cluster-by with same key
[ https://issues.apache.org/jira/browse/HIVE-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-2329: Attachment: HIVE-2329.0.8.0.patch Not using map aggregation, fails to execute group-by after cluster-by with same key --- Key: HIVE-2329 URL: https://issues.apache.org/jira/browse/HIVE-2329 Project: Hive Issue Type: Bug Affects Versions: 0.8.0 Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-2329.0.8.0.patch, HIVE-2329.1.patch, HIVE-2329.2.patch hive.map.aggr=false select Q1.key_int1, sum(Q1.key_int1), sum(distinct Q1.key_int1) from (select * from t1 cluster by key_int1) Q1 group by Q1.key_int1 resulted.. FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask from hadoop logs.. Caused by: java.lang.RuntimeException: cannot find field key from [] at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:321) at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:119) at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:82) at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:198) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433) I think the problem is caused by ReduceSinkDeDuplication, removing RS which was providing rs.key for GBY operation. If child of child RS is a GBY, we should bypass the optimization. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
[
https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2332:
Attachment: (was: HIVE-2332.1.patch)
If all of the parameters of distinct functions are exists in group by
columns, query fails in runtime
-
Key: HIVE-2332
URL: https://issues.apache.org/jira/browse/HIVE-2332
Project: Hive
Issue Type: Bug
Reporter: Navis
Assignee: Navis
Priority: Critical
Attachments: HIVE-2332.0.8.0.patch
select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
fails with message..
{code}
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
{code}
hadoop says..
{code}
Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
{code}
I think the deficient number of key expression, compared to number of key
column, is the problem, which should be equal or more.
Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
[
https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2332:
Attachment: (was: HIVE-2332.2.patch)
If all of the parameters of distinct functions are exists in group by
columns, query fails in runtime
-
Key: HIVE-2332
URL: https://issues.apache.org/jira/browse/HIVE-2332
Project: Hive
Issue Type: Bug
Reporter: Navis
Assignee: Navis
Priority: Critical
Attachments: HIVE-2332.0.8.0.patch
select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
fails with message..
{code}
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
{code}
hadoop says..
{code}
Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
{code}
I think the deficient number of key expression, compared to number of key
column, is the problem, which should be equal or more.
Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2332) If all of the parameters of distinct functions are exists in group by columns, query fails in runtime
[
https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-2332:
Attachment: HIVE-2332.0.8.0.patch
If all of the parameters of distinct functions are exists in group by
columns, query fails in runtime
-
Key: HIVE-2332
URL: https://issues.apache.org/jira/browse/HIVE-2332
Project: Hive
Issue Type: Bug
Reporter: Navis
Assignee: Navis
Priority: Critical
Attachments: HIVE-2332.0.8.0.patch
select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
fails with message..
{code}
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
{code}
hadoop says..
{code}
Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
at
org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
at
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
{code}
I think the deficient number of key expression, compared to number of key
column, is the problem, which should be equal or more.
Would it be solved if add some key expression? I'll try.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
