[jira] [Commented] (HIVE-8613) percentile_approx raise a comparator error
[
https://issues.apache.org/jira/browse/HIVE-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14256366#comment-14256366
]
Navis commented on HIVE-8613:
-
Ok, I got it. Instead of disabling CBO on all group-bys, I've disabled CBO on
the test. Reattaching patch for test.
percentile_approx raise a comparator error
--
Key: HIVE-8613
URL: https://issues.apache.org/jira/browse/HIVE-8613
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Nicolas Lalevée
Assignee: Navis
Attachments: HIVE-8613.1.patch.txt, HIVE-8613.2.patch.txt,
HIVE-8613.3.patch.txt
I have implemented an UADF which wrap around percentile_approx, and I have
been hit by the following stack trace:
{noformat}
Caused by: java.lang.IllegalArgumentException: Comparison method violates its
general contract!
at java.util.ComparableTimSort.mergeLo(ComparableTimSort.java:714)
at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:451)
at java.util.ComparableTimSort.mergeCollapse(ComparableTimSort.java:376)
at java.util.ComparableTimSort.sort(ComparableTimSort.java:182)
at java.util.ComparableTimSort.sort(ComparableTimSort.java:146)
at java.util.Arrays.sort(Arrays.java:472)
at java.util.Collections.sort(Collections.java:155)
at
org.apache.hadoop.hive.ql.udf.generic.NumericHistogram.merge(NumericHistogram.java:161)
{noformat}
In my use case, with my special UADF, there were Double.NaN
Implementing a proper double comparaison with Double.compare in
com.goojet.mapred.hive.udf.NumericHistogram.Coord.compareTo(Object) fixes it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8613) percentile_approx raise a comparator error
[
https://issues.apache.org/jira/browse/HIVE-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-8613:
Attachment: HIVE-8613.3.patch.txt
percentile_approx raise a comparator error
--
Key: HIVE-8613
URL: https://issues.apache.org/jira/browse/HIVE-8613
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Nicolas Lalevée
Assignee: Navis
Attachments: HIVE-8613.1.patch.txt, HIVE-8613.2.patch.txt,
HIVE-8613.3.patch.txt
I have implemented an UADF which wrap around percentile_approx, and I have
been hit by the following stack trace:
{noformat}
Caused by: java.lang.IllegalArgumentException: Comparison method violates its
general contract!
at java.util.ComparableTimSort.mergeLo(ComparableTimSort.java:714)
at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:451)
at java.util.ComparableTimSort.mergeCollapse(ComparableTimSort.java:376)
at java.util.ComparableTimSort.sort(ComparableTimSort.java:182)
at java.util.ComparableTimSort.sort(ComparableTimSort.java:146)
at java.util.Arrays.sort(Arrays.java:472)
at java.util.Collections.sort(Collections.java:155)
at
org.apache.hadoop.hive.ql.udf.generic.NumericHistogram.merge(NumericHistogram.java:161)
{noformat}
In my use case, with my special UADF, there were Double.NaN
Implementing a proper double comparaison with Double.compare in
com.goojet.mapred.hive.udf.NumericHistogram.Coord.compareTo(Object) fixes it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9195) CBO changes constant to column type
Navis created HIVE-9195:
---
Summary: CBO changes constant to column type
Key: HIVE-9195
URL: https://issues.apache.org/jira/browse/HIVE-9195
Project: Hive
Issue Type: Bug
Components: CBO
Reporter: Navis
Making testcase for HIVE-8613, I've found CBO changes constant expr to column
expr. For example (only in test mode).
{code}
CREATE TABLE bucket (key double, value string) CLUSTERED BY (key) SORTED BY
(key DESC) INTO 4 BUCKETS STORED AS TEXTFILE;
load data local inpath '../../data/files/srcsortbucket1outof4.txt' INTO TABLE
bucket;
load data local inpath '../../data/files/srcsortbucket2outof4.txt' INTO TABLE
bucket;
load data local inpath '../../data/files/srcsortbucket3outof4.txt' INTO TABLE
bucket;
load data local inpath '../../data/files/srcsortbucket4outof4.txt' INTO TABLE
bucket;
select percentile_approx(case when key 100 then cast('NaN' as double) else
key end, 0.5) from bucket;
{code}
It works in shell but in TestCliDriver, that induces argument type exception
creating udaf evaluator, which expects constant OI for second argument.
{noformat}
2014-12-22 17:03:31,433 ERROR parse.SemanticAnalyzer
(SemanticAnalyzer.java:analyzeInternal(10102)) - CBO failed, skipping CBO.
org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: The second argument
must be a constant, but double was passed instead.
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFPercentileApprox.getEvaluator(GenericUDAFPercentileApprox.java:146)
at
org.apache.hadoop.hive.ql.exec.FunctionRegistry.getGenericUDAFEvaluator(FunctionRegistry.java:1160)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:3794)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanMapGroupByOperator(SemanticAnalyzer.java:4467)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanMapAggrNoSkew(SemanticAnalyzer.java:5536)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8884)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9745)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9638)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10086)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224)
at
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:419)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1107)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1155)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1034)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:206)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:158)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:369)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:304)
at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:877)
at
org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:136)
at
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23(TestCliDriver.java:120)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9195) CBO changes constant to column type
[
https://issues.apache.org/jira/browse/HIVE-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9195:
Status: Patch Available (was: Open)
CBO changes constant to column type
---
Key: HIVE-9195
URL: https://issues.apache.org/jira/browse/HIVE-9195
Project: Hive
Issue Type: Bug
Components: CBO
Reporter: Navis
Attachments: HIVE-9195.1.patch.txt
Making testcase for HIVE-8613, I've found CBO changes constant expr to column
expr. For example (only in test mode).
{code}
CREATE TABLE bucket (key double, value string) CLUSTERED BY (key) SORTED BY
(key DESC) INTO 4 BUCKETS STORED AS TEXTFILE;
load data local inpath '../../data/files/srcsortbucket1outof4.txt' INTO TABLE
bucket;
load data local inpath '../../data/files/srcsortbucket2outof4.txt' INTO TABLE
bucket;
load data local inpath '../../data/files/srcsortbucket3outof4.txt' INTO TABLE
bucket;
load data local inpath '../../data/files/srcsortbucket4outof4.txt' INTO TABLE
bucket;
select percentile_approx(case when key 100 then cast('NaN' as double) else
key end, 0.5) from bucket;
{code}
It works in shell but in TestCliDriver, that induces argument type exception
creating udaf evaluator, which expects constant OI for second argument.
{noformat}
2014-12-22 17:03:31,433 ERROR parse.SemanticAnalyzer
(SemanticAnalyzer.java:analyzeInternal(10102)) - CBO failed, skipping CBO.
org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: The second argument
must be a constant, but double was passed instead.
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFPercentileApprox.getEvaluator(GenericUDAFPercentileApprox.java:146)
at
org.apache.hadoop.hive.ql.exec.FunctionRegistry.getGenericUDAFEvaluator(FunctionRegistry.java:1160)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:3794)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanMapGroupByOperator(SemanticAnalyzer.java:4467)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanMapAggrNoSkew(SemanticAnalyzer.java:5536)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8884)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9745)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9638)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10086)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224)
at
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:419)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1107)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1155)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1034)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:206)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:158)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:369)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:304)
at
org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:877)
at
org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:136)
at
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23(TestCliDriver.java:120)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9195) CBO changes constant to column type
[
https://issues.apache.org/jira/browse/HIVE-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9195:
Attachment: HIVE-9195.1.patch.txt
Quick shot
CBO changes constant to column type
---
Key: HIVE-9195
URL: https://issues.apache.org/jira/browse/HIVE-9195
Project: Hive
Issue Type: Bug
Components: CBO
Reporter: Navis
Attachments: HIVE-9195.1.patch.txt
Making testcase for HIVE-8613, I've found CBO changes constant expr to column
expr. For example (only in test mode).
{code}
CREATE TABLE bucket (key double, value string) CLUSTERED BY (key) SORTED BY
(key DESC) INTO 4 BUCKETS STORED AS TEXTFILE;
load data local inpath '../../data/files/srcsortbucket1outof4.txt' INTO TABLE
bucket;
load data local inpath '../../data/files/srcsortbucket2outof4.txt' INTO TABLE
bucket;
load data local inpath '../../data/files/srcsortbucket3outof4.txt' INTO TABLE
bucket;
load data local inpath '../../data/files/srcsortbucket4outof4.txt' INTO TABLE
bucket;
select percentile_approx(case when key 100 then cast('NaN' as double) else
key end, 0.5) from bucket;
{code}
It works in shell but in TestCliDriver, that induces argument type exception
creating udaf evaluator, which expects constant OI for second argument.
{noformat}
2014-12-22 17:03:31,433 ERROR parse.SemanticAnalyzer
(SemanticAnalyzer.java:analyzeInternal(10102)) - CBO failed, skipping CBO.
org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: The second argument
must be a constant, but double was passed instead.
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFPercentileApprox.getEvaluator(GenericUDAFPercentileApprox.java:146)
at
org.apache.hadoop.hive.ql.exec.FunctionRegistry.getGenericUDAFEvaluator(FunctionRegistry.java:1160)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getGenericUDAFEvaluator(SemanticAnalyzer.java:3794)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanMapGroupByOperator(SemanticAnalyzer.java:4467)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genGroupByPlanMapAggrNoSkew(SemanticAnalyzer.java:5536)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8884)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9745)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9638)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10086)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224)
at
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:419)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1107)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1155)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1034)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:206)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:158)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:369)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:304)
at
org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:877)
at
org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:136)
at
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23(TestCliDriver.java:120)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9172) Merging HIVE-5871 into LazySimpleSerDe
[ https://issues.apache.org/jira/browse/HIVE-9172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-9172: Attachment: HIVE-9172.3.patch.txt Fixed test fails Merging HIVE-5871 into LazySimpleSerDe -- Key: HIVE-9172 URL: https://issues.apache.org/jira/browse/HIVE-9172 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-9172.1.patch.txt, HIVE-9172.2.patch.txt, HIVE-9172.3.patch.txt Merging multi character support for field delimiter to LazySimpleSerDe -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-9180) Input parsing failing for ALTER TABLE when database and table are specified
[ https://issues.apache.org/jira/browse/HIVE-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis resolved HIVE-9180. - Resolution: Fixed Fixed in hive-0.14.0 Input parsing failing for ALTER TABLE when database and table are specified --- Key: HIVE-9180 URL: https://issues.apache.org/jira/browse/HIVE-9180 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.12.0 Environment: Cloudera Hadoop distribution CDH5.1.3 Reporter: Brian Kessler Priority: Minor The following appears to fail due to an input parsing error ALTER TABLE database.table SET FILEFORMAT RCFile; but can be worked around by using the database so it does not need to be explicitly specified in the command at the hive interpreter. USE database; ALTER TABLE table SET FILEFORMAT RCFile; See below for stack trace followed by successful workaround. hive ALTER TABLE dev.bk_smp_advertiser_events_rcfile SET FILEFORMAT RCFile; MismatchedTokenException(220!=93) at org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617) at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115) at org.apache.hadoop.hive.ql.parse.HiveParser.alterStatementSuffixExchangePartition(HiveParser.java:12896) at org.apache.hadoop.hive.ql.parse.HiveParser.alterTableStatementSuffix(HiveParser.java:6744) at org.apache.hadoop.hive.ql.parse.HiveParser.alterStatement(HiveParser.java:6447) at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:2129) at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1344) at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:983) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:190) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:434) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:352) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:995) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1038) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:790) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:623) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) FAILED: ParseException line 1:48 mismatched input 'SET' expecting KW_EXCHANGE near 'bk_smp_advertiser_events_rcfile' in alter exchange partition hive use dev; OK Time taken: 0.033 seconds hive ALTER TABLE bk_smp_advertiser_events_rcfile SET FILEFORMAT RCFile; OK Time taken: 0.299 seconds hive -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-8613) percentile_approx raise a comparator error
[
https://issues.apache.org/jira/browse/HIVE-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis reassigned HIVE-8613:
---
Assignee: Navis (was: Nicolas Lalevée)
percentile_approx raise a comparator error
--
Key: HIVE-8613
URL: https://issues.apache.org/jira/browse/HIVE-8613
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Nicolas Lalevée
Assignee: Navis
Attachments: HIVE-8613.1.patch.txt, HIVE-8613.2.patch.txt
I have implemented an UADF which wrap around percentile_approx, and I have
been hit by the following stack trace:
{noformat}
Caused by: java.lang.IllegalArgumentException: Comparison method violates its
general contract!
at java.util.ComparableTimSort.mergeLo(ComparableTimSort.java:714)
at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:451)
at java.util.ComparableTimSort.mergeCollapse(ComparableTimSort.java:376)
at java.util.ComparableTimSort.sort(ComparableTimSort.java:182)
at java.util.ComparableTimSort.sort(ComparableTimSort.java:146)
at java.util.Arrays.sort(Arrays.java:472)
at java.util.Collections.sort(Collections.java:155)
at
org.apache.hadoop.hive.ql.udf.generic.NumericHistogram.merge(NumericHistogram.java:161)
{noformat}
In my use case, with my special UADF, there were Double.NaN
Implementing a proper double comparaison with Double.compare in
com.goojet.mapred.hive.udf.NumericHistogram.Coord.compareTo(Object) fixes it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8613) percentile_approx raise a comparator error
[
https://issues.apache.org/jira/browse/HIVE-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-8613:
Attachment: HIVE-8613.2.patch.txt
Added a test
percentile_approx raise a comparator error
--
Key: HIVE-8613
URL: https://issues.apache.org/jira/browse/HIVE-8613
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Nicolas Lalevée
Assignee: Nicolas Lalevée
Attachments: HIVE-8613.1.patch.txt, HIVE-8613.2.patch.txt
I have implemented an UADF which wrap around percentile_approx, and I have
been hit by the following stack trace:
{noformat}
Caused by: java.lang.IllegalArgumentException: Comparison method violates its
general contract!
at java.util.ComparableTimSort.mergeLo(ComparableTimSort.java:714)
at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:451)
at java.util.ComparableTimSort.mergeCollapse(ComparableTimSort.java:376)
at java.util.ComparableTimSort.sort(ComparableTimSort.java:182)
at java.util.ComparableTimSort.sort(ComparableTimSort.java:146)
at java.util.Arrays.sort(Arrays.java:472)
at java.util.Collections.sort(Collections.java:155)
at
org.apache.hadoop.hive.ql.udf.generic.NumericHistogram.merge(NumericHistogram.java:161)
{noformat}
In my use case, with my special UADF, there were Double.NaN
Implementing a proper double comparaison with Double.compare in
com.goojet.mapred.hive.udf.NumericHistogram.Coord.compareTo(Object) fixes it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9154) Cache pathToPartitionInfo in context aware record reader
[ https://issues.apache.org/jira/browse/HIVE-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14255460#comment-14255460 ] Navis commented on HIVE-9154: - [~xuefuz] This patch had a list of failed tests and seemed not have any following fix. How could it be committed to trunk? Anyway, this caused fail of nonmr_fetch and virtual_column. Cache pathToPartitionInfo in context aware record reader Key: HIVE-9154 URL: https://issues.apache.org/jira/browse/HIVE-9154 Project: Hive Issue Type: Bug Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: spark-branch, 0.15.0 Attachments: HIVE-9154.1-spark.patch, HIVE-9154.1-spark.patch, HIVE-9154.2.patch This is similar to HIVE-9127. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9172) Merging HIVE-5871 into LazySimpleSerDe
[ https://issues.apache.org/jira/browse/HIVE-9172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14255461#comment-14255461 ] Navis commented on HIVE-9172: - Test fails are caused by HIVE-9154. Seemed ready to be reviewed. Merging HIVE-5871 into LazySimpleSerDe -- Key: HIVE-9172 URL: https://issues.apache.org/jira/browse/HIVE-9172 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-9172.1.patch.txt, HIVE-9172.2.patch.txt, HIVE-9172.3.patch.txt Merging multi character support for field delimiter to LazySimpleSerDe -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9172) Merging HIVE-5871 into LazySimpleSerDe
Navis created HIVE-9172: --- Summary: Merging HIVE-5871 into LazySimpleSerDe Key: HIVE-9172 URL: https://issues.apache.org/jira/browse/HIVE-9172 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Navis Assignee: Navis Priority: Minor Merging multi character support for field delimiter to LazySimpleSerDe -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9172) Merging HIVE-5871 into LazySimpleSerDe
[ https://issues.apache.org/jira/browse/HIVE-9172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-9172: Status: Patch Available (was: Open) Merging HIVE-5871 into LazySimpleSerDe -- Key: HIVE-9172 URL: https://issues.apache.org/jira/browse/HIVE-9172 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-9172.1.patch.txt Merging multi character support for field delimiter to LazySimpleSerDe -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9172) Merging HIVE-5871 into LazySimpleSerDe
[ https://issues.apache.org/jira/browse/HIVE-9172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-9172: Attachment: HIVE-9172.1.patch.txt Running preliminary test to confirm that things are not broken badly. Merging HIVE-5871 into LazySimpleSerDe -- Key: HIVE-9172 URL: https://issues.apache.org/jira/browse/HIVE-9172 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-9172.1.patch.txt Merging multi character support for field delimiter to LazySimpleSerDe -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9172) Merging HIVE-5871 into LazySimpleSerDe
[ https://issues.apache.org/jira/browse/HIVE-9172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-9172: Attachment: HIVE-9172.2.patch.txt Merging HIVE-5871 into LazySimpleSerDe -- Key: HIVE-9172 URL: https://issues.apache.org/jira/browse/HIVE-9172 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-9172.1.patch.txt, HIVE-9172.2.patch.txt Merging multi character support for field delimiter to LazySimpleSerDe -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9141) HiveOnTez: mix of union all, distinct, group by generates error
[ https://issues.apache.org/jira/browse/HIVE-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252656#comment-14252656 ] Navis commented on HIVE-9141: - [~vikram.dixit] My patch was just a quick shot (as I stated submitting the patch) thinking that parents of root should reference the work of root operator instead of currently evolving work. As it's committed already, you can try that in following issue. Frankly, I don't know well on tez compiling. HiveOnTez: mix of union all, distinct, group by generates error --- Key: HIVE-9141 URL: https://issues.apache.org/jira/browse/HIVE-9141 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.15.0 Reporter: Pengcheng Xiong Assignee: Navis Fix For: 0.15.0 Attachments: HIVE-9141.1.patch.txt Here is the way to produce it: in Hive q test setting (with src table) set hive.execution.engine=tez; SELECT key, value FROM ( SELECT key, value FROM src UNION ALL SELECT key, key as value FROM ( SELECT distinct key FROM ( SELECT key, value FROM (SELECT key, value FROM src UNION ALL SELECT key, value FROM src )t1 group by key, value )t2 )t3 )t4 group by key, value; will generate 2014-12-16 23:19:13,593 ERROR ql.Driver (SessionState.java:printError(834)) - FAILED: ClassCastException org.apache.hadoop.hive.ql.plan.MapWork cannot be cast to org.apache.hadoop.hive.ql.plan.ReduceWork java.lang.ClassCastException: org.apache.hadoop.hive.ql.plan.MapWork cannot be cast to org.apache.hadoop.hive.ql.plan.ReduceWork at org.apache.hadoop.hive.ql.parse.GenTezWork.process(GenTezWork.java:361) at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:87) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.startWalking(GenTezWorkWalker.java:69) at org.apache.hadoop.hive.ql.parse.TezCompiler.generateTaskTree(TezCompiler.java:368) at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:202) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10202) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:419) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1107) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1155) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1034) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:206) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:158) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:369) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:304) at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:834) at org.apache.hadoop.hive.cli.TestMiniTezCliDriver.runTest(TestMiniTezCliDriver.java:136) at org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_uniontez2(TestMiniTezCliDriver.java:120) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-9150) Unrelated types are compared in GenTezWork#getFollowingWorkIndex()
[
https://issues.apache.org/jira/browse/HIVE-9150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis resolved HIVE-9150.
-
Resolution: Fixed
Fix Version/s: 0.15.0
Unrelated types are compared in GenTezWork#getFollowingWorkIndex()
--
Key: HIVE-9150
URL: https://issues.apache.org/jira/browse/HIVE-9150
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Fix For: 0.15.0
Here is the code:
{code}
if (tezWork.getEdgeProperty(unionWork,
baseWork).equals(TezEdgeProperty.EdgeType.CONTAINS)) {
{code}
getEdgeProperty() returns TezEdgeProperty which is compared with
TezEdgeProperty$EdgeType
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-7024) Escape control characters for explain result
[ https://issues.apache.org/jira/browse/HIVE-7024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-7024: Resolution: Fixed Fix Version/s: 0.15.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks Sergey! Escape control characters for explain result Key: HIVE-7024 URL: https://issues.apache.org/jira/browse/HIVE-7024 Project: Hive Issue Type: Bug Reporter: Navis Assignee: Navis Priority: Trivial Fix For: 0.15.0 Attachments: HIVE-7024.1.patch.txt, HIVE-7024.2.patch.txt, HIVE-7024.3.patch.txt, HIVE-7024.4.patch.txt, HIVE-7024.5.patch.txt, HIVE-7024.6.patch.txt Comments for columns are now delimited by 0x00, which is binary and make git refuse to make proper diff file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8960) ParsingException in the WHERE statement with a Sub Query
[ https://issues.apache.org/jira/browse/HIVE-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252759#comment-14252759 ] Navis commented on HIVE-8960: - Simply, it's not supported. ParsingException in the WHERE statement with a Sub Query Key: HIVE-8960 URL: https://issues.apache.org/jira/browse/HIVE-8960 Project: Hive Issue Type: Bug Components: Parser Affects Versions: 0.13.0 Environment: Secured HDP 2.1.3 with Hive 0.13.0 Reporter: Rémy SAISSY Comparison with a Sub query in a WHERE statement does not work. Given that id_chargement is an integer: USE db1; SELECT * FROM tbl1 a WHERE a.id_chargement (SELECT MAX(b.id_chargement) FROM tbl2 b); or SELECT * FROM tbl1 a WHERE a.id_chargement (SELECT b.id_chargement FROM tbl2 b LIMIT 1); Both return the following parsing error: Error: Error while compiling statement: FAILED: ParseException line 1:88 cannot recognize input near 'SELECT' 'b' '.' in expression specification (state=42000,code=4) java.sql.SQLException: Error while compiling statement: FAILED: ParseException line 1:88 cannot recognize input near 'SELECT' 'b' '.' in expression specification at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:121) at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:109) at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:231) at org.apache.hive.beeline.Commands.execute(Commands.java:736) at org.apache.hive.beeline.Commands.sql(Commands.java:657) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:804) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:659) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:368) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:351) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9141) HiveOnTez: mix of union all, distinct, group by generates error
[ https://issues.apache.org/jira/browse/HIVE-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-9141: Attachment: HIVE-9141.1.patch.txt HiveOnTez: mix of union all, distinct, group by generates error --- Key: HIVE-9141 URL: https://issues.apache.org/jira/browse/HIVE-9141 Project: Hive Issue Type: Bug Reporter: Pengcheng Xiong Attachments: HIVE-9141.1.patch.txt Here is the way to produce it: in Hive q test setting (with src table) set hive.execution.engine=tez; SELECT key, value FROM ( SELECT key, value FROM src UNION ALL SELECT key, key as value FROM ( SELECT distinct key FROM ( SELECT key, value FROM (SELECT key, value FROM src UNION ALL SELECT key, value FROM src )t1 group by key, value )t2 )t3 )t4 group by key, value; will generate 2014-12-16 23:19:13,593 ERROR ql.Driver (SessionState.java:printError(834)) - FAILED: ClassCastException org.apache.hadoop.hive.ql.plan.MapWork cannot be cast to org.apache.hadoop.hive.ql.plan.ReduceWork java.lang.ClassCastException: org.apache.hadoop.hive.ql.plan.MapWork cannot be cast to org.apache.hadoop.hive.ql.plan.ReduceWork at org.apache.hadoop.hive.ql.parse.GenTezWork.process(GenTezWork.java:361) at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:87) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.startWalking(GenTezWorkWalker.java:69) at org.apache.hadoop.hive.ql.parse.TezCompiler.generateTaskTree(TezCompiler.java:368) at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:202) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10202) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:419) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1107) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1155) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1034) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:206) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:158) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:369) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:304) at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:834) at org.apache.hadoop.hive.cli.TestMiniTezCliDriver.runTest(TestMiniTezCliDriver.java:136) at org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_uniontez2(TestMiniTezCliDriver.java:120) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9141) HiveOnTez: mix of union all, distinct, group by generates error
[ https://issues.apache.org/jira/browse/HIVE-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-9141: Assignee: Navis Status: Patch Available (was: Open) Quick shot. HiveOnTez: mix of union all, distinct, group by generates error --- Key: HIVE-9141 URL: https://issues.apache.org/jira/browse/HIVE-9141 Project: Hive Issue Type: Bug Reporter: Pengcheng Xiong Assignee: Navis Attachments: HIVE-9141.1.patch.txt Here is the way to produce it: in Hive q test setting (with src table) set hive.execution.engine=tez; SELECT key, value FROM ( SELECT key, value FROM src UNION ALL SELECT key, key as value FROM ( SELECT distinct key FROM ( SELECT key, value FROM (SELECT key, value FROM src UNION ALL SELECT key, value FROM src )t1 group by key, value )t2 )t3 )t4 group by key, value; will generate 2014-12-16 23:19:13,593 ERROR ql.Driver (SessionState.java:printError(834)) - FAILED: ClassCastException org.apache.hadoop.hive.ql.plan.MapWork cannot be cast to org.apache.hadoop.hive.ql.plan.ReduceWork java.lang.ClassCastException: org.apache.hadoop.hive.ql.plan.MapWork cannot be cast to org.apache.hadoop.hive.ql.plan.ReduceWork at org.apache.hadoop.hive.ql.parse.GenTezWork.process(GenTezWork.java:361) at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:87) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.startWalking(GenTezWorkWalker.java:69) at org.apache.hadoop.hive.ql.parse.TezCompiler.generateTaskTree(TezCompiler.java:368) at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:202) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10202) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:419) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1107) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1155) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1034) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:206) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:158) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:369) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:304) at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:834) at org.apache.hadoop.hive.cli.TestMiniTezCliDriver.runTest(TestMiniTezCliDriver.java:136) at org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_uniontez2(TestMiniTezCliDriver.java:120) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9141) HiveOnTez: mix of union all, distinct, group by generates error
[ https://issues.apache.org/jira/browse/HIVE-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-9141: Attachment: (was: HIVE-9141.1.patch.txt) HiveOnTez: mix of union all, distinct, group by generates error --- Key: HIVE-9141 URL: https://issues.apache.org/jira/browse/HIVE-9141 Project: Hive Issue Type: Bug Reporter: Pengcheng Xiong Assignee: Navis Attachments: HIVE-9141.1.patch.txt Here is the way to produce it: in Hive q test setting (with src table) set hive.execution.engine=tez; SELECT key, value FROM ( SELECT key, value FROM src UNION ALL SELECT key, key as value FROM ( SELECT distinct key FROM ( SELECT key, value FROM (SELECT key, value FROM src UNION ALL SELECT key, value FROM src )t1 group by key, value )t2 )t3 )t4 group by key, value; will generate 2014-12-16 23:19:13,593 ERROR ql.Driver (SessionState.java:printError(834)) - FAILED: ClassCastException org.apache.hadoop.hive.ql.plan.MapWork cannot be cast to org.apache.hadoop.hive.ql.plan.ReduceWork java.lang.ClassCastException: org.apache.hadoop.hive.ql.plan.MapWork cannot be cast to org.apache.hadoop.hive.ql.plan.ReduceWork at org.apache.hadoop.hive.ql.parse.GenTezWork.process(GenTezWork.java:361) at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:87) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.startWalking(GenTezWorkWalker.java:69) at org.apache.hadoop.hive.ql.parse.TezCompiler.generateTaskTree(TezCompiler.java:368) at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:202) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10202) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:419) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1107) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1155) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1034) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:206) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:158) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:369) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:304) at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:834) at org.apache.hadoop.hive.cli.TestMiniTezCliDriver.runTest(TestMiniTezCliDriver.java:136) at org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_uniontez2(TestMiniTezCliDriver.java:120) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9141) HiveOnTez: mix of union all, distinct, group by generates error
[ https://issues.apache.org/jira/browse/HIVE-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-9141: Attachment: HIVE-9141.1.patch.txt HiveOnTez: mix of union all, distinct, group by generates error --- Key: HIVE-9141 URL: https://issues.apache.org/jira/browse/HIVE-9141 Project: Hive Issue Type: Bug Reporter: Pengcheng Xiong Assignee: Navis Attachments: HIVE-9141.1.patch.txt Here is the way to produce it: in Hive q test setting (with src table) set hive.execution.engine=tez; SELECT key, value FROM ( SELECT key, value FROM src UNION ALL SELECT key, key as value FROM ( SELECT distinct key FROM ( SELECT key, value FROM (SELECT key, value FROM src UNION ALL SELECT key, value FROM src )t1 group by key, value )t2 )t3 )t4 group by key, value; will generate 2014-12-16 23:19:13,593 ERROR ql.Driver (SessionState.java:printError(834)) - FAILED: ClassCastException org.apache.hadoop.hive.ql.plan.MapWork cannot be cast to org.apache.hadoop.hive.ql.plan.ReduceWork java.lang.ClassCastException: org.apache.hadoop.hive.ql.plan.MapWork cannot be cast to org.apache.hadoop.hive.ql.plan.ReduceWork at org.apache.hadoop.hive.ql.parse.GenTezWork.process(GenTezWork.java:361) at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:87) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103) at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.startWalking(GenTezWorkWalker.java:69) at org.apache.hadoop.hive.ql.parse.TezCompiler.generateTaskTree(TezCompiler.java:368) at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:202) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10202) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:419) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1107) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1155) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1034) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:206) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:158) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:369) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:304) at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:834) at org.apache.hadoop.hive.cli.TestMiniTezCliDriver.runTest(TestMiniTezCliDriver.java:136) at org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_uniontez2(TestMiniTezCliDriver.java:120) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7024) Escape control characters for explain result
[ https://issues.apache.org/jira/browse/HIVE-7024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-7024: Attachment: HIVE-7024.6.patch.txt Escape control characters for explain result Key: HIVE-7024 URL: https://issues.apache.org/jira/browse/HIVE-7024 Project: Hive Issue Type: Bug Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-7024.1.patch.txt, HIVE-7024.2.patch.txt, HIVE-7024.3.patch.txt, HIVE-7024.4.patch.txt, HIVE-7024.5.patch.txt, HIVE-7024.6.patch.txt Comments for columns are now delimited by 0x00, which is binary and make git refuse to make proper diff file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9113) Explain on query failed with NPE
[
https://issues.apache.org/jira/browse/HIVE-9113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9113:
Resolution: Fixed
Fix Version/s: 0.15.0
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks, Szehon!
Explain on query failed with NPE
Key: HIVE-9113
URL: https://issues.apache.org/jira/browse/HIVE-9113
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Chao
Assignee: Navis
Fix For: 0.15.0
Attachments: HIVE-9113.1.patch.txt
Run explain on the following query:
{noformat}
select p.p_partkey, li.l_suppkey
from (select distinct l_partkey as p_partkey from lineitem) p join lineitem
li on p.p_partkey = li.l_partkey
where li.l_linenumber = 1 and
li.l_orderkey in (select l_orderkey where l_linenumber = li.l_linenumber)
;
{noformat}
gave me NPE:
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.parse.QBSubQuery.validateAndRewriteAST(QBSubQuery.java:516)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFilterPlan(SemanticAnalyzer.java:2605)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8866)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9745)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9638)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10125)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:221)
at
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:221)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:420)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:306)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1108)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1045)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1035)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:199)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:151)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:362)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:720)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:639)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
Is this query invalid? If so, it should at least give some explanations, not
just a plain NPE message, and left user clueless.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8848) data loading from text files or text file processing doesn't handle nulls correctly
[ https://issues.apache.org/jira/browse/HIVE-8848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-8848: Resolution: Fixed Fix Version/s: 0.15.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Sergey! data loading from text files or text file processing doesn't handle nulls correctly --- Key: HIVE-8848 URL: https://issues.apache.org/jira/browse/HIVE-8848 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Navis Fix For: 0.15.0 Attachments: HIVE-8848.01.patch, HIVE-8848.2.patch.txt, HIVE-8848.3.patch.txt, HIVE-8848.4.patch.txt, HIVE-8848.patch I am not sure how nulls are supposed to be stored in text tables, but after loading some data with null or NULL strings, or x00 characters, we get bunch of annoying logging from LazyPrimitive that data is not in INT format and was converted to null, with data being null (string saying null, I assume from the code). Either load should load them as nulls, or there should be some defined way to load nulls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9160) Suspicious comparing logic in LazyPrimitive
Navis created HIVE-9160:
---
Summary: Suspicious comparing logic in LazyPrimitive
Key: HIVE-9160
URL: https://issues.apache.org/jira/browse/HIVE-9160
Project: Hive
Issue Type: Bug
Reporter: Navis
Priority: Minor
{code}
@Override
public boolean equals(Object obj) {
if (!(obj instanceof LazyPrimitive?, ?)) {
return false;
}
if (data == obj) {
return true;
}
if (data == null || obj == null) {
return false;
}
return data.equals(((LazyPrimitive?, ?) obj).getWritableObject());
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9160) Suspicious comparing logic in LazyPrimitive
[
https://issues.apache.org/jira/browse/HIVE-9160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9160:
Attachment: HIVE-9160.1.patch.txt
Suspicious comparing logic in LazyPrimitive
---
Key: HIVE-9160
URL: https://issues.apache.org/jira/browse/HIVE-9160
Project: Hive
Issue Type: Bug
Reporter: Navis
Priority: Minor
Attachments: HIVE-9160.1.patch.txt
{code}
@Override
public boolean equals(Object obj) {
if (!(obj instanceof LazyPrimitive?, ?)) {
return false;
}
if (data == obj) {
return true;
}
if (data == null || obj == null) {
return false;
}
return data.equals(((LazyPrimitive?, ?) obj).getWritableObject());
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9160) Suspicious comparing logic in LazyPrimitive
[
https://issues.apache.org/jira/browse/HIVE-9160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9160:
Assignee: Navis
Status: Patch Available (was: Open)
Suspicious comparing logic in LazyPrimitive
---
Key: HIVE-9160
URL: https://issues.apache.org/jira/browse/HIVE-9160
Project: Hive
Issue Type: Bug
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-9160.1.patch.txt
{code}
@Override
public boolean equals(Object obj) {
if (!(obj instanceof LazyPrimitive?, ?)) {
return false;
}
if (data == obj) {
return true;
}
if (data == null || obj == null) {
return false;
}
return data.equals(((LazyPrimitive?, ?) obj).getWritableObject());
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9160) Suspicious comparing logic in LazyPrimitive
[
https://issues.apache.org/jira/browse/HIVE-9160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9160:
Attachment: HIVE-9160.1.patch.txt
Suspicious comparing logic in LazyPrimitive
---
Key: HIVE-9160
URL: https://issues.apache.org/jira/browse/HIVE-9160
Project: Hive
Issue Type: Bug
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-9160.1.patch.txt
{code}
@Override
public boolean equals(Object obj) {
if (!(obj instanceof LazyPrimitive?, ?)) {
return false;
}
if (data == obj) {
return true;
}
if (data == null || obj == null) {
return false;
}
return data.equals(((LazyPrimitive?, ?) obj).getWritableObject());
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9160) Suspicious comparing logic in LazyPrimitive
[
https://issues.apache.org/jira/browse/HIVE-9160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9160:
Attachment: (was: HIVE-9160.1.patch.txt)
Suspicious comparing logic in LazyPrimitive
---
Key: HIVE-9160
URL: https://issues.apache.org/jira/browse/HIVE-9160
Project: Hive
Issue Type: Bug
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-9160.1.patch.txt
{code}
@Override
public boolean equals(Object obj) {
if (!(obj instanceof LazyPrimitive?, ?)) {
return false;
}
if (data == obj) {
return true;
}
if (data == null || obj == null) {
return false;
}
return data.equals(((LazyPrimitive?, ?) obj).getWritableObject());
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7977) Avoid creating serde for partitions if possible in FetchTask
[ https://issues.apache.org/jira/browse/HIVE-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251314#comment-14251314 ] Navis commented on HIVE-7977: - Update RB (don't know the reason but ok now). Avoid creating serde for partitions if possible in FetchTask Key: HIVE-7977 URL: https://issues.apache.org/jira/browse/HIVE-7977 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-7977.1.patch.txt, HIVE-7977.2.patch.txt, HIVE-7977.3.patch.txt, HIVE-7977.4.patch.txt, HIVE-7977.5.patch.txt, HIVE-7977.6.patch.txt Currently, FetchTask creates SerDe instance thrice for each partition, which can be avoided if it's same with table SerDe. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9096) GenericUDF may be left unclosed in PartitionPrune#visitCall()
[
https://issues.apache.org/jira/browse/HIVE-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9096:
Resolution: Fixed
Fix Version/s: 0.15.0
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks Niranjan Singh, for the contribution.
GenericUDF may be left unclosed in PartitionPrune#visitCall()
-
Key: HIVE-9096
URL: https://issues.apache.org/jira/browse/HIVE-9096
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Fix For: 0.15.0
Attachments: HIVE-9096.patch
In
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/PartitionPrune.java
:
{code}
GenericUDF hiveUDF = SqlFunctionConverter.getHiveUDF(call.getOperator(),
call.getType(), call.operands.size());
if (hiveUDF != null
!FunctionRegistry.isDeterministic(hiveUDF)) {
return null;
}
{code}
hiveUDF should be closed upon return.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9123) Query with join fails with NPE when using join auto conversion
[
https://issues.apache.org/jira/browse/HIVE-9123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251327#comment-14251327
]
Navis commented on HIVE-9123:
-
[~kgs] Cannot reproduce in trunk. Could you try with hive-0.14.1?
Query with join fails with NPE when using join auto conversion
--
Key: HIVE-9123
URL: https://issues.apache.org/jira/browse/HIVE-9123
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.1
Environment: CDH5 with Hive 0.13.1
Reporter: Kamil Gorlo
I have two simple tables:
desc kgorlo_comm;
| col_name | data_type | comment |
| id| bigint | |
| dest_id | bigint | |
desc kgorlo_log;
| col_name | data_type | comment |
| id| bigint | |
| dest_id | bigint | |
| tstamp| bigint | |
With data:
select * from kgorlo_comm;
| kgorlo_comm.id | kgorlo_comm.dest_id |
| 1 | 2|
| 2 | 1|
| 1 | 3|
| 2 | 3|
| 3 | 5|
| 4 | 5|
select * from kgorlo_log;
| kgorlo_log.id | kgorlo_log.dest_id | kgorlo_log.tstamp |
| 1 | 2 | 0 |
| 1 | 3 | 0 |
| 1 | 5 | 0 |
| 3 | 1 | 0 |
Following query fails in second stage of execution:
bq. select v.id, v.dest_id from kgorlo_log v join (select id, dest_id,
count(*) as wiad from kgorlo_comm group by id, dest_id)com1 on com1.id=v.id
and com1.dest_id=v.dest_id;
with following exception:
{quote}
2014-12-16 17:09:17,629 ERROR [uber-SubtaskRunner]
org.apache.hadoop.hive.ql.exec.MapJoinOperator: Unxpected exception: null
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.getRefKey(MapJoinOperator.java:198)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.computeMapJoinKey(MapJoinOperator.java:186)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:216)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:92)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:540)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370)
at
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:295)
at
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:181)
at
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:224)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-12-16 17:09:17,659 FATAL [uber-SubtaskRunner]
org.apache.hadoop.hive.ql.exec.mr.ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {_col0:1,_col1:2}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:550)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370)
at
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:295)
at
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:181)
at
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:224)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
[jira] [Updated] (HIVE-9000) LAST_VALUE Window function returns wrong results
[
https://issues.apache.org/jira/browse/HIVE-9000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9000:
Description:
LAST_VALUE Windowing function has been returning bad results, as far as I can
tell from day 1.
And, it seems like the tests are also asserting that LAST_VALUE gives the wrong
result.
Here's the test output:
https://github.com/apache/hive/blob/branch-0.14/ql/src/test/results/clientpositive/windowing_navfn.q.out#L587
The query is:
{code}
select t, s, i, last_value(i) over (partition by t order by s) from over10k
where (s = 'oscar allen' or s = 'oscar carson') and t = 10
{code}
The result is:
{code}
t si last_value(i)
---
10 oscar allen 65662 65662
10 oscar carson65549 65549
{code}
{{LAST_VALUE( i )}} should have returned 65549 in both records, instead it
simply ends up returning i.
Another way you can make sure LAST_VALUE is bad is to verify it's result
against LEAD(i,1) over (partition by t order by s). LAST_VALUE being last value
should always be more (in terms of the specified 'order by s') than the lead by
1. While this doesn't directly apply to the above query, if the result set had
more rows, you would clearly see records where lead is higher than last_value
which is semantically incorrect.
was:
LAST_VALUE Windowing function has been returning bad results, as far as I can
tell from day 1.
And, it seems like the tests are also asserting that LAST_VALUE gives the wrong
result.
Here's the test output:
https://github.com/apache/hive/blob/branch-0.14/ql/src/test/results/clientpositive/windowing_navfn.q.out#L587
The query is:
{code}
select t, s, i, last_value(i) over (partition by t order by s)
{code}
The result is:
{code}
t si last_value(i)
---
10 oscar allen 65662 65662
10 oscar carson65549 65549
{code}
{{LAST_VALUE( i )}} should have returned 65549 in both records, instead it
simply ends up returning i.
Another way you can make sure LAST_VALUE is bad is to verify it's result
against LEAD(i,1) over (partition by t order by s). LAST_VALUE being last value
should always be more (in terms of the specified 'order by s') than the lead by
1. While this doesn't directly apply to the above query, if the result set had
more rows, you would clearly see records where lead is higher than last_value
which is semantically incorrect.
LAST_VALUE Window function returns wrong results
Key: HIVE-9000
URL: https://issues.apache.org/jira/browse/HIVE-9000
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Affects Versions: 0.13.1
Reporter: Mark Grover
Priority: Critical
Fix For: 0.14.1
LAST_VALUE Windowing function has been returning bad results, as far as I can
tell from day 1.
And, it seems like the tests are also asserting that LAST_VALUE gives the
wrong result.
Here's the test output:
https://github.com/apache/hive/blob/branch-0.14/ql/src/test/results/clientpositive/windowing_navfn.q.out#L587
The query is:
{code}
select t, s, i, last_value(i) over (partition by t order by s) from over10k
where (s = 'oscar allen' or s = 'oscar carson') and t = 10
{code}
The result is:
{code}
t si last_value(i)
---
10oscar allen 65662 65662
10oscar carson65549 65549
{code}
{{LAST_VALUE( i )}} should have returned 65549 in both records, instead it
simply ends up returning i.
Another way you can make sure LAST_VALUE is bad is to verify it's result
against LEAD(i,1) over (partition by t order by s). LAST_VALUE being last
value should always be more (in terms of the specified 'order by s') than the
lead by 1. While this doesn't directly apply to the above query, if the
result set had more rows, you would clearly see records where lead is higher
than last_value which is semantically incorrect.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9138) Add some explain to PTF operator
Navis created HIVE-9138: --- Summary: Add some explain to PTF operator Key: HIVE-9138 URL: https://issues.apache.org/jira/browse/HIVE-9138 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial PTFOperator does not explain anything in explain statement, making it hard to understand the internal works. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9138) Add some explain to PTF operator
[ https://issues.apache.org/jira/browse/HIVE-9138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-9138: Status: Patch Available (was: Open) Add some explain to PTF operator Key: HIVE-9138 URL: https://issues.apache.org/jira/browse/HIVE-9138 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-9138.1.patch.txt PTFOperator does not explain anything in explain statement, making it hard to understand the internal works. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9138) Add some explain to PTF operator
[ https://issues.apache.org/jira/browse/HIVE-9138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-9138: Attachment: HIVE-9138.1.patch.txt Add some explain to PTF operator Key: HIVE-9138 URL: https://issues.apache.org/jira/browse/HIVE-9138 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-9138.1.patch.txt PTFOperator does not explain anything in explain statement, making it hard to understand the internal works. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9138) Add some explain to PTF operator
[
https://issues.apache.org/jira/browse/HIVE-9138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14249296#comment-14249296
]
Navis commented on HIVE-9138:
-
Now it shows something like this.
{noformat}
PTF Operator
windowing: true
query input definition
input alias: ptf_0
type: WINDOWING
window table definition
input alias: ptf_1
name: windowingtablefunction
order by: _col1
partition by: _col0
window definition
alias: _wcol0
arguments: _col1
name: rank
window function: GenericUDAFRankEvaluator
window frame: PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
{noformat}
Add some explain to PTF operator
Key: HIVE-9138
URL: https://issues.apache.org/jira/browse/HIVE-9138
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-9138.1.patch.txt
PTFOperator does not explain anything in explain statement, making it hard to
understand the internal works.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9129) Migrate to newer Calcite snapshot, where ByteString is now in org.apache.calcite.avatica.util
[ https://issues.apache.org/jira/browse/HIVE-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-9129: Resolution: Fixed Fix Version/s: 0.15.0 Status: Resolved (was: Patch Available) I've committed this to trunk before 24 hour probation ends because trunk build is broken. Thanks Julian and Ashutosh, for the patch. Migrate to newer Calcite snapshot, where ByteString is now in org.apache.calcite.avatica.util - Key: HIVE-9129 URL: https://issues.apache.org/jira/browse/HIVE-9129 Project: Hive Issue Type: Bug Reporter: Julian Hyde Assignee: Ashutosh Chauhan Fix For: 0.15.0 Attachments: HIVE-9129.1.patch ByteString has moved from org.apache.calcite.avatica to org.apache.calcite.avatica.util. To ease the migration, I have added a shim method RexLiteral.byteValue(), so that ASTBuilder does not need to reference ByteString explicitly. The attached patch allows Hive to run against avatica-1.0.0-incubating-SNAPSHOT both before and after the change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-5744) Implement support for BETWEEN in SELECT list
[ https://issues.apache.org/jira/browse/HIVE-5744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14249331#comment-14249331 ] Navis commented on HIVE-5744: - I apparently I've forgot to update result of vector_between_in. Others are well-known not-related. Could anyone review this? Implement support for BETWEEN in SELECT list Key: HIVE-5744 URL: https://issues.apache.org/jira/browse/HIVE-5744 Project: Hive Issue Type: Sub-task Reporter: Eric Hanson Assignee: Navis Attachments: HIVE-5744.1.patch.txt, HIVE-5744.2.patch.txt Queries like SELECT col1 BETWEEN 0 and 10 from T; fail in vectorized mode. Support needs to be implemented for a BETWEEN expression in the SELECT list, comparable to how it was added for comparison operators (, , ...). These were done by adding new, templates that return a value for a comparison instead of applying a filter. See ColumnCompareScalar.txt under ql/src/gen for an example. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9113) Explain on query failed with NPE
[
https://issues.apache.org/jira/browse/HIVE-9113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9113:
Assignee: Navis
Status: Patch Available (was: Open)
Explain on query failed with NPE
Key: HIVE-9113
URL: https://issues.apache.org/jira/browse/HIVE-9113
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Chao
Assignee: Navis
Attachments: HIVE-9113.1.patch.txt
Run explain on the following query:
{noformat}
select p.p_partkey, li.l_suppkey
from (select distinct l_partkey as p_partkey from lineitem) p join lineitem
li on p.p_partkey = li.l_partkey
where li.l_linenumber = 1 and
li.l_orderkey in (select l_orderkey where l_linenumber = li.l_linenumber)
;
{noformat}
gave me NPE:
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.parse.QBSubQuery.validateAndRewriteAST(QBSubQuery.java:516)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFilterPlan(SemanticAnalyzer.java:2605)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8866)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9745)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9638)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10125)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:221)
at
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:221)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:420)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:306)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1108)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1045)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1035)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:199)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:151)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:362)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:720)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:639)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
Is this query invalid? If so, it should at least give some explanations, not
just a plain NPE message, and left user clueless.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9113) Explain on query failed with NPE
[
https://issues.apache.org/jira/browse/HIVE-9113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9113:
Attachment: HIVE-9113.1.patch.txt
Explain on query failed with NPE
Key: HIVE-9113
URL: https://issues.apache.org/jira/browse/HIVE-9113
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Chao
Attachments: HIVE-9113.1.patch.txt
Run explain on the following query:
{noformat}
select p.p_partkey, li.l_suppkey
from (select distinct l_partkey as p_partkey from lineitem) p join lineitem
li on p.p_partkey = li.l_partkey
where li.l_linenumber = 1 and
li.l_orderkey in (select l_orderkey where l_linenumber = li.l_linenumber)
;
{noformat}
gave me NPE:
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.parse.QBSubQuery.validateAndRewriteAST(QBSubQuery.java:516)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFilterPlan(SemanticAnalyzer.java:2605)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8866)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9745)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9638)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10125)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:221)
at
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:221)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:420)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:306)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1108)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1045)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1035)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:199)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:151)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:362)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:720)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:639)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
Is this query invalid? If so, it should at least give some explanations, not
just a plain NPE message, and left user clueless.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9000) LAST_VALUE Window function returns wrong results
[
https://issues.apache.org/jira/browse/HIVE-9000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14249520#comment-14249520
]
Navis commented on HIVE-9000:
-
[~mgrover] I think it's correct. You can acquire expected result by,
{code}
select t, s, i, last_value(i) over (partition by t order by s RANGE BETWEEN
UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)
from over10k where (s = 'oscar allen' or s = 'oscar carson') and t = 10;
{code}
LAST_VALUE Window function returns wrong results
Key: HIVE-9000
URL: https://issues.apache.org/jira/browse/HIVE-9000
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Affects Versions: 0.13.1
Reporter: Mark Grover
Priority: Critical
Fix For: 0.14.1
LAST_VALUE Windowing function has been returning bad results, as far as I can
tell from day 1.
And, it seems like the tests are also asserting that LAST_VALUE gives the
wrong result.
Here's the test output:
https://github.com/apache/hive/blob/branch-0.14/ql/src/test/results/clientpositive/windowing_navfn.q.out#L587
The query is:
{code}
select t, s, i, last_value(i) over (partition by t order by s) from over10k
where (s = 'oscar allen' or s = 'oscar carson') and t = 10
{code}
The result is:
{code}
t si last_value(i)
---
10oscar allen 65662 65662
10oscar carson65549 65549
{code}
{{LAST_VALUE( i )}} should have returned 65549 in both records, instead it
simply ends up returning i.
Another way you can make sure LAST_VALUE is bad is to verify it's result
against LEAD(i,1) over (partition by t order by s). LAST_VALUE being last
value should always be more (in terms of the specified 'order by s') than the
lead by 1. While this doesn't directly apply to the above query, if the
result set had more rows, you would clearly see records where lead is higher
than last_value which is semantically incorrect.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9130) vector_partition_diff_num_cols result is not updated after CBO upgrade
[ https://issues.apache.org/jira/browse/HIVE-9130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-9130: Resolution: Fixed Fix Version/s: 0.15.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Sergey. vector_partition_diff_num_cols result is not updated after CBO upgrade --- Key: HIVE-9130 URL: https://issues.apache.org/jira/browse/HIVE-9130 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Trivial Fix For: 0.15.0 Attachments: HIVE-9130.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7024) Escape control characters for explain result
[ https://issues.apache.org/jira/browse/HIVE-7024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-7024: Attachment: HIVE-7024.5.patch.txt IntelliJ ate all trailing spaces. Resubmitting. Escape control characters for explain result Key: HIVE-7024 URL: https://issues.apache.org/jira/browse/HIVE-7024 Project: Hive Issue Type: Bug Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-7024.1.patch.txt, HIVE-7024.2.patch.txt, HIVE-7024.3.patch.txt, HIVE-7024.4.patch.txt, HIVE-7024.5.patch.txt Comments for columns are now delimited by 0x00, which is binary and make git refuse to make proper diff file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-9107) Non-lowercase field names in structs causes NullPointerException
[ https://issues.apache.org/jira/browse/HIVE-9107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis resolved HIVE-9107. - Resolution: Duplicate Feel free open this issue if it happens again in hive-0.14.0 Non-lowercase field names in structs causes NullPointerException Key: HIVE-9107 URL: https://issues.apache.org/jira/browse/HIVE-9107 Project: Hive Issue Type: Bug Affects Versions: 0.13.1 Reporter: Jim Pivarski If an HQL query references a struct field with mixed or upper case, Hive throws a NullPointerException instead of giving a better error message or simply lower-casing the name. For example, if I have a struct in column mystruct with a field named myfield, a query like select mystruct.MyField from tablename; passes the local initialize (it submits an M-R job) but the remote initialize jobs throw NullPointerExceptions. The exception is on line 61 of org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator, which is right after the field name is extracted and not forced to be lower-case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7977) Avoid creating serde for partitions if possible in FetchTask
[ https://issues.apache.org/jira/browse/HIVE-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14249556#comment-14249556 ] Navis commented on HIVE-7977: - I think it's ready. But I cannot update/upload git diff file to RB. Is something changed? Avoid creating serde for partitions if possible in FetchTask Key: HIVE-7977 URL: https://issues.apache.org/jira/browse/HIVE-7977 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-7977.1.patch.txt, HIVE-7977.2.patch.txt, HIVE-7977.3.patch.txt, HIVE-7977.4.patch.txt, HIVE-7977.5.patch.txt, HIVE-7977.6.patch.txt Currently, FetchTask creates SerDe instance thrice for each partition, which can be avoided if it's same with table SerDe. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9076) incompatFileSet in AbstractFileMergeOperator should be marked to skip task id check
[
https://issues.apache.org/jira/browse/HIVE-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9076:
Attachment: HIVE-9076.4.patch.txt
Fixed NPE
incompatFileSet in AbstractFileMergeOperator should be marked to skip task id
check
---
Key: HIVE-9076
URL: https://issues.apache.org/jira/browse/HIVE-9076
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-9076.1.patch.txt, HIVE-9076.2.patch.txt,
HIVE-9076.3.patch.txt, HIVE-9076.4.patch.txt
In some file composition, AbstractFileMergeOperator removes incompatible
files. For example,
{noformat}
00_0 (v12)
00_0_copy_1 (v12)
00_1 (v11)
00_1_copy_1 (v11)
00_1_copy_2 (v11)
00_2 (v12)
{noformat}
00_1 (v11) will be removed because 00 is assigned to new merged file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-7111) Extend join transitivity PPD to non-column expressions
[ https://issues.apache.org/jira/browse/HIVE-7111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14249569#comment-14249569 ] Navis commented on HIVE-7111: - [~leftylev] Seemed not need any documentation. And it's extension of PPD rather than join optimization. Extend join transitivity PPD to non-column expressions -- Key: HIVE-7111 URL: https://issues.apache.org/jira/browse/HIVE-7111 Project: Hive Issue Type: Task Components: Query Processor Reporter: Navis Assignee: Navis Priority: Minor Fix For: 0.15.0 Attachments: HIVE-7111.1.patch.txt, HIVE-7111.2.patch.txt, HIVE-7111.2.patch.txt, HIVE-7111.3.patch.txt, HIVE-7111.4.patch.txt Join transitive in PPD only supports column expressions, but it's possible to extend this to generic expressions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9096) GenericUDF may be left unclosed in PartitionPrune#visitCall()
[
https://issues.apache.org/jira/browse/HIVE-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14249576#comment-14249576
]
Navis commented on HIVE-9096:
-
+1
GenericUDF may be left unclosed in PartitionPrune#visitCall()
-
Key: HIVE-9096
URL: https://issues.apache.org/jira/browse/HIVE-9096
Project: Hive
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Attachments: HIVE-9096.patch
In
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/PartitionPrune.java
:
{code}
GenericUDF hiveUDF = SqlFunctionConverter.getHiveUDF(call.getOperator(),
call.getType(), call.operands.size());
if (hiveUDF != null
!FunctionRegistry.isDeterministic(hiveUDF)) {
return null;
}
{code}
hiveUDF should be closed upon return.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-7977) Avoid creating serde for partitions if possible in FetchTask
[ https://issues.apache.org/jira/browse/HIVE-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-7977: Attachment: HIVE-7977.4.patch.txt Avoid creating serde for partitions if possible in FetchTask Key: HIVE-7977 URL: https://issues.apache.org/jira/browse/HIVE-7977 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-7977.1.patch.txt, HIVE-7977.2.patch.txt, HIVE-7977.3.patch.txt, HIVE-7977.4.patch.txt Currently, FetchTask creates SerDe instance thrice for each partition, which can be avoided if it's same with table SerDe. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7977) Avoid creating serde for partitions if possible in FetchTask
[ https://issues.apache.org/jira/browse/HIVE-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246449#comment-14246449 ] Navis commented on HIVE-7977: - [~ashutoshc] leaved RB link. thanks. Avoid creating serde for partitions if possible in FetchTask Key: HIVE-7977 URL: https://issues.apache.org/jira/browse/HIVE-7977 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-7977.1.patch.txt, HIVE-7977.2.patch.txt, HIVE-7977.3.patch.txt, HIVE-7977.4.patch.txt Currently, FetchTask creates SerDe instance thrice for each partition, which can be avoided if it's same with table SerDe. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8803) DESC SCHEMA DATABASE-NAME is not working
[ https://issues.apache.org/jira/browse/HIVE-8803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-8803: Fix Version/s: 0.15.0 DESC SCHEMA DATABASE-NAME is not working -- Key: HIVE-8803 URL: https://issues.apache.org/jira/browse/HIVE-8803 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Fix For: 0.15.0 Attachments: HIVE-8803.1.patch.txt, HIVE-8803.1.patch.txt Found that documenting HIVE-6601 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7977) Avoid creating serde for partitions if possible in FetchTask
[ https://issues.apache.org/jira/browse/HIVE-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-7977: Attachment: HIVE-7977.5.patch.txt Rebased to trunk Avoid creating serde for partitions if possible in FetchTask Key: HIVE-7977 URL: https://issues.apache.org/jira/browse/HIVE-7977 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-7977.1.patch.txt, HIVE-7977.2.patch.txt, HIVE-7977.3.patch.txt, HIVE-7977.4.patch.txt, HIVE-7977.5.patch.txt Currently, FetchTask creates SerDe instance thrice for each partition, which can be avoided if it's same with table SerDe. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7977) Avoid creating serde for partitions if possible in FetchTask
[ https://issues.apache.org/jira/browse/HIVE-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-7977: Attachment: HIVE-7977.6.patch.txt Incurred some unnecessary codes. Reattaching patch for test. Avoid creating serde for partitions if possible in FetchTask Key: HIVE-7977 URL: https://issues.apache.org/jira/browse/HIVE-7977 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-7977.1.patch.txt, HIVE-7977.2.patch.txt, HIVE-7977.3.patch.txt, HIVE-7977.4.patch.txt, HIVE-7977.5.patch.txt, HIVE-7977.6.patch.txt Currently, FetchTask creates SerDe instance thrice for each partition, which can be avoided if it's same with table SerDe. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9076) incompatFileSet in AbstractFileMergeOperator should be marked to skip task id check
[
https://issues.apache.org/jira/browse/HIVE-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9076:
Attachment: HIVE-9076.3.patch.txt
Fixed test fails
incompatFileSet in AbstractFileMergeOperator should be marked to skip task id
check
---
Key: HIVE-9076
URL: https://issues.apache.org/jira/browse/HIVE-9076
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-9076.1.patch.txt, HIVE-9076.2.patch.txt,
HIVE-9076.3.patch.txt
In some file composition, AbstractFileMergeOperator removes incompatible
files. For example,
{noformat}
00_0 (v12)
00_0_copy_1 (v12)
00_1 (v11)
00_1_copy_1 (v11)
00_1_copy_2 (v11)
00_2 (v12)
{noformat}
00_1 (v11) will be removed because 00 is assigned to new merged file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-7270) SerDe Properties are not considered by show create table Command
[ https://issues.apache.org/jira/browse/HIVE-7270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-7270: Resolution: Fixed Fix Version/s: 0.15.0 Status: Resolved (was: Patch Available) Committed to trunk SerDe Properties are not considered by show create table Command Key: HIVE-7270 URL: https://issues.apache.org/jira/browse/HIVE-7270 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.13.1 Reporter: R J Assignee: Navis Priority: Minor Fix For: 0.15.0 Attachments: HIVE-7270.1.patch.txt, HIVE-7270.2.patch.txt, HIVE-7270.3.patch.txt The HIVE table DDl generated by show create table target_table command does not contain SerDe properties of the target table even though it contain specific SerDe properties. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7248) UNION ALL in hive returns incorrect results on Hbase backed table
[ https://issues.apache.org/jira/browse/HIVE-7248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-7248: Attachment: HIVE-7248.3.patch.txt Rebased to trunk UNION ALL in hive returns incorrect results on Hbase backed table - Key: HIVE-7248 URL: https://issues.apache.org/jira/browse/HIVE-7248 Project: Hive Issue Type: Bug Components: HBase Handler Affects Versions: 0.12.0, 0.13.0, 0.13.1 Reporter: Mala Chikka Kempanna Assignee: Navis Attachments: HIVE-7248.1.patch.txt, HIVE-7248.2.patch.txt, HIVE-7248.3.patch.txt The issue can be recreated with following steps 1) In hbase create 'TABLE_EMP','default' 2) On hive sudo -u hive hive CREATE EXTERNAL TABLE TABLE_EMP(FIRST_NAME string,LAST_NAME string,CDS_UPDATED_DATE string,CDS_PK string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES(hbase.columns.mapping = default:FIRST_NAME,default:LAST_NAME,default:CDS_UPDATED_DATE,:key, hbase.scan.cache = 500, hbase.scan.cacheblocks = false ) TBLPROPERTIES(hbase.table.name = TABLE_EMP,'serialization.null.format'=''); 3) On hbase insert the following data put 'TABLE_EMP', '1', 'default:FIRST_NAME', 'Srini' put 'TABLE_EMP', '1', 'default:LAST_NAME', 'P' put 'TABLE_EMP', '1', 'default:CDS_UPDATED_DATE', '2014-06-16 00:00:00' put 'TABLE_EMP', '2', 'default:FIRST_NAME', 'Aravind' put 'TABLE_EMP', '2', 'default:LAST_NAME', 'K' put 'TABLE_EMP', '2', 'default:CDS_UPDATED_DATE', '2014-06-16 00:00:00' 4) On hive execute the following query hive SELECT * FROM ( SELECT CDS_PK FROM TABLE_EMP WHERE CDS_PK = '0' AND CDS_PK = '9' AND CDS_UPDATED_DATE IS NOT NULL UNION ALL SELECT CDS_PK FROM TABLE_EMP WHERE CDS_PK = 'a' AND CDS_PK = 'z' AND CDS_UPDATED_DATE IS NOT NULL )t ; 5) Output of the query 1 1 2 2 6) Output of just SELECT CDS_PK FROM TABLE_EMP WHERE CDS_PK = '0' AND CDS_PK = '9' AND CDS_UPDATED_DATE IS NOT NULL is 1 2 7) Output of just SELECT CDS_PK FROM TABLE_EMP WHERE CDS_PK = 'a' AND CDS_PK = 'z' AND CDS_UPDATED_DATE IS NOT NULL Empty 8) UNION is used to combine the result from multiple SELECT statements into a single result set. Hive currently only supports UNION ALL (bag union), in which duplicates are not eliminated Accordingly above query should return output 1 2 instead it is giving wrong output 1 1 2 2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7024) Escape control characters for explain result
[ https://issues.apache.org/jira/browse/HIVE-7024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-7024: Attachment: HIVE-7024.4.patch.txt Missed +1. Reattaching patch for test. Escape control characters for explain result Key: HIVE-7024 URL: https://issues.apache.org/jira/browse/HIVE-7024 Project: Hive Issue Type: Bug Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-7024.1.patch.txt, HIVE-7024.2.patch.txt, HIVE-7024.3.patch.txt, HIVE-7024.4.patch.txt Comments for columns are now delimited by 0x00, which is binary and make git refuse to make proper diff file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-5744) Implement support for BETWEEN in SELECT list
[ https://issues.apache.org/jira/browse/HIVE-5744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-5744: Attachment: HIVE-5744.2.patch.txt Rebased to trunk Implement support for BETWEEN in SELECT list Key: HIVE-5744 URL: https://issues.apache.org/jira/browse/HIVE-5744 Project: Hive Issue Type: Sub-task Reporter: Eric Hanson Assignee: Navis Attachments: HIVE-5744.1.patch.txt, HIVE-5744.2.patch.txt Queries like SELECT col1 BETWEEN 0 and 10 from T; fail in vectorized mode. Support needs to be implemented for a BETWEEN expression in the SELECT list, comparable to how it was added for comparison operators (, , ...). These were done by adding new, templates that return a value for a comparison instead of applying a filter. See ColumnCompareScalar.txt under ql/src/gen for an example. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-4790) MapredLocalTask task does not make virtual columns
[
https://issues.apache.org/jira/browse/HIVE-4790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-4790:
Attachment: HIVE-4790.11.patch.txt
Rebased to trunk
MapredLocalTask task does not make virtual columns
--
Key: HIVE-4790
URL: https://issues.apache.org/jira/browse/HIVE-4790
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: D11511.3.patch, D11511.4.patch, HIVE-4790.10.patch.txt,
HIVE-4790.11.patch.txt, HIVE-4790.5.patch.txt, HIVE-4790.6.patch.txt,
HIVE-4790.7.patch.txt, HIVE-4790.8.patch.txt, HIVE-4790.9.patch.txt,
HIVE-4790.D11511.1.patch, HIVE-4790.D11511.2.patch
From mailing list,
http://www.mail-archive.com/user@hive.apache.org/msg08264.html
{noformat}
SELECT *,b.BLOCK__OFFSET__INSIDE__FILE FROM a JOIN b ON
b.rownumber = a.number;
fails with this error:
SELECT *,b.BLOCK__OFFSET__INSIDE__FILE FROM a JOIN b ON b.rownumber =
a.number;
Automatically selecting local only mode for query
Total MapReduce jobs = 1
setting HADOOP_USER_NAMEpmarron
13/06/25 10:52:56 WARN conf.HiveConf: DEPRECATED: Configuration property
hive.metastore.local no longer has any effect. Make sure to provide a valid
value for hive.metastore.uris if you are connecting to a remote metastore.
Execution log at: /tmp/pmarron/.log
2013-06-25 10:52:56 Starting to launch local task to process map join;
maximum memory = 932118528
java.lang.RuntimeException: cannot find field block__offset__inside__file
from [0:rownumber, 1:offset]
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:366)
at
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldRef(LazySimpleStructObjectInspector.java:168)
at
org.apache.hadoop.hive.serde2.objectinspector.DelegatedStructObjectInspector.getStructFieldRef(DelegatedStructObjectInspector.java:74)
at
org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:57)
at
org.apache.hadoop.hive.ql.exec.JoinUtil.getObjectInspectorsFromEvaluators(JoinUtil.java:68)
at
org.apache.hadoop.hive.ql.exec.HashTableSinkOperator.initializeOp(HashTableSinkOperator.java:222)
at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:186)
at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.MapredLocalTask.initializeOperators(MapredLocalTask.java:394)
at
org.apache.hadoop.hive.ql.exec.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:277)
at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:676)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Execution failed with exit status: 2
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9077) Set completer in CliDriver is not working
[
https://issues.apache.org/jira/browse/HIVE-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9077:
Resolution: Fixed
Fix Version/s: 0.15.0
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks Ferdinand Xu and Brock Noland, for the review!
Set completer in CliDriver is not working
-
Key: HIVE-9077
URL: https://issues.apache.org/jira/browse/HIVE-9077
Project: Hive
Issue Type: Bug
Components: CLI
Reporter: Navis
Assignee: Navis
Priority: Trivial
Fix For: 0.15.0
Attachments: HIVE-9077.1.patch.txt, HIVE-9077.2.patch.txt
NO PRECOMMIT TESTS
Seemed broken in HIVE-8609
{noformat}
hive (default) set Exception in thread main java.lang.NullPointerException
at
jline.console.completer.ArgumentCompleter$AbstractArgumentDelimiter.delimit(ArgumentCompleter.java:283)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:116)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:152)
at org.apache.hadoop.hive.cli.CliDriver$6.complete(CliDriver.java:567)
at jline.console.ConsoleReader.complete(ConsoleReader.java:3261)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2621)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2269)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:639)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8400) Fix building and packaging hwi war file
[ https://issues.apache.org/jira/browse/HIVE-8400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-8400: Fix Version/s: (was: 0.14.0) 0.15.0 Fix building and packaging hwi war file --- Key: HIVE-8400 URL: https://issues.apache.org/jira/browse/HIVE-8400 Project: Hive Issue Type: Bug Components: Web UI Affects Versions: 0.13.1 Reporter: Pankit Thapar Fix For: 0.15.0 Attachments: HIVE-8400.1.patch hive 0.11 used to have hwi-0.11.war file but it seems that hive 13 is not configured to build a war file, instead it builds a jar file for hwi. A fix for this would be to change jar to war in hwi/pom.xml. Diff is below : diff --git a/hwi/pom.xml b/hwi/pom.xml index 35c124b..88d83fb 100644 --- a/hwi/pom.xml +++ b/hwi/pom.xml @@ -24,7 +24,7 @@ /parent artifactIdhive-hwi/artifactId - packagingjar/packaging + packagingwar/packaging nameHive HWI/name Please let me know if jar was intended or is it actually a bug so that I can submit a patch for the fix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9091) Add additional unit tests for HiveSQLException
[ https://issues.apache.org/jira/browse/HIVE-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246214#comment-14246214 ] Navis commented on HIVE-9091: - Always welcome on more tests. +1 Add additional unit tests for HiveSQLException -- Key: HIVE-9091 URL: https://issues.apache.org/jira/browse/HIVE-9091 Project: Hive Issue Type: Test Components: JDBC Reporter: Aihua Xu Assignee: Aihua Xu Priority: Minor Labels: easyfix, test Fix For: 0.15.0 Attachments: HIVE-9091.1.patch Original Estimate: 12h Remaining Estimate: 12h Change the function scope for the functions in HiveSQLException to be private, including toString(Throwable cause, StackTraceElement[] parent) , enroll(Throwable ex, StackTraceElement[] trace, int max), toStackTrace(ListString details, StackTraceElement[] parent, int index) since these serve as helper function internally. Add comments for the public functions in HiveSQLException. To add additional unit test cases for HiveSQLException such as exception with unknown source, test TStatus conversion, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7641) INSERT ... SELECT with no source table leads to NPE
[
https://issues.apache.org/jira/browse/HIVE-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-7641:
Attachment: HIVE-7641.4.patch.txt
Rerun test
INSERT ... SELECT with no source table leads to NPE
---
Key: HIVE-7641
URL: https://issues.apache.org/jira/browse/HIVE-7641
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.13.1
Reporter: Lenni Kuff
Assignee: Navis
Attachments: HIVE-7641.1.patch.txt, HIVE-7641.2.patch.txt,
HIVE-7641.3.patch.txt, HIVE-7641.4.patch.txt
When no source table is provided for an INSERT statement Hive fails with NPE.
{code}
0: jdbc:hive2://localhost:11050/default create table test_tbl(i int);
No rows affected (0.333 seconds)
0: jdbc:hive2://localhost:11050/default insert into table test_tbl select 1;
Error: Error while compiling statement: FAILED: NullPointerException null
(state=42000,code=4)
-- Get a NPE even when using incorrect syntax (no TABLE keyword)
0: jdbc:hive2://localhost:11050/default insert into test_tbl select 1;
Error: Error while compiling statement: FAILED: NullPointerException null
(state=42000,code=4)
-- Works when a source table is provided
0: jdbc:hive2://localhost:11050/default insert into table test_tbl select 1
from foo;
No rows affected (5.751 seconds)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-7351) ANALYZE TABLE statement fails on postgres metastore
[
https://issues.apache.org/jira/browse/HIVE-7351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-7351:
Attachment: HIVE-7351.2.patch.txt
Rerun test
ANALYZE TABLE statement fails on postgres metastore
---
Key: HIVE-7351
URL: https://issues.apache.org/jira/browse/HIVE-7351
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.13.0, 0.13.1
Environment: postgresSQL
Reporter: Damien Carol
Assignee: Navis
Priority: Minor
Labels: metastore, postgres
Attachments: HIVE-7351.1.patch.txt, HIVE-7351.2.patch.txt
Metastore code use method {{PreparedStatement.setQueryTimeout(int)}} of JDBC
Driver :
Current JDBC driver doesn't implements this method.
{noformat}
2014-07-07 17:52:38,239 ERROR
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during JDBC
connection to
jdbc:postgresql://nc-h04:5432/metastore?user=hiveuserpassword=mvsmt4521.
org.postgresql.util.PSQLException: Method
org.postgresql.jdbc4.Jdbc4PreparedStatement.setQueryTimeout(int) is not yet
implemented.
at org.postgresql.Driver.notImplemented(Driver.java:753)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.setQueryTimeout(AbstractJdbc2Statement.java:666)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher$1.run(JDBCStatsPublisher.java:80)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher$1.run(JDBCStatsPublisher.java:77)
at
org.apache.hadoop.hive.ql.exec.Utilities.executeWithRetry(Utilities.java:2637)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher.connect(JDBCStatsPublisher.java:96)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.publishStats(TableScanOperator.java:280)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.closeOp(TableScanOperator.java:226)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:583)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:595)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-7270) SerDe Properties are not considered by show create table Command
[ https://issues.apache.org/jira/browse/HIVE-7270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-7270: Attachment: HIVE-7270.3.patch.txt SerDe Properties are not considered by show create table Command Key: HIVE-7270 URL: https://issues.apache.org/jira/browse/HIVE-7270 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.13.1 Reporter: R J Assignee: Navis Priority: Minor Attachments: HIVE-7270.1.patch.txt, HIVE-7270.2.patch.txt, HIVE-7270.3.patch.txt The HIVE table DDl generated by show create table target_table command does not contain SerDe properties of the target table even though it contain specific SerDe properties. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-7270) SerDe Properties are not considered by show create table Command
[ https://issues.apache.org/jira/browse/HIVE-7270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243855#comment-14243855 ] Navis commented on HIVE-7270: - [~jdere] Sorry, I've missed your comment. Pending for test. SerDe Properties are not considered by show create table Command Key: HIVE-7270 URL: https://issues.apache.org/jira/browse/HIVE-7270 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.13.1 Reporter: R J Assignee: Navis Priority: Minor Attachments: HIVE-7270.1.patch.txt, HIVE-7270.2.patch.txt, HIVE-7270.3.patch.txt The HIVE table DDl generated by show create table target_table command does not contain SerDe properties of the target table even though it contain specific SerDe properties. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9077) Set completer in CliDriver is not working
[
https://issues.apache.org/jira/browse/HIVE-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243525#comment-14243525
]
Navis commented on HIVE-9077:
-
[~brocknoland] Could you review this when time allows? Thanks.
Set completer in CliDriver is not working
-
Key: HIVE-9077
URL: https://issues.apache.org/jira/browse/HIVE-9077
Project: Hive
Issue Type: Bug
Components: CLI
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-9077.1.patch.txt, HIVE-9077.2.patch.txt
NO PRECOMMIT TESTS
Seemed broken in HIVE-8609
{noformat}
hive (default) set Exception in thread main java.lang.NullPointerException
at
jline.console.completer.ArgumentCompleter$AbstractArgumentDelimiter.delimit(ArgumentCompleter.java:283)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:116)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:152)
at org.apache.hadoop.hive.cli.CliDriver$6.complete(CliDriver.java:567)
at jline.console.ConsoleReader.complete(ConsoleReader.java:3261)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2621)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2269)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:639)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9076) incompatFileSet in AbstractFileMergeOperator should be marked to skip task id check
[
https://issues.apache.org/jira/browse/HIVE-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9076:
Attachment: HIVE-9076.2.patch.txt
Preliminary test
incompatFileSet in AbstractFileMergeOperator should be marked to skip task id
check
---
Key: HIVE-9076
URL: https://issues.apache.org/jira/browse/HIVE-9076
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-9076.1.patch.txt, HIVE-9076.2.patch.txt
In some file composition, AbstractFileMergeOperator removes incompatible
files. For example,
{noformat}
00_0 (v12)
00_0_copy_1 (v12)
00_1 (v11)
00_1_copy_1 (v11)
00_1_copy_2 (v11)
00_2 (v12)
{noformat}
00_1 (v11) will be removed because 00 is assigned to new merged file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8814) Support custom virtual columns from serde implementation
[ https://issues.apache.org/jira/browse/HIVE-8814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-8814: Attachment: HIVE-8814.4.patch.txt Rolled back some part to accommodate to tez usecase. Support custom virtual columns from serde implementation Key: HIVE-8814 URL: https://issues.apache.org/jira/browse/HIVE-8814 Project: Hive Issue Type: Improvement Components: Serializers/Deserializers Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-8814.1.patch.txt, HIVE-8814.2.patch.txt, HIVE-8814.3.patch.txt, HIVE-8814.4.patch.txt Currently, virtual columns are fixed in hive. But some serdes can provide more virtual columns if needed. Idea from https://issues.apache.org/jira/browse/HIVE-7513?focusedCommentId=14073912page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14073912 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8357) Path type entities should use qualified path rather than string
[ https://issues.apache.org/jira/browse/HIVE-8357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-8357: Attachment: HIVE-8357.6.patch.txt Addressed comments Path type entities should use qualified path rather than string --- Key: HIVE-8357 URL: https://issues.apache.org/jira/browse/HIVE-8357 Project: Hive Issue Type: Improvement Components: Authorization Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-8357.1.patch.txt, HIVE-8357.2.patch.txt, HIVE-8357.3.patch.txt, HIVE-8357.4.patch.txt, HIVE-8357.5.patch.txt, HIVE-8357.6.patch.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-7981) alias of compound aggregation functions fails in having clause
[ https://issues.apache.org/jira/browse/HIVE-7981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-7981: Attachment: HIVE-7981.5.patch.txt Fixed test fails alias of compound aggregation functions fails in having clause -- Key: HIVE-7981 URL: https://issues.apache.org/jira/browse/HIVE-7981 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: eyal gruss Assignee: Navis Priority: Minor Attachments: HIVE-7981.1.patch.txt, HIVE-7981.2.patch.txt, HIVE-7981.3.patch.txt, HIVE-7981.4.patch.txt, HIVE-7981.5.patch.txt hive select max(time)-min(time) as span from mytable group by name having span0; FAILED: SemanticException [Error 10025]: Line 1:92 Expression not in GROUP BY key '0' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9076) incompatFileSet in AbstractFileMergeOperator should be marked to skip task id check
Navis created HIVE-9076:
---
Summary: incompatFileSet in AbstractFileMergeOperator should be
marked to skip task id check
Key: HIVE-9076
URL: https://issues.apache.org/jira/browse/HIVE-9076
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
In some file composition, AbstractFileMergeOperator removes incompatible files.
For example,
{noformat}
00_0 (v12)
00_0_copy_1 (v12)
00_1 (v11)
00_1_copy_1 (v11)
00_1_copy_2 (v11)
00_2 (v12)
{noformat}
00_1 (v11) will be removed because 00 is assigned to new merged file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9076) incompatFileSet in AbstractFileMergeOperator should be marked to skip task id check
[
https://issues.apache.org/jira/browse/HIVE-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242082#comment-14242082
]
Navis commented on HIVE-9076:
-
Result
{noformat}
hive (default) dfs -ls ${hiveconf:hive.metastore.warehouse.dir}/orc_merge5b/;
Found 3 items
-rwxr-xr-x 1 navis supergroup 1203 2014-12-11 12:42
/user/hive/warehouse/orc_merge5b/00_0
-rwxr-xr-x 1 navis supergroup600 2014-12-11 12:33
/user/hive/warehouse/orc_merge5b/00_1_copy1
-rwxr-xr-x 1 navis supergroup600 2014-12-11 12:33
/user/hive/warehouse/orc_merge5b/00_1_copy2
{noformat}
log
{noformat}
2014-12-11 12:42:14,843 INFO exec.Task (SessionState.java:printInfo(825)) -
Starting Job = job_1418171849140_0008, Tracking URL =
http://localhost:8088/proxy/application_1418171849140_0008/
2014-12-11 12:42:14,843 INFO exec.Task (SessionState.java:printInfo(825)) -
Kill Command = /home/navis/hadoop-0.20/bin/hadoop job -kill
job_1418171849140_0008
2014-12-11 12:42:17,010 INFO exec.Task (SessionState.java:printInfo(825)) -
Hadoop job information for null: number of mappers: 0; number of reducers: 0
2014-12-11 12:42:17,064 WARN mapreduce.Counters
(AbstractCounters.java:getGroup(234)) - Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
2014-12-11 12:42:17,065 INFO exec.Task (SessionState.java:printInfo(825)) -
2014-12-11 12:42:17,060 null map = 0%, reduce = 0%
2014-12-11 12:42:21,200 WARN mapreduce.Counters
(AbstractCounters.java:getGroup(234)) - Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
2014-12-11 12:42:21,201 INFO exec.Task (SessionState.java:printInfo(825)) -
2014-12-11 12:42:21,200 null map = 100%, reduce = 0%
2014-12-11 12:42:21,206 WARN mapreduce.Counters
(AbstractCounters.java:getGroup(234)) - Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
2014-12-11 12:42:21,210 INFO exec.Task (SessionState.java:printInfo(825)) -
Ended Job = job_1418171849140_0008
2014-12-11 12:42:21,224 WARN exec.Utilities
(Utilities.java:removeTempOrDuplicateFiles(1977)) - Duplicate taskid file
removed:
hdfs://localhost:9000/tmp/hive/navis/80443669-b835-4a09-a73f-8b783d104f61/hive_2014-12-11_12-42-12_963_1945093075009681620-1/_tmp.-ext-1/00_1
with length 600. Existing file:
hdfs://localhost:9000/tmp/hive/navis/80443669-b835-4a09-a73f-8b783d104f61/hive_2014-12-11_12-42-12_963_1945093075009681620-1/_tmp.-ext-1/00_0
with length 1203
2014-12-11 12:42:21,225 INFO exec.AbstractFileMergeOperator
(Utilities.java:mvFileToFinalPath(1805)) - Moving tmp dir:
hdfs://localhost:9000/tmp/hive/navis/80443669-b835-4a09-a73f-8b783d104f61/hive_2014-12-11_12-42-12_963_1945093075009681620-1/_tmp.-ext-1
to:
hdfs://localhost:9000/tmp/hive/navis/80443669-b835-4a09-a73f-8b783d104f61/hive_2014-12-11_12-42-12_963_1945093075009681620-1/-ext-1
2014-12-11 12:42:21,233 INFO exec.AbstractFileMergeOperator
(AbstractFileMergeOperator.java:jobCloseOp(247)) - jobCloseOp moved merged
files to output dir:
hdfs://localhost:9000/tmp/hive/navis/80443669-b835-4a09-a73f-8b783d104f61/hive_2014-12-11_12-42-12_963_1945093075009681620-1/-ext-1
{noformat}
incompatFileSet in AbstractFileMergeOperator should be marked to skip task id
check
---
Key: HIVE-9076
URL: https://issues.apache.org/jira/browse/HIVE-9076
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
In some file composition, AbstractFileMergeOperator removes incompatible
files. For example,
{noformat}
00_0 (v12)
00_0_copy_1 (v12)
00_1 (v11)
00_1_copy_1 (v11)
00_1_copy_2 (v11)
00_2 (v12)
{noformat}
00_1 (v11) will be removed because 00 is assigned to new merged file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9076) incompatFileSet in AbstractFileMergeOperator should be marked to skip task id check
[
https://issues.apache.org/jira/browse/HIVE-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9076:
Attachment: HIVE-9076.1.patch.txt
incompatFileSet in AbstractFileMergeOperator should be marked to skip task id
check
---
Key: HIVE-9076
URL: https://issues.apache.org/jira/browse/HIVE-9076
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-9076.1.patch.txt
In some file composition, AbstractFileMergeOperator removes incompatible
files. For example,
{noformat}
00_0 (v12)
00_0_copy_1 (v12)
00_1 (v11)
00_1_copy_1 (v11)
00_1_copy_2 (v11)
00_2 (v12)
{noformat}
00_1 (v11) will be removed because 00 is assigned to new merged file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9076) incompatFileSet in AbstractFileMergeOperator should be marked to skip task id check
[
https://issues.apache.org/jira/browse/HIVE-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9076:
Status: Patch Available (was: Open)
incompatFileSet in AbstractFileMergeOperator should be marked to skip task id
check
---
Key: HIVE-9076
URL: https://issues.apache.org/jira/browse/HIVE-9076
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-9076.1.patch.txt
In some file composition, AbstractFileMergeOperator removes incompatible
files. For example,
{noformat}
00_0 (v12)
00_0_copy_1 (v12)
00_1 (v11)
00_1_copy_1 (v11)
00_1_copy_2 (v11)
00_2 (v12)
{noformat}
00_1 (v11) will be removed because 00 is assigned to new merged file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9076) incompatFileSet in AbstractFileMergeOperator should be marked to skip task id check
[
https://issues.apache.org/jira/browse/HIVE-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242133#comment-14242133
]
Navis commented on HIVE-9076:
-
Test always makes file composition like below,
{noformat}
00_0
00_0_copy_1
00_0_copy_2
00_0_copy_3
00_0_copy_4
00_0_copy_5
{noformat}
The composition in description is caused by a bug which I've made (I'm in
progress on HIVE-8814). But I think this should be fixed first.
incompatFileSet in AbstractFileMergeOperator should be marked to skip task id
check
---
Key: HIVE-9076
URL: https://issues.apache.org/jira/browse/HIVE-9076
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-9076.1.patch.txt
In some file composition, AbstractFileMergeOperator removes incompatible
files. For example,
{noformat}
00_0 (v12)
00_0_copy_1 (v12)
00_1 (v11)
00_1_copy_1 (v11)
00_1_copy_2 (v11)
00_2 (v12)
{noformat}
00_1 (v11) will be removed because 00 is assigned to new merged file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9077) Set completer in CliDriver is not working
[
https://issues.apache.org/jira/browse/HIVE-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9077:
Attachment: HIVE-9077.1.patch.txt
Set completer in CliDriver is not working
-
Key: HIVE-9077
URL: https://issues.apache.org/jira/browse/HIVE-9077
Project: Hive
Issue Type: Bug
Components: CLI
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-9077.1.patch.txt
Seemed broken in HIVE-8609
{noformat}
hive (default) set Exception in thread main java.lang.NullPointerException
at
jline.console.completer.ArgumentCompleter$AbstractArgumentDelimiter.delimit(ArgumentCompleter.java:283)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:116)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:152)
at org.apache.hadoop.hive.cli.CliDriver$6.complete(CliDriver.java:567)
at jline.console.ConsoleReader.complete(ConsoleReader.java:3261)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2621)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2269)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:639)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9077) Set completer in CliDriver is not working
Navis created HIVE-9077:
---
Summary: Set completer in CliDriver is not working
Key: HIVE-9077
URL: https://issues.apache.org/jira/browse/HIVE-9077
Project: Hive
Issue Type: Bug
Components: CLI
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-9077.1.patch.txt
Seemed broken in HIVE-8609
{noformat}
hive (default) set Exception in thread main java.lang.NullPointerException
at
jline.console.completer.ArgumentCompleter$AbstractArgumentDelimiter.delimit(ArgumentCompleter.java:283)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:116)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:152)
at org.apache.hadoop.hive.cli.CliDriver$6.complete(CliDriver.java:567)
at jline.console.ConsoleReader.complete(ConsoleReader.java:3261)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2621)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2269)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:639)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9077) Set completer in CliDriver is not working
[
https://issues.apache.org/jira/browse/HIVE-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9077:
Description:
NO PRECOMMIT TESTS
Seemed broken in HIVE-8609
{noformat}
hive (default) set Exception in thread main java.lang.NullPointerException
at
jline.console.completer.ArgumentCompleter$AbstractArgumentDelimiter.delimit(ArgumentCompleter.java:283)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:116)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:152)
at org.apache.hadoop.hive.cli.CliDriver$6.complete(CliDriver.java:567)
at jline.console.ConsoleReader.complete(ConsoleReader.java:3261)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2621)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2269)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:639)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
was:
Seemed broken in HIVE-8609
{noformat}
hive (default) set Exception in thread main java.lang.NullPointerException
at
jline.console.completer.ArgumentCompleter$AbstractArgumentDelimiter.delimit(ArgumentCompleter.java:283)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:116)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:152)
at org.apache.hadoop.hive.cli.CliDriver$6.complete(CliDriver.java:567)
at jline.console.ConsoleReader.complete(ConsoleReader.java:3261)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2621)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2269)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:639)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
Set completer in CliDriver is not working
-
Key: HIVE-9077
URL: https://issues.apache.org/jira/browse/HIVE-9077
Project: Hive
Issue Type: Bug
Components: CLI
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-9077.1.patch.txt
NO PRECOMMIT TESTS
Seemed broken in HIVE-8609
{noformat}
hive (default) set Exception in thread main java.lang.NullPointerException
at
jline.console.completer.ArgumentCompleter$AbstractArgumentDelimiter.delimit(ArgumentCompleter.java:283)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:116)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:152)
at org.apache.hadoop.hive.cli.CliDriver$6.complete(CliDriver.java:567)
at jline.console.ConsoleReader.complete(ConsoleReader.java:3261)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2621)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2269)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:639)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9077) Set completer in CliDriver is not working
[
https://issues.apache.org/jira/browse/HIVE-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9077:
Status: Patch Available (was: Open)
Set completer in CliDriver is not working
-
Key: HIVE-9077
URL: https://issues.apache.org/jira/browse/HIVE-9077
Project: Hive
Issue Type: Bug
Components: CLI
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-9077.1.patch.txt
NO PRECOMMIT TESTS
Seemed broken in HIVE-8609
{noformat}
hive (default) set Exception in thread main java.lang.NullPointerException
at
jline.console.completer.ArgumentCompleter$AbstractArgumentDelimiter.delimit(ArgumentCompleter.java:283)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:116)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:152)
at org.apache.hadoop.hive.cli.CliDriver$6.complete(CliDriver.java:567)
at jline.console.ConsoleReader.complete(ConsoleReader.java:3261)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2621)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2269)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:639)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9077) Set completer in CliDriver is not working
[
https://issues.apache.org/jira/browse/HIVE-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-9077:
Attachment: HIVE-9077.2.patch.txt
Set completer in CliDriver is not working
-
Key: HIVE-9077
URL: https://issues.apache.org/jira/browse/HIVE-9077
Project: Hive
Issue Type: Bug
Components: CLI
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-9077.1.patch.txt, HIVE-9077.2.patch.txt
NO PRECOMMIT TESTS
Seemed broken in HIVE-8609
{noformat}
hive (default) set Exception in thread main java.lang.NullPointerException
at
jline.console.completer.ArgumentCompleter$AbstractArgumentDelimiter.delimit(ArgumentCompleter.java:283)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:116)
at
jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:152)
at org.apache.hadoop.hive.cli.CliDriver$6.complete(CliDriver.java:567)
at jline.console.ConsoleReader.complete(ConsoleReader.java:3261)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2621)
at jline.console.ConsoleReader.readLine(ConsoleReader.java:2269)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:639)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HIVE-4923) Provide compile + run method for HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis reopened HIVE-4923: - Seemed still useful for some cases. especially with HIVE-3946. Provide compile + run method for HiveServer2 Key: HIVE-4923 URL: https://issues.apache.org/jira/browse/HIVE-4923 Project: Hive Issue Type: Improvement Components: HiveServer2 Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-4923.2.patch.txt, HIVE-4923.D11781.1.patch Make additional thrift method for asynchronous execution. execute=compile+run This addresses same thing with HIVE-4569 but with simpler implementation. It's just for reference (We are using this for months). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-4923) Provide compile + run method for HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-4923: Status: Patch Available (was: Reopened) Provide compile + run method for HiveServer2 Key: HIVE-4923 URL: https://issues.apache.org/jira/browse/HIVE-4923 Project: Hive Issue Type: Improvement Components: HiveServer2 Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-4923.2.patch.txt, HIVE-4923.D11781.1.patch Make additional thrift method for asynchronous execution. execute=compile+run This addresses same thing with HIVE-4569 but with simpler implementation. It's just for reference (We are using this for months). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-4923) Provide compile + run method for HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-4923: Attachment: HIVE-4923.2.patch.txt Provide compile + run method for HiveServer2 Key: HIVE-4923 URL: https://issues.apache.org/jira/browse/HIVE-4923 Project: Hive Issue Type: Improvement Components: HiveServer2 Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-4923.2.patch.txt, HIVE-4923.D11781.1.patch Make additional thrift method for asynchronous execution. execute=compile+run This addresses same thing with HIVE-4569 but with simpler implementation. It's just for reference (We are using this for months). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-4321) Add Compile/Execute support to Hive Server
[ https://issues.apache.org/jira/browse/HIVE-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis resolved HIVE-4321. - Resolution: Won't Fix HiveServer1 will be deprecated in next major release (hive-0.15.0 or hive-1.0) Add Compile/Execute support to Hive Server -- Key: HIVE-4321 URL: https://issues.apache.org/jira/browse/HIVE-4321 Project: Hive Issue Type: Bug Components: HiveServer2, Thrift API Reporter: Sarah Parra Assignee: Sarah Parra Attachments: CompileExecute.patch Adds support for query compilation in Hive Server 2 and adds Thrift support for compile/execute APIs. This enables scenarios that need to compile a query before it is executed, e.g. and ODBC driver that implements SQLPrepare/SQLExecute. This is commonly used for a client that needs metadata for the query before it is executed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9018) SHOW GRANT ROLE in Hive should return grant_time in human readable format
[ https://issues.apache.org/jira/browse/HIVE-9018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14239225#comment-14239225 ] Navis commented on HIVE-9018: - I think this kind of formatting should be configured in client side. SHOW GRANT ROLE in Hive should return grant_time in human readable format --- Key: HIVE-9018 URL: https://issues.apache.org/jira/browse/HIVE-9018 Project: Hive Issue Type: Improvement Reporter: Dapeng Sun Priority: Minor Attachments: HIVE-9018.003.patch, HIVE-9018.004.patch, HIVE-9018.005.patch, HIVE-9018.006.patch, HIVE-9018.007.patch, HIVE-9018.007.patch, HIVE-9018.008.patch, HIVE-9018.008.patch Currently, SHOW GRANT ROLE will return the 'grant_time' in microseconds since epoch. It would be nice if this were in human readable format. Current output: 1411801585902000 Desired output: Sat, Sep 27 2014 00:06:25.902 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9050) NULL values for empty strings when joining with ORC table
[ https://issues.apache.org/jira/browse/HIVE-9050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14239252#comment-14239252 ] Navis commented on HIVE-9050: - Cannot reproduce with trunk. Could try with hive-0.14.0? NULL values for empty strings when joining with ORC table - Key: HIVE-9050 URL: https://issues.apache.org/jira/browse/HIVE-9050 Project: Hive Issue Type: Bug Components: SQL Affects Versions: 0.13.0 Environment: CentOS release 6.4 (Final), Hortonworks 2.1, Tez Hive 0.13.0.2.1.3.0-563 Subversion git://ip-10-0-0-91/grid/0/jenkins/workspace/BIGTOP-HDP_RPM_REPO-baikal-GA-centos6/bigtop/build/hive/rpm/BUILD/h ive-0.13.0.2.1.3.0 -r a738a76c72d6d9dd304691faada57a94429256bc Compiled by jenkins on Thu Jun 26 18:28:50 EDT 2014 From source with checksum 4dbd99dd254f0c521ad8ab072045325d Reporter: Usein Faradzhev When ORC table contains an empty strings and the SQL query contains at least one join a hive returns NULL instead of empty values. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8564) DROP TABLE IF EXISTS throws exception if the table does not exist.
[ https://issues.apache.org/jira/browse/HIVE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14240451#comment-14240451 ] Navis commented on HIVE-8564: - I'm pretty sure fails are not related to this. Will be committed shortly after. DROP TABLE IF EXISTS throws exception if the table does not exist. Key: HIVE-8564 URL: https://issues.apache.org/jira/browse/HIVE-8564 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.1 Reporter: Ben Assignee: Navis Priority: Minor Attachments: HIVE-8564.1.patch.txt, HIVE-8564.2.patch.txt DROP TABLE IF EXISTS throws exception if the table does not exist. I tried set hive.exec.drop.ignorenonexistent=true, and it made no difference. hive DROP TABLE IF EXISTS testdb.mytable; 14/10/22 15:48:29 ERROR metadata.Hive: NoSuchObjectException(message:testdb.mytable table not found) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29338) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29306) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:29237) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table(ThriftHiveMetastore.java:1036) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table(ThriftHiveMetastore.java:1022) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:997) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89) at com.sun.proxy.$Proxy7.getTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:975) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:930) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:917) at org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3846) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:306) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1503) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1270) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1088) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) OK -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8564) DROP TABLE IF EXISTS throws exception if the table does not exist.
[ https://issues.apache.org/jira/browse/HIVE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-8564: Resolution: Fixed Fix Version/s: 0.15.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks Ashutosh, for the review. DROP TABLE IF EXISTS throws exception if the table does not exist. Key: HIVE-8564 URL: https://issues.apache.org/jira/browse/HIVE-8564 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.1 Reporter: Ben Assignee: Navis Priority: Minor Fix For: 0.15.0 Attachments: HIVE-8564.1.patch.txt, HIVE-8564.2.patch.txt DROP TABLE IF EXISTS throws exception if the table does not exist. I tried set hive.exec.drop.ignorenonexistent=true, and it made no difference. hive DROP TABLE IF EXISTS testdb.mytable; 14/10/22 15:48:29 ERROR metadata.Hive: NoSuchObjectException(message:testdb.mytable table not found) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29338) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29306) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:29237) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table(ThriftHiveMetastore.java:1036) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table(ThriftHiveMetastore.java:1022) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:997) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89) at com.sun.proxy.$Proxy7.getTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:975) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:930) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:917) at org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3846) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:306) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1503) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1270) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1088) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) OK -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8538) Fully qualified table error when adding partition
[
https://issues.apache.org/jira/browse/HIVE-8538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238982#comment-14238982
]
Navis commented on HIVE-8538:
-
This is fixed in hive-0.14, by HIVE-4064, HIVE-7681, etc. Could you check that?
Fully qualified table error when adding partition
-
Key: HIVE-8538
URL: https://issues.apache.org/jira/browse/HIVE-8538
Project: Hive
Issue Type: Improvement
Affects Versions: 0.13.0
Environment: HDP 2.1
Reporter: Hari Sekhon
Priority: Minor
When adding a partition using a fully qualified table name I get the
following error, works fine if I use database and then re-issue the query
with a non-qualified table name. Other queries such as select do work with
fully qualified table names so should make this allow qualified table names
for convenience and consistency.
{code}
hive alter table myDB.myTable add partition (log_date='-xx-xx');
MismatchedTokenException(27!=96)
at
org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617)
at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115)
at
org.apache.hadoop.hive.ql.parse.HiveParser.alterStatementSuffixExchangePartition(HiveParser.java:13264)
at
org.apache.hadoop.hive.ql.parse.HiveParser.alterTableStatementSuffix(HiveParser.java:6851)
at
org.apache.hadoop.hive.ql.parse.HiveParser.alterStatement(HiveParser.java:6552)
at
org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:2189)
at
org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1398)
at
org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1036)
at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:199)
at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:409)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
FAILED: ParseException line 1:32 mismatched input 'add' expecting KW_EXCHANGE
near 'myTable' in alter exchange partition
hive use myDB;
OK
Time taken: 0.009 seconds
hive alter table myTable add partition (log_date='-xx-xx');
OK
Time taken: 0.157 seconds
{code}
Thanks
Hari Sekhon
http://www.linkedin.com/in/harisekhon
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-8814) Support custom virtual columns from serde implementation
[ https://issues.apache.org/jira/browse/HIVE-8814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-8814: Attachment: HIVE-8814.3.patch.txt Fixed test fails Support custom virtual columns from serde implementation Key: HIVE-8814 URL: https://issues.apache.org/jira/browse/HIVE-8814 Project: Hive Issue Type: Improvement Components: Serializers/Deserializers Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-8814.1.patch.txt, HIVE-8814.2.patch.txt, HIVE-8814.3.patch.txt Currently, virtual columns are fixed in hive. But some serdes can provide more virtual columns if needed. Idea from https://issues.apache.org/jira/browse/HIVE-7513?focusedCommentId=14073912page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14073912 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8564) DROP TABLE IF EXISTS throws exception if the table does not exist.
[ https://issues.apache.org/jira/browse/HIVE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-8564: Attachment: HIVE-8564.2.patch.txt Submitting same patch to kick test DROP TABLE IF EXISTS throws exception if the table does not exist. Key: HIVE-8564 URL: https://issues.apache.org/jira/browse/HIVE-8564 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.1 Reporter: Ben Assignee: Navis Priority: Minor Attachments: HIVE-8564.1.patch.txt, HIVE-8564.2.patch.txt DROP TABLE IF EXISTS throws exception if the table does not exist. I tried set hive.exec.drop.ignorenonexistent=true, and it made no difference. hive DROP TABLE IF EXISTS testdb.mytable; 14/10/22 15:48:29 ERROR metadata.Hive: NoSuchObjectException(message:testdb.mytable table not found) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29338) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29306) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:29237) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table(ThriftHiveMetastore.java:1036) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table(ThriftHiveMetastore.java:1022) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:997) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89) at com.sun.proxy.$Proxy7.getTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:975) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:930) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:917) at org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3846) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:306) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1503) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1270) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1088) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) OK -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8564) DROP TABLE IF EXISTS throws exception if the table does not exist.
[ https://issues.apache.org/jira/browse/HIVE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-8564: Status: Patch Available (was: Open) DROP TABLE IF EXISTS throws exception if the table does not exist. Key: HIVE-8564 URL: https://issues.apache.org/jira/browse/HIVE-8564 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.1 Reporter: Ben Assignee: Navis Priority: Minor Attachments: HIVE-8564.1.patch.txt, HIVE-8564.2.patch.txt DROP TABLE IF EXISTS throws exception if the table does not exist. I tried set hive.exec.drop.ignorenonexistent=true, and it made no difference. hive DROP TABLE IF EXISTS testdb.mytable; 14/10/22 15:48:29 ERROR metadata.Hive: NoSuchObjectException(message:testdb.mytable table not found) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29338) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29306) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:29237) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table(ThriftHiveMetastore.java:1036) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table(ThriftHiveMetastore.java:1022) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:997) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89) at com.sun.proxy.$Proxy7.getTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:975) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:930) at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:917) at org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3846) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:306) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1503) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1270) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1088) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) OK -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HIVE-8466) nonReserved keywords can not be used as table alias
[ https://issues.apache.org/jira/browse/HIVE-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis resolved HIVE-8466. - Resolution: Won't Fix Keep current behavior to minimize confusion. Feel free to open this issue whenever it's needed to be. nonReserved keywords can not be used as table alias --- Key: HIVE-8466 URL: https://issues.apache.org/jira/browse/HIVE-8466 Project: Hive Issue Type: Bug Components: SQL Affects Versions: 0.12.0, 0.13.0, 0.14.0, 0.13.1 Reporter: Wan Chang Assignee: Navis Priority: Minor Attachments: HIVE-8466.1.patch, HIVE-8466.2.patch.txt, HIVE-8466.3.patch.txt There is a small mistake in the patch of issue HIVE-2906. See the change of FromClauseParser.g -: tabname=tableName (ts=tableSample)? (KW_AS? alias=identifier)? -- ^(TOK_TABREF $tabname $ts? $alias?) +: tabname=tableName (props=tableProperties)? (ts=tableSample)? (KW_AS? alias=Identifier)? +- ^(TOK_TABREF $tabname $props? $ts? $alias?) With the 'identifier' changed to 'Identifier' we can not use nonReserved keywords as table alias. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8848) data loading from text files or text file processing doesn't handle nulls correctly
[ https://issues.apache.org/jira/browse/HIVE-8848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-8848: Attachment: HIVE-8848.4.patch.txt Fixed test fails rebased to trunk. Hard to make time for hive recently. data loading from text files or text file processing doesn't handle nulls correctly --- Key: HIVE-8848 URL: https://issues.apache.org/jira/browse/HIVE-8848 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Navis Attachments: HIVE-8848.01.patch, HIVE-8848.2.patch.txt, HIVE-8848.3.patch.txt, HIVE-8848.4.patch.txt, HIVE-8848.patch I am not sure how nulls are supposed to be stored in text tables, but after loading some data with null or NULL strings, or x00 characters, we get bunch of annoying logging from LazyPrimitive that data is not in INT format and was converted to null, with data being null (string saying null, I assume from the code). Either load should load them as nulls, or there should be some defined way to load nulls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8832) SessionState.getUserFromAuthenticator() should be used instead of SessionState.getName()
[ https://issues.apache.org/jira/browse/HIVE-8832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-8832: Attachment: HIVE-8832.2.patch.txt Rebased to trunk SessionState.getUserFromAuthenticator() should be used instead of SessionState.getName() Key: HIVE-8832 URL: https://issues.apache.org/jira/browse/HIVE-8832 Project: Hive Issue Type: Bug Components: Authentication Reporter: Navis Assignee: Navis Attachments: HIVE-8832.1.patch.txt, HIVE-8832.2.patch.txt It's not valid sometimes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-8814) Support custom virtual columns from serde implementation
[ https://issues.apache.org/jira/browse/HIVE-8814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-8814: Attachment: HIVE-8814.2.patch.txt Kick test Support custom virtual columns from serde implementation Key: HIVE-8814 URL: https://issues.apache.org/jira/browse/HIVE-8814 Project: Hive Issue Type: Improvement Components: Serializers/Deserializers Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-8814.1.patch.txt, HIVE-8814.2.patch.txt Currently, virtual columns are fixed in hive. But some serdes can provide more virtual columns if needed. Idea from https://issues.apache.org/jira/browse/HIVE-7513?focusedCommentId=14073912page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14073912 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
