[jira] [Updated] (HIVE-21119) String UDAF and count distinct in the same select give error
[
https://issues.apache.org/jira/browse/HIVE-21119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Shetye updated HIVE-21119:
---
Labels: (was: plannin)
> String UDAF and count distinct in the same select give error
>
>
> Key: HIVE-21119
> URL: https://issues.apache.org/jira/browse/HIVE-21119
> Project: Hive
> Issue Type: Bug
>Reporter: Ravi Shetye
>Priority: Major
> Attachments: MaxUDA.java, run.log
>
>
> With the attached UDAF the following query crashes on hive.
> CRASHES
> {noformat}
> select rs_max(genderkey),count(distinct genderkey) from
> as_adventure.dimgender;
> {noformat}
> WORKS
> {noformat}
> select rs_max(genderkey) from as_adventure.dimgender;
> {noformat}
> The table looks like
> {noformat}
> 0: jdbc:hive2://localhost:1> select * from dimgender;
> OK
> INFO : Compiling
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7):
> select * from dimgender
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:dimgender.genderkey, type:string,
> comment:null), FieldSchema(name:dimgender.gendername, type:string,
> comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7);
> Time taken: 0.2 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Executing
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7):
> select * from dimgender
> INFO : Completed executing
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7);
> Time taken: 0.004 seconds
> INFO : OK
> INFO : Concurrency mode is disabled, not creating a lock manager
> +--+---+
> | dimgender.genderkey | dimgender.gendername |
> +--+---+
> | M| Male |
> | F| Female|
> | U| Unisex|
> +--+---+
> {noformat}
> {noformat}
> Vertex failed, vertexName=Reducer 2, vertexId=vertex_1547169244949_0024_2_01,
> diagnostics=[Task failed, taskId=task_1547169244949_0024_2_01_00,
> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
> failure ) :
> attempt_1547169244949_0024_2_01_00_0:java.lang.RuntimeException:
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row (tag=0)
> {"key":{"_col0":"F"},"value":{"_col0":"F"}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> {noformat}
> ...
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to
> execute method public boolean
> com.sample.MaxUDA$Evaluator.merge(java.lang.String) with arguments
> {F}:argument type mismatch
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFBridge$GenericUDAFBridgeEvaluator.merge(GenericUDAFBridge.java:176)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:216)
> {noformat}
> PLAN
> {noformat}
> ++
> | Explain |
> ++
> | Plan optimized by CBO. |
> ||
> | Vertex dependency in root stage|
> | Reducer 2 <- Map 1 (SIMPLE_EDGE) |
> | Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE)|
> ||
> | Stage-0|
> | Fetch Operator |
> | limit:-1 |
> | Stage-1|
> | Reducer 3|
> | File Output Operator [FS_6] |
> | Group By Operator [GBY_12] (rows=1 width=368) |
> |
[jira] [Updated] (HIVE-21119) String UDAF and count distinct in the same select give error
[
https://issues.apache.org/jira/browse/HIVE-21119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Shetye updated HIVE-21119:
---
Component/s: Query Processor
Query Planning
> String UDAF and count distinct in the same select give error
>
>
> Key: HIVE-21119
> URL: https://issues.apache.org/jira/browse/HIVE-21119
> Project: Hive
> Issue Type: Bug
> Components: Query Planning, Query Processor
>Reporter: Ravi Shetye
>Priority: Major
> Attachments: MaxUDA.java, run.log
>
>
> With the attached UDAF the following query crashes on hive.
> CRASHES
> {noformat}
> select rs_max(genderkey),count(distinct genderkey) from
> as_adventure.dimgender;
> {noformat}
> WORKS
> {noformat}
> select rs_max(genderkey) from as_adventure.dimgender;
> {noformat}
> The table looks like
> {noformat}
> 0: jdbc:hive2://localhost:1> select * from dimgender;
> OK
> INFO : Compiling
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7):
> select * from dimgender
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:dimgender.genderkey, type:string,
> comment:null), FieldSchema(name:dimgender.gendername, type:string,
> comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7);
> Time taken: 0.2 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Executing
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7):
> select * from dimgender
> INFO : Completed executing
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7);
> Time taken: 0.004 seconds
> INFO : OK
> INFO : Concurrency mode is disabled, not creating a lock manager
> +--+---+
> | dimgender.genderkey | dimgender.gendername |
> +--+---+
> | M| Male |
> | F| Female|
> | U| Unisex|
> +--+---+
> {noformat}
> {noformat}
> Vertex failed, vertexName=Reducer 2, vertexId=vertex_1547169244949_0024_2_01,
> diagnostics=[Task failed, taskId=task_1547169244949_0024_2_01_00,
> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
> failure ) :
> attempt_1547169244949_0024_2_01_00_0:java.lang.RuntimeException:
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row (tag=0)
> {"key":{"_col0":"F"},"value":{"_col0":"F"}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> {noformat}
> ...
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to
> execute method public boolean
> com.sample.MaxUDA$Evaluator.merge(java.lang.String) with arguments
> {F}:argument type mismatch
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFBridge$GenericUDAFBridgeEvaluator.merge(GenericUDAFBridge.java:176)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:216)
> {noformat}
> PLAN
> {noformat}
> ++
> | Explain |
> ++
> | Plan optimized by CBO. |
> ||
> | Vertex dependency in root stage|
> | Reducer 2 <- Map 1 (SIMPLE_EDGE) |
> | Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE)|
> ||
> | Stage-0|
> | Fetch Operator |
> | limit:-1 |
> | Stage-1|
> | Reducer 3|
> | File Output Operator [FS_
[jira] [Updated] (HIVE-21119) String UDAF and count distinct in the same select give error
[
https://issues.apache.org/jira/browse/HIVE-21119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Shetye updated HIVE-21119:
---
Labels: plannin (was: wrongresults)
> String UDAF and count distinct in the same select give error
>
>
> Key: HIVE-21119
> URL: https://issues.apache.org/jira/browse/HIVE-21119
> Project: Hive
> Issue Type: Bug
>Reporter: Ravi Shetye
>Priority: Major
> Labels: plannin
> Attachments: MaxUDA.java, run.log
>
>
> With the attached UDAF the following query crashes on hive.
> CRASHES
> {noformat}
> select rs_max(genderkey),count(distinct genderkey) from
> as_adventure.dimgender;
> {noformat}
> WORKS
> {noformat}
> select rs_max(genderkey) from as_adventure.dimgender;
> {noformat}
> The table looks like
> {noformat}
> 0: jdbc:hive2://localhost:1> select * from dimgender;
> OK
> INFO : Compiling
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7):
> select * from dimgender
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:dimgender.genderkey, type:string,
> comment:null), FieldSchema(name:dimgender.gendername, type:string,
> comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7);
> Time taken: 0.2 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Executing
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7):
> select * from dimgender
> INFO : Completed executing
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7);
> Time taken: 0.004 seconds
> INFO : OK
> INFO : Concurrency mode is disabled, not creating a lock manager
> +--+---+
> | dimgender.genderkey | dimgender.gendername |
> +--+---+
> | M| Male |
> | F| Female|
> | U| Unisex|
> +--+---+
> {noformat}
> {noformat}
> Vertex failed, vertexName=Reducer 2, vertexId=vertex_1547169244949_0024_2_01,
> diagnostics=[Task failed, taskId=task_1547169244949_0024_2_01_00,
> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
> failure ) :
> attempt_1547169244949_0024_2_01_00_0:java.lang.RuntimeException:
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row (tag=0)
> {"key":{"_col0":"F"},"value":{"_col0":"F"}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> {noformat}
> ...
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to
> execute method public boolean
> com.sample.MaxUDA$Evaluator.merge(java.lang.String) with arguments
> {F}:argument type mismatch
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFBridge$GenericUDAFBridgeEvaluator.merge(GenericUDAFBridge.java:176)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:216)
> {noformat}
> PLAN
> {noformat}
> ++
> | Explain |
> ++
> | Plan optimized by CBO. |
> ||
> | Vertex dependency in root stage|
> | Reducer 2 <- Map 1 (SIMPLE_EDGE) |
> | Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE)|
> ||
> | Stage-0|
> | Fetch Operator |
> | limit:-1 |
> | Stage-1|
> | Reducer 3|
> | File Output Operator [FS_6] |
> | Group By Opera
[jira] [Updated] (HIVE-21119) String UDAF and count distinct in the same select give error
[
https://issues.apache.org/jira/browse/HIVE-21119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Shetye updated HIVE-21119:
---
Description:
With the attached UDAF the following query crashes on hive.
CRASHES
{noformat}
select rs_max(genderkey),count(distinct genderkey) from as_adventure.dimgender;
{noformat}
WORKS
{noformat}
select rs_max(genderkey) from as_adventure.dimgender;
{noformat}
The table looks like
{noformat}
0: jdbc:hive2://localhost:1> select * from dimgender;
OK
INFO : Compiling
command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7):
select * from dimgender
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial = false)
INFO : Returning Hive schema:
Schema(fieldSchemas:[FieldSchema(name:dimgender.genderkey, type:string,
comment:null), FieldSchema(name:dimgender.gendername, type:string,
comment:null)], properties:null)
INFO : Completed compiling
command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7); Time
taken: 0.2 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing
command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7):
select * from dimgender
INFO : Completed executing
command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7); Time
taken: 0.004 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
+--+---+
| dimgender.genderkey | dimgender.gendername |
+--+---+
| M| Male |
| F| Female|
| U| Unisex|
+--+---+
{noformat}
{noformat}
Vertex failed, vertexName=Reducer 2, vertexId=vertex_1547169244949_0024_2_01,
diagnostics=[Task failed, taskId=task_1547169244949_0024_2_01_00,
diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
failure ) :
attempt_1547169244949_0024_2_01_00_0:java.lang.RuntimeException:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row (tag=0)
{"key":{"_col0":"F"},"value":{"_col0":"F"}}
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
{noformat}
...
{noformat}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute
method public boolean com.sample.MaxUDA$Evaluator.merge(java.lang.String) with
arguments {F}:argument type mismatch
at
org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFBridge$GenericUDAFBridgeEvaluator.merge(GenericUDAFBridge.java:176)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:216)
{noformat}
PLAN
{noformat}
++
| Explain |
++
| Plan optimized by CBO. |
||
| Vertex dependency in root stage|
| Reducer 2 <- Map 1 (SIMPLE_EDGE) |
| Reducer 3 <- Reducer 2 (CUSTOM_SIMPLE_EDGE)|
||
| Stage-0|
| Fetch Operator |
| limit:-1 |
| Stage-1|
| Reducer 3|
| File Output Operator [FS_6] |
| Group By Operator [GBY_12] (rows=1 width=368) |
|
Output:["_col0","_col1"],aggregations:["rs_max(VALUE._col0)","count(VALUE._col1)"]
|
| <-Reducer 2 [CUSTOM_SIMPLE_EDGE] |
| PARTITION_ONLY_SHUFFLE [RS_11] |
| Group By Operator [GBY_10] (rows=1 width=368) |
|
Output:["_col0","_col1"],aggregations:["rs_max(_col1)","count(_col0)"] |
| Group By Operator [GBY_9] (rows=3 width=2) |
|
Output:["_col0","_col1"],aggregations:["rs_max(VALUE._col0)"],keys:KEY._col0 |
| <-Map 1 [SIMPLE_EDGE]|
|
[jira] [Updated] (HIVE-21119) String UDAF and count distinct in the same select give error
[
https://issues.apache.org/jira/browse/HIVE-21119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Shetye updated HIVE-21119:
---
Attachment: run.log
> String UDAF and count distinct in the same select give error
>
>
> Key: HIVE-21119
> URL: https://issues.apache.org/jira/browse/HIVE-21119
> Project: Hive
> Issue Type: Bug
>Reporter: Ravi Shetye
>Priority: Major
> Labels: wrongresults
> Attachments: MaxUDA.java, run.log
>
>
> With the attached UDAF the following query crashes on hive.
> CRASHES
> {noformat}
> select rs_max(genderkey),count(distinct genderkey) from
> as_adventure.dimgender;
> {noformat}
> WORKS
> {noformat}
> select rs_max(genderkey) from as_adventure.dimgender;
> {noformat}
> The table looks like
> {noformat}
> 0: jdbc:hive2://localhost:1> select * from dimgender;
> OK
> INFO : Compiling
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7):
> select * from dimgender
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:dimgender.genderkey, type:string,
> comment:null), FieldSchema(name:dimgender.gendername, type:string,
> comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7);
> Time taken: 0.2 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Executing
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7):
> select * from dimgender
> INFO : Completed executing
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7);
> Time taken: 0.004 seconds
> INFO : OK
> INFO : Concurrency mode is disabled, not creating a lock manager
> +--+---+
> | dimgender.genderkey | dimgender.gendername |
> +--+---+
> | M| Male |
> | F| Female|
> | U| Unisex|
> +--+---+
> {noformat}
> {noformat}
> Vertex failed, vertexName=Reducer 2, vertexId=vertex_1547169244949_0024_2_01,
> diagnostics=[Task failed, taskId=task_1547169244949_0024_2_01_00,
> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
> failure ) :
> attempt_1547169244949_0024_2_01_00_0:java.lang.RuntimeException:
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row (tag=0)
> {"key":{"_col0":"F"},"value":{"_col0":"F"}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> {noformat}
> ...
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to
> execute method public boolean
> com.sample.MaxUDA$Evaluator.merge(java.lang.String) with arguments
> {F}:argument type mismatch
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFBridge$GenericUDAFBridgeEvaluator.merge(GenericUDAFBridge.java:176)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:216)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21119) String UDAF and count distinct in the same select give error
[
https://issues.apache.org/jira/browse/HIVE-21119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Shetye updated HIVE-21119:
---
Attachment: MaxUDA.java
> String UDAF and count distinct in the same select give error
>
>
> Key: HIVE-21119
> URL: https://issues.apache.org/jira/browse/HIVE-21119
> Project: Hive
> Issue Type: Bug
>Reporter: Ravi Shetye
>Priority: Major
> Labels: wrongresults
> Attachments: MaxUDA.java
>
>
> With the attached UDAF the following query crashes on hive.
> CRASHES
> {noformat}
> select rs_max(genderkey),count(distinct genderkey) from
> as_adventure.dimgender;
> {noformat}
> WORKS
> {noformat}
> select rs_max(genderkey) from as_adventure.dimgender;
> {noformat}
> The table looks like
> {noformat}
> 0: jdbc:hive2://localhost:1> select * from dimgender;
> OK
> INFO : Compiling
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7):
> select * from dimgender
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:dimgender.genderkey, type:string,
> comment:null), FieldSchema(name:dimgender.gendername, type:string,
> comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7);
> Time taken: 0.2 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Executing
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7):
> select * from dimgender
> INFO : Completed executing
> command(queryId=hive_20190111225125_486e6e6b-97fa-4dda-9688-a733180bcfe7);
> Time taken: 0.004 seconds
> INFO : OK
> INFO : Concurrency mode is disabled, not creating a lock manager
> +--+---+
> | dimgender.genderkey | dimgender.gendername |
> +--+---+
> | M| Male |
> | F| Female|
> | U| Unisex|
> +--+---+
> {noformat}
> {noformat}
> Vertex failed, vertexName=Reducer 2, vertexId=vertex_1547169244949_0024_2_01,
> diagnostics=[Task failed, taskId=task_1547169244949_0024_2_01_00,
> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
> failure ) :
> attempt_1547169244949_0024_2_01_00_0:java.lang.RuntimeException:
> java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Hive Runtime Error while processing row (tag=0)
> {"key":{"_col0":"F"},"value":{"_col0":"F"}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> {noformat}
> ...
> {noformat}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to
> execute method public boolean
> com.sample.MaxUDA$Evaluator.merge(java.lang.String) with arguments
> {F}:argument type mismatch
> at
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFBridge$GenericUDAFBridgeEvaluator.merge(GenericUDAFBridge.java:176)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:216)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21066) unexpected scemantic exception
[
https://issues.apache.org/jira/browse/HIVE-21066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Shetye updated HIVE-21066:
---
Summary: unexpected scemantic exception (was: unexpected schematic
exception)
> unexpected scemantic exception
> --
>
> Key: HIVE-21066
> URL: https://issues.apache.org/jira/browse/HIVE-21066
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Ravi Shetye
>Priority: Major
>
> *WORKS*
> {noformat}
> 0: jdbc:hive2://localhost:1> select c1=c1 from (select "a" as c1) t1;
> +---+
> | _c0 |
> +---+
> | true |
> +---+
> {noformat}
> {noformat}
> 0: jdbc:hive2://localhost:1> select c1 in ("a","b") a from (select "a" c1
> )t1 ;
> +---+
> | a |
> +---+
> | true |
> +---+
> {noformat}
> {noformat}
> 0: jdbc:hive2://localhost:1> select true in (true) a from (select "a" c1
> )t1;
> +---+
> | a |
> +---+
> | true |
> +---+
> {noformat}
> *DOES NOT WORK*
> {noformat}
> 0: jdbc:hive2://localhost:1> select c1=c1 in (true) a from (select "a" c1
> )t1 ;
> Error: Error while compiling statement: FAILED: SemanticException Line 0:-1
> Wrong arguments 'true': The arguments for IN should be the same type! Types
> are: {string IN (boolean)} (state=42000,code=4)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21066) unexpected SemanticException
[
https://issues.apache.org/jira/browse/HIVE-21066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Shetye updated HIVE-21066:
---
Affects Version/s: (was: 3.0.0)
2.3.0
> unexpected SemanticException
>
>
> Key: HIVE-21066
> URL: https://issues.apache.org/jira/browse/HIVE-21066
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Ravi Shetye
>Priority: Major
>
> *WORKS*
> {noformat}
> 0: jdbc:hive2://localhost:1> select c1=c1 from (select "a" as c1) t1;
> +---+
> | _c0 |
> +---+
> | true |
> +---+
> {noformat}
> {noformat}
> 0: jdbc:hive2://localhost:1> select c1 in ("a","b") a from (select "a" c1
> )t1 ;
> +---+
> | a |
> +---+
> | true |
> +---+
> {noformat}
> {noformat}
> 0: jdbc:hive2://localhost:1> select true in (true) a from (select "a" c1
> )t1;
> +---+
> | a |
> +---+
> | true |
> +---+
> {noformat}
> *DOES NOT WORK*
> {noformat}
> 0: jdbc:hive2://localhost:1> select c1=c1 in (true) a from (select "a" c1
> )t1 ;
> Error: Error while compiling statement: FAILED: SemanticException Line 0:-1
> Wrong arguments 'true': The arguments for IN should be the same type! Types
> are: {string IN (boolean)} (state=42000,code=4)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21066) unexpected SemanticException
[
https://issues.apache.org/jira/browse/HIVE-21066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Shetye updated HIVE-21066:
---
Summary: unexpected SemanticException (was: unexpected scemantic exception)
> unexpected SemanticException
>
>
> Key: HIVE-21066
> URL: https://issues.apache.org/jira/browse/HIVE-21066
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Ravi Shetye
>Priority: Major
>
> *WORKS*
> {noformat}
> 0: jdbc:hive2://localhost:1> select c1=c1 from (select "a" as c1) t1;
> +---+
> | _c0 |
> +---+
> | true |
> +---+
> {noformat}
> {noformat}
> 0: jdbc:hive2://localhost:1> select c1 in ("a","b") a from (select "a" c1
> )t1 ;
> +---+
> | a |
> +---+
> | true |
> +---+
> {noformat}
> {noformat}
> 0: jdbc:hive2://localhost:1> select true in (true) a from (select "a" c1
> )t1;
> +---+
> | a |
> +---+
> | true |
> +---+
> {noformat}
> *DOES NOT WORK*
> {noformat}
> 0: jdbc:hive2://localhost:1> select c1=c1 in (true) a from (select "a" c1
> )t1 ;
> Error: Error while compiling statement: FAILED: SemanticException Line 0:-1
> Wrong arguments 'true': The arguments for IN should be the same type! Types
> are: {string IN (boolean)} (state=42000,code=4)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21066) unexpected schematic exception
[
https://issues.apache.org/jira/browse/HIVE-21066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Shetye updated HIVE-21066:
---
Environment: (was:
{noformat}
*no* further _formatting_ is done here
{noformat}
)
> unexpected schematic exception
> --
>
> Key: HIVE-21066
> URL: https://issues.apache.org/jira/browse/HIVE-21066
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Ravi Shetye
>Priority: Major
>
> *WORKS*
> {noformat}
> 0: jdbc:hive2://localhost:1> select c1=c1 from (select "a" as c1) t1;
> +---+
> | _c0 |
> +---+
> | true |
> +---+
> {noformat}
> {noformat}
> 0: jdbc:hive2://localhost:1> select c1 in ("a","b") a from (select "a" c1
> )t1 ;
> +---+
> | a |
> +---+
> | true |
> +---+
> {noformat}
> {noformat}
> 0: jdbc:hive2://localhost:1> select true in (true) a from (select "a" c1
> )t1;
> +---+
> | a |
> +---+
> | true |
> +---+
> {noformat}
> *DOES NOT WORK*
> {noformat}
> 0: jdbc:hive2://localhost:1> select c1=c1 in (true) a from (select "a" c1
> )t1 ;
> Error: Error while compiling statement: FAILED: SemanticException Line 0:-1
> Wrong arguments 'true': The arguments for IN should be the same type! Types
> are: {string IN (boolean)} (state=42000,code=4)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21062) Hive 2.X auto converts a round decimal to integer
[
https://issues.apache.org/jira/browse/HIVE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16728503#comment-16728503
]
Ravi Shetye commented on HIVE-21062:
{noformat}
it seem it just trims away zeros - and packages it into the minimal type it
fits.{noformat}
Yes, that is precisely what is happening, but would you agree that "packages it
into the minimal type it fits" might not be the right thing to do?
for 1.200 hive prunes the insignificant zeros, and retains 1.2
{code:java}
0: jdbc:hive2://localhost:1> select 1.200 ;
+--+
| _c0 |
+--+
| 1.2 |
+--+
1 row selected (0.221 seconds)
0: jdbc:hive2://localhost:1>
{code}
on older version of hive select 1.0 would return 1.0.
on hive 1.2
{code:java}
0: jdbc:hive2://node1.ubuntu.localdomain:2181> select 1.0 ;
+--+--+
| _c0 |
+--+--+
| 1.0 |
+--+--+
{code}
corresponding plan
{noformat}
+--+--+
| Explain|
+--+--+
| ABSTRACT SYNTAX TREE:|
| |
| TOK_QUERY|
|TOK_INSERT|
| TOK_DESTINATION|
| TOK_DIR |
| TOK_TMP_FILE |
| TOK_SELECT |
| TOK_SELEXPR |
| 1.0 |
| |
| |
| STAGE DEPENDENCIES: |
| Stage-0 is a root stage|
| |
| STAGE PLANS: |
| Stage: Stage-0 |
| Fetch Operator |
| limit: -1 |
| Processor Tree:|
| TableScan|
| alias: _dummy_table|
| Row Limit Per Split: 1 |
| GatherStats: false |
| Select Operator|
| expressions: 1.0 (type: double) |
| outputColumnNames: _col0 |
| ListSink |
| |
+--+--+
{noformat}
> Hive 2.X auto converts a round decimal to integer
> -
>
> Key: HIVE-21062
> URL: https://issues.apache.org/jira/browse/HIVE-21062
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.3.4
>Reporter: Ravi Shetye
>Priority: Major
>
> {noformat}
> 0: jdbc:hive2://localhost:1> select 1.0 ;
> +--+
> | _c0 |
> +--+
> | 1|
> +--+
> {noformat}
> It happens with parameters to functions as well
> {code:java}
> 0: jdbc:hive2://localhost:1> select CONCAT("hi",1.0) ;
> +--+
> | _c0 |
> +--+
> | hi1 |
> +--+
> {code}
> Hive reads 1.0 as decimal(1,0), and then may be is converting it to int ?
> {code:java}
> 0: jdbc:hive2://localhost:1> EXPLAIN EXTENDED select 1.0;
> +--+
> | Explain |
> +--+
> | STAGE DEPENDENCIES: |
> | Stage-0 is a root stage|
> | |
> | STAGE PLANS: |
> | Stage: Stage-0 |
> | Fetch Operator |
> | limit: -1 |
> | Processor Tree:|
> | TableScan|
> | alias: _dummy_table|
> | Row Limit Per Split: 1 |
> | GatherStats: false |
> | Select Operator|
> | expressions: 1 (type: decimal(1,0)) |
> | outputColumnNames: _col0 |
> | ListSink |
> | |
> +--+
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21062) Hive 2.X auto converts a round decimal to integer
[
https://issues.apache.org/jira/browse/HIVE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Shetye updated HIVE-21062:
---
Summary: Hive 2.X auto converts a round decimal to integer (was: Hive 3.0
auto converts a round decimal to integer)
> Hive 2.X auto converts a round decimal to integer
> -
>
> Key: HIVE-21062
> URL: https://issues.apache.org/jira/browse/HIVE-21062
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.3.4
>Reporter: Ravi Shetye
>Priority: Major
>
> {noformat}
> 0: jdbc:hive2://localhost:1> select 1.0 ;
> +--+
> | _c0 |
> +--+
> | 1|
> +--+
> {noformat}
> It happens with parameters to functions as well
> {code:java}
> 0: jdbc:hive2://localhost:1> select CONCAT("hi",1.0) ;
> +--+
> | _c0 |
> +--+
> | hi1 |
> +--+
> {code}
> Hive reads 1.0 as decimal(1,0), and then may be is converting it to int ?
> {code:java}
> 0: jdbc:hive2://localhost:1> EXPLAIN EXTENDED select 1.0;
> +--+
> | Explain |
> +--+
> | STAGE DEPENDENCIES: |
> | Stage-0 is a root stage|
> | |
> | STAGE PLANS: |
> | Stage: Stage-0 |
> | Fetch Operator |
> | limit: -1 |
> | Processor Tree:|
> | TableScan|
> | alias: _dummy_table|
> | Row Limit Per Split: 1 |
> | GatherStats: false |
> | Select Operator|
> | expressions: 1 (type: decimal(1,0)) |
> | outputColumnNames: _col0 |
> | ListSink |
> | |
> +--+
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
