[jira] [Created] (HIVE-7453) Partition Pruning enhancements 1
Laljo John Pullokkaran created HIVE-7453: Summary: Partition Pruning enhancements 1 Key: HIVE-7453 URL: https://issues.apache.org/jira/browse/HIVE-7453 Project: Hive Issue Type: Sub-task Reporter: Laljo John Pullokkaran Assignee: Laljo John Pullokkaran 1. Handle type casts 2. Handle Literal Conversion for Partition Pruning expressions -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 23387: HIVE-6806: Native avro support
On July 19, 2014, 12:43 a.m., David Chen wrote: serde/src/test/org/apache/hadoop/hive/serde2/avro/TestTypeInfoToSchema.java, line 294 https://reviews.apache.org/r/23387/diff/9/?file=634616#file634616line294 It would improve maintainability to keep the test schemas in separate .avsc files under serde/src/test/resources rather than inline in the file. You can use Guava's Resources class to get the file and construct the schema. For example: Schema expectedSchema = new Schema.Parser().parse( Resources.getResource(record1.avsc).openStream()); Ashish Singh wrote: David, as the tests have a lot common in their schema I am using a method to generate the common schema part and each test only provides a part of schema that is specific to the test. This made my tests have much less LOC. If I create a .avsc file for each test it will much more cumbersome for both maintaining and adding new tests. Hi Ashish, sorry I was a bit unclear. The tests for the individual data types are fine. I thought that moving just the two large schemas into their own files may make them easier to maintain since keeping them inline requires a large number of escape characters. - David --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23387/#review48168 --- On July 19, 2014, 5:11 a.m., Ashish Singh wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23387/ --- (Updated July 19, 2014, 5:11 a.m.) Review request for hive. Bugs: HIVE-6806 https://issues.apache.org/jira/browse/HIVE-6806 Repository: hive-git Description --- HIVE-6806: Native avro support Diffs - ql/src/java/org/apache/hadoop/hive/ql/io/AvroStorageFormatDescriptor.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/io/IOConstants.java 1bae0a8fee04049f90b16d813ff4c96707b349c8 ql/src/main/resources/META-INF/services/org.apache.hadoop.hive.ql.io.StorageFormatDescriptor a23ff115512da5fe3167835a88d582c427585b8e ql/src/test/org/apache/hadoop/hive/ql/io/TestStorageFormatDescriptor.java d53ebc65174d66bfeee25fd2891c69c78f9137ee ql/src/test/queries/clientpositive/avro_compression_enabled_native.q PRE-CREATION ql/src/test/queries/clientpositive/avro_decimal_native.q PRE-CREATION ql/src/test/queries/clientpositive/avro_joins_native.q PRE-CREATION ql/src/test/queries/clientpositive/avro_native.q PRE-CREATION ql/src/test/queries/clientpositive/avro_partitioned_native.q PRE-CREATION ql/src/test/queries/clientpositive/avro_schema_evolution_native.q PRE-CREATION ql/src/test/results/clientpositive/avro_compression_enabled_native.q.out PRE-CREATION ql/src/test/results/clientpositive/avro_decimal_native.q.out PRE-CREATION ql/src/test/results/clientpositive/avro_joins_native.q.out PRE-CREATION ql/src/test/results/clientpositive/avro_native.q.out PRE-CREATION ql/src/test/results/clientpositive/avro_partitioned_native.q.out PRE-CREATION ql/src/test/results/clientpositive/avro_schema_evolution_native.q.out PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java 0db12437406170686a21b6055d83156fe5d6a55f serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerDe.java 1fe31e0034f8988d03a0c51a90904bb93e7cb157 serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 4564e75d9bfc73f8e10f160e2535f1a08b90ff79 serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java PRE-CREATION serde/src/test/org/apache/hadoop/hive/serde2/avro/TestTypeInfoToSchema.java PRE-CREATION Diff: https://reviews.apache.org/r/23387/diff/ Testing --- Added qTests and unit tests Thanks, Ashish Singh
[jira] [Commented] (HIVE-7404) Revoke privilege should support revoking of grant option
[
https://issues.apache.org/jira/browse/HIVE-7404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067428#comment-14067428
]
Lefty Leverenz commented on HIVE-7404:
--
Thanks for the docs, [~jdere]. I added version information to the SQL Standard
Based Authorization wiki, but held off on the wiki for default Authorization
because I don't know which jira added WITH ADMIN OPTION to GRANT ROLE. Was
that added in 0.13.0 along with the clause in SQL standard based authorization?
(That's the earliest I've found it in the test suite, but I can't tell whether
it's SQL standard or default, or both.)
Revoke privilege should support revoking of grant option
Key: HIVE-7404
URL: https://issues.apache.org/jira/browse/HIVE-7404
Project: Hive
Issue Type: Sub-task
Components: Authorization
Reporter: Jason Dere
Assignee: Jason Dere
Fix For: 0.14.0
Attachments: HIVE-7404.1.patch, HIVE-7404.2.patch
Similar to HIVE-6252, but for grant option on privileges:
{noformat}
REVOKE GRANT OPTION FOR privilege ON object FROM USER user
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7357) Add vectorized support for BINARY data type
[
https://issues.apache.org/jira/browse/HIVE-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067440#comment-14067440
]
Hive QA commented on HIVE-7357:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12656676/HIVE-7357.2.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 5718 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_stats_counter
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_fail_8
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/860/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/860/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-860/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12656676
Add vectorized support for BINARY data type
---
Key: HIVE-7357
URL: https://issues.apache.org/jira/browse/HIVE-7357
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7357.1.patch, HIVE-7357.2.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Review Request 23387: HIVE-6806: Native avro support
On July 18, 2014, 11:42 p.m., Lars Francke wrote: serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java, line 269 https://reviews.apache.org/r/23387/diff/9/?file=634613#file634613line269 final is not used elsewhere in this file for local variables and I think it's true for most of the rest of Hive's code. Ashish Singh wrote: That does not mean using final for local variables is wrong. Convention over correctness? Unless there is a strong reason for not declaring variables, not supposed to be modified, as final, I would argue using final is correct. I'll obviously leave it up to you and it's just my personal opinion but Hive's codebase is hard enough to understand as it is for newcomers. Adding confusion by introducing multiple coding styles doesn't help even if your way may be strictly better it may still hurt in the long run. - Lars --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23387/#review48167 --- On July 19, 2014, 5:11 a.m., Ashish Singh wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23387/ --- (Updated July 19, 2014, 5:11 a.m.) Review request for hive. Bugs: HIVE-6806 https://issues.apache.org/jira/browse/HIVE-6806 Repository: hive-git Description --- HIVE-6806: Native avro support Diffs - ql/src/java/org/apache/hadoop/hive/ql/io/AvroStorageFormatDescriptor.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/io/IOConstants.java 1bae0a8fee04049f90b16d813ff4c96707b349c8 ql/src/main/resources/META-INF/services/org.apache.hadoop.hive.ql.io.StorageFormatDescriptor a23ff115512da5fe3167835a88d582c427585b8e ql/src/test/org/apache/hadoop/hive/ql/io/TestStorageFormatDescriptor.java d53ebc65174d66bfeee25fd2891c69c78f9137ee ql/src/test/queries/clientpositive/avro_compression_enabled_native.q PRE-CREATION ql/src/test/queries/clientpositive/avro_decimal_native.q PRE-CREATION ql/src/test/queries/clientpositive/avro_joins_native.q PRE-CREATION ql/src/test/queries/clientpositive/avro_native.q PRE-CREATION ql/src/test/queries/clientpositive/avro_partitioned_native.q PRE-CREATION ql/src/test/queries/clientpositive/avro_schema_evolution_native.q PRE-CREATION ql/src/test/results/clientpositive/avro_compression_enabled_native.q.out PRE-CREATION ql/src/test/results/clientpositive/avro_decimal_native.q.out PRE-CREATION ql/src/test/results/clientpositive/avro_joins_native.q.out PRE-CREATION ql/src/test/results/clientpositive/avro_native.q.out PRE-CREATION ql/src/test/results/clientpositive/avro_partitioned_native.q.out PRE-CREATION ql/src/test/results/clientpositive/avro_schema_evolution_native.q.out PRE-CREATION serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java 0db12437406170686a21b6055d83156fe5d6a55f serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerDe.java 1fe31e0034f8988d03a0c51a90904bb93e7cb157 serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 4564e75d9bfc73f8e10f160e2535f1a08b90ff79 serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java PRE-CREATION serde/src/test/org/apache/hadoop/hive/serde2/avro/TestTypeInfoToSchema.java PRE-CREATION Diff: https://reviews.apache.org/r/23387/diff/ Testing --- Added qTests and unit tests Thanks, Ashish Singh
[jira] [Commented] (HIVE-7450) Database should inherit perms of warehouse dir
[
https://issues.apache.org/jira/browse/HIVE-7450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067454#comment-14067454
]
Hive QA commented on HIVE-7450:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12656681/HIVE-7450.patch
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 5748 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_fail_8
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/861/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/861/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-861/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12656681
Database should inherit perms of warehouse dir
--
Key: HIVE-7450
URL: https://issues.apache.org/jira/browse/HIVE-7450
Project: Hive
Issue Type: Bug
Components: Security
Reporter: Szehon Ho
Assignee: Szehon Ho
Attachments: HIVE-7450.patch
One more ask: the database directory should inherit permission and extended
ACL's of the hive warehouse directory.
As table dirs are inheriting it, theres no reason that database dirs
shouldn't inherit it.
Behavior is governed by hive.warehouse.subdir.inherit.perms flag.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7385) Optimize for empty relation scans
[
https://issues.apache.org/jira/browse/HIVE-7385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067469#comment-14067469
]
Hive QA commented on HIVE-7385:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12656687/HIVE-7385.2.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 5748 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_fail_8
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Json
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/862/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/862/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-862/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12656687
Optimize for empty relation scans
-
Key: HIVE-7385
URL: https://issues.apache.org/jira/browse/HIVE-7385
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Attachments: HIVE-7385.1.patch, HIVE-7385.2.patch, HIVE-7385.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7385) Optimize for empty relation scans
[ https://issues.apache.org/jira/browse/HIVE-7385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7385: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Optimize for empty relation scans - Key: HIVE-7385 URL: https://issues.apache.org/jira/browse/HIVE-7385 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.14.0 Attachments: HIVE-7385.1.patch, HIVE-7385.2.patch, HIVE-7385.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7054) Support ELT UDF in vectorized mode
[ https://issues.apache.org/jira/browse/HIVE-7054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7054: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Deepesh! Support ELT UDF in vectorized mode -- Key: HIVE-7054 URL: https://issues.apache.org/jira/browse/HIVE-7054 Project: Hive Issue Type: New Feature Components: Vectorization Affects Versions: 0.14.0 Reporter: Deepesh Khandelwal Assignee: Deepesh Khandelwal Fix For: 0.14.0 Attachments: HIVE-7054.2.patch, HIVE-7054.3.patch, HIVE-7054.4.patch, HIVE-7054.5.patch, HIVE-7054.patch Implement support for ELT udf in vectorized execution mode. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7385) Optimize for empty relation scans
[ https://issues.apache.org/jira/browse/HIVE-7385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-7385: - Labels: TODOC14 (was: ) Optimize for empty relation scans - Key: HIVE-7385 URL: https://issues.apache.org/jira/browse/HIVE-7385 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Labels: TODOC14 Fix For: 0.14.0 Attachments: HIVE-7385.1.patch, HIVE-7385.2.patch, HIVE-7385.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7385) Optimize for empty relation scans
[ https://issues.apache.org/jira/browse/HIVE-7385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067675#comment-14067675 ] Lefty Leverenz commented on HIVE-7385: -- This adds configuration parameter *hive.optimize.null.scan* with a default of true. It needs to be documented in the wiki for 0.14.0. * [Configuration Properties | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties] Optimize for empty relation scans - Key: HIVE-7385 URL: https://issues.apache.org/jira/browse/HIVE-7385 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Labels: TODOC14 Fix For: 0.14.0 Attachments: HIVE-7385.1.patch, HIVE-7385.2.patch, HIVE-7385.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7357) Add vectorized support for BINARY data type
[ https://issues.apache.org/jira/browse/HIVE-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067721#comment-14067721 ] Matt McCline commented on HIVE-7357: None of these failures are related to my changes. The change is ready. Add vectorized support for BINARY data type --- Key: HIVE-7357 URL: https://issues.apache.org/jira/browse/HIVE-7357 Project: Hive Issue Type: Sub-task Reporter: Matt McCline Assignee: Matt McCline Attachments: HIVE-7357.1.patch, HIVE-7357.2.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7422) Array out of bounds exception involving ql.exec.vector.expressions.aggregates.gen.VectorUDAFAvgDouble
[
https://issues.apache.org/jira/browse/HIVE-7422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-7422:
---
Status: Patch Available (was: In Progress)
Array out of bounds exception involving
ql.exec.vector.expressions.aggregates.gen.VectorUDAFAvgDouble
-
Key: HIVE-7422
URL: https://issues.apache.org/jira/browse/HIVE-7422
Project: Hive
Issue Type: Bug
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7422.1.patch, HIVE-7422.2.patch, TestWithORC.zip,
fail_119.sql
One of several found by Raj Bains.
M/R or Tez.
{code}
set hive.vectorized.execution.enabled=true;
{code}
Query:
{code}
SELECT `Starbucks`.`product` AS `none_product_nk`, AVG(CAST(50 AS DOUBLE))
AS `avg_x_ok`, AVG(CAST(50 AS DOUBLE)) AS `avg_y_ok` FROM
`default`.`testv1_Starbucks` `Starbucks` GROUP BY `Starbucks`.`product` ;
{code}
Stack trace:
{code}
Caused by: java.lang.ArrayIndexOutOfBoundsException: 50
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.CastLongToDouble.evaluate(CastLongToDouble.java:50)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFAvgDouble.aggregateInputSelection(VectorUDAFAvgDouble.java:139)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processAggregators(VectorGroupByOperator.java:121)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeHashAggregate.processBatch(VectorGroupByOperator.java:295)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.processOp(VectorGroupByOperator.java:711)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:139)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7422) Array out of bounds exception involving ql.exec.vector.expressions.aggregates.gen.VectorUDAFAvgDouble
[
https://issues.apache.org/jira/browse/HIVE-7422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-7422:
---
Status: In Progress (was: Patch Available)
Array out of bounds exception involving
ql.exec.vector.expressions.aggregates.gen.VectorUDAFAvgDouble
-
Key: HIVE-7422
URL: https://issues.apache.org/jira/browse/HIVE-7422
Project: Hive
Issue Type: Bug
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7422.1.patch, HIVE-7422.2.patch, TestWithORC.zip,
fail_119.sql
One of several found by Raj Bains.
M/R or Tez.
{code}
set hive.vectorized.execution.enabled=true;
{code}
Query:
{code}
SELECT `Starbucks`.`product` AS `none_product_nk`, AVG(CAST(50 AS DOUBLE))
AS `avg_x_ok`, AVG(CAST(50 AS DOUBLE)) AS `avg_y_ok` FROM
`default`.`testv1_Starbucks` `Starbucks` GROUP BY `Starbucks`.`product` ;
{code}
Stack trace:
{code}
Caused by: java.lang.ArrayIndexOutOfBoundsException: 50
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.CastLongToDouble.evaluate(CastLongToDouble.java:50)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFAvgDouble.aggregateInputSelection(VectorUDAFAvgDouble.java:139)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processAggregators(VectorGroupByOperator.java:121)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeHashAggregate.processBatch(VectorGroupByOperator.java:295)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.processOp(VectorGroupByOperator.java:711)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:139)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7422) Array out of bounds exception involving ql.exec.vector.expressions.aggregates.gen.VectorUDAFAvgDouble
[
https://issues.apache.org/jira/browse/HIVE-7422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-7422:
---
Attachment: HIVE-7422.2.patch
Array out of bounds exception involving
ql.exec.vector.expressions.aggregates.gen.VectorUDAFAvgDouble
-
Key: HIVE-7422
URL: https://issues.apache.org/jira/browse/HIVE-7422
Project: Hive
Issue Type: Bug
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7422.1.patch, HIVE-7422.2.patch, TestWithORC.zip,
fail_119.sql
One of several found by Raj Bains.
M/R or Tez.
{code}
set hive.vectorized.execution.enabled=true;
{code}
Query:
{code}
SELECT `Starbucks`.`product` AS `none_product_nk`, AVG(CAST(50 AS DOUBLE))
AS `avg_x_ok`, AVG(CAST(50 AS DOUBLE)) AS `avg_y_ok` FROM
`default`.`testv1_Starbucks` `Starbucks` GROUP BY `Starbucks`.`product` ;
{code}
Stack trace:
{code}
Caused by: java.lang.ArrayIndexOutOfBoundsException: 50
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.CastLongToDouble.evaluate(CastLongToDouble.java:50)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFAvgDouble.aggregateInputSelection(VectorUDAFAvgDouble.java:139)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processAggregators(VectorGroupByOperator.java:121)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeHashAggregate.processBatch(VectorGroupByOperator.java:295)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.processOp(VectorGroupByOperator.java:711)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:139)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7357) Add vectorized support for BINARY data type
[ https://issues.apache.org/jira/browse/HIVE-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-7357: --- Status: In Progress (was: Patch Available) Add vectorized support for BINARY data type --- Key: HIVE-7357 URL: https://issues.apache.org/jira/browse/HIVE-7357 Project: Hive Issue Type: Sub-task Reporter: Matt McCline Assignee: Matt McCline Attachments: HIVE-7357.1.patch, HIVE-7357.2.patch, HIVE-7357.3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7357) Add vectorized support for BINARY data type
[ https://issues.apache.org/jira/browse/HIVE-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-7357: --- Status: Patch Available (was: In Progress) Add vectorized support for BINARY data type --- Key: HIVE-7357 URL: https://issues.apache.org/jira/browse/HIVE-7357 Project: Hive Issue Type: Sub-task Reporter: Matt McCline Assignee: Matt McCline Attachments: HIVE-7357.1.patch, HIVE-7357.2.patch, HIVE-7357.3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7357) Add vectorized support for BINARY data type
[ https://issues.apache.org/jira/browse/HIVE-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067737#comment-14067737 ] Matt McCline commented on HIVE-7357: Rebased changes and submitted new patch. The file itests/qtest/testconfiguration.properties is becoming a hot update spot. Add vectorized support for BINARY data type --- Key: HIVE-7357 URL: https://issues.apache.org/jira/browse/HIVE-7357 Project: Hive Issue Type: Sub-task Reporter: Matt McCline Assignee: Matt McCline Attachments: HIVE-7357.1.patch, HIVE-7357.2.patch, HIVE-7357.3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7421) Null pointer exception involving ql.exec.vector.expressions.StringConcatColScalar.evaluate
[
https://issues.apache.org/jira/browse/HIVE-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067764#comment-14067764
]
Matt McCline commented on HIVE-7421:
Made this change.
Now there are exceptions being thrown and caught below in the same call
sequence.
{code}
java.lang.IllegalArgumentException
at java.sql.Date.valueOf(Date.java:138)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFDateString$1.evaluate(VectorUDFDateString.java:38)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.StringUnaryUDF.evaluate(StringUnaryUDF.java:101)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.StringConcatColScalar.evaluate(StringConcatColScalar.java:50)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.FilterStringColEqualStringScalar.evaluate(FilterStringColEqualStringScalar.java:48)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.FilterExprOrExpr.evaluate(FilterExprOrExpr.java:87)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.FilterExprAndExpr.evaluate(FilterExprAndExpr.java:37)
at
org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.processOp(VectorFilterOperator.java:91)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:695)
{code}
Null pointer exception involving
ql.exec.vector.expressions.StringConcatColScalar.evaluate
--
Key: HIVE-7421
URL: https://issues.apache.org/jira/browse/HIVE-7421
Project: Hive
Issue Type: Bug
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: TestWithORC.zip, fail_47.sql, fail_62.sql, fail_932.sql
One of several found by Raj Bains.
M/R or Tez.
{code}
set hive.vectorized.execution.enabled=true;
{code}
Seems very similar to https://issues.apache.org/jira/browse/HIVE-6649
Query:
{code}
SELECT FLOOR((7 + DATEDIFF(`Staples`.`order_date_`,
CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING), '-01-01 00:00:00'))
+pmod(8 + pmod(datediff(CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING),
'-01-01 00:00:00'), '1995-01-01'), 7) - 2, 7) ) / 7) AS `wk_order_date_ok`,
SUM(`Staples`.`sales_total`) AS `sum_sales_total_ok` FROM
`default`.`testv1_Staples` `Staples` GROUP BY FLOOR((7 +
DATEDIFF(`Staples`.`order_date_`, CONCAT(CAST(YEAR(`Staples`.`order_date_`)
AS STRING), '-01-01 00:00:00')) +pmod(8 +
pmod(datediff(CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING), '-01-01
00:00:00'), '1995-01-01'), 7) - 2, 7) ) / 7) ;
{code}
Stack trace:
{code}
Caused by: java.lang.NullPointerException
at java.lang.System.arraycopy(Native Method)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setConcat(BytesColumnVector.java:190)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.StringConcatColScalar.evaluate(StringConcatColScalar.java:78)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFDateDiffColCol.evaluate(VectorUDFDateDiffColCol.java:59)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.LongScalarAddLongColumn.evaluate(LongScalarAddLongColumn.java:65)
at
[jira] [Commented] (HIVE-7357) Add vectorized support for BINARY data type
[
https://issues.apache.org/jira/browse/HIVE-7357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067796#comment-14067796
]
Hive QA commented on HIVE-7357:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12656766/HIVE-7357.3.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 5751 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_8
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_fail_8
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/864/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/864/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-864/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12656766
Add vectorized support for BINARY data type
---
Key: HIVE-7357
URL: https://issues.apache.org/jira/browse/HIVE-7357
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7357.1.patch, HIVE-7357.2.patch, HIVE-7357.3.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7421) Null pointer exception involving ql.exec.vector.expressions.StringConcatColScalar.evaluate
[
https://issues.apache.org/jira/browse/HIVE-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067801#comment-14067801
]
Matt McCline commented on HIVE-7421:
The dates being passed to VectorUDFDateString which calls Date.valueOf include
the time 00:00:00, which isn't allowed...
{code}
14/07/19 21:29:35 INFO vector.VectorGroupByOperator: VectorUDFDateString string
passed to Date.valueOf = '2002-12-31 00:00:00'
{code}
Is this an error in the query or in our execution of it?
Null pointer exception involving
ql.exec.vector.expressions.StringConcatColScalar.evaluate
--
Key: HIVE-7421
URL: https://issues.apache.org/jira/browse/HIVE-7421
Project: Hive
Issue Type: Bug
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: TestWithORC.zip, fail_47.sql, fail_62.sql, fail_932.sql
One of several found by Raj Bains.
M/R or Tez.
{code}
set hive.vectorized.execution.enabled=true;
{code}
Seems very similar to https://issues.apache.org/jira/browse/HIVE-6649
Query:
{code}
SELECT FLOOR((7 + DATEDIFF(`Staples`.`order_date_`,
CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING), '-01-01 00:00:00'))
+pmod(8 + pmod(datediff(CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING),
'-01-01 00:00:00'), '1995-01-01'), 7) - 2, 7) ) / 7) AS `wk_order_date_ok`,
SUM(`Staples`.`sales_total`) AS `sum_sales_total_ok` FROM
`default`.`testv1_Staples` `Staples` GROUP BY FLOOR((7 +
DATEDIFF(`Staples`.`order_date_`, CONCAT(CAST(YEAR(`Staples`.`order_date_`)
AS STRING), '-01-01 00:00:00')) +pmod(8 +
pmod(datediff(CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING), '-01-01
00:00:00'), '1995-01-01'), 7) - 2, 7) ) / 7) ;
{code}
Stack trace:
{code}
Caused by: java.lang.NullPointerException
at java.lang.System.arraycopy(Native Method)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setConcat(BytesColumnVector.java:190)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.StringConcatColScalar.evaluate(StringConcatColScalar.java:78)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFDateDiffColCol.evaluate(VectorUDFDateDiffColCol.java:59)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.LongScalarAddLongColumn.evaluate(LongScalarAddLongColumn.java:65)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.LongColAddLongColumn.evaluate(LongColAddLongColumn.java:52)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.LongColDivideLongScalar.evaluate(LongColDivideLongScalar.java:52)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.FuncFloorDoubleToLong.evaluate(FuncFloorDoubleToLong.java:47)
at
org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapperBatch.evaluateBatch(VectorHashKeyWrapperBatch.java:147)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeHashAggregate.processBatch(VectorGroupByOperator.java:289)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.processOp(VectorGroupByOperator.java:711)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:139)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7421) Null pointer exception involving ql.exec.vector.expressions.StringConcatColScalar.evaluate
[
https://issues.apache.org/jira/browse/HIVE-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067803#comment-14067803
]
Matt McCline commented on HIVE-7421:
Non-vectorized query produces output.
{code}
1-URGENT
2-HIGH
3-MEDIUM
4-NOT SPECIFIED
5-LOW
{code}
Null pointer exception involving
ql.exec.vector.expressions.StringConcatColScalar.evaluate
--
Key: HIVE-7421
URL: https://issues.apache.org/jira/browse/HIVE-7421
Project: Hive
Issue Type: Bug
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: TestWithORC.zip, fail_47.sql, fail_62.sql, fail_932.sql
One of several found by Raj Bains.
M/R or Tez.
{code}
set hive.vectorized.execution.enabled=true;
{code}
Seems very similar to https://issues.apache.org/jira/browse/HIVE-6649
Query:
{code}
SELECT FLOOR((7 + DATEDIFF(`Staples`.`order_date_`,
CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING), '-01-01 00:00:00'))
+pmod(8 + pmod(datediff(CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING),
'-01-01 00:00:00'), '1995-01-01'), 7) - 2, 7) ) / 7) AS `wk_order_date_ok`,
SUM(`Staples`.`sales_total`) AS `sum_sales_total_ok` FROM
`default`.`testv1_Staples` `Staples` GROUP BY FLOOR((7 +
DATEDIFF(`Staples`.`order_date_`, CONCAT(CAST(YEAR(`Staples`.`order_date_`)
AS STRING), '-01-01 00:00:00')) +pmod(8 +
pmod(datediff(CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING), '-01-01
00:00:00'), '1995-01-01'), 7) - 2, 7) ) / 7) ;
{code}
Stack trace:
{code}
Caused by: java.lang.NullPointerException
at java.lang.System.arraycopy(Native Method)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setConcat(BytesColumnVector.java:190)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.StringConcatColScalar.evaluate(StringConcatColScalar.java:78)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFDateDiffColCol.evaluate(VectorUDFDateDiffColCol.java:59)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.LongScalarAddLongColumn.evaluate(LongScalarAddLongColumn.java:65)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.LongColAddLongColumn.evaluate(LongColAddLongColumn.java:52)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.LongColDivideLongScalar.evaluate(LongColDivideLongScalar.java:52)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.FuncFloorDoubleToLong.evaluate(FuncFloorDoubleToLong.java:47)
at
org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapperBatch.evaluateBatch(VectorHashKeyWrapperBatch.java:147)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeHashAggregate.processBatch(VectorGroupByOperator.java:289)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.processOp(VectorGroupByOperator.java:711)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:139)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7422) Array out of bounds exception involving ql.exec.vector.expressions.aggregates.gen.VectorUDAFAvgDouble
[
https://issues.apache.org/jira/browse/HIVE-7422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067806#comment-14067806
]
Matt McCline commented on HIVE-7422:
Casts of Long and Double constants were being evaluated with functions like
CastLongToDouble which doesn't work in some vector cases.
Instead, detect the constant and fold it into a constant vector so it will work.
Array out of bounds exception involving
ql.exec.vector.expressions.aggregates.gen.VectorUDAFAvgDouble
-
Key: HIVE-7422
URL: https://issues.apache.org/jira/browse/HIVE-7422
Project: Hive
Issue Type: Bug
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: HIVE-7422.1.patch, HIVE-7422.2.patch, TestWithORC.zip,
fail_119.sql
One of several found by Raj Bains.
M/R or Tez.
{code}
set hive.vectorized.execution.enabled=true;
{code}
Query:
{code}
SELECT `Starbucks`.`product` AS `none_product_nk`, AVG(CAST(50 AS DOUBLE))
AS `avg_x_ok`, AVG(CAST(50 AS DOUBLE)) AS `avg_y_ok` FROM
`default`.`testv1_Starbucks` `Starbucks` GROUP BY `Starbucks`.`product` ;
{code}
Stack trace:
{code}
Caused by: java.lang.ArrayIndexOutOfBoundsException: 50
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.CastLongToDouble.evaluate(CastLongToDouble.java:50)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFAvgDouble.aggregateInputSelection(VectorUDAFAvgDouble.java:139)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processAggregators(VectorGroupByOperator.java:121)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeHashAggregate.processBatch(VectorGroupByOperator.java:295)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.processOp(VectorGroupByOperator.java:711)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:139)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7421) Null pointer exception involving ql.exec.vector.expressions.StringConcatColScalar.evaluate
[
https://issues.apache.org/jira/browse/HIVE-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067811#comment-14067811
]
Jitendra Nath Pandey commented on HIVE-7421:
bq. Is this an error in the query or in our execution of it?
It is possible that the data contains a string that cannot be parsed as a
DATE and those strings are parsed into null value. VectorUDFDateString actually
catches the IllegalArgumentException and returns null, therefore the query
should not fail.
Is the query failing in vectorized path? Or is it returning wrong result?
Null pointer exception involving
ql.exec.vector.expressions.StringConcatColScalar.evaluate
--
Key: HIVE-7421
URL: https://issues.apache.org/jira/browse/HIVE-7421
Project: Hive
Issue Type: Bug
Reporter: Matt McCline
Assignee: Matt McCline
Attachments: TestWithORC.zip, fail_47.sql, fail_62.sql, fail_932.sql
One of several found by Raj Bains.
M/R or Tez.
{code}
set hive.vectorized.execution.enabled=true;
{code}
Seems very similar to https://issues.apache.org/jira/browse/HIVE-6649
Query:
{code}
SELECT FLOOR((7 + DATEDIFF(`Staples`.`order_date_`,
CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING), '-01-01 00:00:00'))
+pmod(8 + pmod(datediff(CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING),
'-01-01 00:00:00'), '1995-01-01'), 7) - 2, 7) ) / 7) AS `wk_order_date_ok`,
SUM(`Staples`.`sales_total`) AS `sum_sales_total_ok` FROM
`default`.`testv1_Staples` `Staples` GROUP BY FLOOR((7 +
DATEDIFF(`Staples`.`order_date_`, CONCAT(CAST(YEAR(`Staples`.`order_date_`)
AS STRING), '-01-01 00:00:00')) +pmod(8 +
pmod(datediff(CONCAT(CAST(YEAR(`Staples`.`order_date_`) AS STRING), '-01-01
00:00:00'), '1995-01-01'), 7) - 2, 7) ) / 7) ;
{code}
Stack trace:
{code}
Caused by: java.lang.NullPointerException
at java.lang.System.arraycopy(Native Method)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setConcat(BytesColumnVector.java:190)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.StringConcatColScalar.evaluate(StringConcatColScalar.java:78)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFDateDiffColCol.evaluate(VectorUDFDateDiffColCol.java:59)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.LongScalarAddLongColumn.evaluate(LongScalarAddLongColumn.java:65)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.LongColAddLongColumn.evaluate(LongColAddLongColumn.java:52)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.LongColDivideLongScalar.evaluate(LongColDivideLongScalar.java:52)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.FuncFloorDoubleToLong.evaluate(FuncFloorDoubleToLong.java:47)
at
org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapperBatch.evaluateBatch(VectorHashKeyWrapperBatch.java:147)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeHashAggregate.processBatch(VectorGroupByOperator.java:289)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.processOp(VectorGroupByOperator.java:711)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:139)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
