[jira] [Commented] (HIVE-20762) NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small.
[
https://issues.apache.org/jira/browse/HIVE-20762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654744#comment-16654744
]
Hive QA commented on HIVE-20762:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
15s{color} | {color:blue} standalone-metastore/metastore-common in master has
28 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} hcatalog/server-extensions in master has 1 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
17s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
17s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 17s{color}
| {color:red} server-extensions in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} The patch metastore-common passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} hcatalog/server-extensions: The patch generated 0
new + 4 unchanged - 1 fixed = 4 total (was 5) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
17s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 5s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14536/dev-support/hive-personality.sh
|
| git revision | master / e8b87bf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14536/yetus/patch-mvninstall-hcatalog_server-extensions.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14536/yetus/patch-compile-hcatalog_server-extensions.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14536/yetus/patch-compile-hcatalog_server-extensions.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14536/yetus/patch-findbugs-hcatalog_server-extensions.txt
|
| modules | C: standalone-metastore/metastore-common hcatalog/server-extensions
U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14536/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small.
> ---
>
> Key: HIVE-20762
> URL: https://issues.apache.org/jira/browse/HIVE-20762
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
[jira] [Commented] (HIVE-20679) DDL operations on hive might create large messages for DBNotification
[ https://issues.apache.org/jira/browse/HIVE-20679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654749#comment-16654749 ] Sankar Hariappan commented on HIVE-20679: - +1, pending tests > DDL operations on hive might create large messages for DBNotification > - > > Key: HIVE-20679 > URL: https://issues.apache.org/jira/browse/HIVE-20679 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: anishek >Assignee: anishek >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20679.1.patch, HIVE-20679.2.patch, > HIVE-20679.3.patch, HIVE-20679.4.patch, HIVE-20679.5.patch, > HIVE-20679.6.patch, HIVE-20679.7.patch, a.sql, b.sql > > > Certain type of ddl operations might create large messages as part of > DBNoitification, this might lead to the rdbms throwing an error when storing > the message since its size is to large. It will also increase the footprint > of the rdbms space usage. > We should try store compressed messages to allow handling these situations. > Edit: For notification_log table the message column for all supported > databases can store messages from 2GB to 4GB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20771) LazyBinarySerDe fails on empty structs.
[
https://issues.apache.org/jira/browse/HIVE-20771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Clemens Valiente reassigned HIVE-20771:
---
> LazyBinarySerDe fails on empty structs.
> ---
>
> Key: HIVE-20771
> URL: https://issues.apache.org/jira/browse/HIVE-20771
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
>Reporter: Clemens Valiente
>Assignee: Clemens Valiente
>Priority: Minor
>
> {code:java}
> CREATE TABLE cvaliente.structtest AS
> SELECT named_struct();
> SHOW CREATE TABLE cvaliente.structtest;
> SELECT * FROM cvaliente.structtest ORDER BY rand();
> {code}
> The resulting schema is:
> {code:sql}
> CREATE TABLE `cvaliente.structtest`(
> `_c0` struct<>)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.mapred.TextInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
> LOCATION
> 'hdfs://nameservice1/user/cvaliente/cvaliente/structtest2'
> TBLPROPERTIES (
> 'COLUMN_STATS_ACCURATE'='true',
> 'numFiles'='1',
> 'numRows'='1',
> 'rawDataSize'='0',
> 'totalSize'='1',
> 'transient_lastDdlTime'='1539781607');
> {code}
> Between the MAP and REDUCE phase hive serializes to LazyBinaryStruct and when
> trying to read the same object back the {{SELECT}} query above fails:
> {code}
> 2018-10-17 14:32:02,298 [FATAL] [TezChild] |tez.ReduceRecordSource|:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":0.13508293503238622},"value":{"_col0":{}}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:338)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:259)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:169)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:164)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating
> VALUE._col0
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:82)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:329)
> ... 17 more
> Caused by: java.lang.RuntimeException: length should be positive!
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryNonPrimitive.init(LazyBinaryNonPrimitive.java:54)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.init(LazyBinaryStruct.java:95)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.uncheckedGetField(LazyBinaryStruct.java:264)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.getField(LazyBinaryStruct.java:201)
> at
> org.apache.hadoop.hive.serde2.lazybinary.objectinspector.LazyBinaryStructObjectInspector.getStructFieldData(LazyBinaryStructObjectInspector.java:64)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator._evaluate(ExprNodeColumnEvaluator.java:98)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:77)
> ... 18 more
> {code}
> this is because the LazyBinaryNonPrimitive doesn't allow for empty structs in
> https:
[jira] [Comment Edited] (HIVE-20679) DDL operations on hive might create large messages for DBNotification
[ https://issues.apache.org/jira/browse/HIVE-20679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654749#comment-16654749 ] Sankar Hariappan edited comment on HIVE-20679 at 10/18/18 7:27 AM: --- +1, pending tests Also, please take care of Findbugs issues. was (Author: sankarh): +1, pending tests > DDL operations on hive might create large messages for DBNotification > - > > Key: HIVE-20679 > URL: https://issues.apache.org/jira/browse/HIVE-20679 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: anishek >Assignee: anishek >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20679.1.patch, HIVE-20679.2.patch, > HIVE-20679.3.patch, HIVE-20679.4.patch, HIVE-20679.5.patch, > HIVE-20679.6.patch, HIVE-20679.7.patch, a.sql, b.sql > > > Certain type of ddl operations might create large messages as part of > DBNoitification, this might lead to the rdbms throwing an error when storing > the message since its size is to large. It will also increase the footprint > of the rdbms space usage. > We should try store compressed messages to allow handling these situations. > Edit: For notification_log table the message column for all supported > databases can store messages from 2GB to 4GB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20490) UDAF: Add an `approx_distinct` to Hive
[ https://issues.apache.org/jira/browse/HIVE-20490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654765#comment-16654765 ] Nishant Bangarwa commented on HIVE-20490: - +1 > UDAF: Add an `approx_distinct` to Hive > -- > > Key: HIVE-20490 > URL: https://issues.apache.org/jira/browse/HIVE-20490 > Project: Hive > Issue Type: Improvement > Components: UDF >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-20490.1.patch, HIVE-20490.2.patch > > > This gives a direct pathway to call the equivalent of Hive's default > compute_stats() UDAF which computes a HyperLogLog bitVector + approx distinct > on a column. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20762) NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small.
[
https://issues.apache.org/jira/browse/HIVE-20762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654828#comment-16654828
]
Hive QA commented on HIVE-20762:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944391/HIVE-20762.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15072 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=196)
[druidmini_masking.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_joins.q,druid_timestamptz.q]
org.apache.hive.hcatalog.listener.TestDbNotificationListener.org.apache.hive.hcatalog.listener.TestDbNotificationListener
(batchId=264)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14536/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14536/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14536/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944391 - PreCommit-HIVE-Build
> NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small.
> ---
>
> Key: HIVE-20762
> URL: https://issues.apache.org/jira/browse/HIVE-20762
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Metastore
>Affects Versions: 4.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Minor
> Labels: Metastore
> Attachments: HIVE-20762.01.patch
>
>
> NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small. It
> should be set to several hours or else the number of metastore calls would be
> too high and impact other operations.
> Make it configurable item and set it as 2Hrs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20770) Need improvement in hive for ACID properties and tables
[ https://issues.apache.org/jira/browse/HIVE-20770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654787#comment-16654787 ] Daniel Voros commented on HIVE-20770: - Hey [~pritambhandare], Thank you for your interest! Could you please forward your message to the mailing list (**[u...@hive.apache.org|mailto:u...@hive.apache.org])? Jira is supposed to be used for reporting bugs and we use the mailing list for general questions. You'll also reach a broader audience there. Thanks, Daniel > Need improvement in hive for ACID properties and tables > --- > > Key: HIVE-20770 > URL: https://issues.apache.org/jira/browse/HIVE-20770 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: pritam >Assignee: Daniel Voros >Priority: Major > > Hello Team, > In current version of Apache Hive if we set ACID properties it can not be > revert as well as > Apache Spark does not support Hive ACID table. If it is possible to revert > ACID properties from hive table and read-write Hive table from spark-scala > please let me know. > If there is no provision for above conditions it is important to add features > and improvement in next Apache Hive version. It will be very helpful for all > in distributed batch processing. > I am eager to hear from you. Thank you all in advance for such a great batch > processing tool. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20679) DDL operations on hive might create large messages for DBNotification
[ https://issues.apache.org/jira/browse/HIVE-20679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-20679: --- Attachment: (was: HIVE-20679.7.patch) > DDL operations on hive might create large messages for DBNotification > - > > Key: HIVE-20679 > URL: https://issues.apache.org/jira/browse/HIVE-20679 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: anishek >Assignee: anishek >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20679.1.patch, HIVE-20679.2.patch, > HIVE-20679.3.patch, HIVE-20679.4.patch, HIVE-20679.5.patch, > HIVE-20679.6.patch, a.sql, b.sql > > > Certain type of ddl operations might create large messages as part of > DBNoitification, this might lead to the rdbms throwing an error when storing > the message since its size is to large. It will also increase the footprint > of the rdbms space usage. > We should try store compressed messages to allow handling these situations. > Edit: For notification_log table the message column for all supported > databases can store messages from 2GB to 4GB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20771) LazyBinarySerDe fails on empty structs.
[
https://issues.apache.org/jira/browse/HIVE-20771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Clemens Valiente updated HIVE-20771:
Affects Version/s: 3.1.0
Attachment: HIVE-20771.patch
Status: Patch Available (was: Open)
> LazyBinarySerDe fails on empty structs.
> ---
>
> Key: HIVE-20771
> URL: https://issues.apache.org/jira/browse/HIVE-20771
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
>Affects Versions: 3.1.0
>Reporter: Clemens Valiente
>Assignee: Clemens Valiente
>Priority: Minor
> Attachments: HIVE-20771.patch
>
>
> {code:java}
> CREATE TABLE cvaliente.structtest AS
> SELECT named_struct();
> SHOW CREATE TABLE cvaliente.structtest;
> SELECT * FROM cvaliente.structtest ORDER BY rand();
> {code}
> The resulting schema is:
> {code:sql}
> CREATE TABLE `cvaliente.structtest`(
> `_c0` struct<>)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.mapred.TextInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
> LOCATION
> 'hdfs://nameservice1/user/cvaliente/cvaliente/structtest2'
> TBLPROPERTIES (
> 'COLUMN_STATS_ACCURATE'='true',
> 'numFiles'='1',
> 'numRows'='1',
> 'rawDataSize'='0',
> 'totalSize'='1',
> 'transient_lastDdlTime'='1539781607');
> {code}
> Between the MAP and REDUCE phase hive serializes to LazyBinaryStruct and when
> trying to read the same object back the {{SELECT}} query above fails:
> {code}
> 2018-10-17 14:32:02,298 [FATAL] [TezChild] |tez.ReduceRecordSource|:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":0.13508293503238622},"value":{"_col0":{}}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:338)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:259)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:169)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:164)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating
> VALUE._col0
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:82)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:329)
> ... 17 more
> Caused by: java.lang.RuntimeException: length should be positive!
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryNonPrimitive.init(LazyBinaryNonPrimitive.java:54)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.init(LazyBinaryStruct.java:95)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.uncheckedGetField(LazyBinaryStruct.java:264)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.getField(LazyBinaryStruct.java:201)
> at
> org.apache.hadoop.hive.serde2.lazybinary.objectinspector.LazyBinaryStructObjectInspector.getStructFieldData(LazyBinaryStructObjectInspector.java:64)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator._evaluate(ExprNodeColumnEvaluator.java:98)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
> at
> org.apache.ha
[jira] [Updated] (HIVE-20771) LazyBinarySerDe fails on empty structs.
[
https://issues.apache.org/jira/browse/HIVE-20771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Clemens Valiente updated HIVE-20771:
Affects Version/s: 1.2.2
2.3.2
> LazyBinarySerDe fails on empty structs.
> ---
>
> Key: HIVE-20771
> URL: https://issues.apache.org/jira/browse/HIVE-20771
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
>Affects Versions: 1.2.2, 2.3.2, 3.1.0
>Reporter: Clemens Valiente
>Assignee: Clemens Valiente
>Priority: Minor
> Attachments: HIVE-20771.patch
>
>
> {code:java}
> CREATE TABLE cvaliente.structtest AS
> SELECT named_struct();
> SHOW CREATE TABLE cvaliente.structtest;
> SELECT * FROM cvaliente.structtest ORDER BY rand();
> {code}
> The resulting schema is:
> {code:sql}
> CREATE TABLE `cvaliente.structtest`(
> `_c0` struct<>)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.mapred.TextInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
> LOCATION
> 'hdfs://nameservice1/user/cvaliente/cvaliente/structtest2'
> TBLPROPERTIES (
> 'COLUMN_STATS_ACCURATE'='true',
> 'numFiles'='1',
> 'numRows'='1',
> 'rawDataSize'='0',
> 'totalSize'='1',
> 'transient_lastDdlTime'='1539781607');
> {code}
> Between the MAP and REDUCE phase hive serializes to LazyBinaryStruct and when
> trying to read the same object back the {{SELECT}} query above fails:
> {code}
> 2018-10-17 14:32:02,298 [FATAL] [TezChild] |tez.ReduceRecordSource|:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":0.13508293503238622},"value":{"_col0":{}}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:338)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:259)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:169)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:164)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating
> VALUE._col0
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:82)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:329)
> ... 17 more
> Caused by: java.lang.RuntimeException: length should be positive!
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryNonPrimitive.init(LazyBinaryNonPrimitive.java:54)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.init(LazyBinaryStruct.java:95)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.uncheckedGetField(LazyBinaryStruct.java:264)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.getField(LazyBinaryStruct.java:201)
> at
> org.apache.hadoop.hive.serde2.lazybinary.objectinspector.LazyBinaryStructObjectInspector.getStructFieldData(LazyBinaryStructObjectInspector.java:64)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator._evaluate(ExprNodeColumnEvaluator.java:98)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(Select
[jira] [Updated] (HIVE-20679) DDL operations on hive might create large messages for DBNotification
[ https://issues.apache.org/jira/browse/HIVE-20679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-20679: --- Attachment: HIVE-20679.8.patch > DDL operations on hive might create large messages for DBNotification > - > > Key: HIVE-20679 > URL: https://issues.apache.org/jira/browse/HIVE-20679 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: anishek >Assignee: anishek >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20679.1.patch, HIVE-20679.2.patch, > HIVE-20679.3.patch, HIVE-20679.4.patch, HIVE-20679.5.patch, > HIVE-20679.6.patch, HIVE-20679.8.patch, a.sql, b.sql > > > Certain type of ddl operations might create large messages as part of > DBNoitification, this might lead to the rdbms throwing an error when storing > the message since its size is to large. It will also increase the footprint > of the rdbms space usage. > We should try store compressed messages to allow handling these situations. > Edit: For notification_log table the message column for all supported > databases can store messages from 2GB to 4GB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20771) LazyBinarySerDe fails on empty structs.
[
https://issues.apache.org/jira/browse/HIVE-20771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-20771:
--
Labels: pull-request-available (was: )
> LazyBinarySerDe fails on empty structs.
> ---
>
> Key: HIVE-20771
> URL: https://issues.apache.org/jira/browse/HIVE-20771
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
>Affects Versions: 1.2.2, 2.3.2, 3.1.0
>Reporter: Clemens Valiente
>Assignee: Clemens Valiente
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-20771.patch
>
>
> {code:java}
> CREATE TABLE cvaliente.structtest AS
> SELECT named_struct();
> SHOW CREATE TABLE cvaliente.structtest;
> SELECT * FROM cvaliente.structtest ORDER BY rand();
> {code}
> The resulting schema is:
> {code:sql}
> CREATE TABLE `cvaliente.structtest`(
> `_c0` struct<>)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.mapred.TextInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
> LOCATION
> 'hdfs://nameservice1/user/cvaliente/cvaliente/structtest2'
> TBLPROPERTIES (
> 'COLUMN_STATS_ACCURATE'='true',
> 'numFiles'='1',
> 'numRows'='1',
> 'rawDataSize'='0',
> 'totalSize'='1',
> 'transient_lastDdlTime'='1539781607');
> {code}
> Between the MAP and REDUCE phase hive serializes to LazyBinaryStruct and when
> trying to read the same object back the {{SELECT}} query above fails:
> {code}
> 2018-10-17 14:32:02,298 [FATAL] [TezChild] |tez.ReduceRecordSource|:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":0.13508293503238622},"value":{"_col0":{}}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:338)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:259)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:169)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:164)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating
> VALUE._col0
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:82)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:329)
> ... 17 more
> Caused by: java.lang.RuntimeException: length should be positive!
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryNonPrimitive.init(LazyBinaryNonPrimitive.java:54)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.init(LazyBinaryStruct.java:95)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.uncheckedGetField(LazyBinaryStruct.java:264)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.getField(LazyBinaryStruct.java:201)
> at
> org.apache.hadoop.hive.serde2.lazybinary.objectinspector.LazyBinaryStructObjectInspector.getStructFieldData(LazyBinaryStructObjectInspector.java:64)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator._evaluate(ExprNodeColumnEvaluator.java:98)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> at
> org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
> at
> org.apache.hadoop.hive.ql.exec.Sel
[jira] [Commented] (HIVE-20771) LazyBinarySerDe fails on empty structs.
[
https://issues.apache.org/jira/browse/HIVE-20771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654834#comment-16654834
]
ASF GitHub Bot commented on HIVE-20771:
---
GitHub user cvaliente opened a pull request:
https://github.com/apache/hive/pull/450
[HIVE-20771] LazyBinarySerDe fails on empty structs.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cvaliente/hive HIVE-20771
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/hive/pull/450.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #450
commit c4b839f610e4a57c497cfc108823d8da1b466fa7
Author: Clemens Valiente
Date: 2018-10-18T08:08:08Z
HIVE-20771 enable LazyBinarySerDe to read fields with empty structs
> LazyBinarySerDe fails on empty structs.
> ---
>
> Key: HIVE-20771
> URL: https://issues.apache.org/jira/browse/HIVE-20771
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
>Affects Versions: 1.2.2, 2.3.2, 3.1.0
>Reporter: Clemens Valiente
>Assignee: Clemens Valiente
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-20771.patch
>
>
> {code:java}
> CREATE TABLE cvaliente.structtest AS
> SELECT named_struct();
> SHOW CREATE TABLE cvaliente.structtest;
> SELECT * FROM cvaliente.structtest ORDER BY rand();
> {code}
> The resulting schema is:
> {code:sql}
> CREATE TABLE `cvaliente.structtest`(
> `_c0` struct<>)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.mapred.TextInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
> LOCATION
> 'hdfs://nameservice1/user/cvaliente/cvaliente/structtest2'
> TBLPROPERTIES (
> 'COLUMN_STATS_ACCURATE'='true',
> 'numFiles'='1',
> 'numRows'='1',
> 'rawDataSize'='0',
> 'totalSize'='1',
> 'transient_lastDdlTime'='1539781607');
> {code}
> Between the MAP and REDUCE phase hive serializes to LazyBinaryStruct and when
> trying to read the same object back the {{SELECT}} query above fails:
> {code}
> 2018-10-17 14:32:02,298 [FATAL] [TezChild] |tez.ReduceRecordSource|:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":0.13508293503238622},"value":{"_col0":{}}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:338)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:259)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:169)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:164)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating
> VALUE._col0
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:82)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:329)
> ... 17 more
> Caused by: java.lang.RuntimeException: length should be positive!
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryNonPrimitive.init(LazyBinaryNonPrimitive.java:54)
> at
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.init(LazyBinaryStruc
[jira] [Updated] (HIVE-20549) Allow user set query tag, and kill query with tag
[
https://issues.apache.org/jira/browse/HIVE-20549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-20549:
--
Attachment: HIVE-20549.6.patch
> Allow user set query tag, and kill query with tag
> -
>
> Key: HIVE-20549
> URL: https://issues.apache.org/jira/browse/HIVE-20549
> Project: Hive
> Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-20549.1.patch, HIVE-20549.2.patch,
> HIVE-20549.3.patch, HIVE-20549.4.patch, HIVE-20549.5.patch, HIVE-20549.6.patch
>
>
> HIVE-19924 add capacity for replication job set a query tag and kill the
> replication distcp job with the tag. Here I make it more general, user can
> set arbitrary "hive.query.tag" in sql script, and kill query with the tag.
> Hive will cancel the corresponding operation in hs2, along with Tez/MR
> application launched for the query. For example:
> {code}
> set hive.query.tag=mytag;
> select . -- long running query
> {code}
> In another session:
> {code}
> kill query 'mytag';
> {code}
> There're limitations in the implementation:
> 1. No tag duplication check. There's nothing to prevent conflicting tag for
> same user, and kill query will kill queries share the same tag. However, kill
> query will not kill queries from different user unless admin. So different
> user might share the same tag
> 2. In multiple hs2 environment, kill statement should be issued to all hs2 to
> make sure the corresponding operation is canceled. When beeline/jdbc connects
> to hs2 using regular way (zookeeper url), the session will connect to random
> hs2, which might be different than the hs2 where query run on. User can use
> HiveConnection.getAllUrls or beeline --getUrlsFromBeelineSite (HIVE-20507) to
> get a list of all hs2 instances.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20549) Allow user set query tag, and kill query with tag
[
https://issues.apache.org/jira/browse/HIVE-20549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-20549:
--
Status: Patch Available (was: Reopened)
> Allow user set query tag, and kill query with tag
> -
>
> Key: HIVE-20549
> URL: https://issues.apache.org/jira/browse/HIVE-20549
> Project: Hive
> Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-20549.1.patch, HIVE-20549.2.patch,
> HIVE-20549.3.patch, HIVE-20549.4.patch, HIVE-20549.5.patch, HIVE-20549.6.patch
>
>
> HIVE-19924 add capacity for replication job set a query tag and kill the
> replication distcp job with the tag. Here I make it more general, user can
> set arbitrary "hive.query.tag" in sql script, and kill query with the tag.
> Hive will cancel the corresponding operation in hs2, along with Tez/MR
> application launched for the query. For example:
> {code}
> set hive.query.tag=mytag;
> select . -- long running query
> {code}
> In another session:
> {code}
> kill query 'mytag';
> {code}
> There're limitations in the implementation:
> 1. No tag duplication check. There's nothing to prevent conflicting tag for
> same user, and kill query will kill queries share the same tag. However, kill
> query will not kill queries from different user unless admin. So different
> user might share the same tag
> 2. In multiple hs2 environment, kill statement should be issued to all hs2 to
> make sure the corresponding operation is canceled. When beeline/jdbc connects
> to hs2 using regular way (zookeeper url), the session will connect to random
> hs2, which might be different than the hs2 where query run on. User can use
> HiveConnection.getAllUrls or beeline --getUrlsFromBeelineSite (HIVE-20507) to
> get a list of all hs2 instances.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18767) Some alterPartitions invocations throw 'NumberFormatException: null'
[
https://issues.apache.org/jira/browse/HIVE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654854#comment-16654854
]
Peter Vary commented on HIVE-18767:
---
[~massdos...@gmail.com]: I think HIVE-20191 changed the patch file naming
convention. This might be the issue... Could you please check?
I am very much in awe of your persistence and will do my best to help you out
getting this stuff into every branch, so please prevail :D
> Some alterPartitions invocations throw 'NumberFormatException: null'
>
>
> Key: HIVE-18767
> URL: https://issues.apache.org/jira/browse/HIVE-18767
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.3.3, 3.1.0, 4.0.0, 3.2.0
>Reporter: Yuming Wang
>Assignee: Mass Dosage
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.4
>
> Attachments: HIVE-18767-branch-2.3.patch, HIVE-18767-branch-2.patch,
> HIVE-18767-branch-3.1.patch, HIVE-18767-branch-3.patch, HIVE-18767.1.patch,
> HIVE-18767.2-branch-2.3.patch, HIVE-18767.2-branch-2.patch,
> HIVE-18767.2-branch-3.1.patch, HIVE-18767.2.patch,
> HIVE-18767.3-branch-3.1.patch, HIVE-18767.3.patch,
> HIVE-18767.4-branch-3.1.patch, HIVE-18767.4.patch, HIVE-18767.5.patch,
> HIVE-18767.6.patch
>
>
> Error messages:
> {noformat}
> [info] Cause: java.lang.NumberFormatException: null
> [info] at java.lang.Long.parseLong(Long.java:552)
> [info] at java.lang.Long.parseLong(Long.java:631)
> [info] at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.isFastStatsSame(MetaStoreUtils.java:315)
> [info] at
> org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartitions(HiveAlterHandler.java:605)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3837)
> [info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [info] at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [info] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [info] at java.lang.reflect.Method.invoke(Method.java:498)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> [info] at
> com.sun.proxy.$Proxy23.alter_partitions_with_environment_context(Unknown
> Source)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_partitions(HiveMetaStoreClient.java:1527)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18767) Some alterPartitions invocations throw 'NumberFormatException: null'
[
https://issues.apache.org/jira/browse/HIVE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mass Dosage updated HIVE-18767:
---
Target Version/s: 3.1.0, 2.3.3, 4.0.0, 3.2.0 (was: 2.3.3, 3.1.0, 4.0.0,
3.2.0)
Status: In Progress (was: Patch Available)
> Some alterPartitions invocations throw 'NumberFormatException: null'
>
>
> Key: HIVE-18767
> URL: https://issues.apache.org/jira/browse/HIVE-18767
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.0, 2.3.3, 4.0.0, 3.2.0
>Reporter: Yuming Wang
>Assignee: Mass Dosage
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.4
>
> Attachments: HIVE-18767-branch-2.3.patch, HIVE-18767-branch-2.patch,
> HIVE-18767-branch-3.1.patch, HIVE-18767-branch-3.patch, HIVE-18767.1.patch,
> HIVE-18767.2-branch-2.3.patch, HIVE-18767.2-branch-2.patch,
> HIVE-18767.2-branch-3.1.patch, HIVE-18767.2.patch,
> HIVE-18767.3-branch-3.1.patch, HIVE-18767.3.patch,
> HIVE-18767.4-branch-3.1.patch, HIVE-18767.4.branch-3.1.patch,
> HIVE-18767.4.patch, HIVE-18767.5.patch, HIVE-18767.6.patch
>
>
> Error messages:
> {noformat}
> [info] Cause: java.lang.NumberFormatException: null
> [info] at java.lang.Long.parseLong(Long.java:552)
> [info] at java.lang.Long.parseLong(Long.java:631)
> [info] at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.isFastStatsSame(MetaStoreUtils.java:315)
> [info] at
> org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartitions(HiveAlterHandler.java:605)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3837)
> [info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [info] at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [info] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [info] at java.lang.reflect.Method.invoke(Method.java:498)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> [info] at
> com.sun.proxy.$Proxy23.alter_partitions_with_environment_context(Unknown
> Source)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_partitions(HiveMetaStoreClient.java:1527)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18767) Some alterPartitions invocations throw 'NumberFormatException: null'
[
https://issues.apache.org/jira/browse/HIVE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mass Dosage updated HIVE-18767:
---
Attachment: HIVE-18767.4.branch-3.1.patch
> Some alterPartitions invocations throw 'NumberFormatException: null'
>
>
> Key: HIVE-18767
> URL: https://issues.apache.org/jira/browse/HIVE-18767
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.3.3, 3.1.0, 4.0.0, 3.2.0
>Reporter: Yuming Wang
>Assignee: Mass Dosage
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.4
>
> Attachments: HIVE-18767-branch-2.3.patch, HIVE-18767-branch-2.patch,
> HIVE-18767-branch-3.1.patch, HIVE-18767-branch-3.patch, HIVE-18767.1.patch,
> HIVE-18767.2-branch-2.3.patch, HIVE-18767.2-branch-2.patch,
> HIVE-18767.2-branch-3.1.patch, HIVE-18767.2.patch,
> HIVE-18767.3-branch-3.1.patch, HIVE-18767.3.patch,
> HIVE-18767.4-branch-3.1.patch, HIVE-18767.4.branch-3.1.patch,
> HIVE-18767.4.patch, HIVE-18767.5.patch, HIVE-18767.6.patch
>
>
> Error messages:
> {noformat}
> [info] Cause: java.lang.NumberFormatException: null
> [info] at java.lang.Long.parseLong(Long.java:552)
> [info] at java.lang.Long.parseLong(Long.java:631)
> [info] at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.isFastStatsSame(MetaStoreUtils.java:315)
> [info] at
> org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartitions(HiveAlterHandler.java:605)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3837)
> [info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [info] at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [info] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [info] at java.lang.reflect.Method.invoke(Method.java:498)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> [info] at
> com.sun.proxy.$Proxy23.alter_partitions_with_environment_context(Unknown
> Source)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_partitions(HiveMetaStoreClient.java:1527)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18767) Some alterPartitions invocations throw 'NumberFormatException: null'
[
https://issues.apache.org/jira/browse/HIVE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mass Dosage updated HIVE-18767:
---
Target Version/s: 3.1.0, 2.3.3, 4.0.0, 3.2.0 (was: 2.3.3, 3.1.0, 4.0.0,
3.2.0)
Status: Patch Available (was: In Progress)
Added patch file as "HIVE-18767.4.branch-3.1.patch" which adheres to new patch
naming convention.
> Some alterPartitions invocations throw 'NumberFormatException: null'
>
>
> Key: HIVE-18767
> URL: https://issues.apache.org/jira/browse/HIVE-18767
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.0, 2.3.3, 4.0.0, 3.2.0
>Reporter: Yuming Wang
>Assignee: Mass Dosage
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.4
>
> Attachments: HIVE-18767-branch-2.3.patch, HIVE-18767-branch-2.patch,
> HIVE-18767-branch-3.1.patch, HIVE-18767-branch-3.patch, HIVE-18767.1.patch,
> HIVE-18767.2-branch-2.3.patch, HIVE-18767.2-branch-2.patch,
> HIVE-18767.2-branch-3.1.patch, HIVE-18767.2.patch,
> HIVE-18767.3-branch-3.1.patch, HIVE-18767.3.patch,
> HIVE-18767.4-branch-3.1.patch, HIVE-18767.4.branch-3.1.patch,
> HIVE-18767.4.patch, HIVE-18767.5.patch, HIVE-18767.6.patch
>
>
> Error messages:
> {noformat}
> [info] Cause: java.lang.NumberFormatException: null
> [info] at java.lang.Long.parseLong(Long.java:552)
> [info] at java.lang.Long.parseLong(Long.java:631)
> [info] at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.isFastStatsSame(MetaStoreUtils.java:315)
> [info] at
> org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartitions(HiveAlterHandler.java:605)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3837)
> [info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [info] at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [info] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [info] at java.lang.reflect.Method.invoke(Method.java:498)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> [info] at
> com.sun.proxy.$Proxy23.alter_partitions_with_environment_context(Unknown
> Source)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_partitions(HiveMetaStoreClient.java:1527)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654884#comment-16654884
]
Hive QA commented on HIVE-20707:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
24s{color} | {color:blue} standalone-metastore/metastore-common in master has
28 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
51s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
4s{color} | {color:blue} standalone-metastore/metastore-server in master has
182 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
48s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
30s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 40 new + 515 unchanged - 6
fixed = 555 total (was 521) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
26s{color} | {color:red} standalone-metastore/metastore-common generated 1 new
+ 28 unchanged - 0 fixed = 29 total (was 28) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} hbase-handler in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} hive-unit in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s{color} | {color:green} ql generated 0 new + 2316 unchanged - 2 fixed = 2316
total (was 2318) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
13s{color} | {color:red} standalone-metastore/metastore-server generated 4 new
+ 182 unchanged - 0 fixed = 186 total (was 182) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore_metastore-server generated 1 new
+ 49 unchanged - 0 fixed = 50 total (was 49) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-common |
| | org.apache.hadoop.hive.metastore.CheckResult$PartitionResult defines
[jira] [Updated] (HIVE-20762) NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small.
[ https://issues.apache.org/jira/browse/HIVE-20762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20762: Attachment: HIVE-20762.02.patch > NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small. > --- > > Key: HIVE-20762 > URL: https://issues.apache.org/jira/browse/HIVE-20762 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Metastore >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Minor > Labels: Metastore > Attachments: HIVE-20762.01.patch, HIVE-20762.02.patch > > > NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small. It > should be set to several hours or else the number of metastore calls would be > too high and impact other operations. > Make it configurable item and set it as 2Hrs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20762) NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small.
[ https://issues.apache.org/jira/browse/HIVE-20762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20762: Status: Open (was: Patch Available) > NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small. > --- > > Key: HIVE-20762 > URL: https://issues.apache.org/jira/browse/HIVE-20762 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Metastore >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Minor > Labels: Metastore > Attachments: HIVE-20762.01.patch, HIVE-20762.02.patch > > > NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small. It > should be set to several hours or else the number of metastore calls would be > too high and impact other operations. > Make it configurable item and set it as 2Hrs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20762) NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small.
[ https://issues.apache.org/jira/browse/HIVE-20762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20762: Status: Patch Available (was: Open) 02.patch fixed test failure. Test code only change. > NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small. > --- > > Key: HIVE-20762 > URL: https://issues.apache.org/jira/browse/HIVE-20762 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Metastore >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Minor > Labels: Metastore > Attachments: HIVE-20762.01.patch, HIVE-20762.02.patch > > > NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small. It > should be set to several hours or else the number of metastore calls would be > too high and impact other operations. > Make it configurable item and set it as 2Hrs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654900#comment-16654900
]
Hive QA commented on HIVE-20707:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944396/HIVE-20707.5.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 15106 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_discovery]
(batchId=24)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[table_nonprintable]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=168)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14537/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14537/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14537/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944396 - PreCommit-HIVE-Build
> Automatic partition management
> --
>
> Key: HIVE-20707
> URL: https://issues.apache.org/jira/browse/HIVE-20707
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch,
> HIVE-20707.2.patch, HIVE-20707.4.patch, HIVE-20707.5.patch
>
>
> In current scenario, to add partitions for external tables to metastore, MSCK
> REPAIR command has to be executed manually. To avoid this manual step,
> external tables can be specified a table property based on which a background
> metastore thread can sync partitions periodically. Tables can also be
> specified with partition retention period. Any partition whose age exceeds
> the retention period will be dropped automatically.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20617) Fix type of constants in IN expressions to have correct type
[
https://issues.apache.org/jira/browse/HIVE-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654903#comment-16654903
]
Hive QA commented on HIVE-20617:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944405/HIVE-20617.08.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14538/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14538/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14538/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12944405/HIVE-20617.08.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944405 - PreCommit-HIVE-Build
> Fix type of constants in IN expressions to have correct type
>
>
> Key: HIVE-20617
> URL: https://issues.apache.org/jira/browse/HIVE-20617
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20617.01.patch, HIVE-20617.02.patch,
> HIVE-20617.03.patch, HIVE-20617.05.patch, HIVE-20617.06.patch,
> HIVE-20617.07.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.08.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.08.patch, HIVE-20617.08.patch
>
>
> In statements like {{struct(a,b) IN (const struct('x','y'), ... )}} the
> comparision in UDFIn may fail because if a or b is of char/varchar type the
> constants will retain string type - especially after PointlookupOptimizer
> compaction.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[
https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654922#comment-16654922
]
Hive QA commented on HIVE-20765:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
59s{color} | {color:blue} standalone-metastore/metastore-server in master has
182 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14539/dev-support/hive-personality.sh
|
| git revision | master / e8b87bf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14539/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> fetch partitions for txn stats validation in get_aggr_stats with one call
> -
>
> Key: HIVE-20765
> URL: https://issues.apache.org/jira/browse/HIVE-20765
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20765.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[
https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655011#comment-16655011
]
Hive QA commented on HIVE-20765:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944417/HIVE-20765.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 15096 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats3] (batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats4] (batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_wm]
(batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_part2] (batchId=21)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_no_buckets]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_bhif]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sample10_mm]
(batchId=173)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14539/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14539/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14539/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944417 - PreCommit-HIVE-Build
> fetch partitions for txn stats validation in get_aggr_stats with one call
> -
>
> Key: HIVE-20765
> URL: https://issues.apache.org/jira/browse/HIVE-20765
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20765.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[
https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655012#comment-16655012
]
Hive QA commented on HIVE-20765:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944417/HIVE-20765.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14540/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14540/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14540/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12944417/HIVE-20765.patch was
found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944417 - PreCommit-HIVE-Build
> fetch partitions for txn stats validation in get_aggr_stats with one call
> -
>
> Key: HIVE-20765
> URL: https://issues.apache.org/jira/browse/HIVE-20765
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20765.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) Local SparkContext is shared between user sessions and should be closed only when there is no active
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655048#comment-16655048
]
Hive QA commented on HIVE-20737:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
54s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} ql: The patch generated 0 new + 6 unchanged - 1
fixed = 6 total (was 7) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14541/dev-support/hive-personality.sh
|
| git revision | master / e8b87bf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14541/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Local SparkContext is shared between user sessions and should be closed only
> when there is no active
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.10.patch,
> HIVE-20737.11.patch, HIVE-20737.12.patch, HIVE-20737.2.patch,
> HIVE-20737.5.patch, HIVE-20737.6.patch, HIVE-20737.7.patch,
> HIVE-20737.8.patch, HIVE-20737.9.patch
>
>
> 1. Local SparkContext is shared between user sessions and should be closed
> only when there is no active.
> 2. Possible race condition in SparkSession.open() in case when user queries
> run in parallel within the same session.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) Local SparkContext is shared between user sessions and should be closed only when there is no active
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655074#comment-16655074
]
Hive QA commented on HIVE-20737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944435/HIVE-20737.12.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15097 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14541/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14541/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14541/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944435 - PreCommit-HIVE-Build
> Local SparkContext is shared between user sessions and should be closed only
> when there is no active
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.10.patch,
> HIVE-20737.11.patch, HIVE-20737.12.patch, HIVE-20737.2.patch,
> HIVE-20737.5.patch, HIVE-20737.6.patch, HIVE-20737.7.patch,
> HIVE-20737.8.patch, HIVE-20737.9.patch
>
>
> 1. Local SparkContext is shared between user sessions and should be closed
> only when there is no active.
> 2. Possible race condition in SparkSession.open() in case when user queries
> run in parallel within the same session.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20767) Multiple project between join operators may affect join reordering using constraints
[
https://issues.apache.org/jira/browse/HIVE-20767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655115#comment-16655115
]
Hive QA commented on HIVE-20767:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14542/dev-support/hive-personality.sh
|
| git revision | master / e8b87bf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14542/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Multiple project between join operators may affect join reordering using
> constraints
>
>
> Key: HIVE-20767
> URL: https://issues.apache.org/jira/browse/HIVE-20767
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20767.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20742) SparkSessionManagerImpl maintenance thread only cleans up session once
[ https://issues.apache.org/jira/browse/HIVE-20742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655159#comment-16655159 ] Peter Vary commented on HIVE-20742: --- +1 [~asinkovits]: Please create follow up jiras to test the cases mentioned by [~stakiar]. Thanks, Peter > SparkSessionManagerImpl maintenance thread only cleans up session once > -- > > Key: HIVE-20742 > URL: https://issues.apache.org/jira/browse/HIVE-20742 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Antal Sinkovits >Assignee: Antal Sinkovits >Priority: Major > Attachments: HIVE-20742.1.patch, HIVE-20742.2.patch > > > If there is a reconnect at the client session, the SparkSessionManagerImpl > doesn't puts it back in the created sessions, so it will not time out the > second time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20767) Multiple project between join operators may affect join reordering using constraints
[
https://issues.apache.org/jira/browse/HIVE-20767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655176#comment-16655176
]
Hive QA commented on HIVE-20767:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/1293/HIVE-20767.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 15096 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bool_unknown]
(batchId=39)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[lineage2]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mapjoin_hint]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_rewrite_7]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_scalar]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_select]
(batchId=169)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_scalar_multi_rows]
(batchId=99)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar]
(batchId=128)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select]
(batchId=128)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query54]
(batchId=272)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query54]
(batchId=270)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14542/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14542/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14542/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 1293 - PreCommit-HIVE-Build
> Multiple project between join operators may affect join reordering using
> constraints
>
>
> Key: HIVE-20767
> URL: https://issues.apache.org/jira/browse/HIVE-20767
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20767.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20767) Multiple project between join operators may affect join reordering using constraints
[
https://issues.apache.org/jira/browse/HIVE-20767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655178#comment-16655178
]
Hive QA commented on HIVE-20767:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/1293/HIVE-20767.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14543/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14543/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14543/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/1293/HIVE-20767.patch was
found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 1293 - PreCommit-HIVE-Build
> Multiple project between join operators may affect join reordering using
> constraints
>
>
> Key: HIVE-20767
> URL: https://issues.apache.org/jira/browse/HIVE-20767
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20767.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20748) Disable materialized view rewriting when plan pattern is not allowed
[
https://issues.apache.org/jira/browse/HIVE-20748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655209#comment-16655209
]
Hive QA commented on HIVE-20748:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
48s{color} | {color:red} ql: The patch generated 9 new + 1062 unchanged - 3
fixed = 1071 total (was 1065) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
2s{color} | {color:red} ql generated 1 new + 2317 unchanged - 1 fixed = 2318
total (was 2318) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 49s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Dead store to tt in
org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelOptCachedQueryValidator.visit(TableScan)
At
HiveRelOptCachedQueryValidator.java:org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelOptCachedQueryValidator.visit(TableScan)
At HiveRelOptCachedQueryValidator.java:[line 82] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14544/dev-support/hive-personality.sh
|
| git revision | master / e8b87bf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14544/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14544/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14544/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Disable materialized view rewriting when plan pattern is not allowed
>
>
> Key: HIVE-20748
> URL: https://issues.apache.org/jira/browse/HIVE-20748
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20748.01.patch, HIVE-20748.01.patch,
> HIVE-20748.patch
>
>
> For instance, currently rewriting algorithm does not support some operators.
> Or we cannot have non-deterministic function in the MV definition. In those
> cases, we should fail either when we try to create the MV with rewriting
> enabled, or when when we enable the rewriting for a MV already created.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20748) Disable materialized view rewriting when plan pattern is not allowed
[
https://issues.apache.org/jira/browse/HIVE-20748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655248#comment-16655248
]
Hive QA commented on HIVE-20748:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/1292/HIVE-20748.01.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15100 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14544/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14544/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14544/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 1292 - PreCommit-HIVE-Build
> Disable materialized view rewriting when plan pattern is not allowed
>
>
> Key: HIVE-20748
> URL: https://issues.apache.org/jira/browse/HIVE-20748
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20748.01.patch, HIVE-20748.01.patch,
> HIVE-20748.patch
>
>
> For instance, currently rewriting algorithm does not support some operators.
> Or we cannot have non-deterministic function in the MV definition. In those
> cases, we should fail either when we try to create the MV with rewriting
> enabled, or when when we enable the rewriting for a MV already created.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20477) OptimizedSql is not shown if the expression contains INs
[ https://issues.apache.org/jira/browse/HIVE-20477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20477: Attachment: HIVE-20477.01.patch > OptimizedSql is not shown if the expression contains INs > > > Key: HIVE-20477 > URL: https://issues.apache.org/jira/browse/HIVE-20477 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20477.01.patch, HIVE-20477.01.patch, > HIVE-20477.01wip01.patch > > > This ticket should fix HiveIn to be able to unparse; currently if an IN is > unparsed - there are some exceptions because HiveIn is a special operator; > but doesn't have unparse implemented. > CALCITE-2444 is also needed to fix rel2sql to be able to process INs which > are not there represent a subquery. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20617) Fix type of constants in IN expressions to have correct type
[
https://issues.apache.org/jira/browse/HIVE-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-20617:
Attachment: HIVE-20617.08.patch
> Fix type of constants in IN expressions to have correct type
>
>
> Key: HIVE-20617
> URL: https://issues.apache.org/jira/browse/HIVE-20617
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20617.01.patch, HIVE-20617.02.patch,
> HIVE-20617.03.patch, HIVE-20617.05.patch, HIVE-20617.06.patch,
> HIVE-20617.07.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.08.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.08.patch, HIVE-20617.08.patch, HIVE-20617.08.patch
>
>
> In statements like {{struct(a,b) IN (const struct('x','y'), ... )}} the
> comparision in UDFIn may fail because if a or b is of char/varchar type the
> constants will retain string type - especially after PointlookupOptimizer
> compaction.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20395) Parallelize files move in the ql.metadata.Hive#replaceFiles
[ https://issues.apache.org/jira/browse/HIVE-20395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-20395: - Attachment: HIVE-20395.10-branch-3.patch Status: Patch Available (was: Reopened) > Parallelize files move in the ql.metadata.Hive#replaceFiles > --- > > Key: HIVE-20395 > URL: https://issues.apache.org/jira/browse/HIVE-20395 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0 >Reporter: Igor Dvorzhak >Assignee: Igor Dvorzhak >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20395.10-branch-3.patch, > HIVE-20395.10-branch-3.patch, HIVE-20395.10.patch, > HIVE-20395.11-branch-2.patch > > > Files move is not parallelized in `replaceFiles` method and could be very > slow on Cloud Object Stores. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HIVE-20395) Parallelize files move in the ql.metadata.Hive#replaceFiles
[ https://issues.apache.org/jira/browse/HIVE-20395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak reopened HIVE-20395: -- > Parallelize files move in the ql.metadata.Hive#replaceFiles > --- > > Key: HIVE-20395 > URL: https://issues.apache.org/jira/browse/HIVE-20395 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0 >Reporter: Igor Dvorzhak >Assignee: Igor Dvorzhak >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20395.10-branch-3.patch, > HIVE-20395.10-branch-3.patch, HIVE-20395.10.patch, > HIVE-20395.11-branch-2.patch > > > Files move is not parallelized in `replaceFiles` method and could be very > slow on Cloud Object Stores. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20395) Parallelize files move in the ql.metadata.Hive#replaceFiles
[ https://issues.apache.org/jira/browse/HIVE-20395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-20395: - Attachment: (was: HIVE-20395.10-branch-3.patch) > Parallelize files move in the ql.metadata.Hive#replaceFiles > --- > > Key: HIVE-20395 > URL: https://issues.apache.org/jira/browse/HIVE-20395 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0 >Reporter: Igor Dvorzhak >Assignee: Igor Dvorzhak >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20395.10-branch-3.patch, HIVE-20395.10.patch, > HIVE-20395.11-branch-2.patch > > > Files move is not parallelized in `replaceFiles` method and could be very > slow on Cloud Object Stores. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20768) Adding Tumbling Window UDF
[
https://issues.apache.org/jira/browse/HIVE-20768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655290#comment-16655290
]
Hive QA commented on HIVE-20768:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14545/dev-support/hive-personality.sh
|
| git revision | master / e8b87bf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14545/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Adding Tumbling Window UDF
> --
>
> Key: HIVE-20768
> URL: https://issues.apache.org/jira/browse/HIVE-20768
> Project: Hive
> Issue Type: New Feature
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20768.patch
>
>
> Goal is to provide a UDF that truncates a timestamp to a beginning of a
> tumbling window interval.
> {code}
> /**
> * Tumbling windows are a series of fixed-sized, non-overlapping and
> contiguous time intervals.
> * Tumbling windows are inclusive start exclusive end.
> * By default the beginning instant of fist window is Epoch 0 Thu Jan 01
> 00:00:00 1970 UTC.
> * Optionally users may provide a different origin as a timestamp arg3.
> *
> * This an example of series of window with an interval of 5 seconds and
> origin Epoch 0 Thu Jan 01 00:00:00 1970 UTC:
> *
> *
> * interval 1 interval 2interval 3
> * Jan 01 00:00:00 Jan 01 00:00:05 Jan 01 00:00:10
> * 0 -- 4 : 5 --- 9: 10 --- 14
> *
> * This UDF rounds timestamp agr1 to the beginning of window interval where
> it belongs to.
> *
> */
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20768) Adding Tumbling Window UDF
[
https://issues.apache.org/jira/browse/HIVE-20768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655364#comment-16655364
]
Hive QA commented on HIVE-20768:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944450/HIVE-20768.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15097 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[show_functions]
(batchId=79)
org.apache.hadoop.hive.ql.exec.spark.TestSparkSessionTimeout.testMultiSessionSparkSessionTimeout
(batchId=249)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14545/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14545/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14545/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944450 - PreCommit-HIVE-Build
> Adding Tumbling Window UDF
> --
>
> Key: HIVE-20768
> URL: https://issues.apache.org/jira/browse/HIVE-20768
> Project: Hive
> Issue Type: New Feature
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20768.patch
>
>
> Goal is to provide a UDF that truncates a timestamp to a beginning of a
> tumbling window interval.
> {code}
> /**
> * Tumbling windows are a series of fixed-sized, non-overlapping and
> contiguous time intervals.
> * Tumbling windows are inclusive start exclusive end.
> * By default the beginning instant of fist window is Epoch 0 Thu Jan 01
> 00:00:00 1970 UTC.
> * Optionally users may provide a different origin as a timestamp arg3.
> *
> * This an example of series of window with an interval of 5 seconds and
> origin Epoch 0 Thu Jan 01 00:00:00 1970 UTC:
> *
> *
> * interval 1 interval 2interval 3
> * Jan 01 00:00:00 Jan 01 00:00:05 Jan 01 00:00:10
> * 0 -- 4 : 5 --- 9: 10 --- 14
> *
> * This UDF rounds timestamp agr1 to the beginning of window interval where
> it belongs to.
> *
> */
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) Local SparkContext is shared between user sessions and should be closed only when there is no active
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655398#comment-16655398
]
Sahil Takiar commented on HIVE-20737:
-
{quote} As the separate JIRA we are planning to refactor SparkSession
open/submit functionality to be an atomic unit. {quote} Can you explain what
this means?
> Local SparkContext is shared between user sessions and should be closed only
> when there is no active
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.10.patch,
> HIVE-20737.11.patch, HIVE-20737.12.patch, HIVE-20737.2.patch,
> HIVE-20737.5.patch, HIVE-20737.6.patch, HIVE-20737.7.patch,
> HIVE-20737.8.patch, HIVE-20737.9.patch
>
>
> 1. Local SparkContext is shared between user sessions and should be closed
> only when there is no active.
> 2. Possible race condition in SparkSession.open() in case when user queries
> run in parallel within the same session.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20752) In case of LLAP start failure add info how to find YARN logs
[
https://issues.apache.org/jira/browse/HIVE-20752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655399#comment-16655399
]
Hive QA commented on HIVE-20752:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} llap-server in master has 84 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} llap-server: The patch generated 21 new + 67 unchanged
- 30 fixed = 88 total (was 97) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} llap-server generated 0 new + 83 unchanged - 1 fixed
= 83 total (was 84) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14546/dev-support/hive-personality.sh
|
| git revision | master / e8b87bf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14546/yetus/diff-checkstyle-llap-server.txt
|
| modules | C: llap-server U: llap-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14546/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> In case of LLAP start failure add info how to find YARN logs
>
>
> Key: HIVE-20752
> URL: https://issues.apache.org/jira/browse/HIVE-20752
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20752.01.patch, HIVE-20752.02.patch,
> HIVE-20752.03.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) Local SparkContext is shared between user sessions and should be closed only when there is no active
[ https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655435#comment-16655435 ] Denys Kuzmenko commented on HIVE-20737: --- Opening of a Spark session and Job submission should be done as an atomic operation. Not to have case when we submit something having already closed session > Local SparkContext is shared between user sessions and should be closed only > when there is no active > > > Key: HIVE-20737 > URL: https://issues.apache.org/jira/browse/HIVE-20737 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20737.1.patch, HIVE-20737.10.patch, > HIVE-20737.11.patch, HIVE-20737.12.patch, HIVE-20737.2.patch, > HIVE-20737.5.patch, HIVE-20737.6.patch, HIVE-20737.7.patch, > HIVE-20737.8.patch, HIVE-20737.9.patch > > > 1. Local SparkContext is shared between user sessions and should be closed > only when there is no active. > 2. Possible race condition in SparkSession.open() in case when user queries > run in parallel within the same session. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[
https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655439#comment-16655439
]
Sahil Takiar commented on HIVE-20512:
-
A few comments:
(1) I think the logging should be done in a separate thread so that we don't
have to invoke {{logMemoryInfo()}} for each record, which can add significant
overhead to per-record processing
(2) I think we should start with a lower interval, something like 15 seconds
You could try and add a unit test that logs to a string buffers, and then parse
that string buffer in a unit test. However, I don't think its necessary.
CC: [~asinkovits]
> Improve record and memory usage logging in SparkRecordHandler
> -
>
> Key: HIVE-20512
> URL: https://issues.apache.org/jira/browse/HIVE-20512
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Attachments: HIVE-20512.1.patch
>
>
> We currently log memory usage and # of records processed in Spark tasks, but
> we should improve the methodology for how frequently we log this info.
> Currently we use the following code:
> {code:java}
> private long getNextLogThreshold(long currentThreshold) {
> // A very simple counter to keep track of number of rows processed by the
> // reducer. It dumps
> // every 1 million times, and quickly before that
> if (currentThreshold >= 100) {
> return currentThreshold + 100;
> }
> return 10 * currentThreshold;
> }
> {code}
> The issue is that after a while, the increase by 10x factor means that you
> have to process a huge # of records before this gets triggered.
> A better approach would be to log this info at a given interval. This would
> help in debugging tasks that are seemingly hung.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20737) Local SparkContext is shared between user sessions and should be closed only when there is no active
[ https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655435#comment-16655435 ] Denys Kuzmenko edited comment on HIVE-20737 at 10/18/18 3:41 PM: - Opening/getting an opened Spark session and Job submission should be done as an atomic operation. Not to have case when we submit something having already closed session was (Author: dkuzmenko): Opening of a Spark session and Job submission should be done as an atomic operation. Not to have case when we submit something having already closed session > Local SparkContext is shared between user sessions and should be closed only > when there is no active > > > Key: HIVE-20737 > URL: https://issues.apache.org/jira/browse/HIVE-20737 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20737.1.patch, HIVE-20737.10.patch, > HIVE-20737.11.patch, HIVE-20737.12.patch, HIVE-20737.2.patch, > HIVE-20737.5.patch, HIVE-20737.6.patch, HIVE-20737.7.patch, > HIVE-20737.8.patch, HIVE-20737.9.patch > > > 1. Local SparkContext is shared between user sessions and should be closed > only when there is no active. > 2. Possible race condition in SparkSession.open() in case when user queries > run in parallel within the same session. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20737) Local SparkContext is shared between user sessions and should be closed only when there is no active
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655458#comment-16655458
]
Sahil Takiar edited comment on HIVE-20737 at 10/18/18 3:49 PM:
---
{quote}Opening of a Spark session and Job submission should be done as an
atomic operation.
{quote}
Well we intentionally don't do that. HoS by design de-couples opening a Spark
session and submitting a job. There are a few reasons:
(1) In order to support static allocation in Spark, we have to open a session
before we even have a job to submit (e.g. see {{SetSparkReducerParallelism}})
(2) At some point I think we should implement HIVE-17927; the reason is that
opening a Spark session causes a Spark application to be created, which
requires resource negotiation with YARN and the spawning of the Spark driver,
which takes a non-trivial amount of time
{quote}Not to have case when we submit something having already closed session
{quote}
We could just re-open the session if we try to submit a job on a closed session.
was (Author: stakiar):
{quote} Opening of a Spark session and Job submission should be done as an
atomic operation. {quote}
Well we intentionally don't do that. HoS by design de-couples opening a Spark
session and submitting a job. There are a few reasons:
(1) In order to support static allocation in Spark, we have to open a session
before we even have a job to submit (e.g. see {{SetSparkReducerParallelism}})
(2) At some point I think we should implement HIVE-17927; the reason is that
opening a Spark session causes a Spark application to be created, which
requires resource negotiation with YARN and the spawning of the Spark driver,
which takes a non-trivial amount of time
{quote} Not to have case when we submit something having already closed session
{quote}
We could just re-open the session if we try to submit a job on a closed session.
> Local SparkContext is shared between user sessions and should be closed only
> when there is no active
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.10.patch,
> HIVE-20737.11.patch, HIVE-20737.12.patch, HIVE-20737.2.patch,
> HIVE-20737.5.patch, HIVE-20737.6.patch, HIVE-20737.7.patch,
> HIVE-20737.8.patch, HIVE-20737.9.patch
>
>
> 1. Local SparkContext is shared between user sessions and should be closed
> only when there is no active.
> 2. Possible race condition in SparkSession.open() in case when user queries
> run in parallel within the same session.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) Local SparkContext is shared between user sessions and should be closed only when there is no active
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655458#comment-16655458
]
Sahil Takiar commented on HIVE-20737:
-
{quote} Opening of a Spark session and Job submission should be done as an
atomic operation. {quote}
Well we intentionally don't do that. HoS by design de-couples opening a Spark
session and submitting a job. There are a few reasons:
(1) In order to support static allocation in Spark, we have to open a session
before we even have a job to submit (e.g. see {{SetSparkReducerParallelism}})
(2) At some point I think we should implement HIVE-17927; the reason is that
opening a Spark session causes a Spark application to be created, which
requires resource negotiation with YARN and the spawning of the Spark driver,
which takes a non-trivial amount of time
{quote} Not to have case when we submit something having already closed session
{quote}
We could just re-open the session if we try to submit a job on a closed session.
> Local SparkContext is shared between user sessions and should be closed only
> when there is no active
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.10.patch,
> HIVE-20737.11.patch, HIVE-20737.12.patch, HIVE-20737.2.patch,
> HIVE-20737.5.patch, HIVE-20737.6.patch, HIVE-20737.7.patch,
> HIVE-20737.8.patch, HIVE-20737.9.patch
>
>
> 1. Local SparkContext is shared between user sessions and should be closed
> only when there is no active.
> 2. Possible race condition in SparkSession.open() in case when user queries
> run in parallel within the same session.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) Local SparkContext is shared between user sessions and should be closed only when there is no active
[ https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655464#comment-16655464 ] Sahil Takiar commented on HIVE-20737: - Making getting an opened Spark Session + submitting a Spark job an atomic operation makes sense to me. > Local SparkContext is shared between user sessions and should be closed only > when there is no active > > > Key: HIVE-20737 > URL: https://issues.apache.org/jira/browse/HIVE-20737 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-20737.1.patch, HIVE-20737.10.patch, > HIVE-20737.11.patch, HIVE-20737.12.patch, HIVE-20737.2.patch, > HIVE-20737.5.patch, HIVE-20737.6.patch, HIVE-20737.7.patch, > HIVE-20737.8.patch, HIVE-20737.9.patch > > > 1. Local SparkContext is shared between user sessions and should be closed > only when there is no active. > 2. Possible race condition in SparkSession.open() in case when user queries > run in parallel within the same session. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20488) SparkSubmitSparkClient#launchDriver should parse exceptions, not just errors
[
https://issues.apache.org/jira/browse/HIVE-20488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-20488:
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
> SparkSubmitSparkClient#launchDriver should parse exceptions, not just errors
>
>
> Key: HIVE-20488
> URL: https://issues.apache.org/jira/browse/HIVE-20488
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20488.1.patch
>
>
> In {{SparkSubmitSparkClient#launchDriver}} we parse the stdout / stderr of
> {{bin/spark-submit}} for strings that contain "Error", but we should also
> look for "Exception".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20488) SparkSubmitSparkClient#launchDriver should parse exceptions, not just errors
[
https://issues.apache.org/jira/browse/HIVE-20488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655467#comment-16655467
]
Sahil Takiar commented on HIVE-20488:
-
Pushed to master.
> SparkSubmitSparkClient#launchDriver should parse exceptions, not just errors
>
>
> Key: HIVE-20488
> URL: https://issues.apache.org/jira/browse/HIVE-20488
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20488.1.patch
>
>
> In {{SparkSubmitSparkClient#launchDriver}} we parse the stdout / stderr of
> {{bin/spark-submit}} for strings that contain "Error", but we should also
> look for "Exception".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20752) In case of LLAP start failure add info how to find YARN logs
[
https://issues.apache.org/jira/browse/HIVE-20752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655498#comment-16655498
]
Hive QA commented on HIVE-20752:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944451/HIVE-20752.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15101 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14546/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14546/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14546/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944451 - PreCommit-HIVE-Build
> In case of LLAP start failure add info how to find YARN logs
>
>
> Key: HIVE-20752
> URL: https://issues.apache.org/jira/browse/HIVE-20752
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20752.01.patch, HIVE-20752.02.patch,
> HIVE-20752.03.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20737) Local SparkContext is shared between user sessions and should be closed only when there is no active
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655501#comment-16655501
]
Denys Kuzmenko commented on HIVE-20737:
---
Ok, then we will do this in scope of use case where we call {{close()}} while
{{open()}} is being run.
Should I create a JIRA, or you already have one for that? Actually, I would
need more details on this.
> Local SparkContext is shared between user sessions and should be closed only
> when there is no active
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.10.patch,
> HIVE-20737.11.patch, HIVE-20737.12.patch, HIVE-20737.2.patch,
> HIVE-20737.5.patch, HIVE-20737.6.patch, HIVE-20737.7.patch,
> HIVE-20737.8.patch, HIVE-20737.9.patch
>
>
> 1. Local SparkContext is shared between user sessions and should be closed
> only when there is no active.
> 2. Possible race condition in SparkSession.open() in case when user queries
> run in parallel within the same session.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20737) Local SparkContext is shared between user sessions and should be closed only when there is no active
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655501#comment-16655501
]
Denys Kuzmenko edited comment on HIVE-20737 at 10/18/18 4:09 PM:
-
Ok, then we will do this in scope of use case where we call {{close()}} while
{{open()}} is being run. It would require significant refactor.
Should I create a JIRA, or you already have one for that? Actually, I would
need more details on this.
was (Author: dkuzmenko):
Ok, then we will do this in scope of use case where we call {{close()}} while
{{open()}} is being run.
Should I create a JIRA, or you already have one for that? Actually, I would
need more details on this.
> Local SparkContext is shared between user sessions and should be closed only
> when there is no active
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.10.patch,
> HIVE-20737.11.patch, HIVE-20737.12.patch, HIVE-20737.2.patch,
> HIVE-20737.5.patch, HIVE-20737.6.patch, HIVE-20737.7.patch,
> HIVE-20737.8.patch, HIVE-20737.9.patch
>
>
> 1. Local SparkContext is shared between user sessions and should be closed
> only when there is no active.
> 2. Possible race condition in SparkSession.open() in case when user queries
> run in parallel within the same session.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20737) Local SparkContext is shared between user sessions and should be closed only when there is no active
[
https://issues.apache.org/jira/browse/HIVE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655501#comment-16655501
]
Denys Kuzmenko edited comment on HIVE-20737 at 10/18/18 4:09 PM:
-
Ok, we will tackle this in scope of use case where we call {{close()}} while
{{open()}} is being run. It would require significant refactor.
Should I create a JIRA, or you already have one for that? Actually, I would
need more details on this.
was (Author: dkuzmenko):
Ok, then we will do this in scope of use case where we call {{close()}} while
{{open()}} is being run. It would require significant refactor.
Should I create a JIRA, or you already have one for that? Actually, I would
need more details on this.
> Local SparkContext is shared between user sessions and should be closed only
> when there is no active
>
>
> Key: HIVE-20737
> URL: https://issues.apache.org/jira/browse/HIVE-20737
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-20737.1.patch, HIVE-20737.10.patch,
> HIVE-20737.11.patch, HIVE-20737.12.patch, HIVE-20737.2.patch,
> HIVE-20737.5.patch, HIVE-20737.6.patch, HIVE-20737.7.patch,
> HIVE-20737.8.patch, HIVE-20737.9.patch
>
>
> 1. Local SparkContext is shared between user sessions and should be closed
> only when there is no active.
> 2. Possible race condition in SparkSession.open() in case when user queries
> run in parallel within the same session.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20490) UDAF: Add an `approx_distinct` to Hive
[
https://issues.apache.org/jira/browse/HIVE-20490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655549#comment-16655549
]
Hive QA commented on HIVE-20490:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 8 new + 72 unchanged - 0 fixed
= 80 total (was 72) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 15 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
59s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14547/dev-support/hive-personality.sh
|
| git revision | master / d67d52c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14547/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14547/yetus/whitespace-eol.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14547/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: itests ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14547/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> UDAF: Add an `approx_distinct` to Hive
> --
>
> Key: HIVE-20490
> URL: https://issues.apache.org/jira/browse/HIVE-20490
> Project: Hive
> Issue Type: Improvement
> Components: UDF
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-20490.1.patch, HIVE-20490.2.patch
>
>
> This gives a direct pathway to call the equivalent of Hive's default
> compute_stats() UDAF which computes a HyperLogLog bitVector + approx distinct
> on a column.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20735) Address some of the review comments plus Kerberos support
[ https://issues.apache.org/jira/browse/HIVE-20735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655610#comment-16655610 ] slim bouguerra commented on HIVE-20735: --- pushed to master https://github.com/apache/hive/tree/dcaeeb4722553f3256c9d3f5282cc82cae7bc9b4 > Address some of the review comments plus Kerberos support > - > > Key: HIVE-20735 > URL: https://issues.apache.org/jira/browse/HIVE-20735 > Project: Hive > Issue Type: Sub-task > Components: kafka integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20735.patch > > > As part of the review comments we agreed to: > # remove start and end offsets columns > # remove the best effort mode > # make the 2pc as default protocol for EOS > Also this patch will include an additional enhancement to add kerberos > support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20490) UDAF: Add an `approx_distinct` to Hive
[
https://issues.apache.org/jira/browse/HIVE-20490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655612#comment-16655612
]
Hive QA commented on HIVE-20490:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944467/HIVE-20490.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15098 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestGetPartitionsUsingProjectionAndFilterSpecs.testGetPartitionsUsingValuesWithJDO
(batchId=223)
org.apache.hadoop.hive.ql.exec.tez.TestDynamicPartitionPruner.testSingleSourceMultipleFiltersOrdering1
(batchId=313)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14547/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14547/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14547/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944467 - PreCommit-HIVE-Build
> UDAF: Add an `approx_distinct` to Hive
> --
>
> Key: HIVE-20490
> URL: https://issues.apache.org/jira/browse/HIVE-20490
> Project: Hive
> Issue Type: Improvement
> Components: UDF
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-20490.1.patch, HIVE-20490.2.patch
>
>
> This gives a direct pathway to call the equivalent of Hive's default
> compute_stats() UDAF which computes a HyperLogLog bitVector + approx distinct
> on a column.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655624#comment-16655624 ] Vineet Garg commented on HIVE-20703: bq. long executorMem = 40L can be replaced by OperatorDesc.getMaxMemoryAvailable() which give max memory available per container (in case of tez) or executor in case of llap. During compilation getMaxMemoryAvailable() returns zero, perhaps this is set during execution? > Put dynamic sort partition optimization under cost based decision > - > > Key: HIVE-20703 > URL: https://issues.apache.org/jira/browse/HIVE-20703 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20703.1.patch, HIVE-20703.2.patch, > HIVE-20703.3.patch, HIVE-20703.4.patch, HIVE-20703.5.patch, > HIVE-20703.6.patch, HIVE-20703.7.patch, HIVE-20703.8.patch, HIVE-20703.9.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20762) NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small.
[ https://issues.apache.org/jira/browse/HIVE-20762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655631#comment-16655631 ] Thejas M Nair commented on HIVE-20762: -- +1 > NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small. > --- > > Key: HIVE-20762 > URL: https://issues.apache.org/jira/browse/HIVE-20762 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Metastore >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Minor > Labels: Metastore > Attachments: HIVE-20762.01.patch, HIVE-20762.02.patch > > > NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small. It > should be set to several hours or else the number of metastore calls would be > too high and impact other operations. > Make it configurable item and set it as 2Hrs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655644#comment-16655644 ] Vineet Garg commented on HIVE-20703: bq. Can hive.tez.container.size be used for now? Perhaps adding an explicit default in this case if it's not set. Is hive.tez.container.size in MB? I guess we can use it until Prasanth's patch is in. > Put dynamic sort partition optimization under cost based decision > - > > Key: HIVE-20703 > URL: https://issues.apache.org/jira/browse/HIVE-20703 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20703.1.patch, HIVE-20703.2.patch, > HIVE-20703.3.patch, HIVE-20703.4.patch, HIVE-20703.5.patch, > HIVE-20703.6.patch, HIVE-20703.7.patch, HIVE-20703.8.patch, HIVE-20703.9.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20768) Adding Tumbling Window UDF
[
https://issues.apache.org/jira/browse/HIVE-20768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-20768:
--
Attachment: HIVE-20768.patch
> Adding Tumbling Window UDF
> --
>
> Key: HIVE-20768
> URL: https://issues.apache.org/jira/browse/HIVE-20768
> Project: Hive
> Issue Type: New Feature
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20768.patch, HIVE-20768.patch
>
>
> Goal is to provide a UDF that truncates a timestamp to a beginning of a
> tumbling window interval.
> {code}
> /**
> * Tumbling windows are a series of fixed-sized, non-overlapping and
> contiguous time intervals.
> * Tumbling windows are inclusive start exclusive end.
> * By default the beginning instant of fist window is Epoch 0 Thu Jan 01
> 00:00:00 1970 UTC.
> * Optionally users may provide a different origin as a timestamp arg3.
> *
> * This an example of series of window with an interval of 5 seconds and
> origin Epoch 0 Thu Jan 01 00:00:00 1970 UTC:
> *
> *
> * interval 1 interval 2interval 3
> * Jan 01 00:00:00 Jan 01 00:00:05 Jan 01 00:00:10
> * 0 -- 4 : 5 --- 9: 10 --- 14
> *
> * This UDF rounds timestamp agr1 to the beginning of window interval where
> it belongs to.
> *
> */
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[
https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655654#comment-16655654
]
Prasanth Jayachandran commented on HIVE-20703:
--
{quote}During compilation getMaxMemoryAvailable() returns zero, perhaps this is
set during execution?
{quote}
Yes. you are right. This is set in RecordProcessor init which is execution.
Maybe you can copy MemoryInfo class from HIVE-20713 to avoid duplicate. I can
rebase HIVE-20713 after this patch is committed.
> Put dynamic sort partition optimization under cost based decision
> -
>
> Key: HIVE-20703
> URL: https://issues.apache.org/jira/browse/HIVE-20703
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20703.1.patch, HIVE-20703.2.patch,
> HIVE-20703.3.patch, HIVE-20703.4.patch, HIVE-20703.5.patch,
> HIVE-20703.6.patch, HIVE-20703.7.patch, HIVE-20703.8.patch, HIVE-20703.9.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20720) Add partition column option to JDBC handler
[
https://issues.apache.org/jira/browse/HIVE-20720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655675#comment-16655675
]
Hive QA commented on HIVE-20720:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} jdbc-handler in master has 10 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} jdbc-handler: The patch generated 111 new + 64
unchanged - 5 fixed = 175 total (was 69) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
30s{color} | {color:red} jdbc-handler generated 5 new + 7 unchanged - 3 fixed =
12 total (was 10) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:jdbc-handler |
| | Dead store to dbAccessor in
org.apache.hive.storage.jdbc.JdbcSerDe.initialize(Configuration, Properties)
At
JdbcSerDe.java:org.apache.hive.storage.jdbc.JdbcSerDe.initialize(Configuration,
Properties) At JdbcSerDe.java:[line 77] |
| |
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getBounds(Configuration,
String, boolean, boolean) may fail to clean up java.sql.ResultSet Obligation
to clean up resource created at GenericJdbcDatabaseAccessor.java:fail to clean
up java.sql.ResultSet Obligation to clean up resource created at
GenericJdbcDatabaseAccessor.java:[line 376] is not discharged |
| |
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getBounds(Configuration,
String, boolean, boolean) may fail to clean up java.sql.Statement Obligation
to clean up resource created at GenericJdbcDatabaseAccessor.java:fail to clean
up java.sql.Statement Obligation to clean up resource created at
GenericJdbcDatabaseAccessor.java:[line 375] is not discharged |
| |
org.apache.hive.storage.jdbc.dao.GenericJdbcDatabaseAccessor.getRecordIterator(Configuration,
String, String, String, int, int) may fail to clean up java.sql.Statement on
checked exception Obligation to clean up resource created at
GenericJdbcDatabaseAccessor.java:int) may fail to clean up java.sql.Statement
on checked exception Obligation to clean up resource created at
GenericJdbcDatabaseAccessor.java:[line 168] is not discharged |
| | A prepared statement is generated from a
[jira] [Commented] (HIVE-20748) Disable materialized view rewriting when plan pattern is not allowed
[ https://issues.apache.org/jira/browse/HIVE-20748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655676#comment-16655676 ] Jesus Camacho Rodriguez commented on HIVE-20748: [~ashutoshc], could you take a look? Thanks > Disable materialized view rewriting when plan pattern is not allowed > > > Key: HIVE-20748 > URL: https://issues.apache.org/jira/browse/HIVE-20748 > Project: Hive > Issue Type: Bug > Components: Materialized views >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20748.01.patch, HIVE-20748.01.patch, > HIVE-20748.patch > > > For instance, currently rewriting algorithm does not support some operators. > Or we cannot have non-deterministic function in the MV definition. In those > cases, we should fail either when we try to create the MV with rewriting > enabled, or when when we enable the rewriting for a MV already created. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20701) Allow HiveStreaming to receive a key value to commit atomically together with the transaction
[ https://issues.apache.org/jira/browse/HIVE-20701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655686#comment-16655686 ] Prasanth Jayachandran commented on HIVE-20701: -- nit: long tableId = conn.getMSC().getTable(conn.getTable().getDbName(), conn.getTable().getTableName()).getId(); This tableId does not change within a streaming connection right? Maybe this can be a variable in streaming connection or transaction batch since we already have the table object (also avoids additional metastore call during commit). Looks good otherwise. +1 > Allow HiveStreaming to receive a key value to commit atomically together with > the transaction > - > > Key: HIVE-20701 > URL: https://issues.apache.org/jira/browse/HIVE-20701 > Project: Hive > Issue Type: Improvement >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Attachments: HIVE-20701.1.patch, HIVE-20701.2.patch, > HIVE-20701.3.patch, HIVE-20701.4.patch, HIVE-20701.5.patch > > > Following up with HIVE-20538 it'd be nice to be able to use this feature with > hive streaming -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20703: --- Status: Open (was: Patch Available) > Put dynamic sort partition optimization under cost based decision > - > > Key: HIVE-20703 > URL: https://issues.apache.org/jira/browse/HIVE-20703 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20703.1.patch, HIVE-20703.10.patch, > HIVE-20703.2.patch, HIVE-20703.3.patch, HIVE-20703.4.patch, > HIVE-20703.5.patch, HIVE-20703.6.patch, HIVE-20703.7.patch, > HIVE-20703.8.patch, HIVE-20703.9.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20703: --- Attachment: HIVE-20703.10.patch > Put dynamic sort partition optimization under cost based decision > - > > Key: HIVE-20703 > URL: https://issues.apache.org/jira/browse/HIVE-20703 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20703.1.patch, HIVE-20703.10.patch, > HIVE-20703.2.patch, HIVE-20703.3.patch, HIVE-20703.4.patch, > HIVE-20703.5.patch, HIVE-20703.6.patch, HIVE-20703.7.patch, > HIVE-20703.8.patch, HIVE-20703.9.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20703: --- Status: Patch Available (was: Open) > Put dynamic sort partition optimization under cost based decision > - > > Key: HIVE-20703 > URL: https://issues.apache.org/jira/browse/HIVE-20703 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20703.1.patch, HIVE-20703.10.patch, > HIVE-20703.2.patch, HIVE-20703.3.patch, HIVE-20703.4.patch, > HIVE-20703.5.patch, HIVE-20703.6.patch, HIVE-20703.7.patch, > HIVE-20703.8.patch, HIVE-20703.9.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20720) Add partition column option to JDBC handler
[
https://issues.apache.org/jira/browse/HIVE-20720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655707#comment-16655707
]
Hive QA commented on HIVE-20720:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944482/HIVE-20720.9.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15106 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.exec.spark.TestSparkSessionTimeout.testMultiSessionSparkSessionTimeout
(batchId=249)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14548/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14548/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14548/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944482 - PreCommit-HIVE-Build
> Add partition column option to JDBC handler
> ---
>
> Key: HIVE-20720
> URL: https://issues.apache.org/jira/browse/HIVE-20720
> Project: Hive
> Issue Type: New Feature
> Components: StorageHandler
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-20720.1.patch, HIVE-20720.2.patch,
> HIVE-20720.3.patch, HIVE-20720.4.patch, HIVE-20720.5.patch,
> HIVE-20720.6.patch, HIVE-20720.7.patch, HIVE-20720.8.patch, HIVE-20720.9.patch
>
>
> Currently JdbcStorageHandler does not split input in Tez. The reason is
> numSplit of JdbcInputFormat.getSplits can only pass via "mapreduce.job.maps"
> in Tez. And "mapreduce.job.maps" is not a valid param if authorizer(eg.
> SQLStdAuth) is in use. User ends up always use 1 split.
> We need to rely on this new feature if we want to support multi-splits. Here
> is my proposal:
> 1. Specify partitionColumn/numPartitions, and optional lowerBound/upperBound
> in tblproperties if user want to split jdbc data source. In case
> lowerBound/upperBound is not specified, JdbcStorageHandler will run max/min
> query to get this in planner. We can currently limit partitionColumn to only
> numeric/date/timestamp column for simplicity
> 2. If partitionColumn/numPartitions are not specified, don't split input
> 3. Splits are equal intervals without respect to data distribution
> 4. There is also a "hive.sql.query.split" flag vetos the split (can be set
> manually or automatically by calcite)
> 5. If partitionColumn is not defined, but numPartitions is defined, use
> original limit/offset logic (however, don't rely on numSplit).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20707) Automatic partition management
[ https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-20707: - Attachment: HIVE-20707.6.patch > Automatic partition management > -- > > Key: HIVE-20707 > URL: https://issues.apache.org/jira/browse/HIVE-20707 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch, > HIVE-20707.2.patch, HIVE-20707.4.patch, HIVE-20707.5.patch, HIVE-20707.6.patch > > > In current scenario, to add partitions for external tables to metastore, MSCK > REPAIR command has to be executed manually. To avoid this manual step, > external tables can be specified a table property based on which a background > metastore thread can sync partitions periodically. Tables can also be > specified with partition retention period. Any partition whose age exceeds > the retention period will be dropped automatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[ https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655714#comment-16655714 ] Prasanth Jayachandran commented on HIVE-20707: -- druid test failure looks strange. Running it locally is passing. > Automatic partition management > -- > > Key: HIVE-20707 > URL: https://issues.apache.org/jira/browse/HIVE-20707 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch, > HIVE-20707.2.patch, HIVE-20707.4.patch, HIVE-20707.5.patch, HIVE-20707.6.patch > > > In current scenario, to add partitions for external tables to metastore, MSCK > REPAIR command has to be executed manually. To avoid this manual step, > external tables can be specified a table property based on which a background > metastore thread can sync partitions periodically. Tables can also be > specified with partition retention period. Any partition whose age exceeds > the retention period will be dropped automatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655718#comment-16655718 ] Vineet Garg commented on HIVE-20703: [~prasanth_j] I have added the MemoryInfo class and updated the logic to get to executor/container memory. Can you take a look at the patch? > Put dynamic sort partition optimization under cost based decision > - > > Key: HIVE-20703 > URL: https://issues.apache.org/jira/browse/HIVE-20703 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20703.1.patch, HIVE-20703.10.patch, > HIVE-20703.2.patch, HIVE-20703.3.patch, HIVE-20703.4.patch, > HIVE-20703.5.patch, HIVE-20703.6.patch, HIVE-20703.7.patch, > HIVE-20703.8.patch, HIVE-20703.9.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20720) Add partition column option to JDBC handler
[ https://issues.apache.org/jira/browse/HIVE-20720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-20720: -- Attachment: HIVE-20720.10.patch > Add partition column option to JDBC handler > --- > > Key: HIVE-20720 > URL: https://issues.apache.org/jira/browse/HIVE-20720 > Project: Hive > Issue Type: New Feature > Components: StorageHandler >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20720.1.patch, HIVE-20720.10.patch, > HIVE-20720.2.patch, HIVE-20720.3.patch, HIVE-20720.4.patch, > HIVE-20720.5.patch, HIVE-20720.6.patch, HIVE-20720.7.patch, > HIVE-20720.8.patch, HIVE-20720.9.patch > > > Currently JdbcStorageHandler does not split input in Tez. The reason is > numSplit of JdbcInputFormat.getSplits can only pass via "mapreduce.job.maps" > in Tez. And "mapreduce.job.maps" is not a valid param if authorizer(eg. > SQLStdAuth) is in use. User ends up always use 1 split. > We need to rely on this new feature if we want to support multi-splits. Here > is my proposal: > 1. Specify partitionColumn/numPartitions, and optional lowerBound/upperBound > in tblproperties if user want to split jdbc data source. In case > lowerBound/upperBound is not specified, JdbcStorageHandler will run max/min > query to get this in planner. We can currently limit partitionColumn to only > numeric/date/timestamp column for simplicity > 2. If partitionColumn/numPartitions are not specified, don't split input > 3. Splits are equal intervals without respect to data distribution > 4. There is also a "hive.sql.query.split" flag vetos the split (can be set > manually or automatically by calcite) > 5. If partitionColumn is not defined, but numPartitions is defined, use > original limit/offset logic (however, don't rely on numSplit). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20572) Change default value of hive.tez.llap.min.reducer.per.executor
[ https://issues.apache.org/jira/browse/HIVE-20572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-20572: Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master. > Change default value of hive.tez.llap.min.reducer.per.executor > -- > > Key: HIVE-20572 > URL: https://issues.apache.org/jira/browse/HIVE-20572 > Project: Hive > Issue Type: Improvement > Components: Configuration >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20572.2.patch, HIVE-20572.3.patch, HIVE-20572.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20701) Allow HiveStreaming to receive a key value to commit atomically together with the transaction
[ https://issues.apache.org/jira/browse/HIVE-20701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-20701: --- Status: Open (was: Patch Available) > Allow HiveStreaming to receive a key value to commit atomically together with > the transaction > - > > Key: HIVE-20701 > URL: https://issues.apache.org/jira/browse/HIVE-20701 > Project: Hive > Issue Type: Improvement >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Attachments: HIVE-20701.1.patch, HIVE-20701.2.patch, > HIVE-20701.3.patch, HIVE-20701.4.patch, HIVE-20701.5.patch, HIVE-20701.6.patch > > > Following up with HIVE-20538 it'd be nice to be able to use this feature with > hive streaming -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20701) Allow HiveStreaming to receive a key value to commit atomically together with the transaction
[ https://issues.apache.org/jira/browse/HIVE-20701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-20701: --- Attachment: HIVE-20701.6.patch Status: Patch Available (was: Open) > Allow HiveStreaming to receive a key value to commit atomically together with > the transaction > - > > Key: HIVE-20701 > URL: https://issues.apache.org/jira/browse/HIVE-20701 > Project: Hive > Issue Type: Improvement >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Attachments: HIVE-20701.1.patch, HIVE-20701.2.patch, > HIVE-20701.3.patch, HIVE-20701.4.patch, HIVE-20701.5.patch, HIVE-20701.6.patch > > > Following up with HIVE-20538 it'd be nice to be able to use this feature with > hive streaming -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655729#comment-16655729 ] Prasanth Jayachandran commented on HIVE-20703: -- lgtm, +1 > Put dynamic sort partition optimization under cost based decision > - > > Key: HIVE-20703 > URL: https://issues.apache.org/jira/browse/HIVE-20703 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20703.1.patch, HIVE-20703.10.patch, > HIVE-20703.2.patch, HIVE-20703.3.patch, HIVE-20703.4.patch, > HIVE-20703.5.patch, HIVE-20703.6.patch, HIVE-20703.7.patch, > HIVE-20703.8.patch, HIVE-20703.9.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20735) Address some of the review comments plus Kerberos support
[ https://issues.apache.org/jira/browse/HIVE-20735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-20735: Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) > Address some of the review comments plus Kerberos support > - > > Key: HIVE-20735 > URL: https://issues.apache.org/jira/browse/HIVE-20735 > Project: Hive > Issue Type: Sub-task > Components: kafka integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20735.patch > > > As part of the review comments we agreed to: > # remove start and end offsets columns > # remove the best effort mode > # make the 2pc as default protocol for EOS > Also this patch will include an additional enhancement to add kerberos > support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20763) Add google cloud storage (gs) to the exim uri schema whitelist
[
https://issues.apache.org/jira/browse/HIVE-20763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655731#comment-16655731
]
Hive QA commented on HIVE-20763:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14549/dev-support/hive-personality.sh
|
| git revision | master / dcaeeb4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common U: common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14549/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add google cloud storage (gs) to the exim uri schema whitelist
> --
>
> Key: HIVE-20763
> URL: https://issues.apache.org/jira/browse/HIVE-20763
> Project: Hive
> Issue Type: Task
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20763.01.patch
>
>
> import/export is enabled for s3a by default. Ideally this list should include
> other cloud storage options. This Jira adds Google Storage to the list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20752) In case of LLAP start failure add info how to find YARN logs
[ https://issues.apache.org/jira/browse/HIVE-20752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655751#comment-16655751 ] Ashutosh Chauhan commented on HIVE-20752: - [~sershe] Can you please review this? > In case of LLAP start failure add info how to find YARN logs > > > Key: HIVE-20752 > URL: https://issues.apache.org/jira/browse/HIVE-20752 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20752.01.patch, HIVE-20752.02.patch, > HIVE-20752.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20767) Multiple project between join operators may affect join reordering using constraints
[ https://issues.apache.org/jira/browse/HIVE-20767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655762#comment-16655762 ] Jesus Camacho Rodriguez commented on HIVE-20767: [~ashutoshc], [~vgarg], could you take a look? This basically adds Project merge rule to the block with Project-Join transpose. As Project was getting in the middle, reordering of full join tree was not happening. Verified that this fixes reordering issue. Thanks > Multiple project between join operators may affect join reordering using > constraints > > > Key: HIVE-20767 > URL: https://issues.apache.org/jira/browse/HIVE-20767 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20767.01.patch, HIVE-20767.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20767) Multiple project between join operators may affect join reordering using constraints
[ https://issues.apache.org/jira/browse/HIVE-20767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20767: --- Attachment: HIVE-20767.01.patch > Multiple project between join operators may affect join reordering using > constraints > > > Key: HIVE-20767 > URL: https://issues.apache.org/jira/browse/HIVE-20767 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20767.01.patch, HIVE-20767.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20752) In case of LLAP start failure add info how to find YARN logs
[ https://issues.apache.org/jira/browse/HIVE-20752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655770#comment-16655770 ] Sergey Shelukhin commented on HIVE-20752: - +1 > In case of LLAP start failure add info how to find YARN logs > > > Key: HIVE-20752 > URL: https://issues.apache.org/jira/browse/HIVE-20752 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20752.01.patch, HIVE-20752.02.patch, > HIVE-20752.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20763) Add google cloud storage (gs) to the exim uri schema whitelist
[
https://issues.apache.org/jira/browse/HIVE-20763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655794#comment-16655794
]
Hive QA commented on HIVE-20763:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944399/HIVE-20763.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15097 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14549/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14549/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14549/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944399 - PreCommit-HIVE-Build
> Add google cloud storage (gs) to the exim uri schema whitelist
> --
>
> Key: HIVE-20763
> URL: https://issues.apache.org/jira/browse/HIVE-20763
> Project: Hive
> Issue Type: Task
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20763.01.patch
>
>
> import/export is enabled for s3a by default. Ideally this list should include
> other cloud storage options. This Jira adds Google Storage to the list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20772) record per-task CPU counters in LLAP
[ https://issues.apache.org/jira/browse/HIVE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-20772: --- > record per-task CPU counters in LLAP > > > Key: HIVE-20772 > URL: https://issues.apache.org/jira/browse/HIVE-20772 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20772) record per-task CPU counters in LLAP
[ https://issues.apache.org/jira/browse/HIVE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-20772: Attachment: HIVE-20772.patch > record per-task CPU counters in LLAP > > > Key: HIVE-20772 > URL: https://issues.apache.org/jira/browse/HIVE-20772 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20772.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20772) record per-task CPU counters in LLAP
[ https://issues.apache.org/jira/browse/HIVE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655821#comment-16655821 ] Sergey Shelukhin commented on HIVE-20772: - [~prasanth_j] can you take a look? > record per-task CPU counters in LLAP > > > Key: HIVE-20772 > URL: https://issues.apache.org/jira/browse/HIVE-20772 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20772.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20772) record per-task CPU counters in LLAP
[ https://issues.apache.org/jira/browse/HIVE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655820#comment-16655820 ] Sergey Shelukhin commented on HIVE-20772: - A quick patch... I'd probably need to run it in a cluster to make sure it works correctly in a realistic deployment. > record per-task CPU counters in LLAP > > > Key: HIVE-20772 > URL: https://issues.apache.org/jira/browse/HIVE-20772 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20772.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20772) record per-task CPU counters in LLAP
[ https://issues.apache.org/jira/browse/HIVE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-20772: Status: Patch Available (was: Open) > record per-task CPU counters in LLAP > > > Key: HIVE-20772 > URL: https://issues.apache.org/jira/browse/HIVE-20772 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20772.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20523) Improve table statistics for Parquet format
[ https://issues.apache.org/jira/browse/HIVE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655826#comment-16655826 ] Zoltan Haindrich commented on HIVE-20523: - +1 makes sense to me ; its way better estimate than 1 :) I think you will need to update the output of some other parquet tests > Improve table statistics for Parquet format > --- > > Key: HIVE-20523 > URL: https://issues.apache.org/jira/browse/HIVE-20523 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: George Pachitariu >Assignee: George Pachitariu >Priority: Minor > Attachments: HIVE-20523.1.patch, HIVE-20523.2.patch, > HIVE-20523.3.patch, HIVE-20523.4.patch, HIVE-20523.5.patch, > HIVE-20523.6.patch, HIVE-20523.patch > > > Right now, in the table basic statistics, the *raw data size* for a row with > any data type in the Parquet format is 1. This is an underestimated value > when columns are complex data structures, like arrays. > Having tables with underestimated raw data size makes Hive assign less > containers (mappers/reducers) to it, making the overall query slower. > Heavy underestimation also makes Hive choose MapJoin instead of the > ShuffleJoin that can fail with OOM errors. > In this patch, I compute the columns data size better, taking into account > complex structures. I followed the Writer implementation for the ORC format. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20772) record per-task CPU counters in LLAP
[ https://issues.apache.org/jira/browse/HIVE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655829#comment-16655829 ] Prasanth Jayachandran commented on HIVE-20772: -- tezCounters.findCounter(LlapExecutorCounters.EXECUTOR_CPU_NS).increment(cpuTime); Should this be set instead of increment? If we are reusing the executor thread (I hope we are not), increment will give aggregate whereas set will just give just for that thread. Looks good otherwise. > record per-task CPU counters in LLAP > > > Key: HIVE-20772 > URL: https://issues.apache.org/jira/browse/HIVE-20772 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20772.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20772) record per-task CPU counters in LLAP
[
https://issues.apache.org/jira/browse/HIVE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655832#comment-16655832
]
Sergey Shelukhin commented on HIVE-20772:
-
Hmm, I was assuming based on other counters there (FS ones) that this object is
scoped per task... e.g. {noformat} tezCounters =
taskRunner2Callable.addAndGetTezCounter(FileSystemCounter.class.getName());
} else if (actualCallable instanceof TezCounterSource) { {noformat}
> record per-task CPU counters in LLAP
>
>
> Key: HIVE-20772
> URL: https://issues.apache.org/jira/browse/HIVE-20772
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20772.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20679) DDL operations on hive might create large messages for DBNotification
[
https://issues.apache.org/jira/browse/HIVE-20679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655835#comment-16655835
]
Hive QA commented on HIVE-20679:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
16s{color} | {color:blue} standalone-metastore/metastore-common in master has
28 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} hcatalog/server-extensions in master has 1 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
54s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
2s{color} | {color:blue} standalone-metastore/metastore-server in master has
182 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} testutils/ptest2 in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
15s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
18s{color} | {color:red} hcatalog-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
36s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
32s{color} | {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
17s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
36s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 17s{color}
| {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 36s{color}
| {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} hcatalog/server-extensions: The patch generated 1 new
+ 23 unchanged - 2 fixed = 24 total (was 25) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} itests/hive-unit: The patch generated 34 new + 814
unchanged - 36 fixed = 848 total (was 850) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 5 new + 36 unchanged - 5 fixed
= 41 total (was 41) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
16s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
37s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} find
[jira] [Commented] (HIVE-20772) record per-task CPU counters in LLAP
[ https://issues.apache.org/jira/browse/HIVE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655838#comment-16655838 ] Prasanth Jayachandran commented on HIVE-20772: -- oh yeah.. make sense :) +1 > record per-task CPU counters in LLAP > > > Key: HIVE-20772 > URL: https://issues.apache.org/jira/browse/HIVE-20772 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20772.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20756) Disable SARG leaf creation for date column until ORC-135
[ https://issues.apache.org/jira/browse/HIVE-20756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-20756: Resolution: Fixed Fix Version/s: 1.3.0 Status: Resolved (was: Patch Available) Committed to branch-1. Thanks for the patch! > Disable SARG leaf creation for date column until ORC-135 > > > Key: HIVE-20756 > URL: https://issues.apache.org/jira/browse/HIVE-20756 > Project: Hive > Issue Type: Bug >Affects Versions: 2.1.1 >Reporter: Chiran Ravani >Assignee: Chiran Ravani >Priority: Major > Fix For: 1.3.0 > > Attachments: HIVE-20756.01-branch-1.patch, > HIVE-20756.01-branch-1.patch, HIVE-20756.1.patch > > > Until ORC-135 is committed and orc version is updated in hive, disable SARG > creation for date columns in hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20679) DDL operations on hive might create large messages for DBNotification
[
https://issues.apache.org/jira/browse/HIVE-20679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655846#comment-16655846
]
Hive QA commented on HIVE-20679:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944493/HIVE-20679.8.patch
{color:green}SUCCESS:{color} +1 due to 11 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15184 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=154)
[intersect_all.q,unionDistinct_1.q,table_nonprintable.q,orc_llap_counters1.q,mm_cttas.q,whroot_external1.q,global_limit.q,cte_2.q,rcfile_createas1.q,dynamic_partition_pruning_2.q,intersect_merge.q,results_cache_diff_fs.q,cttl.q,parallel_colstats.q,load_hdfs_file_with_space_in_the_name.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14550/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14550/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14550/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944493 - PreCommit-HIVE-Build
> DDL operations on hive might create large messages for DBNotification
> -
>
> Key: HIVE-20679
> URL: https://issues.apache.org/jira/browse/HIVE-20679
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: anishek
>Assignee: anishek
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20679.1.patch, HIVE-20679.2.patch,
> HIVE-20679.3.patch, HIVE-20679.4.patch, HIVE-20679.5.patch,
> HIVE-20679.6.patch, HIVE-20679.8.patch, a.sql, b.sql
>
>
> Certain type of ddl operations might create large messages as part of
> DBNoitification, this might lead to the rdbms throwing an error when storing
> the message since its size is to large. It will also increase the footprint
> of the rdbms space usage.
> We should try store compressed messages to allow handling these situations.
> Edit: For notification_log table the message column for all supported
> databases can store messages from 2GB to 4GB
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20304) When hive.optimize.skewjoin and hive.auto.convert.join are both set to true, and the execution engine is mr, same stage may launch twice due to the wrong generated plan
[
https://issues.apache.org/jira/browse/HIVE-20304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655854#comment-16655854
]
Hive QA commented on HIVE-20304:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12934290/HIVE-20304.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14551/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14551/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14551/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-10-18 20:47:37.181
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-14551/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-10-18 20:47:37.184
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
db72a86..7fecb6f branch-1 -> origin/branch-1
+ git reset --hard HEAD
HEAD is now at f89274a HIVE-20572 : Change default value of
hive.tez.llap.min.reducer.per.executor
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at f89274a HIVE-20572 : Change default value of
hive.tez.llap.min.reducer.per.executor
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-10-18 20:47:38.202
+ rm -rf ../yetus_PreCommit-HIVE-Build-14551
+ mkdir ../yetus_PreCommit-HIVE-Build-14551
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-14551
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14551/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/test/results/clientpositive/runtime_skewjoin_mapjoin_spark.q.out:29
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/runtime_skewjoin_mapjoin_spark.q.out' with
conflicts.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:367: trailing whitespace.
$hdt$_0:$hdt$_1:tbl3
/data/hiveptest/working/scratch/build.patch:371: trailing whitespace.
$hdt$_0:$hdt$_1:tbl3
/data/hiveptest/working/scratch/build.patch:426: trailing whitespace.
$hdt$_0:$hdt$_2:tbl4
/data/hiveptest/working/scratch/build.patch:430: trailing whitespace.
$hdt$_0:$hdt$_2:tbl4
/data/hiveptest/working/scratch/build.patch:475: trailing whitespace.
$hdt$_0:$hdt$_3:tbl5
error: patch failed:
ql/src/test/results/clientpositive/runtime_skewjoin_mapjoin_spark.q.out:29
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/runtime_skewjoin_mapjoin_spark.q.out' with
conflicts.
U ql/src/test/results/clientpositive/runtime_skewjoin_mapjoin_spark.q.out
warning: squelched 13 whitespace errors
warning: 18 lines add whitespace errors.
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-14551
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12934290 - PreCommit-HIVE-Build
> When hive.optimize.skewjoin and hive.auto.convert.join are both set to true,
> and the execution engine is mr, same stage may launch twice due to the wrong
> generated plan
>
>
> Key: HIVE-20304
> URL: https://issues.apache.org/jira/browse/HIVE-20304
> Project: Hive
> Issue Type: Bug
> Compon
