[jira] [Commented] (HIVE-17272) when hive.vectorized.execution.enabled is true, query on empty partitioned table fails with NPE
[
https://issues.apache.org/jira/browse/HIVE-17272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124816#comment-16124816
]
Hive QA commented on HIVE-17272:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12881639/HIVE-17272.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 381 failed/errored test(s), 10966 tests
executed
*Failed tests:*
{noformat}
TestTxnCommands2WithSplitUpdateAndVectorization - did not produce a TEST-*.xml
file (likely timed out) (batchId=280)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_vectorization]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_vectorization_partition]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_vectorization_project]
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constprog_when_case]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[foldts] (batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_acid] (batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_reader] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_uncompressed]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mergejoin] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[optimize_filter_literal]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_int_type_promotion]
(batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_vectorization_ppd]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_no_row_serde]
(batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_types_non_dictionary_encoding_vectorization]
(batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_types_vectorization]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[tez_join_hash]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_acid3]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_aggregate_9]
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_aggregate_without_gby]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_between_columns]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_binary_join_groupby]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_bround]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_bucket]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_cast_constant]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_char_2]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_char_4]
(batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_char_cast]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_char_mapjoin1]
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_char_simple]
(batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_coalesce]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_coalesce_2]
(batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_coalesce_3]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_complex_join]
(batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_const]
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_count]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_custom_udf_configure]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_data_types]
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_date_1]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_10_0]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_1]
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_3]
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_4]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_5]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_6]
(batchId=13)
[jira] [Updated] (HIVE-17301) Make JSONMessageFactory.getTObj method thread safe
[ https://issues.apache.org/jira/browse/HIVE-17301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-17301: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Patch pushed to master. > Make JSONMessageFactory.getTObj method thread safe > -- > > Key: HIVE-17301 > URL: https://issues.apache.org/jira/browse/HIVE-17301 > Project: Hive > Issue Type: Bug > Components: Metastore >Reporter: Tao Li >Assignee: Tao Li > Fix For: 3.0.0 > > Attachments: HIVE-17301.1.patch > > > This static method is using a singleton instance of TDeserializer, which is > not thread safe. Instead we want to create a new instance per method call. > This class is lightweight, so it should be fine from perf perspective. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17272) when hive.vectorized.execution.enabled is true, query on empty partitioned table fails with NPE
[
https://issues.apache.org/jira/browse/HIVE-17272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-17272:
Attachment: HIVE-17272.1.patch
> when hive.vectorized.execution.enabled is true, query on empty partitioned
> table fails with NPE
> ---
>
> Key: HIVE-17272
> URL: https://issues.apache.org/jira/browse/HIVE-17272
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 2.1.1
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-17272.1.patch
>
>
> {noformat}
> set hive.vectorized.execution.enabled=true;
> CREATE TABLE `tab`(`x` int) PARTITIONED BY ( `y` int) stored as parquet;
> select * from tab t1 join tab t2 where t1.x=t2.x;
> {noformat}
> The query fails with the following exception.
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.createAndInitPartitionContext(VectorMapOperator.java:386)
> ~[hive-exec-2.3.0.jar:2.3.0]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.internalSetChildren(VectorMapOperator.java:559)
> ~[hive-exec-2.3.0.jar:2.3.0]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.setChildren(VectorMapOperator.java:474)
> ~[hive-exec-2.3.0.jar:2.3.0]
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:106)
> ~[hive-exec-2.3.0.jar:2.3.0]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
> ~[hadoop-common-2.6.0.jar:?]
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
> ~[hadoop-common-2.6.0.jar:?]
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
> ~[hadoop-common-2.6.0.jar:?]
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> ~[hadoop-core-2.6.0-mr1-cdh5.4.2.jar:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
> ~[hadoop-common-2.6.0.jar:?]
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
> ~[hadoop-common-2.6.0.jar:?]
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
> ~[hadoop-common-2.6.0.jar:?]
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:413)
> ~[hadoop-core-2.6.0-mr1-cdh5.4.2.jar:?]
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
> ~[hadoop-core-2.6.0-mr1-cdh5.4.2.jar:?]
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:268)
> ~[hadoop-core-2.6.0-mr1-cdh5.4.2.jar:?]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[?:1.8.0_101]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[?:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[?:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[?:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_101]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17272) when hive.vectorized.execution.enabled is true, query on empty partitioned table fails with NPE
[
https://issues.apache.org/jira/browse/HIVE-17272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-17272:
Status: Patch Available (was: Open)
> when hive.vectorized.execution.enabled is true, query on empty partitioned
> table fails with NPE
> ---
>
> Key: HIVE-17272
> URL: https://issues.apache.org/jira/browse/HIVE-17272
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 2.1.1
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-17272.1.patch
>
>
> {noformat}
> set hive.vectorized.execution.enabled=true;
> CREATE TABLE `tab`(`x` int) PARTITIONED BY ( `y` int) stored as parquet;
> select * from tab t1 join tab t2 where t1.x=t2.x;
> {noformat}
> The query fails with the following exception.
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.createAndInitPartitionContext(VectorMapOperator.java:386)
> ~[hive-exec-2.3.0.jar:2.3.0]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.internalSetChildren(VectorMapOperator.java:559)
> ~[hive-exec-2.3.0.jar:2.3.0]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.setChildren(VectorMapOperator.java:474)
> ~[hive-exec-2.3.0.jar:2.3.0]
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:106)
> ~[hive-exec-2.3.0.jar:2.3.0]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
> ~[hadoop-common-2.6.0.jar:?]
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
> ~[hadoop-common-2.6.0.jar:?]
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
> ~[hadoop-common-2.6.0.jar:?]
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> ~[hadoop-core-2.6.0-mr1-cdh5.4.2.jar:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_101]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_101]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_101]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
> ~[hadoop-common-2.6.0.jar:?]
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
> ~[hadoop-common-2.6.0.jar:?]
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
> ~[hadoop-common-2.6.0.jar:?]
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:413)
> ~[hadoop-core-2.6.0-mr1-cdh5.4.2.jar:?]
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
> ~[hadoop-core-2.6.0-mr1-cdh5.4.2.jar:?]
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:268)
> ~[hadoop-core-2.6.0-mr1-cdh5.4.2.jar:?]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[?:1.8.0_101]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[?:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[?:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[?:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_101]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17089) make acid 2.0 the default
[
https://issues.apache.org/jira/browse/HIVE-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124770#comment-16124770
]
Hive QA commented on HIVE-17089:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12881634/HIVE-17089.14.patch
{color:green}SUCCESS:{color} +1 due to 13 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 10968 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=235)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=222)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=180)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6374/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6374/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6374/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12881634 - PreCommit-HIVE-Build
> make acid 2.0 the default
> -
>
> Key: HIVE-17089
> URL: https://issues.apache.org/jira/browse/HIVE-17089
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-17089.01.patch, HIVE-17089.03.patch,
> HIVE-17089.05.patch, HIVE-17089.06.patch, HIVE-17089.07.patch,
> HIVE-17089.10.patch, HIVE-17089.10.patch, HIVE-17089.11.patch,
> HIVE-17089.12.patch, HIVE-17089.13.patch, HIVE-17089.14.patch
>
>
> acid 2.0 is introduced in HIVE-14035. It replaces Update events with a
> combination of Delete + Insert events. This now makes U=D+I the default (and
> only) supported acid table type in Hive 3.0.
> The expectation for upgrade is that Major compaction has to be run on all
> acid tables in the existing Hive cluster and that no new writes to these
> table take place since the start of compaction (Need to add a mechanism to
> put a table in read-only mode - this way it can still be read while it's
> being compacted). Then upgrade to Hive 3.0 can take place.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17089) make acid 2.0 the default
[ https://issues.apache.org/jira/browse/HIVE-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-17089: -- Attachment: HIVE-17089.14.patch > make acid 2.0 the default > - > > Key: HIVE-17089 > URL: https://issues.apache.org/jira/browse/HIVE-17089 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-17089.01.patch, HIVE-17089.03.patch, > HIVE-17089.05.patch, HIVE-17089.06.patch, HIVE-17089.07.patch, > HIVE-17089.10.patch, HIVE-17089.10.patch, HIVE-17089.11.patch, > HIVE-17089.12.patch, HIVE-17089.13.patch, HIVE-17089.14.patch > > > acid 2.0 is introduced in HIVE-14035. It replaces Update events with a > combination of Delete + Insert events. This now makes U=D+I the default (and > only) supported acid table type in Hive 3.0. > The expectation for upgrade is that Major compaction has to be run on all > acid tables in the existing Hive cluster and that no new writes to these > table take place since the start of compaction (Need to add a mechanism to > put a table in read-only mode - this way it can still be read while it's > being compacted). Then upgrade to Hive 3.0 can take place. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17308) Improvement in join cardinality estimation
[
https://issues.apache.org/jira/browse/HIVE-17308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124756#comment-16124756
]
Hive QA commented on HIVE-17308:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12881632/HIVE-17308.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 11004 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_alt_syntax]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_cond_pushdown_2]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_cond_pushdown_4]
(batchId=79)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=100)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query17]
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query24]
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query25]
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query29]
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query50]
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query54]
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query64]
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query72]
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query85]
(batchId=235)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_alt_syntax]
(batchId=135)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_cond_pushdown_2]
(batchId=126)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_cond_pushdown_4]
(batchId=136)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=222)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=180)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6373/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6373/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6373/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 29 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12881632 - PreCommit-HIVE-Build
> Improvement in join cardinality estimation
> --
>
> Key: HIVE-17308
> URL: https://issues.apache.org/jira/browse/HIVE-17308
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-17308.1.patch
>
>
> Currently during logical planning join cardinality is estimated assuming no
> correlation among join keys (This estimation is done using exponential
> backoff). Physical planning on the other hand consider correlation for multi
> keys and uses different estimation. We should consider correlation during
> logical planning as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17308) Improvement in join cardinality estimation
[ https://issues.apache.org/jira/browse/HIVE-17308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17308: --- Status: Patch Available (was: Open) > Improvement in join cardinality estimation > -- > > Key: HIVE-17308 > URL: https://issues.apache.org/jira/browse/HIVE-17308 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17308.1.patch > > > Currently during logical planning join cardinality is estimated assuming no > correlation among join keys (This estimation is done using exponential > backoff). Physical planning on the other hand consider correlation for multi > keys and uses different estimation. We should consider correlation during > logical planning as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17308) Improvement in join cardinality estimation
[ https://issues.apache.org/jira/browse/HIVE-17308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124738#comment-16124738 ] Vineet Garg commented on HIVE-17308: First patch introduces different cardinality estimation if there are multiple join keys (since hive doesn't have any way to figure out if there is correlation we always assume correlation). > Improvement in join cardinality estimation > -- > > Key: HIVE-17308 > URL: https://issues.apache.org/jira/browse/HIVE-17308 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17308.1.patch > > > Currently during logical planning join cardinality is estimated assuming no > correlation among join keys (This estimation is done using exponential > backoff). Physical planning on the other hand consider correlation for multi > keys and uses different estimation. We should consider correlation during > logical planning as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17308) Improvement in join cardinality estimation
[ https://issues.apache.org/jira/browse/HIVE-17308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17308: --- Attachment: HIVE-17308.1.patch > Improvement in join cardinality estimation > -- > > Key: HIVE-17308 > URL: https://issues.apache.org/jira/browse/HIVE-17308 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17308.1.patch > > > Currently during logical planning join cardinality is estimated assuming no > correlation among join keys (This estimation is done using exponential > backoff). Physical planning on the other hand consider correlation for multi > keys and uses different estimation. We should consider correlation during > logical planning as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-17308) Improvement in join cardinality estimation
[ https://issues.apache.org/jira/browse/HIVE-17308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-17308: -- > Improvement in join cardinality estimation > -- > > Key: HIVE-17308 > URL: https://issues.apache.org/jira/browse/HIVE-17308 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > > Currently during logical planning join cardinality is estimated assuming no > correlation among join keys (This estimation is done using exponential > backoff). Physical planning on the other hand consider correlation for multi > keys and uses different estimation. We should consider correlation during > logical planning as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17089) make acid 2.0 the default
[
https://issues.apache.org/jira/browse/HIVE-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124727#comment-16124727
]
Hive QA commented on HIVE-17089:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12881630/HIVE-17089.13.patch
{color:green}SUCCESS:{color} +1 due to 13 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 10968 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=235)
org.apache.hadoop.hive.ql.io.orc.TestOrcRecordUpdater.testWriter (batchId=264)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=180)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6372/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6372/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6372/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12881630 - PreCommit-HIVE-Build
> make acid 2.0 the default
> -
>
> Key: HIVE-17089
> URL: https://issues.apache.org/jira/browse/HIVE-17089
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-17089.01.patch, HIVE-17089.03.patch,
> HIVE-17089.05.patch, HIVE-17089.06.patch, HIVE-17089.07.patch,
> HIVE-17089.10.patch, HIVE-17089.10.patch, HIVE-17089.11.patch,
> HIVE-17089.12.patch, HIVE-17089.13.patch
>
>
> acid 2.0 is introduced in HIVE-14035. It replaces Update events with a
> combination of Delete + Insert events. This now makes U=D+I the default (and
> only) supported acid table type in Hive 3.0.
> The expectation for upgrade is that Major compaction has to be run on all
> acid tables in the existing Hive cluster and that no new writes to these
> table take place since the start of compaction (Need to add a mechanism to
> put a table in read-only mode - this way it can still be read while it's
> being compacted). Then upgrade to Hive 3.0 can take place.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17290) Should use equals() rather than == to compare strings
[ https://issues.apache.org/jira/browse/HIVE-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124710#comment-16124710 ] ASF GitHub Bot commented on HIVE-17290: --- Github user asfgit closed the pull request at: https://github.com/apache/hive/pull/226 > Should use equals() rather than == to compare strings > - > > Key: HIVE-17290 > URL: https://issues.apache.org/jira/browse/HIVE-17290 > Project: Hive > Issue Type: Bug >Reporter: Oleg Danilov >Priority: Trivial > Fix For: 3.0.0 > > Attachments: HIVE-17290.patch > > > There are number of places, where strings have been compared with == or !=. > Seems like it works now, thanks to string interning, but it would be better > not to be relied upon. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17290) Should use equals() rather than == to compare strings
[ https://issues.apache.org/jira/browse/HIVE-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-17290: -- Resolution: Fixed Assignee: Oleg Danilov Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Patch committed. Thanks Oleg. > Should use equals() rather than == to compare strings > - > > Key: HIVE-17290 > URL: https://issues.apache.org/jira/browse/HIVE-17290 > Project: Hive > Issue Type: Bug >Reporter: Oleg Danilov >Assignee: Oleg Danilov >Priority: Trivial > Fix For: 3.0.0 > > Attachments: HIVE-17290.patch > > > There are number of places, where strings have been compared with == or !=. > Seems like it works now, thanks to string interning, but it would be better > not to be relied upon. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17290) Should use equals() rather than == to compare strings
[ https://issues.apache.org/jira/browse/HIVE-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124709#comment-16124709 ] Alan Gates commented on HIVE-17290: --- +1 > Should use equals() rather than == to compare strings > - > > Key: HIVE-17290 > URL: https://issues.apache.org/jira/browse/HIVE-17290 > Project: Hive > Issue Type: Bug >Reporter: Oleg Danilov >Priority: Trivial > Attachments: HIVE-17290.patch > > > There are number of places, where strings have been compared with == or !=. > Seems like it works now, thanks to string interning, but it would be better > not to be relied upon. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17241) Change metastore classes to not use the shims
[
https://issues.apache.org/jira/browse/HIVE-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124704#comment-16124704
]
Hive QA commented on HIVE-17241:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12881627/HIVE-17241.patch
{color:green}SUCCESS:{color} +1 due to 24 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 11017 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[create_merge_compressed]
(batchId=240)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[udtf_output_on_close]
(batchId=234)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=99)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=235)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=222)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=180)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testHttpRetryOnServerIdleTimeout
(batchId=228)
org.apache.hive.minikdc.TestSSLWithMiniKdc.org.apache.hive.minikdc.TestSSLWithMiniKdc
(batchId=241)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6371/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6371/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6371/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12881627 - PreCommit-HIVE-Build
> Change metastore classes to not use the shims
> -
>
> Key: HIVE-17241
> URL: https://issues.apache.org/jira/browse/HIVE-17241
> Project: Hive
> Issue Type: Sub-task
> Components: Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
> Attachments: HIVE-17241.patch
>
>
> As part of moving the metastore into a standalone package, it will no longer
> have access to the shims. This means we need to either copy them or access
> the underlying Hadoop operations directly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17089) make acid 2.0 the default
[ https://issues.apache.org/jira/browse/HIVE-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-17089: -- Attachment: HIVE-17089.13.patch > make acid 2.0 the default > - > > Key: HIVE-17089 > URL: https://issues.apache.org/jira/browse/HIVE-17089 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-17089.01.patch, HIVE-17089.03.patch, > HIVE-17089.05.patch, HIVE-17089.06.patch, HIVE-17089.07.patch, > HIVE-17089.10.patch, HIVE-17089.10.patch, HIVE-17089.11.patch, > HIVE-17089.12.patch, HIVE-17089.13.patch > > > acid 2.0 is introduced in HIVE-14035. It replaces Update events with a > combination of Delete + Insert events. This now makes U=D+I the default (and > only) supported acid table type in Hive 3.0. > The expectation for upgrade is that Major compaction has to be run on all > acid tables in the existing Hive cluster and that no new writes to these > table take place since the start of compaction (Need to add a mechanism to > put a table in read-only mode - this way it can still be read while it's > being compacted). Then upgrade to Hive 3.0 can take place. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17307) Change the metastore to not use the metrics code in hive/common
[ https://issues.apache.org/jira/browse/HIVE-17307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124696#comment-16124696 ] Alan Gates commented on HIVE-17307: --- I propose that we move the metastore to use the codahale metrics directly. The reason that the current Metrics interface was created was to allow the legacy metrics system to co-exist with the new codahale metrics. As far as I know Yahoo was the only user of the legacy metrics system. I have spoken to [~mithun] and [~cdrome] and they are fine with not continuing support for the legacy metrics in the standalone metastore. Removing the Metrics interface and using codahale metrics directly is a much simpler solution. > Change the metastore to not use the metrics code in hive/common > --- > > Key: HIVE-17307 > URL: https://issues.apache.org/jira/browse/HIVE-17307 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Reporter: Alan Gates >Assignee: Alan Gates > > As we move code into the standalone metastore module, it cannot use the > metrics in hive-common. We could copy the current Metrics interface or we > could change the metastore code to directly use codahale metrics. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17089) make acid 2.0 the default
[
https://issues.apache.org/jira/browse/HIVE-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124694#comment-16124694
]
Hive QA commented on HIVE-17089:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12881612/HIVE-17089.12.patch
{color:green}SUCCESS:{color} +1 due to 13 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 10968 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[insert_overwrite_local_directory_1]
(batchId=240)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=100)
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testRecordReaderIncompleteDelta
(batchId=264)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=180)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6370/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6370/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6370/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12881612 - PreCommit-HIVE-Build
> make acid 2.0 the default
> -
>
> Key: HIVE-17089
> URL: https://issues.apache.org/jira/browse/HIVE-17089
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-17089.01.patch, HIVE-17089.03.patch,
> HIVE-17089.05.patch, HIVE-17089.06.patch, HIVE-17089.07.patch,
> HIVE-17089.10.patch, HIVE-17089.10.patch, HIVE-17089.11.patch,
> HIVE-17089.12.patch
>
>
> acid 2.0 is introduced in HIVE-14035. It replaces Update events with a
> combination of Delete + Insert events. This now makes U=D+I the default (and
> only) supported acid table type in Hive 3.0.
> The expectation for upgrade is that Major compaction has to be run on all
> acid tables in the existing Hive cluster and that no new writes to these
> table take place since the start of compaction (Need to add a mechanism to
> put a table in read-only mode - this way it can still be read while it's
> being compacted). Then upgrade to Hive 3.0 can take place.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-17307) Change the metastore to not use the metrics code in hive/common
[ https://issues.apache.org/jira/browse/HIVE-17307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates reassigned HIVE-17307: - > Change the metastore to not use the metrics code in hive/common > --- > > Key: HIVE-17307 > URL: https://issues.apache.org/jira/browse/HIVE-17307 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Reporter: Alan Gates >Assignee: Alan Gates > > As we move code into the standalone metastore module, it cannot use the > metrics in hive-common. We could copy the current Metrics interface or we > could change the metastore code to directly use codahale metrics. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17241) Change metastore classes to not use the shims
[ https://issues.apache.org/jira/browse/HIVE-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-17241: -- Status: Patch Available (was: Open) > Change metastore classes to not use the shims > - > > Key: HIVE-17241 > URL: https://issues.apache.org/jira/browse/HIVE-17241 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Reporter: Alan Gates >Assignee: Alan Gates > Attachments: HIVE-17241.patch > > > As part of moving the metastore into a standalone package, it will no longer > have access to the shims. This means we need to either copy them or access > the underlying Hadoop operations directly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17241) Change metastore classes to not use the shims
[ https://issues.apache.org/jira/browse/HIVE-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-17241: -- Attachment: HIVE-17241.patch > Change metastore classes to not use the shims > - > > Key: HIVE-17241 > URL: https://issues.apache.org/jira/browse/HIVE-17241 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Reporter: Alan Gates >Assignee: Alan Gates > Attachments: HIVE-17241.patch > > > As part of moving the metastore into a standalone package, it will no longer > have access to the shims. This means we need to either copy them or access > the underlying Hadoop operations directly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17305) New insert overwrite dynamic partitions qtest need to have the golden file regenerated
[
https://issues.apache.org/jira/browse/HIVE-17305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124671#comment-16124671
]
Hive QA commented on HIVE-17305:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12881604/HIVE-17305.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 11004 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=240)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=99)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=235)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=235)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=180)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testHttpRetryOnServerIdleTimeout
(batchId=228)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6369/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6369/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6369/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12881604 - PreCommit-HIVE-Build
> New insert overwrite dynamic partitions qtest need to have the golden file
> regenerated
> --
>
> Key: HIVE-17305
> URL: https://issues.apache.org/jira/browse/HIVE-17305
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Affects Versions: 3.0.0
>Reporter: Barna Zsombor Klara
>Assignee: Barna Zsombor Klara
>Priority: Trivial
> Attachments: HIVE-17305.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17304) ThreadMXBean based memory allocation monitory for hash table loader
[
https://issues.apache.org/jira/browse/HIVE-17304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124653#comment-16124653
]
Hive QA commented on HIVE-17304:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12881589/HIVE-17304.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 11004 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=240)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=235)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=180)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testConcurrentStatements (batchId=228)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6368/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6368/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6368/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12881589 - PreCommit-HIVE-Build
> ThreadMXBean based memory allocation monitory for hash table loader
> ---
>
> Key: HIVE-17304
> URL: https://issues.apache.org/jira/browse/HIVE-17304
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-17304.1.patch
>
>
> Hash table memory monitoring is based on java data model which can be
> unreliable because of various reasons (wrong object size estimation, adding
> new variables to any class without accounting its size for memory monitoring,
> etc.). We can use allocation size per thread that is provided by ThreadMXBean
> and fallback to DataModel in case if JDK doesn't support thread based
> allocations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17089) make acid 2.0 the default
[ https://issues.apache.org/jira/browse/HIVE-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-17089: -- Attachment: HIVE-17089.12.patch > make acid 2.0 the default > - > > Key: HIVE-17089 > URL: https://issues.apache.org/jira/browse/HIVE-17089 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Attachments: HIVE-17089.01.patch, HIVE-17089.03.patch, > HIVE-17089.05.patch, HIVE-17089.06.patch, HIVE-17089.07.patch, > HIVE-17089.10.patch, HIVE-17089.10.patch, HIVE-17089.11.patch, > HIVE-17089.12.patch > > > acid 2.0 is introduced in HIVE-14035. It replaces Update events with a > combination of Delete + Insert events. This now makes U=D+I the default (and > only) supported acid table type in Hive 3.0. > The expectation for upgrade is that Major compaction has to be run on all > acid tables in the existing Hive cluster and that no new writes to these > table take place since the start of compaction (Need to add a mechanism to > put a table in read-only mode - this way it can still be read while it's > being compacted). Then upgrade to Hive 3.0 can take place. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17303) Missmatch between roaring bitmap library used by druid and the one coming from tez
[
https://issues.apache.org/jira/browse/HIVE-17303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124587#comment-16124587
]
slim bouguerra commented on HIVE-17303:
---
Druid is using 0.5.18, Spark 0.5.11 and Tez 0.4.9
{code}
org.apache.tez:tez-runtime-library:jar:0.8.4.2.6.2.0-SNAPSHOT:compile [INFO] |
+- org.roaringbitmap:RoaringBitmap:jar:0.4.9:compile
{code}
spark is using 0.5.11
{code}
INFO] +- org.apache.spark:spark-core_2.10:jar:1.6.0:compile
[INFO] | +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile
[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.5.11:compile
{code}
{code}
io.druid:druid-processing:jar:0.9.2.2.6.2.0-SNAPSHOT:compile
[INFO] | | \- org.roaringbitmap:RoaringBitmap:jar:0.5.18:compile
{code}
> Missmatch between roaring bitmap library used by druid and the one coming
> from tez
> --
>
> Key: HIVE-17303
> URL: https://issues.apache.org/jira/browse/HIVE-17303
> Project: Hive
> Issue Type: Bug
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
> Attachments: HIVE-17303.patch
>
>
> {code}
>
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.NoSuchMethodError:
> org.roaringbitmap.buffer.MutableRoaringBitmap.runOptimize()Z
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
> at
> org.apache.hadoop.hive.druid.io.DruidRecordWriter.pushSegments(DruidRecordWriter.java:165)
> ... 25 more
> Caused by: java.lang.NoSuchMethodError:
> org.roaringbitmap.buffer.MutableRoaringBitmap.runOptimize()Z
> at
> org.apache.hive.druid.com.metamx.collections.bitmap.WrappedRoaringBitmap.toImmutableBitmap(WrappedRoaringBitmap.java:65)
> at
> org.apache.hive.druid.com.metamx.collections.bitmap.RoaringBitmapFactory.makeImmutableBitmap(RoaringBitmapFactory.java:88)
> at
> org.apache.hive.druid.io.druid.segment.StringDimensionMergerV9.writeIndexes(StringDimensionMergerV9.java:348)
> at
> org.apache.hive.druid.io.druid.segment.IndexMergerV9.makeIndexFiles(IndexMergerV9.java:218)
> at
> org.apache.hive.druid.io.druid.segment.IndexMerger.merge(IndexMerger.java:438)
> at
> org.apache.hive.druid.io.druid.segment.IndexMerger.persist(IndexMerger.java:186)
> at
> org.apache.hive.druid.io.druid.segment.IndexMerger.persist(IndexMerger.java:152)
> at
> org.apache.hive.druid.io.druid.segment.realtime.appenderator.AppenderatorImpl.persistHydrant(AppenderatorImpl.java:996)
> at
> org.apache.hive.druid.io.druid.segment.realtime.appenderator.AppenderatorImpl.access$200(AppenderatorImpl.java:93)
> at
> org.apache.hive.druid.io.druid.segment.realtime.appenderator.AppenderatorImpl$2.doCall(AppenderatorImpl.java:385)
> at
> org.apache.hive.druid.io.druid.common.guava.ThreadRenamingCallable.call(ThreadRenamingCallable.java:44)
> ... 4 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:89, Vertex vertex_1502470020457_0005_12_05 [Reducer 2]
> killed/failed due to:OWN_TASK_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:0 (state=08S01,code=2)
> Options
> Attachments
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HIVE-17303) Missmatch between roaring bitmap library used by druid and the one coming from tez
[
https://issues.apache.org/jira/browse/HIVE-17303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124587#comment-16124587
]
slim bouguerra edited comment on HIVE-17303 at 8/12/17 2:31 PM:
Druid is using 0.5.18, Spark 0.5.11 and Tez 0.4.9
{code}
org.apache.tez:tez-runtime-library:jar:0.8.4.2.6.2.0-SNAPSHOT:compile [INFO] |
+- org.roaringbitmap:RoaringBitmap:jar:0.4.9:compile
{code}
{code}
INFO] +- org.apache.spark:spark-core_2.10:jar:1.6.0:compile
[INFO] | +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile
[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.5.11:compile
{code}
{code}
io.druid:druid-processing:jar:0.9.2.2.6.2.0-SNAPSHOT:compile
[INFO] | | \- org.roaringbitmap:RoaringBitmap:jar:0.5.18:compile
{code}
was (Author: bslim):
Druid is using 0.5.18, Spark 0.5.11 and Tez 0.4.9
{code}
org.apache.tez:tez-runtime-library:jar:0.8.4.2.6.2.0-SNAPSHOT:compile [INFO] |
+- org.roaringbitmap:RoaringBitmap:jar:0.4.9:compile
{code}
spark is using 0.5.11
{code}
INFO] +- org.apache.spark:spark-core_2.10:jar:1.6.0:compile
[INFO] | +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile
[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.5.11:compile
{code}
{code}
io.druid:druid-processing:jar:0.9.2.2.6.2.0-SNAPSHOT:compile
[INFO] | | \- org.roaringbitmap:RoaringBitmap:jar:0.5.18:compile
{code}
> Missmatch between roaring bitmap library used by druid and the one coming
> from tez
> --
>
> Key: HIVE-17303
> URL: https://issues.apache.org/jira/browse/HIVE-17303
> Project: Hive
> Issue Type: Bug
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
> Attachments: HIVE-17303.patch
>
>
> {code}
>
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.NoSuchMethodError:
> org.roaringbitmap.buffer.MutableRoaringBitmap.runOptimize()Z
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
> at
> org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
> at
> org.apache.hadoop.hive.druid.io.DruidRecordWriter.pushSegments(DruidRecordWriter.java:165)
> ... 25 more
> Caused by: java.lang.NoSuchMethodError:
> org.roaringbitmap.buffer.MutableRoaringBitmap.runOptimize()Z
> at
> org.apache.hive.druid.com.metamx.collections.bitmap.WrappedRoaringBitmap.toImmutableBitmap(WrappedRoaringBitmap.java:65)
> at
> org.apache.hive.druid.com.metamx.collections.bitmap.RoaringBitmapFactory.makeImmutableBitmap(RoaringBitmapFactory.java:88)
> at
> org.apache.hive.druid.io.druid.segment.StringDimensionMergerV9.writeIndexes(StringDimensionMergerV9.java:348)
> at
> org.apache.hive.druid.io.druid.segment.IndexMergerV9.makeIndexFiles(IndexMergerV9.java:218)
> at
> org.apache.hive.druid.io.druid.segment.IndexMerger.merge(IndexMerger.java:438)
> at
> org.apache.hive.druid.io.druid.segment.IndexMerger.persist(IndexMerger.java:186)
> at
> org.apache.hive.druid.io.druid.segment.IndexMerger.persist(IndexMerger.java:152)
> at
> org.apache.hive.druid.io.druid.segment.realtime.appenderator.AppenderatorImpl.persistHydrant(AppenderatorImpl.java:996)
> at
> org.apache.hive.druid.io.druid.segment.realtime.appenderator.AppenderatorImpl.access$200(AppenderatorImpl.java:93)
> at
> org.apache.hive.druid.io.druid.segment.realtime.appenderator.AppenderatorImpl$2.doCall(AppenderatorImpl.java:385)
> at
> org.apache.hive.druid.io.druid.common.guava.ThreadRenamingCallable.call(ThreadRenamingCallable.java:44)
> ... 4 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:89, Vertex vertex_1502470020457_0005_12_05 [Reducer 2]
> killed/failed due to:OWN_TASK_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:0 (state=08S01,code=2)
> Options
> Attachments
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17305) New insert overwrite dynamic partitions qtest need to have the golden file regenerated
[ https://issues.apache.org/jira/browse/HIVE-17305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barna Zsombor Klara updated HIVE-17305: --- Status: Patch Available (was: Open) > New insert overwrite dynamic partitions qtest need to have the golden file > regenerated > -- > > Key: HIVE-17305 > URL: https://issues.apache.org/jira/browse/HIVE-17305 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 3.0.0 >Reporter: Barna Zsombor Klara >Assignee: Barna Zsombor Klara >Priority: Trivial > Attachments: HIVE-17305.01.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17305) New insert overwrite dynamic partitions qtest need to have the golden file regenerated
[ https://issues.apache.org/jira/browse/HIVE-17305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124522#comment-16124522 ] Barna Zsombor Klara commented on HIVE-17305: [~pvary] could you please review? > New insert overwrite dynamic partitions qtest need to have the golden file > regenerated > -- > > Key: HIVE-17305 > URL: https://issues.apache.org/jira/browse/HIVE-17305 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 3.0.0 >Reporter: Barna Zsombor Klara >Assignee: Barna Zsombor Klara >Priority: Trivial > Attachments: HIVE-17305.01.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17305) New insert overwrite dynamic partitions qtest need to have the golden file regenerated
[ https://issues.apache.org/jira/browse/HIVE-17305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124521#comment-16124521 ] Barna Zsombor Klara commented on HIVE-17305: Only one qtest had to have its golden file regenerated. The other two were failing because of a regression. LineageState should retain information on paths even after optimisation or already captured lineage information will not be displayed. > New insert overwrite dynamic partitions qtest need to have the golden file > regenerated > -- > > Key: HIVE-17305 > URL: https://issues.apache.org/jira/browse/HIVE-17305 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 3.0.0 >Reporter: Barna Zsombor Klara >Assignee: Barna Zsombor Klara >Priority: Trivial > Attachments: HIVE-17305.01.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17305) New insert overwrite dynamic partitions qtest need to have the golden file regenerated
[ https://issues.apache.org/jira/browse/HIVE-17305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barna Zsombor Klara updated HIVE-17305: --- Attachment: HIVE-17305.01.patch > New insert overwrite dynamic partitions qtest need to have the golden file > regenerated > -- > > Key: HIVE-17305 > URL: https://issues.apache.org/jira/browse/HIVE-17305 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 3.0.0 >Reporter: Barna Zsombor Klara >Assignee: Barna Zsombor Klara >Priority: Trivial > Attachments: HIVE-17305.01.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-8600) Add option to log explain output for query
[ https://issues.apache.org/jira/browse/HIVE-8600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124504#comment-16124504 ] Lefty Leverenz commented on HIVE-8600: -- [~klcopp] documented *hive.log.explain.output* in the Explain doc and Configuration Properties (thanks, Karen!) and I added a sentence about it in the Hive Logging section of Getting Started. Here's a direct link to the parameter in Configuration Properties: * [hive.log.explain.output | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.log.explain.output] Removed the TODOC15 label. > Add option to log explain output for query > -- > > Key: HIVE-8600 > URL: https://issues.apache.org/jira/browse/HIVE-8600 > Project: Hive > Issue Type: Bug >Reporter: Mohit Sabharwal >Assignee: Mohit Sabharwal > Fix For: 1.1.0 > > Attachments: HIVE-8600.patch > > > When diagnosing issues, it is useful to have the explain output for the query > already available in log4j log. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-8600) Add option to log explain output for query
[ https://issues.apache.org/jira/browse/HIVE-8600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-8600: - Labels: (was: TODOC15) > Add option to log explain output for query > -- > > Key: HIVE-8600 > URL: https://issues.apache.org/jira/browse/HIVE-8600 > Project: Hive > Issue Type: Bug >Reporter: Mohit Sabharwal >Assignee: Mohit Sabharwal > Fix For: 1.1.0 > > Attachments: HIVE-8600.patch > > > When diagnosing issues, it is useful to have the explain output for the query > already available in log4j log. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17267) Make HMS Notification Listeners typesafe
[ https://issues.apache.org/jira/browse/HIVE-17267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124491#comment-16124491 ] Barna Zsombor Klara commented on HIVE-17267: Unit test failures should not be related. > Make HMS Notification Listeners typesafe > > > Key: HIVE-17267 > URL: https://issues.apache.org/jira/browse/HIVE-17267 > Project: Hive > Issue Type: Bug >Reporter: Barna Zsombor Klara >Assignee: Barna Zsombor Klara > Attachments: HIVE-17267.01.patch, HIVE-17267.02.patch, > HIVE-17267.03.patch > > > Currently in the HMS we support two types of notification listeners, > transactional and non-transactional ones. Transactional listeners will only > be invoked if the jdbc transaction finished successfully while > non-transactional ones are supposed to be resilient and will be invoked in > any case, even for failures. > Having the same type for these two is a source of confusion and opens the > door for misconfigurations. We should try to fix this. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-17305) New insert overwrite dynamic partitions qtest need to have the golden file regenerated
[ https://issues.apache.org/jira/browse/HIVE-17305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barna Zsombor Klara reassigned HIVE-17305: -- > New insert overwrite dynamic partitions qtest need to have the golden file > regenerated > -- > > Key: HIVE-17305 > URL: https://issues.apache.org/jira/browse/HIVE-17305 > Project: Hive > Issue Type: Bug > Components: Tests >Affects Versions: 3.0.0 >Reporter: Barna Zsombor Klara >Assignee: Barna Zsombor Klara >Priority: Trivial > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17089) make acid 2.0 the default
[
https://issues.apache.org/jira/browse/HIVE-17089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124480#comment-16124480
]
Hive QA commented on HIVE-17089:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12881561/HIVE-17089.11.patch
{color:green}SUCCESS:{color} +1 due to 13 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 10968 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only]
(batchId=243)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[lateral_view_cp]
(batchId=82)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_mapjoin_only]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=100)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=235)
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning
(batchId=291)
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testRecordReaderIncompleteDelta
(batchId=264)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=180)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=180)
org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithMr.testFetchResultsOfLogWithNoneMode
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6364/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6364/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6364/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12881561 - PreCommit-HIVE-Build
> make acid 2.0 the default
> -
>
> Key: HIVE-17089
> URL: https://issues.apache.org/jira/browse/HIVE-17089
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-17089.01.patch, HIVE-17089.03.patch,
> HIVE-17089.05.patch, HIVE-17089.06.patch, HIVE-17089.07.patch,
> HIVE-17089.10.patch, HIVE-17089.10.patch, HIVE-17089.11.patch
>
>
> acid 2.0 is introduced in HIVE-14035. It replaces Update events with a
> combination of Delete + Insert events. This now makes U=D+I the default (and
> only) supported acid table type in Hive 3.0.
> The expectation for upgrade is that Major compaction has to be run on all
> acid tables in the existing Hive cluster and that no new writes to these
> table take place since the start of compaction (Need to add a mechanism to
> put a table in read-only mode - this way it can still be read while it's
> being compacted). Then upgrade to Hive 3.0 can take place.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
