[jira] [Commented] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[
https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254893#comment-16254893
]
Hive QA commented on HIVE-16756:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897924/HIVE-16756.03.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 11382 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7847/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7847/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7847/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897924 - PreCommit-HIVE-Build
> Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException:
> / by zero"
>
>
> Key: HIVE-16756
> URL: https://issues.apache.org/jira/browse/HIVE-16756
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.3.0
>Reporter: Matt McCline
>Assignee: Vihang Karajgaonkar
>Priority: Critical
> Attachments: HIVE-16756.01.patch, HIVE-16756.02.patch,
> HIVE-16756.03.patch
>
>
> vectorization_div0.q needs to test the long data type testing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[
https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254840#comment-16254840
]
Hive QA commented on HIVE-16756:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897924/HIVE-16756.03.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 11382 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7846/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7846/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7846/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897924 - PreCommit-HIVE-Build
> Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException:
> / by zero"
>
>
> Key: HIVE-16756
> URL: https://issues.apache.org/jira/browse/HIVE-16756
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.3.0
>Reporter: Matt McCline
>Assignee: Vihang Karajgaonkar
>Priority: Critical
> Attachments: HIVE-16756.01.patch, HIVE-16756.02.patch,
> HIVE-16756.03.patch
>
>
> vectorization_div0.q needs to test the long data type testing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HIVE-18080) Performance degradation on VectorizedLogicBench#IfExprLongColumnLongColumnBench when AVX512 is enabled
[
https://issues.apache.org/jira/browse/HIVE-18080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254829#comment-16254829
]
liyunzhang edited comment on HIVE-18080 at 11/16/17 6:41 AM:
-
[~teddy.choi]: I retested
{{VectorizedLogicBench#IfExprLongColumnLongColumnBench}},{{VectorizedLogicBench#IfExprRepeatingLongColumnLongColumnBench}}
and
{{VectorizedLogicBench#IfExprLongColumnRepeatingLongColumnBench}} in AVX1 and
AVX2
the result is
AVX1
{code}
o.a.h.b.v.VectorizedLogicBench.IfExprLongColumnLongColumnBench.bench
avgt 20 1595748.343 ± 16887.073 us/op
o.a.h.b.v.VectorizedLogicBench.IfExprLongColumnRepeatingLongColumnBench.bench
avgt 20 1735827.809 ± 18129.173 us/op
o.a.h.b.v.VectorizedLogicBench.IfExprRepeatingLongColumnLongColumnBench.bench
avgt 20 1768004.314 ± 14489.511 us/op
{code}
AVX2
{code}
o.a.h.b.v.VectorizedLogicBench.IfExprLongColumnLongColumnBench.bench
avgt 20 1691559.843 ± 118986.372 us/op
o.a.h.b.v.VectorizedLogicBench.IfExprLongColumnRepeatingLongColumnBench.bench
avgt 20 1837327.456 ± 76084.038 us/op
o.a.h.b.v.VectorizedLogicBench.IfExprRepeatingLongColumnLongColumnBench.bench
avgt 20 1760544.684 ± 93512.838 us/op
{code}
the test script I used
{code}
export JAVA_HOME=/home/zly/sr601/jmh/jdk-9.0.1/
export PATH=$JAVA_HOME/bin:$PATH
export LD_LIBRARY_PATH=/home/zly/sr601/jmh/jdk-9.0.1/mylib
for i in 0 1 2; do
java -server -XX:UseAVX=1 -jar benchmarks.jar
org.apache.hive.benchmark.vectorization.VectorizedLogicBench * -wi 10 -i 20 -f
1 -bm avgt -tu us >log.logic.avx1.single.$i & export pid=$!
taskset -cp 1 $pid
wait $pid
done
for i in 0 1 2; do
java -server -XX:UseAVX=2 -jar benchmarks.jar
org.apache.hive.benchmark.vectorization.VectorizedLogicBench * -wi 10 -i 20 -f
1 -bm avgt -tu us >log.logic.avx2.single.$i & export pid=$!
taskset -cp 1 $pid
wait $pid
done
{code}
It seems that no much improvement comparing AVX1 and AVX2. Can you spend some
time to help find the root cause? Thanks!
was (Author: kellyzly):
[~teddy.choi]: I retested
{{VectorizedLogicBench#IfExprLongColumnLongColumnBench}},{{VectorizedLogicBench#IfExprRepeatingLongColumnLongColumnBench}}
and
{{VectorizedLogicBench#IfExprLongColumnRepeatingLongColumnBench}} in AVX1 and
AVX2
the result is
AVX1
{code}
o.a.h.b.v.VectorizedLogicBench.IfExprLongColumnLongColumnBench.bench
avgt 20 1595748.343 ± 16887.073 us/op
o.a.h.b.v.VectorizedLogicBench.IfExprLongColumnRepeatingLongColumnBench.bench
avgt 20 1735827.809 ± 18129.173 us/op
o.a.h.b.v.VectorizedLogicBench.IfExprRepeatingLongColumnLongColumnBench.bench
avgt 20 1768004.314 ± 14489.511 us/op
{code}
AVX2
{code}
o.a.h.b.v.VectorizedLogicBench.IfExprLongColumnLongColumnBench.bench
avgt 20 1691559.843 ± 118986.372 us/op
o.a.h.b.v.VectorizedLogicBench.IfExprLongColumnRepeatingLongColumnBench.bench
avgt 20 1837327.456 ± 76084.038 us/op
o.a.h.b.v.VectorizedLogicBench.IfExprRepeatingLongColumnLongColumnBench.bench
avgt 20 1760544.684 ± 93512.838 us/op
{code}
It seems that no much improvement comparing AVX1 and AVX2. Can you spend some
time to help find the root cause? Thanks!
> Performance degradation on
> VectorizedLogicBench#IfExprLongColumnLongColumnBench when AVX512 is enabled
> --
>
> Key: HIVE-18080
> URL: https://issues.apache.org/jira/browse/HIVE-18080
> Project: Hive
> Issue Type: Bug
>Reporter: liyunzhang
>
> Use Xeon(R) Platinum 8180 CPU to test the performance of
> [AVX512|https://en.wikipedia.org/wiki/AVX-512].
> {code}
> #cat /proc/cpuinfo |grep "model name"|head -n 1
> model name: Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz
> {code}
> Before that I have compiled hive with JDK9 as JDK9 enables AVX512
> Use hive microbenchmark(HIVE-10189) to evaluate the performance improvement.
> It seems performance(20%+) in cases in
> {{VectorizedArithmeticBench}},{{VectorizedComparisonBench}},{{VectorizedLikeBench}},{{VectorizedLogicBench}}
> execpt
> {{VectorizedLogicBench#IfExprLongColumnLongColumnBench}},{{VectorizedLogicBench#IfExprRepeatingLongColumnLongColumnBench}}
> and
> {{VectorizedLogicBench#IfExprLongColumnRepeatingLongColumnBench}}.The data is
> like following
> When i use Skylake CPU to evaluate the performance improvement of AVX512.
> I found the performance in VectorizedLogicBench is like following
> || ||AVX2 us/op||AVX512 us/op || (AVX2-AVX512)/AVX2||
> |ColAndColBench|122510| 87014| 28.9%|
> |IfExprLongColumnLongColumnBench | 1325759| 1436073| -8.3% |
> |IfExprLongColumnRepeatingLongColumnBench|1397447|1480450| -5.9%|
> |IfExprRepeatingLongColumnLongColumnBench|1401164|1483062| -5.9% |
>
[jira] [Commented] (HIVE-18080) Performance degradation on VectorizedLogicBench#IfExprLongColumnLongColumnBench when AVX512 is enabled
[
https://issues.apache.org/jira/browse/HIVE-18080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254829#comment-16254829
]
liyunzhang commented on HIVE-18080:
---
[~teddy.choi]: I retested
{{VectorizedLogicBench#IfExprLongColumnLongColumnBench}},{{VectorizedLogicBench#IfExprRepeatingLongColumnLongColumnBench}}
and
{{VectorizedLogicBench#IfExprLongColumnRepeatingLongColumnBench}} in AVX1 and
AVX2
the result is
AVX1
{code}
o.a.h.b.v.VectorizedLogicBench.IfExprLongColumnLongColumnBench.bench
avgt 20 1595748.343 ± 16887.073 us/op
o.a.h.b.v.VectorizedLogicBench.IfExprLongColumnRepeatingLongColumnBench.bench
avgt 20 1735827.809 ± 18129.173 us/op
o.a.h.b.v.VectorizedLogicBench.IfExprRepeatingLongColumnLongColumnBench.bench
avgt 20 1768004.314 ± 14489.511 us/op
{code}
AVX2
{code}
o.a.h.b.v.VectorizedLogicBench.IfExprLongColumnLongColumnBench.bench
avgt 20 1691559.843 ± 118986.372 us/op
o.a.h.b.v.VectorizedLogicBench.IfExprLongColumnRepeatingLongColumnBench.bench
avgt 20 1837327.456 ± 76084.038 us/op
o.a.h.b.v.VectorizedLogicBench.IfExprRepeatingLongColumnLongColumnBench.bench
avgt 20 1760544.684 ± 93512.838 us/op
{code}
It seems that no much improvement comparing AVX1 and AVX2. Can you spend some
time to help find the root cause? Thanks!
> Performance degradation on
> VectorizedLogicBench#IfExprLongColumnLongColumnBench when AVX512 is enabled
> --
>
> Key: HIVE-18080
> URL: https://issues.apache.org/jira/browse/HIVE-18080
> Project: Hive
> Issue Type: Bug
>Reporter: liyunzhang
>
> Use Xeon(R) Platinum 8180 CPU to test the performance of
> [AVX512|https://en.wikipedia.org/wiki/AVX-512].
> {code}
> #cat /proc/cpuinfo |grep "model name"|head -n 1
> model name: Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz
> {code}
> Before that I have compiled hive with JDK9 as JDK9 enables AVX512
> Use hive microbenchmark(HIVE-10189) to evaluate the performance improvement.
> It seems performance(20%+) in cases in
> {{VectorizedArithmeticBench}},{{VectorizedComparisonBench}},{{VectorizedLikeBench}},{{VectorizedLogicBench}}
> execpt
> {{VectorizedLogicBench#IfExprLongColumnLongColumnBench}},{{VectorizedLogicBench#IfExprRepeatingLongColumnLongColumnBench}}
> and
> {{VectorizedLogicBench#IfExprLongColumnRepeatingLongColumnBench}}.The data is
> like following
> When i use Skylake CPU to evaluate the performance improvement of AVX512.
> I found the performance in VectorizedLogicBench is like following
> || ||AVX2 us/op||AVX512 us/op || (AVX2-AVX512)/AVX2||
> |ColAndColBench|122510| 87014| 28.9%|
> |IfExprLongColumnLongColumnBench | 1325759| 1436073| -8.3% |
> |IfExprLongColumnRepeatingLongColumnBench|1397447|1480450| -5.9%|
> |IfExprRepeatingLongColumnLongColumnBench|1401164|1483062| -5.9% |
> |NotColBench|77042.83|51513.28| 33%|
> There are degradation in
> IfExprLongColumnLongColumnBench,IfExprLongColumnRepeatingLongColumnBench,
> IfExprRepeatingLongColumnLongColumnBench, very confused why there is
> degradation on IfExprLongColumnLongColumnBench cases.
> Here we use {{taskset -cp 1 $pid}} to run the benchmark on single core to
> avoid the impact of dynamic CPU frequency scaling.
> my script
> {code}
> export JAVA_HOME=/home/zly/jdk-9.0.1/
> export PATH=$JAVA_HOME/bin:$PATH
> export LD_LIBRARY_PATH=/home/zly/jdk-9.0.1/mylib
> for i in 0 1 2; do
> java -server -XX:UseAVX=3 -jar benchmarks.jar
> org.apache.hive.benchmark.vectorization.VectorizedLogicBench * -wi 10 -i 20
> -f 1 -bm avgt -tu us >log.logic.avx3.single.$i & export pid=$!
> taskset -cp 1 $pid
> wait $pid
> done
> for i in 0 1 2; do
> java -server -XX:UseAVX=2 -jar benchmarks.jar
> org.apache.hive.benchmark.vectorization.VectorizedLogicBench * -wi 10 -i 20
> -f 1 -bm avgt -tu us >log.logic.avx2.single.$i & export pid=$!
> taskset -cp 1 $pid
> wait $pid
> done
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HIVE-12746) when dropping external hive tables，hive metastore should not check the hdfs path write permission
[ https://issues.apache.org/jira/browse/HIVE-12746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-12746. --- Resolution: Duplicate Resolving this as a duplicate of HIVE-9020 > when dropping external hive tables，hive metastore should not check the hdfs > path write permission > - > > Key: HIVE-12746 > URL: https://issues.apache.org/jira/browse/HIVE-12746 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 1.2.1 > Environment: hive1.2.1 hadoop2.6 >Reporter: wangfeng >Priority: Critical > Labels: hdfspermission, metastore > Original Estimate: 168h > Remaining Estimate: 168h > > 1 user1 has readonly permission on hdfs path '/user/www/seller_shop_info'; > 2 user1 create external table seller_shop_info on the hdfs path; > 3 user1 drop the exernal table seller_shop_info > then problem occurred! > hive> drop table seller_shop_info; > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Table metadata > not deleted since hdfs://argo/user/www/seller_shop_info is not writable by > user1) > because when dropping external table,hive doesnot delete hdfs path,so hive > metastore should not check the hdfs write permission -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2
[
https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254790#comment-16254790
]
Hive QA commented on HIVE-18072:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897912/HIVE-18072.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 73 failed/errored test(s), 11138 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=150)
[skewjoinopt15.q,vector_coalesce.q,dp_counter_mm.q,orc_ppd_decimal.q,cbo_rp_lineage2.q,insert_into_with_schema.q,join_emit_interval.q,join1.q,join32_lessSize.q,vector_null_projection.q,vector_groupby_grouping_sets_grouping.q,vector_cast_constant.q,mapjoin2.q,orc_predicate_pushdown.q,cte_mat_4.q,correlationoptimizer4.q,vector_windowing_navfn.q,vectorization_12.q,vector_number_compare_projection.q,orc_merge_incompat3.q,dynamic_semijoin_reduction_sw.q,vector_leftsemi_mapjoin.q,update_all_non_partitioned.q,tez_insert_overwrite_local_directory_1.q,column_table_stats_orc.q,schema_evol_text_vecrow_table.q,vector_if_expr.q,delete_whole_partition.q,bucketsortoptimize_insert_6.q,ctas.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=155)
[mapjoin_emit_interval.q,vector_auto_smb_mapjoin_14.q,deleteAnalyze.q,tez_dml.q,database.q,vector_bround.q,vector_join30.q,smb_mapjoin_6.q,vector_reduce_groupby_decimal.q,vectorized_dynamic_partition_pruning.q,cbo_views.q,vectorization_part.q,dynamic_partition_pruning.q,cte_mat_1.q,cluster.q,vector_char_mapjoin1.q,cte_5.q,subquery_shared_alias.q,vector_decimal_2.q,bucketmapjoin7.q,vector_outer_join3.q,vector_string_concat.q,subquery_nested_subquery.q,limit_pushdown3.q,vector_outer_join2.q,vectorized_context.q,metadata_only_queries.q,auto_sortmerge_join_12.q,union6.q,vectorized_timestamp_funcs.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=159)
[dynamic_semijoin_reduction.q,vectorization_pushdown.q,correlationoptimizer2.q,cbo_subq_not_in.q,cbo_gby_empty.q,resourceplan.q,vectorization_short_regress.q,identity_project_remove_skip.q,mapjoin3.q,cross_product_check_1.q,bucket2.q,cbo_join.q,schema_evol_text_nonvec_part_all_primitive.q,correlationoptimizer6.q,union_remove_26.q,cbo_rp_limit.q,vector_groupby_cube1.q,vectorization_13.q,mapjoin_mapjoin.q,union2.q,groupby2.q,vector_decimal_precision.q,dynpart_sort_opt_vectorization.q,exchgpartition2lel.q,sample10.q,vectorized_timestamp_ints_casts.q,auto_sortmerge_join_2.q,bucketizedhiveinputformat.q,cte_mat_2.q,vectorization_8.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=161)
[load_dyn_part2.q,parquet_types.q,vectorization_5.q,mapjoin46.q,jdbc_handler.q,column_access_stats.q,vector_between_in.q,subquery_in_having.q,vectorized_string_funcs.q,vectorization_1.q,auto_join_filters.q,join0.q,vector_coalesce_3.q,mergejoin.q,stats_noscan_1.q,orc_split_elimination.q,vector_decimal_10_0.q,vector_outer_join0.q,vector_udf1.q,vector_groupby_reduce.q,union8.q,vector_complex_all.q,insert_values_orig_table.q,correlationoptimizer1.q,windowing_windowspec2.q,vector_decimal_mapjoin.q,auto_smb_mapjoin_14.q,vectorized_case.q,vector_join_filters.q,vector_windowing_rank.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=162)
[auto_sortmerge_join_13.q,union_top_level.q,schema_evol_orc_acid_table_update.q,vector_left_outer_join2.q,schema_evol_text_vecrow_part_all_primitive.q,constprog_semijoin.q,update_where_partitioned.q,vectorized_multi_output_select.q,drop_partition_with_stats.q,smb_mapjoin_14.q,skiphf_aggr.q,vectorized_ptf.q,alter_merge_orc.q,vector_char_cast.q,join_reordering_no_stats.q,insert_orig_table.q,join_filters.q,subquery_in.q,schema_evol_text_vec_part_all_primitive.q,auto_sortmerge_join_4.q,bucket_many.q,vectorization_15.q,union3.q,union7.q,vectorization_nested_udf.q,vector_groupby_3.q,insert_values_orig_table_use_metadata.q,vector_mr_diff_schema_alias.q,cbo_rp_gby.q,reduce_deduplicate_extended.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=163)
[jira] [Commented] (HIVE-18080) Performance degradation on VectorizedLogicBench#IfExprLongColumnLongColumnBench when AVX512 is enabled
[
https://issues.apache.org/jira/browse/HIVE-18080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254783#comment-16254783
]
liyunzhang commented on HIVE-18080:
---
[~teddy.choi]: Can you help view it as you are familiar with the code, thanks!
> Performance degradation on
> VectorizedLogicBench#IfExprLongColumnLongColumnBench when AVX512 is enabled
> --
>
> Key: HIVE-18080
> URL: https://issues.apache.org/jira/browse/HIVE-18080
> Project: Hive
> Issue Type: Bug
>Reporter: liyunzhang
>
> Use Xeon(R) Platinum 8180 CPU to test the performance of
> [AVX512|https://en.wikipedia.org/wiki/AVX-512].
> {code}
> #cat /proc/cpuinfo |grep "model name"|head -n 1
> model name: Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz
> {code}
> Before that I have compiled hive with JDK9 as JDK9 enables AVX512
> Use hive microbenchmark(HIVE-10189) to evaluate the performance improvement.
> It seems performance(20%+) in cases in
> {{VectorizedArithmeticBench}},{{VectorizedComparisonBench}},{{VectorizedLikeBench}},{{VectorizedLogicBench}}
> execpt
> {{VectorizedLogicBench#IfExprLongColumnLongColumnBench}},{{VectorizedLogicBench#IfExprRepeatingLongColumnLongColumnBench}}
> and
> {{VectorizedLogicBench#IfExprLongColumnRepeatingLongColumnBench}}.The data is
> like following
> When i use Skylake CPU to evaluate the performance improvement of AVX512.
> I found the performance in VectorizedLogicBench is like following
> || ||AVX2 us/op||AVX512 us/op || (AVX2-AVX512)/AVX2||
> |ColAndColBench|122510| 87014| 28.9%|
> |IfExprLongColumnLongColumnBench | 1325759| 1436073| -8.3% |
> |IfExprLongColumnRepeatingLongColumnBench|1397447|1480450| -5.9%|
> |IfExprRepeatingLongColumnLongColumnBench|1401164|1483062| -5.9% |
> |NotColBench|77042.83|51513.28| 33%|
> There are degradation in
> IfExprLongColumnLongColumnBench,IfExprLongColumnRepeatingLongColumnBench,
> IfExprRepeatingLongColumnLongColumnBench, very confused why there is
> degradation on IfExprLongColumnLongColumnBench cases.
> Here we use {{taskset -cp 1 $pid}} to run the benchmark on single core to
> avoid the impact of dynamic CPU frequency scaling.
> my script
> {code}
> export JAVA_HOME=/home/zly/jdk-9.0.1/
> export PATH=$JAVA_HOME/bin:$PATH
> export LD_LIBRARY_PATH=/home/zly/jdk-9.0.1/mylib
> for i in 0 1 2; do
> java -server -XX:UseAVX=3 -jar benchmarks.jar
> org.apache.hive.benchmark.vectorization.VectorizedLogicBench * -wi 10 -i 20
> -f 1 -bm avgt -tu us >log.logic.avx3.single.$i & export pid=$!
> taskset -cp 1 $pid
> wait $pid
> done
> for i in 0 1 2; do
> java -server -XX:UseAVX=2 -jar benchmarks.jar
> org.apache.hive.benchmark.vectorization.VectorizedLogicBench * -wi 10 -i 20
> -f 1 -bm avgt -tu us >log.logic.avx2.single.$i & export pid=$!
> taskset -cp 1 $pid
> wait $pid
> done
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18077: Status: Patch Available (was: In Progress) > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18077.01.patch, HIVE-18077.02.patch > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18077: Attachment: HIVE-18077.02.patch > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18077.01.patch, HIVE-18077.02.patch > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18077: Status: In Progress (was: Patch Available) > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18077.01.patch > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18079) Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator bit-size
[ https://issues.apache.org/jira/browse/HIVE-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned HIVE-18079: -- Assignee: Gopal V > Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator > bit-size > > > Key: HIVE-18079 > URL: https://issues.apache.org/jira/browse/HIVE-18079 > Project: Hive > Issue Type: Improvement >Reporter: Gopal V >Assignee: Gopal V > > HyperLogLog can merge a 14 bit HLL into a 10 bit HLL bitset, because of its > mathematical hash distribution & construction. > Allow the squashing of a 14 bit HLL -> 10 bit HLL without needing a second > scan over the data-set. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[ https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-16756: --- Attachment: (was: HIVE-16756.03.patch) > Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: > / by zero" > > > Key: HIVE-16756 > URL: https://issues.apache.org/jira/browse/HIVE-16756 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-16756.01.patch, HIVE-16756.02.patch, > HIVE-16756.03.patch > > > vectorization_div0.q needs to test the long data type testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[ https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-16756: --- Attachment: HIVE-16756.03.patch > Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: > / by zero" > > > Key: HIVE-16756 > URL: https://issues.apache.org/jira/browse/HIVE-16756 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-16756.01.patch, HIVE-16756.02.patch, > HIVE-16756.03.patch > > > vectorization_div0.q needs to test the long data type testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[
https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-16756:
---
Attachment: HIVE-16756.03.patch
third version includes a junit test for the newly added class for
{{LongColModuloLongColumn.java}}
> Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException:
> / by zero"
>
>
> Key: HIVE-16756
> URL: https://issues.apache.org/jira/browse/HIVE-16756
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.3.0
>Reporter: Matt McCline
>Assignee: Vihang Karajgaonkar
>Priority: Critical
> Attachments: HIVE-16756.01.patch, HIVE-16756.02.patch,
> HIVE-16756.03.patch
>
>
> vectorization_div0.q needs to test the long data type testing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18071) add HS2 jmx information about pools and current resource plan
[
https://issues.apache.org/jira/browse/HIVE-18071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254743#comment-16254743
]
Hive QA commented on HIVE-18071:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897910/HIVE-18071.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 11381 tests
executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=102)
[explainanalyze_4.q,acid_vectorization_original_tez.q,orc_merge12.q,explainanalyze_2.q,vectorization_div0.q,tez_union_with_udf.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7844/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7844/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7844/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897910 - PreCommit-HIVE-Build
> add HS2 jmx information about pools and current resource plan
> -
>
> Key: HIVE-18071
> URL: https://issues.apache.org/jira/browse/HIVE-18071
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-18071.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2
[ https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18072: Status: Patch Available (was: Open) > WM - fix various bugs based on cluster testing - part 2 > --- > > Key: HIVE-18072 > URL: https://issues.apache.org/jira/browse/HIVE-18072 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-18072.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2
[ https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18072: Attachment: HIVE-18072.patch [~prasanth_j] another one :) I will post an RB for this one > WM - fix various bugs based on cluster testing - part 2 > --- > > Key: HIVE-18072 > URL: https://issues.apache.org/jira/browse/HIVE-18072 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-18072.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17717) Enable rule to push post-aggregations into Druid
[
https://issues.apache.org/jira/browse/HIVE-17717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254694#comment-16254694
]
Hive QA commented on HIVE-17717:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897881/HIVE-17717.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 11388 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7840/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7840/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7840/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897881 - PreCommit-HIVE-Build
> Enable rule to push post-aggregations into Druid
>
>
> Key: HIVE-17717
> URL: https://issues.apache.org/jira/browse/HIVE-17717
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-17717.01.patch, HIVE-17717.patch
>
>
> Enable rule created by CALCITE-1803.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18071) add HS2 jmx information about pools and current resource plan
[ https://issues.apache.org/jira/browse/HIVE-18071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18071: Attachment: HIVE-18071.patch [~prasanth_j] can you take a look? > add HS2 jmx information about pools and current resource plan > - > > Key: HIVE-18071 > URL: https://issues.apache.org/jira/browse/HIVE-18071 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-18071.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18071) add HS2 jmx information about pools and current resource plan
[ https://issues.apache.org/jira/browse/HIVE-18071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18071: Status: Patch Available (was: Open) > add HS2 jmx information about pools and current resource plan > - > > Key: HIVE-18071 > URL: https://issues.apache.org/jira/browse/HIVE-18071 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-18071.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17856) MM tables - IOW is not ACID compliant
[
https://issues.apache.org/jira/browse/HIVE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254682#comment-16254682
]
Steve Yeom commented on HIVE-17856:
---
I have been working on a new version of patch, which will reflect on your
review and
also will clear all (the consistently occurring) mm_all.q and mm_loaddata.q
failures during pre-commit test.
I am in the last phase.
> MM tables - IOW is not ACID compliant
> -
>
> Key: HIVE-17856
> URL: https://issues.apache.org/jira/browse/HIVE-17856
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
> Labels: mm-gap-1
> Attachments: HIVE-17856.1.patch, HIVE-17856.2.patch,
> HIVE-17856.3.patch, HIVE-17856.4.patch, HIVE-17856.5.patch,
> HIVE-17856.6.patch, HIVE-17856.7.patch, HIVE-17856.8.patch, HIVE-17856.9.patch
>
>
> The following tests were removed from mm_all during "integration"... I should
> have never allowed such manner of intergration.
> MM logic should have been kept intact until ACID logic could catch up. Alas,
> here we are.
> {noformat}
> drop table iow0_mm;
> create table iow0_mm(key int) tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow0_mm select key from intermediate;
> insert into table iow0_mm select key + 1 from intermediate;
> select * from iow0_mm order by key;
> insert overwrite table iow0_mm select key + 2 from intermediate;
> select * from iow0_mm order by key;
> drop table iow0_mm;
> drop table iow1_mm;
> create table iow1_mm(key int) partitioned by (key2 int)
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow1_mm partition (key2)
> select key as k1, key from intermediate union all select key as k1, key from
> intermediate;
> insert into table iow1_mm partition (key2)
> select key + 1 as k1, key from intermediate union all select key as k1, key
> from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key from intermediate union all select key + 4 as k1,
> key from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key + 3 from intermediate union all select key + 2 as
> k1, key + 2 from intermediate;
> select * from iow1_mm order by key, key2;
> drop table iow1_mm;
> {noformat}
> {noformat}
> drop table simple_mm;
> create table simple_mm(key int) stored as orc tblproperties
> ("transactional"="true", "transactional_properties"="insert_only");
> insert into table simple_mm select key from intermediate;
> -insert overwrite table simple_mm select key from intermediate;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17856) MM tables - IOW is not ACID compliant
[
https://issues.apache.org/jira/browse/HIVE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-17856:
Attachment: HIVE-17856.9.patch
The same patch again, not sure why HiveQA is having trouble with it.
[~steveyeom2017] can you update review board? looks like the last thing there
are the 4 last comments I left. Do you think it's ready for go w.r.t. test
fixes?
> MM tables - IOW is not ACID compliant
> -
>
> Key: HIVE-17856
> URL: https://issues.apache.org/jira/browse/HIVE-17856
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
> Labels: mm-gap-1
> Attachments: HIVE-17856.1.patch, HIVE-17856.2.patch,
> HIVE-17856.3.patch, HIVE-17856.4.patch, HIVE-17856.5.patch,
> HIVE-17856.6.patch, HIVE-17856.7.patch, HIVE-17856.8.patch, HIVE-17856.9.patch
>
>
> The following tests were removed from mm_all during "integration"... I should
> have never allowed such manner of intergration.
> MM logic should have been kept intact until ACID logic could catch up. Alas,
> here we are.
> {noformat}
> drop table iow0_mm;
> create table iow0_mm(key int) tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow0_mm select key from intermediate;
> insert into table iow0_mm select key + 1 from intermediate;
> select * from iow0_mm order by key;
> insert overwrite table iow0_mm select key + 2 from intermediate;
> select * from iow0_mm order by key;
> drop table iow0_mm;
> drop table iow1_mm;
> create table iow1_mm(key int) partitioned by (key2 int)
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow1_mm partition (key2)
> select key as k1, key from intermediate union all select key as k1, key from
> intermediate;
> insert into table iow1_mm partition (key2)
> select key + 1 as k1, key from intermediate union all select key as k1, key
> from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key from intermediate union all select key + 4 as k1,
> key from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key + 3 from intermediate union all select key + 2 as
> k1, key + 2 from intermediate;
> select * from iow1_mm order by key, key2;
> drop table iow1_mm;
> {noformat}
> {noformat}
> drop table simple_mm;
> create table simple_mm(key int) stored as orc tblproperties
> ("transactional"="true", "transactional_properties"="insert_only");
> insert into table simple_mm select key from intermediate;
> -insert overwrite table simple_mm select key from intermediate;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17193) HoS: don't combine map works that are targets of different DPPs
[
https://issues.apache.org/jira/browse/HIVE-17193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254659#comment-16254659
]
Rui Li commented on HIVE-17193:
---
The test failures are not related.
[~kellyzly], [~stakiar], [~xuefuz] could you take a look? Thanks.
> HoS: don't combine map works that are targets of different DPPs
> ---
>
> Key: HIVE-17193
> URL: https://issues.apache.org/jira/browse/HIVE-17193
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-17193.1.patch, HIVE-17193.2.patch
>
>
> Suppose {{srcpart}} is partitioned by {{ds}}. The following query can trigger
> the issue:
> {code}
> explain
> select * from
> (select srcpart.ds,srcpart.key from srcpart join src on srcpart.ds=src.key)
> a
> join
> (select srcpart.ds,srcpart.key from srcpart join src on
> srcpart.ds=src.value) b
> on a.key=b.key;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18075) verify commands on a cluster
[ https://issues.apache.org/jira/browse/HIVE-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254658#comment-16254658 ] Sergey Shelukhin commented on HIVE-18075: - [~harishjp] fyi. Some of these might be bogus. However some cases are not included in the q file, as far as I see, so they could be real issues. > verify commands on a cluster > > > Key: HIVE-18075 > URL: https://issues.apache.org/jira/browse/HIVE-18075 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin > > I was running the commands in the cluster, with potentially a slightly > outdated version of the DB on mysql (yet, the DB was created with a script > from master and I didn't hit any schema issues, so it's fairly recent); > however, recent master code + the pools commands patch. I've hit the > following issues. > # Cannot drop pool or RP with a mapping (see also 3). > # Cannot drop pool that is set as default (probably correct, but the error > message is bad). > # When I dropped an RP with a mapping, and then created it again with the > same name, the pool creation in that RP would fail with an error that a > unique query returned multiple results. In the DB, there were actually 2 RPs > with the same name. Not sure how exactly that happened, there might have been > intermediate states, but I didn't mess with mysql. I think the name > uniqueness is either missing from some script or doesn't work. > # Setting RP default pool no longer works. I think I might have broken it > with one of the rebases in that area, but it could also be something else (or > like other things, it works in q tests but not on cluster for whatever > reason). > # Resource plan rename doesn't check the disable state. It probably should. > Also need to see for other commands. > Need to figure out which are real issues and which aren't and fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17964) HoS: some spark configs doesn't require re-creating a session
[
https://issues.apache.org/jira/browse/HIVE-17964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-17964:
--
Attachment: HIVE-17964.2.patch
> HoS: some spark configs doesn't require re-creating a session
> -
>
> Key: HIVE-17964
> URL: https://issues.apache.org/jira/browse/HIVE-17964
> Project: Hive
> Issue Type: Improvement
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Minor
> Attachments: HIVE-17964.1.patch, HIVE-17964.2.patch,
> HIVE-17964.2.patch
>
>
> I guess the {{hive.spark.}} configs were initially intended for the RSC.
> Therefore when they're changed, we'll re-create the session for them to take
> effect. There're some configs not related to RSC that also start with
> {{hive.spark.}}. We'd better rename them so that we don't unnecessarily
> re-create sessions, which is usually time consuming.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17361) Support LOAD DATA for transactional tables
[
https://issues.apache.org/jira/browse/HIVE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254646#comment-16254646
]
Hive QA commented on HIVE-17361:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897868/HIVE-17361.12.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 11392 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_default] (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_25]
(batchId=86)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[load_data_into_acid]
(batchId=91)
org.apache.hadoop.hive.ql.TestTxnLoadData.loadData (batchId=254)
org.apache.hadoop.hive.ql.TestTxnLoadData.loadDataPartitioned (batchId=254)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testACIDReaderFooterSerializeWithDeltas
(batchId=267)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testACIDReaderNoFooterSerializeWithDeltas
(batchId=267)
org.apache.hadoop.hive.ql.io.orc.TestInputOutputFormat.testEtlCombinedStrategy
(batchId=267)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testMapNullKey[3]
(batchId=187)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testSyntheticComplexSchema[2]
(batchId=187)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testTupleInBagInTupleInBag[2]
(batchId=187)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7839/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7839/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7839/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897868 - PreCommit-HIVE-Build
> Support LOAD DATA for transactional tables
> --
>
> Key: HIVE-17361
> URL: https://issues.apache.org/jira/browse/HIVE-17361
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Reporter: Wei Zheng
>Assignee: Eugene Koifman
>Priority: Critical
> Attachments: HIVE-17361.07.patch, HIVE-17361.08.patch,
> HIVE-17361.09.patch, HIVE-17361.1.patch, HIVE-17361.10.patch,
> HIVE-17361.11.patch, HIVE-17361.12.patch, HIVE-17361.2.patch,
> HIVE-17361.3.patch, HIVE-17361.4.patch
>
>
> LOAD DATA was not supported since ACID was introduced. Need to fill this gap
> between ACID table and regular hive table.
> Current Documentation is under [DML
> Operations|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations]
> and [Loading files into
> tables|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables]:
> \\
> * Load Data performs very limited validations of the data, in particular it
> uses the input file name which may not be in 0_0 which can break some
> read logic. (Certainly will for Acid).
> * It does not check the schema of the file. This may be a non issue for Acid
> which requires ORC which is self describing so Schema Evolution may handle
> this seamlessly. (Assuming Schema is not too different).
> * It does check that _InputFormat_S are compatible.
> * Bucketed (and thus sorted) tables don't support Load Data (but only if
> hive.strict.checks.bucketing=true (default)). Will keep this restriction for
> Acid.
> * Load Data supports OVERWRITE clause
> * What happens to file permissions/ownership: rename vs copy differences
> \\
> The
[jira] [Comment Edited] (HIVE-18056) CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming
[ https://issues.apache.org/jira/browse/HIVE-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254606#comment-16254606 ] Sergey Shelukhin edited comment on HIVE-18056 at 11/16/17 1:42 AM: --- Hmm... I don't understand the new logic. waitForInitAndBlock seems to be a dup of waitForInit. The difference is that waitForInit doesn't sleep forever to avoid common bugs we've seen with the code that does sleep forever when some changes are made later and the state is modified without waking it up. It still does wait forever (via the while loop), sleeping 100ms on every iteration and re-checking the state. Notify is a nice addition. Also, Wrapper.get already calls waitForInit, so there shouldn't be need to also call waitForInitAndBlock (or if there's a bug in waitForInit that makes it not wait forever, it should be fixed). was (Author: sershe): Hmm... I don't understand the new logic. waitForInitAndBlock seems to be a dup of waitForInit. The difference is that waitForInit doesn't sleep forever to avoid common bugs we've seen with the code that does sleep forever when some changes are made later and the state is modified without waking it up. It still does wait forever (via the while loop), sleeping 100ms on every iteration and re-checking the state. Wrapper.get already calls waitForInit, so there shouldn't be need to also call waitForInitAndBlock (or if there's a bug in waitForInit that makes it not wait forever, it should be fixed). > CachedStore: Have a whitelist/blacklist config to allow selective caching of > tables/partitions and allow read while prewarming > -- > > Key: HIVE-18056 > URL: https://issues.apache.org/jira/browse/HIVE-18056 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-18056.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18077: Status: Patch Available (was: Open) > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18077.01.patch > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18077: Attachment: HIVE-18077.01.patch > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18077.01.patch > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HIVE-18056) CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming
[ https://issues.apache.org/jira/browse/HIVE-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254606#comment-16254606 ] Sergey Shelukhin edited comment on HIVE-18056 at 11/16/17 1:39 AM: --- Hmm... I don't understand the new logic. waitForInitAndBlock seems to be a dup of waitForInit. The difference is that waitForInit doesn't sleep forever to avoid common bugs we've seen with the code that does sleep forever when some changes are made later and the state is modified without waking it up. It still does wait forever (via the while loop), sleeping 100ms on every iteration and re-checking the state. Wrapper.get already calls waitForInit, so there shouldn't be need to also call waitForInitAndBlock (or if there's a bug in waitForInit that makes it not wait forever, it should be fixed). was (Author: sershe): Hmm... I don't understand the new logic. waitForInitAndBlock seems to be a dup of waitForInit. The difference is that waitForInit doesn't sleep forever to avoid common bugs we've seen with the code that does sleep forever when some changes are made later and the state is modified without waking it up. It still does wait forever (via the while loop), sleeping 100ms on every iteration and re-checking the state. Wrapper.get already calls waitForInit, so there shouldn't be need to also call waitForInitAndBlock (or is there's a bug in waitForInit that makes it not wait forever, it should be fixed). > CachedStore: Have a whitelist/blacklist config to allow selective caching of > tables/partitions and allow read while prewarming > -- > > Key: HIVE-18056 > URL: https://issues.apache.org/jira/browse/HIVE-18056 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-18056.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18056) CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming
[ https://issues.apache.org/jira/browse/HIVE-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254606#comment-16254606 ] Sergey Shelukhin commented on HIVE-18056: - Hmm... I don't understand the new logic. waitForInitAndBlock seems to be a dup of waitForInit. The difference is that waitForInit doesn't sleep forever to avoid common bugs we've seen with the code that does sleep forever when some changes are made later and the state is modified without waking it up. It still does loop forever, sleeping 100ms on every iteration. Wrapper.get already calls waitForInit, so there shouldn't be need to also call waitForInitAndBlock (or is there's a bug in waitForInit that makes it not wait forever, it should be fixed). > CachedStore: Have a whitelist/blacklist config to allow selective caching of > tables/partitions and allow read while prewarming > -- > > Key: HIVE-18056 > URL: https://issues.apache.org/jira/browse/HIVE-18056 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-18056.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HIVE-18056) CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming
[ https://issues.apache.org/jira/browse/HIVE-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254606#comment-16254606 ] Sergey Shelukhin edited comment on HIVE-18056 at 11/16/17 1:38 AM: --- Hmm... I don't understand the new logic. waitForInitAndBlock seems to be a dup of waitForInit. The difference is that waitForInit doesn't sleep forever to avoid common bugs we've seen with the code that does sleep forever when some changes are made later and the state is modified without waking it up. It still does wait forever (via the while loop), sleeping 100ms on every iteration and re-checking the state. Wrapper.get already calls waitForInit, so there shouldn't be need to also call waitForInitAndBlock (or is there's a bug in waitForInit that makes it not wait forever, it should be fixed). was (Author: sershe): Hmm... I don't understand the new logic. waitForInitAndBlock seems to be a dup of waitForInit. The difference is that waitForInit doesn't sleep forever to avoid common bugs we've seen with the code that does sleep forever when some changes are made later and the state is modified without waking it up. It still does loop forever, sleeping 100ms on every iteration. Wrapper.get already calls waitForInit, so there shouldn't be need to also call waitForInitAndBlock (or is there's a bug in waitForInit that makes it not wait forever, it should be fixed). > CachedStore: Have a whitelist/blacklist config to allow selective caching of > tables/partitions and allow read while prewarming > -- > > Key: HIVE-18056 > URL: https://issues.apache.org/jira/browse/HIVE-18056 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-18056.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254604#comment-16254604 ] Matt McCline commented on HIVE-18077: - Similar to https://issues.apache.org/jira/browse/HIVE-15857: Vectorization: Add string conversion case for UDFToInteger, etc > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-18077: --- > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17717) Enable rule to push post-aggregations into Druid
[ https://issues.apache.org/jira/browse/HIVE-17717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254601#comment-16254601 ] Ashutosh Chauhan commented on HIVE-17717: - +1 > Enable rule to push post-aggregations into Druid > > > Key: HIVE-17717 > URL: https://issues.apache.org/jira/browse/HIVE-17717 > Project: Hive > Issue Type: New Feature > Components: Druid integration >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-17717.01.patch, HIVE-17717.patch > > > Enable rule created by CALCITE-1803. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[
https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254585#comment-16254585
]
Hive QA commented on HIVE-16756:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897863/HIVE-16756.02.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 11381 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_join2]
(batchId=10)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions
(batchId=233)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7838/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7838/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7838/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897863 - PreCommit-HIVE-Build
> Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException:
> / by zero"
>
>
> Key: HIVE-16756
> URL: https://issues.apache.org/jira/browse/HIVE-16756
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.3.0
>Reporter: Matt McCline
>Assignee: Vihang Karajgaonkar
>Priority: Critical
> Attachments: HIVE-16756.01.patch, HIVE-16756.02.patch
>
>
> vectorization_div0.q needs to test the long data type testing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17361) Support LOAD DATA for transactional tables
[ https://issues.apache.org/jira/browse/HIVE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-17361: -- Description: LOAD DATA was not supported since ACID was introduced. Need to fill this gap between ACID table and regular hive table. Current Documentation is under [DML Operations|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations] and [Loading files into tables|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables]: \\ * Load Data performs very limited validations of the data, in particular it uses the input file name which may not be in 0_0 which can break some read logic. (Certainly will for Acid). * It does not check the schema of the file. This may be a non issue for Acid which requires ORC which is self describing so Schema Evolution may handle this seamlessly. (Assuming Schema is not too different). * It does check that _InputFormat_S are compatible. * Bucketed (and thus sorted) tables don't support Load Data (but only if hive.strict.checks.bucketing=true (default)). Will keep this restriction for Acid. * Load Data supports OVERWRITE clause * What happens to file permissions/ownership: rename vs copy differences \\ The implementation will follow the same idea as in HIVE-14988 and use a base_N/ dir for OVERWRITE clause. \\ How is minor compaction going to handle delta/base with original files? Since delta_8_8/_meta_data is created before files are moved, delta_8_8 becomes visible before it's populated. Is that an issue? was: LOAD DATA was not supported since ACID was introduced. Need to fill this gap between ACID table and regular hive table. Current Documentation is under [DML Operations|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations] and [Loading files into tables|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables]: \\ * Load Data performs very limited validations of the data, in particular it uses the input file name which may not be in 0_0 which can break some read logic. (Certainly will for Acid). * It does not check the schema of the file. This may be a non issue for Acid which requires ORC which is self describing so Schema Evolution may handle this seamlessly. (Assuming Schema is not too different). * It does check that _InputFormat_S are compatible. * Bucketed (and thus sorted) tables don't support Load Data (but only if hive.strict.checks.bucketing=true (default)). Will keep this restriction for Acid. * Load Data supports OVERWRITE clause * What happens to file permissions/ownership: rename vs copy differences \\ The implementation will follow the same idea as in HIVE-14988 and use a base_N/ dir for OVERWRITE clause. \\ How is minor compaction going to handle delta/base with original files? > Support LOAD DATA for transactional tables > -- > > Key: HIVE-17361 > URL: https://issues.apache.org/jira/browse/HIVE-17361 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Wei Zheng >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-17361.07.patch, HIVE-17361.08.patch, > HIVE-17361.09.patch, HIVE-17361.1.patch, HIVE-17361.10.patch, > HIVE-17361.11.patch, HIVE-17361.12.patch, HIVE-17361.2.patch, > HIVE-17361.3.patch, HIVE-17361.4.patch > > > LOAD DATA was not supported since ACID was introduced. Need to fill this gap > between ACID table and regular hive table. > Current Documentation is under [DML > Operations|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations] > and [Loading files into > tables|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables]: > \\ > * Load Data performs very limited validations of the data, in particular it > uses the input file name which may not be in 0_0 which can break some > read logic. (Certainly will for Acid). > * It does not check the schema of the file. This may be a non issue for Acid > which requires ORC which is self describing so Schema Evolution may handle > this seamlessly. (Assuming Schema is not too different). > * It does check that _InputFormat_S are compatible. > * Bucketed (and thus sorted) tables don't support Load Data (but only if > hive.strict.checks.bucketing=true (default)). Will keep this restriction for > Acid. > * Load Data supports OVERWRITE clause > * What happens to file permissions/ownership: rename vs copy differences > \\ > The implementation will follow the same idea as in HIVE-14988 and use a > base_N/ dir for OVERWRITE clause. > \\ > How is minor compaction going to handle
[jira] [Updated] (HIVE-17717) Enable rule to push post-aggregations into Druid
[ https://issues.apache.org/jira/browse/HIVE-17717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-17717: --- Attachment: HIVE-17717.01.patch [~ashutoshc], could you take a look? Thanks https://reviews.apache.org/r/63855/ > Enable rule to push post-aggregations into Druid > > > Key: HIVE-17717 > URL: https://issues.apache.org/jira/browse/HIVE-17717 > Project: Hive > Issue Type: New Feature > Components: Druid integration >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-17717.01.patch, HIVE-17717.patch > > > Enable rule created by CALCITE-1803. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[ https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254534#comment-16254534 ] Matt McCline commented on HIVE-16756: - I see it now -- sorry. > Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: > / by zero" > > > Key: HIVE-16756 > URL: https://issues.apache.org/jira/browse/HIVE-16756 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-16756.01.patch, HIVE-16756.02.patch > > > vectorization_div0.q needs to test the long data type testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18056) CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming
[ https://issues.apache.org/jira/browse/HIVE-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-18056: Attachment: (was: HIVE-18056.1.patch) > CachedStore: Have a whitelist/blacklist config to allow selective caching of > tables/partitions and allow read while prewarming > -- > > Key: HIVE-18056 > URL: https://issues.apache.org/jira/browse/HIVE-18056 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-18056.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18056) CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming
[ https://issues.apache.org/jira/browse/HIVE-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-18056: Attachment: HIVE-18056.1.patch > CachedStore: Have a whitelist/blacklist config to allow selective caching of > tables/partitions and allow read while prewarming > -- > > Key: HIVE-18056 > URL: https://issues.apache.org/jira/browse/HIVE-18056 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-18056.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17954) Implement pool, user, group and trigger to pool management API's.
[ https://issues.apache.org/jira/browse/HIVE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254506#comment-16254506 ] Sergey Shelukhin commented on HIVE-17954: - Left some comments on RB, mostly about the tests > Implement pool, user, group and trigger to pool management API's. > - > > Key: HIVE-17954 > URL: https://issues.apache.org/jira/browse/HIVE-17954 > Project: Hive > Issue Type: Sub-task >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash > Attachments: HIVE-17954.01.patch, HIVE-17954.02.patch, > HIVE-17954.03.patch, HIVE-17954.04.patch, HIVE-17954.05.patch, > HIVE-17954.06.patch, HIVE-17954.07.patch > > > Implement the following commands: > -- Pool management. > CREATE POOL `resource_plan`.`pool_path` WITH > ALLOC_FRACTION `fraction` > QUERY_PARALLELISM `parallelism` > SCHEDULING_POLICY `policy`; > ALTER POOL `resource_plan`.`pool_path` SET > PATH = `new_path`, > ALLOC_FRACTION = `fraction`, > QUERY_PARALLELISM = `parallelism`, > SCHEDULING_POLICY = `policy`; > DROP POOL `resource_plan`.`pool_path`; > -- Trigger to pool mappings. > ALTER RESOURCE PLAN `resource_plan` > ADD TRIGGER `trigger_name` TO `pool_path`; > ALTER RESOURCE PLAN `resource_plan` > DROP TRIGGER `trigger_name` TO `pool_path`; > -- User/Group to pool mappings. > CREATE USER|GROUP MAPPING `resource_plan`.`group_or_user_name` > TO `pool_path` WITH ORDERING `order_no`; > DROP USER|GROUP MAPPING `resource_plan`.`group_or_user_name`; -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[ https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254505#comment-16254505 ] Vihang Karajgaonkar commented on HIVE-16756: Do you mean in qfile? or a separate Junit for the newly added class? The qfile which I modified in the patch includes % operator when denomitor is a long. > Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: > / by zero" > > > Key: HIVE-16756 > URL: https://issues.apache.org/jira/browse/HIVE-16756 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-16756.01.patch, HIVE-16756.02.patch > > > vectorization_div0.q needs to test the long data type testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17717) Enable rule to push post-aggregations into Druid
[
https://issues.apache.org/jira/browse/HIVE-17717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254497#comment-16254497
]
Hive QA commented on HIVE-17717:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12890605/HIVE-17717.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 11388 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_basic3]
(batchId=59)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7837/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7837/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7837/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12890605 - PreCommit-HIVE-Build
> Enable rule to push post-aggregations into Druid
>
>
> Key: HIVE-17717
> URL: https://issues.apache.org/jira/browse/HIVE-17717
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-17717.patch
>
>
> Enable rule created by CALCITE-1803.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HIVE-18056) CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming
[ https://issues.apache.org/jira/browse/HIVE-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254451#comment-16254451 ] Daniel Dai edited comment on HIVE-18056 at 11/15/17 11:43 PM: -- Comments for the HIVE-18056.1.patch: 1. Can you put a note how this comma separated list look like (db.table) by giving an example, also what happen when a table appear in both blacklist and whitelist? 2. I assume capture exception during prewarm (such as table is dropped during prewarm, no need to interrupt prewarm) will be in a separate Jira, is that right? 3. getDatabases, we shall get from db if prewarm is not done, rather than wait 4. getAllTables/getTableMeta/listTableNamesByFilter, if both black/white list are empty, we can get from cache 5. Why we don't handle get_aggr_stats_for with cache? was (Author: daijy): Comments for the HIVE-18056.1.patch: 1. Can you put a note how this comma separated list look like (db.table) by giving an example, also what happen when a table appear in both blacklist and whitelist? 2. I assume capture exception during prewarm will be in a separate Jira, is that right? 3. getDatabases, we shall get from db if prewarm is not done, rather than wait 4. getAllTables/getTableMeta/listTableNamesByFilter, if both black/white list are empty, we can get from cache 5. Why we don't handle get_aggr_stats_for with cache? > CachedStore: Have a whitelist/blacklist config to allow selective caching of > tables/partitions and allow read while prewarming > -- > > Key: HIVE-18056 > URL: https://issues.apache.org/jira/browse/HIVE-18056 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-18056.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[ https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254460#comment-16254460 ] Matt McCline commented on HIVE-16756: - Do we need to add modulus tests in addition to division? > Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: > / by zero" > > > Key: HIVE-16756 > URL: https://issues.apache.org/jira/browse/HIVE-16756 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-16756.01.patch, HIVE-16756.02.patch > > > vectorization_div0.q needs to test the long data type testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18056) CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming
[ https://issues.apache.org/jira/browse/HIVE-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254451#comment-16254451 ] Daniel Dai commented on HIVE-18056: --- Comments for the HIVE-18056.1.patch: 1. Can you put a note how this comma separated list look like (db.table) by giving an example, also what happen when a table appear in both blacklist and whitelist? 2. I assume capture exception during prewarm will be in a separate Jira, is that right? 3. getDatabases, we shall get from db if prewarm is not done, rather than wait 4. getAllTables/getTableMeta/listTableNamesByFilter, if both black/white list are empty, we can get from cache 5. Why we don't handle get_aggr_stats_for with cache? > CachedStore: Have a whitelist/blacklist config to allow selective caching of > tables/partitions and allow read while prewarming > -- > > Key: HIVE-18056 > URL: https://issues.apache.org/jira/browse/HIVE-18056 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-18056.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18069) MetaStoreDirectSql to get tables has misplaced comma
[ https://issues.apache.org/jira/browse/HIVE-18069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18069: --- Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master, thanks for reporting and reviewing [~aihuaxu]! > MetaStoreDirectSql to get tables has misplaced comma > > > Key: HIVE-18069 > URL: https://issues.apache.org/jira/browse/HIVE-18069 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Jesus Camacho Rodriguez > Fix For: 3.0.0 > > Attachments: HIVE-18069.patch > > > Introduced by HIVE-15436. > Cc [~aihuaxu] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17361) Support LOAD DATA for transactional tables
[ https://issues.apache.org/jira/browse/HIVE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-17361: -- Attachment: HIVE-17361.12.patch > Support LOAD DATA for transactional tables > -- > > Key: HIVE-17361 > URL: https://issues.apache.org/jira/browse/HIVE-17361 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Wei Zheng >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-17361.07.patch, HIVE-17361.08.patch, > HIVE-17361.09.patch, HIVE-17361.1.patch, HIVE-17361.10.patch, > HIVE-17361.11.patch, HIVE-17361.12.patch, HIVE-17361.2.patch, > HIVE-17361.3.patch, HIVE-17361.4.patch > > > LOAD DATA was not supported since ACID was introduced. Need to fill this gap > between ACID table and regular hive table. > Current Documentation is under [DML > Operations|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations] > and [Loading files into > tables|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables]: > \\ > * Load Data performs very limited validations of the data, in particular it > uses the input file name which may not be in 0_0 which can break some > read logic. (Certainly will for Acid). > * It does not check the schema of the file. This may be a non issue for Acid > which requires ORC which is self describing so Schema Evolution may handle > this seamlessly. (Assuming Schema is not too different). > * It does check that _InputFormat_S are compatible. > * Bucketed (and thus sorted) tables don't support Load Data (but only if > hive.strict.checks.bucketing=true (default)). Will keep this restriction for > Acid. > * Load Data supports OVERWRITE clause > * What happens to file permissions/ownership: rename vs copy differences > \\ > The implementation will follow the same idea as in HIVE-14988 and use a > base_N/ dir for OVERWRITE clause. > \\ > How is minor compaction going to handle delta/base with original files? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18069) MetaStoreDirectSql to get tables has misplaced comma
[
https://issues.apache.org/jira/browse/HIVE-18069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254390#comment-16254390
]
Hive QA commented on HIVE-18069:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897847/HIVE-18069.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 11387 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_range_multiorder]
(batchId=7)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=198)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7836/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7836/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7836/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897847 - PreCommit-HIVE-Build
> MetaStoreDirectSql to get tables has misplaced comma
>
>
> Key: HIVE-18069
> URL: https://issues.apache.org/jira/browse/HIVE-18069
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Aihua Xu
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-18069.patch
>
>
> Introduced by HIVE-15436.
> Cc [~aihuaxu]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[ https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-16756: --- Attachment: HIVE-16756.02.patch attaching second version based on the above discussion. > Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: > / by zero" > > > Key: HIVE-16756 > URL: https://issues.apache.org/jira/browse/HIVE-16756 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-16756.01.patch, HIVE-16756.02.patch > > > vectorization_div0.q needs to test the long data type testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18074) do not show rejected tasks as killed in query UI
[ https://issues.apache.org/jira/browse/HIVE-18074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18074: Issue Type: Sub-task (was: Bug) Parent: HIVE-17481 > do not show rejected tasks as killed in query UI > > > Key: HIVE-18074 > URL: https://issues.apache.org/jira/browse/HIVE-18074 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin > > Tasks rejected from LLAP because the cluster is full are shown as killed > tasks in the commandline query UI (CLI and beeline). This shouldn't really > happen; killed tasks in the container case means something else, and this > scenario doesn't exist because AM doesn't continuously try to queue tasks. We > could change LLAP queue to use sort of a pull model (would also allow for > better duplicate scheduling), but for now we should fix the UI -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17904) handle internal Tez AM restart in registry and WM
[ https://issues.apache.org/jira/browse/HIVE-17904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-17904: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to master. Thanks for the review! > handle internal Tez AM restart in registry and WM > - > > Key: HIVE-17904 > URL: https://issues.apache.org/jira/browse/HIVE-17904 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 3.0.0 > > Attachments: HIVE-17904.01.patch, HIVE-17904.02.patch, > HIVE-17904.03.patch, HIVE-17904.04.patch, HIVE-17904.05.patch, > HIVE-17904.06.patch, HIVE-17904.patch, HIVE-17904.patch > > > After the plan update patch is committed. The current code doesn't account > very well for it; registry may have races, and an event needs to be added to > WM when some AM resets, at least to make sure we discard the update errors > that pertain to the old AM. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HIVE-18053) Support different table types for MVs
[ https://issues.apache.org/jira/browse/HIVE-18053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-18053 started by Jesus Camacho Rodriguez. -- > Support different table types for MVs > - > > Key: HIVE-18053 > URL: https://issues.apache.org/jira/browse/HIVE-18053 > Project: Hive > Issue Type: Sub-task > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Critical > > MVs backed by MM tables, managed tables, external tables. This might work > already, but we need to add tests. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18075) verify commands on a cluster
[ https://issues.apache.org/jira/browse/HIVE-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18075: Description: I was running the commands in the cluster, with potentially a slightly outdated version of the DB on mysql (yet, the DB was created with a script from master and I didn't hit any schema issues, so it's fairly recent); however, recent master code + the pools commands patch. I've hit the following issues. # Cannot drop pool or RP with a mapping (see also 3). # Cannot drop pool that is set as default (probably correct, but the error message is bad). # When I dropped an RP with a mapping, and then created it again with the same name, the pool creation in that RP would fail with an error that a unique query returned multiple results. In the DB, there were actually 2 RPs with the same name. Not sure how exactly that happened, there might have been intermediate states, but I didn't mess with mysql. I think the name uniqueness is either missing from some script or doesn't work. # Setting RP default pool no longer works. I think I might have broken it with one of the rebases in that area, but it could also be something else (or like other things, it works in q tests but not on cluster for whatever reason). # Resource plan rename doesn't check the disable state. It probably should. Also need to see for other commands. Need to figure out which are real issues and which aren't and fix. was: I was running the commands in the cluster, with potentially a slightly outdated version of the DB on mysql (yet, the DB was created with a script from master and I didn't hit any schema issues, so it's fairly recent); however, recent master code + the pools commands patch. I've hit the following issues. # Cannot drop pool or RP with a mapping (see also 3). # Cannot drop pool that is set as default (probably correct, but the error message is bad). # When I dropped an RP with a mapping, and then created it again with the same name, the pool creation in that RP would fail with an error that a unique query returned multiple results. In the DB, there were actually 2 RPs with the same name. Not sure how exactly that happened, there might have been intermediate states, but I didn't mess with mysql. I think the name uniqueness is either missing from some script or doesn't work. # Setting RP default pool no longer works. I think I might have broken it with one of the rebases in that area, but it could also be something else (or like other things, it works in q tests but not on cluster for whatever reason). # Resource plan rename doesn't check the disable state. It probably should. Also need to see for other commands. Need to figure out which are real issues and which aren't and fix. > verify commands on a cluster > > > Key: HIVE-18075 > URL: https://issues.apache.org/jira/browse/HIVE-18075 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin > > I was running the commands in the cluster, with potentially a slightly > outdated version of the DB on mysql (yet, the DB was created with a script > from master and I didn't hit any schema issues, so it's fairly recent); > however, recent master code + the pools commands patch. I've hit the > following issues. > # Cannot drop pool or RP with a mapping (see also 3). > # Cannot drop pool that is set as default (probably correct, but the error > message is bad). > # When I dropped an RP with a mapping, and then created it again with the > same name, the pool creation in that RP would fail with an error that a > unique query returned multiple results. In the DB, there were actually 2 RPs > with the same name. Not sure how exactly that happened, there might have been > intermediate states, but I didn't mess with mysql. I think the name > uniqueness is either missing from some script or doesn't work. > # Setting RP default pool no longer works. I think I might have broken it > with one of the rebases in that area, but it could also be something else (or > like other things, it works in q tests but not on cluster for whatever > reason). > # Resource plan rename doesn't check the disable state. It probably should. > Also need to see for other commands. > Need to figure out which are real issues and which aren't and fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[ https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254339#comment-16254339 ] Matt McCline commented on HIVE-16756: - Yes, its true we only need the fix the for long % long -> long case. > Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: > / by zero" > > > Key: HIVE-16756 > URL: https://issues.apache.org/jira/browse/HIVE-16756 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-16756.01.patch > > > vectorization_div0.q needs to test the long data type testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18076) killquery doesn't actually work for non-trigger WM kills, or the error message is not propagated
[ https://issues.apache.org/jira/browse/HIVE-18076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-18076: --- > killquery doesn't actually work for non-trigger WM kills, or the error > message is not propagated > > > Key: HIVE-18076 > URL: https://issues.apache.org/jira/browse/HIVE-18076 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > > Not sure what's wrong with it, need to take a look. > It dumps a lot of info about everything being cancelled, instead of a nice > message like triggers do. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18075) verify commands on a cluster
[ https://issues.apache.org/jira/browse/HIVE-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18075: Description: I was running the commands in the cluster, with potentially a slightly outdated version of the DB on mysql (yet, the DB was created with a script from master and I didn't hit any schema issues, so it's fairly recent); however, recent master code + the pools commands patch. I've hit the following issues. # Cannot drop pool or RP with a mapping (see also 3). # Cannot drop pool that is set as default (probably correct, but the error message is bad). # When I dropped an RP with a mapping, and then created it again with the same name, the pool creation in that RP would fail with an error that a unique query returned multiple results. In the DB, there were actually 2 RPs with the same name. Not sure how exactly that happened, there might have been intermediate states, but I didn't mess with mysql. I think the name uniqueness is either missing from some script or doesn't work. # Setting RP default pool no longer works. I think I might have broken it with one of the rebases in that area, but it could also be something else (or like other things, it works in q tests but not on cluster for whatever reason). # Resource plan rename doesn't check the disable state. It probably should. Also need to see for other commands. Need to figure out which are real issues and which aren't and fix. was: I was running the commands in the cluster, with potentially a slightly outdated version of the DB; however, recent master code + the pools commands patch. I've hit the following issues. # Cannot drop pool or RP with a mapping (see also 3). # Cannot drop pool that is set as default (probably correct, but the error message is bad). # When I dropped an RP with a mapping, and then created it again with the same name, the pool creation in that RP would fail with an error that a unique query returned multiple results. In the DB, there were actually 2 RPs with the same name. Not sure how exactly that happened, there might have been intermediate states, but I didn't mess with mysql. I think the name uniqueness is either missing from some script or doesn't work. # Setting RP default pool no longer works. I think I might have broken it with one of the rebases in that area, but it could also be something else (or like other things, it works in q tests but not on cluster for whatever reason). # Resource plan rename doesn't check the disable state. It probably should. Also need to see for other commands. > verify commands on a cluster > > > Key: HIVE-18075 > URL: https://issues.apache.org/jira/browse/HIVE-18075 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin > > I was running the commands in the cluster, with potentially a slightly > outdated version of the DB on mysql (yet, the DB was created with a script > from master and I didn't hit any schema issues, so it's fairly recent); > however, recent master code + the pools commands patch. I've hit the > following issues. > # Cannot drop pool or RP with a mapping (see also 3). > # Cannot drop pool that is set as default (probably correct, but the error > message is bad). > # When I dropped an RP with a mapping, and then created it again with the > same name, the pool creation in that RP would fail with an error that a > unique query returned multiple results. In the DB, there were actually 2 RPs > with the same name. Not sure how exactly that happened, there might have been > intermediate states, but I didn't mess with mysql. I think the name > uniqueness is either missing from some script or doesn't work. > # Setting RP default pool no longer works. I think I might have broken it > with one of the rebases in that area, but it could also be something else (or > like other things, it works in q tests but not on cluster for whatever > reason). > # Resource plan rename doesn't check the disable state. It probably should. > Also need to see for other commands. > Need to figure out which are real issues and which aren't and fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[
https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254295#comment-16254295
]
Vihang Karajgaonkar commented on HIVE-16756:
Actually, after discussing with [~aihuaxu] I realized that
{{LongColDivideLongColumn.java}} doesn't need the fix since it typecasts the
denominator to double. So it should not throw a / by zero exception. For the
other cases of division and modulo when one of the operands is double, the
result is a double so that should work fine too. I guess we should only hit
this issue when we have a long modulo long. Is that understanding correct
[~mmccline]? If yes, I can modify the patch such that it doesn't generate a
LongColModuloLongColumn.java but rather we add it to source code after handling
the / by zero.
> Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException:
> / by zero"
>
>
> Key: HIVE-16756
> URL: https://issues.apache.org/jira/browse/HIVE-16756
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.3.0
>Reporter: Matt McCline
>Assignee: Vihang Karajgaonkar
>Priority: Critical
> Attachments: HIVE-16756.01.patch
>
>
> vectorization_div0.q needs to test the long data type testing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2
[ https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-18072: --- > WM - fix various bugs based on cluster testing - part 2 > --- > > Key: HIVE-18072 > URL: https://issues.apache.org/jira/browse/HIVE-18072 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18071) add HS2 jmx information about pools and current resource plan
[ https://issues.apache.org/jira/browse/HIVE-18071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-18071: --- > add HS2 jmx information about pools and current resource plan > - > > Key: HIVE-18071 > URL: https://issues.apache.org/jira/browse/HIVE-18071 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[ https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254291#comment-16254291 ] Matt McCline commented on HIVE-16756: - +1 LGTM. Thanks for fixing this. > Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: > / by zero" > > > Key: HIVE-16756 > URL: https://issues.apache.org/jira/browse/HIVE-16756 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-16756.01.patch > > > vectorization_div0.q needs to test the long data type testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18070) Merge partitions NDV estimators in batches
[ https://issues.apache.org/jira/browse/HIVE-18070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-18070: -- > Merge partitions NDV estimators in batches > -- > > Key: HIVE-18070 > URL: https://issues.apache.org/jira/browse/HIVE-18070 > Project: Hive > Issue Type: Improvement > Components: Statistics >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17717) Enable rule to push post-aggregations into Druid
[ https://issues.apache.org/jira/browse/HIVE-17717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-17717: --- Status: Patch Available (was: In Progress) > Enable rule to push post-aggregations into Druid > > > Key: HIVE-17717 > URL: https://issues.apache.org/jira/browse/HIVE-17717 > Project: Hive > Issue Type: New Feature > Components: Druid integration >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-17717.patch > > > Enable rule created by CALCITE-1803. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HIVE-17717) Enable rule to push post-aggregations into Druid
[ https://issues.apache.org/jira/browse/HIVE-17717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-17717 started by Jesus Camacho Rodriguez. -- > Enable rule to push post-aggregations into Druid > > > Key: HIVE-17717 > URL: https://issues.apache.org/jira/browse/HIVE-17717 > Project: Hive > Issue Type: New Feature > Components: Druid integration >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-17717.patch > > > Enable rule created by CALCITE-1803. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18069) MetaStoreDirectSql to get tables has misplaced comma
[ https://issues.apache.org/jira/browse/HIVE-18069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254232#comment-16254232 ] Aihua Xu commented on HIVE-18069: - +1. > MetaStoreDirectSql to get tables has misplaced comma > > > Key: HIVE-18069 > URL: https://issues.apache.org/jira/browse/HIVE-18069 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-18069.patch > > > Introduced by HIVE-15436. > Cc [~aihuaxu] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18069) MetaStoreDirectSql to get tables has misplaced comma
[ https://issues.apache.org/jira/browse/HIVE-18069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18069: --- Attachment: HIVE-18069.patch > MetaStoreDirectSql to get tables has misplaced comma > > > Key: HIVE-18069 > URL: https://issues.apache.org/jira/browse/HIVE-18069 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-18069.patch > > > Introduced by HIVE-15436. > Cc [~aihuaxu] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18069) MetaStoreDirectSql to get tables has misplaced comma
[ https://issues.apache.org/jira/browse/HIVE-18069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254225#comment-16254225 ] Aihua Xu commented on HIVE-18069: - Thanks [~jcamachorodriguez] > MetaStoreDirectSql to get tables has misplaced comma > > > Key: HIVE-18069 > URL: https://issues.apache.org/jira/browse/HIVE-18069 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Jesus Camacho Rodriguez > > Introduced by HIVE-15436. > Cc [~aihuaxu] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18069) MetaStoreDirectSql to get tables has misplaced comma
[ https://issues.apache.org/jira/browse/HIVE-18069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18069: --- Status: Patch Available (was: In Progress) > MetaStoreDirectSql to get tables has misplaced comma > > > Key: HIVE-18069 > URL: https://issues.apache.org/jira/browse/HIVE-18069 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Jesus Camacho Rodriguez > > Introduced by HIVE-15436. > Cc [~aihuaxu] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HIVE-18069) MetaStoreDirectSql to get tables has misplaced comma
[ https://issues.apache.org/jira/browse/HIVE-18069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-18069 started by Jesus Camacho Rodriguez. -- > MetaStoreDirectSql to get tables has misplaced comma > > > Key: HIVE-18069 > URL: https://issues.apache.org/jira/browse/HIVE-18069 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Jesus Camacho Rodriguez > > Introduced by HIVE-15436. > Cc [~aihuaxu] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18069) MetaStoreDirectSql to get tables has misplaced comma
[ https://issues.apache.org/jira/browse/HIVE-18069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-18069: -- > MetaStoreDirectSql to get tables has misplaced comma > > > Key: HIVE-18069 > URL: https://issues.apache.org/jira/browse/HIVE-18069 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Jesus Camacho Rodriguez > > Introduced by HIVE-15436. > Cc [~aihuaxu] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-15436) Enhancing metastore APIs to retrieve only materialized views
[
https://issues.apache.org/jira/browse/HIVE-15436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254216#comment-16254216
]
Aihua Xu commented on HIVE-15436:
-
[~jcamachorodriguez] Seems the query for getTables call is not correct. There
is an extra ','.
See the failure
https://builds.apache.org/job/PreCommit-HIVE-Build/7812/testReport/org.apache.hadoop.hive.cli/TestAccumuloCliDriver/testCliDriver_accumulo_queries_/.
{noformat}
2017-11-14T15:35:09,416 WARN [main] metastore.MetaStoreDirectSql: Failed to
execute [SELECT "TBLS"."TBL_NAME", FROM "TBLS" INNER JOIN "DBS" ON
"TBLS"."DB_ID" = "DBS"."DB_ID" WHERE "DBS"."NAME" = ? ] with parameters
[default]
javax.jdo.JDODataStoreException: Error executing SQL query "SELECT
"TBLS"."TBL_NAME", FROM "TBLS" INNER JOIN "DBS" ON "TBLS"."DB_ID" =
"DBS"."DB_ID" WHERE "DBS"."NAME" = ?".
{noformat}
> Enhancing metastore APIs to retrieve only materialized views
>
>
> Key: HIVE-15436
> URL: https://issues.apache.org/jira/browse/HIVE-15436
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Fix For: 3.0.0
>
> Attachments: HIVE-15436.patch
>
>
> Enhancing metastore APIs such that, instead of returning all tables, it can
> return only:
> - views
> - materialized views
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HIVE-17715) Exception when pushing postaggregates into Druid
[
https://issues.apache.org/jira/browse/HIVE-17715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved HIVE-17715.
Resolution: Duplicate
Fix Version/s: 3.0.0
Fixed in CALCITE-2050.
> Exception when pushing postaggregates into Druid
>
>
> Key: HIVE-17715
> URL: https://issues.apache.org/jira/browse/HIVE-17715
> Project: Hive
> Issue Type: Bug
> Components: Druid integration
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Fix For: 3.0.0
>
>
> After Calcite is upgraded to 1.14 and the rule to push post-aggregations to
> Druid is enabled, the following query will fail:
> {code}
> EXPLAIN
> SELECT language, robot, sum(added) - sum(delta) AS a
> FROM druid_table_1
> WHERE extract (week from `__time`) IN (10,11)
> AND robot='Bird Call'
> GROUP BY language, robot;
> {code}
> The error we get is the following:
> {code}
> Cannot add expression of different type to set:
> set type is RecordType(VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary" language, VARCHAR(2147483647) CHARACTER SET
> "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" robot, DOUBLE a) NOT NULL
> expression type is RecordType(VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" language, DOUBLE postagg#0) NOT NULL
> set is
> rel#1507:HiveProject.HIVE.[](input=HepRelVertex#1514,language=$0,robot=CAST(_UTF-16LE'Bird
> Call'):VARCHAR(2147483647) CHARACTER SET "UTF-16LE" COLLATE
> "ISO-8859-1$en_US$primary",a=-($1, $2))
> expression is DruidQuery#1516
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17528) Add more q-tests for Hive-on-Spark with Parquet vectorized reader
[ https://issues.apache.org/jira/browse/HIVE-17528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254145#comment-16254145 ] Vihang Karajgaonkar commented on HIVE-17528: Thanks [~Ferd] for the patch. I reviewed the patch and mostly it looks good except for the one comment which I posted on RB. Can you please check that? > Add more q-tests for Hive-on-Spark with Parquet vectorized reader > - > > Key: HIVE-17528 > URL: https://issues.apache.org/jira/browse/HIVE-17528 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Ferdinand Xu > Attachments: HIVE-17528.1.patch, HIVE-17528.2.patch, > HIVE-17528.3.patch, HIVE-17528.4.patch, HIVE-17528.patch > > > Most of the vectorization related q-tests operate on ORC tables using Tez. It > would be good to add more coverage on a different combination of engine and > file-format. We can model existing q-tests using parquet tables and run it > using TestSparkCliDriver -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17954) Implement pool, user, group and trigger to pool management API's.
[
https://issues.apache.org/jira/browse/HIVE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254092#comment-16254092
]
Hive QA commented on HIVE-17954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897782/HIVE-17954.07.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 11387 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_range_multiorder]
(batchId=7)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7831/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7831/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7831/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897782 - PreCommit-HIVE-Build
> Implement pool, user, group and trigger to pool management API's.
> -
>
> Key: HIVE-17954
> URL: https://issues.apache.org/jira/browse/HIVE-17954
> Project: Hive
> Issue Type: Sub-task
>Reporter: Harish Jaiprakash
>Assignee: Harish Jaiprakash
> Attachments: HIVE-17954.01.patch, HIVE-17954.02.patch,
> HIVE-17954.03.patch, HIVE-17954.04.patch, HIVE-17954.05.patch,
> HIVE-17954.06.patch, HIVE-17954.07.patch
>
>
> Implement the following commands:
> -- Pool management.
> CREATE POOL `resource_plan`.`pool_path` WITH
> ALLOC_FRACTION `fraction`
> QUERY_PARALLELISM `parallelism`
> SCHEDULING_POLICY `policy`;
> ALTER POOL `resource_plan`.`pool_path` SET
> PATH = `new_path`,
> ALLOC_FRACTION = `fraction`,
> QUERY_PARALLELISM = `parallelism`,
> SCHEDULING_POLICY = `policy`;
> DROP POOL `resource_plan`.`pool_path`;
> -- Trigger to pool mappings.
> ALTER RESOURCE PLAN `resource_plan`
> ADD TRIGGER `trigger_name` TO `pool_path`;
> ALTER RESOURCE PLAN `resource_plan`
> DROP TRIGGER `trigger_name` TO `pool_path`;
> -- User/Group to pool mappings.
> CREATE USER|GROUP MAPPING `resource_plan`.`group_or_user_name`
> TO `pool_path` WITH ORDERING `order_no`;
> DROP USER|GROUP MAPPING `resource_plan`.`group_or_user_name`;
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18068) Replace LocalInterval by Interval in Druid storage handler
[ https://issues.apache.org/jira/browse/HIVE-18068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254091#comment-16254091 ] Ashutosh Chauhan commented on HIVE-18068: - +1 pending tests > Replace LocalInterval by Interval in Druid storage handler > -- > > Key: HIVE-18068 > URL: https://issues.apache.org/jira/browse/HIVE-18068 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-18068.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17684) HoS memory issues with MapJoinMemoryExhaustionHandler
[
https://issues.apache.org/jira/browse/HIVE-17684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254068#comment-16254068
]
Misha Dmitriev commented on HIVE-17684:
---
The problem with {{MapJoinMemoryExhaustionHandler}} is a large percentage of
false alarms about memory exhaustion. Without it, however, Hive may go into a
"GC death spiral", where the JVM runs back-to-back full GCs, but doesn't fail
for long enough. Because user threads are unable to run most of the time, the
executor stops responding, and the Spark driver eventually drops it after some
time. This results in hard-to-debug failures, because from the logs it's not
clear why the executor stopped responding.
I recently added the new
https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GcTimeMonitor.java
class to hadoop, which allows the user to accurately monitor the percentage of
time that the JVM spends in GC. When this percentage grows above ~50% over a
1-minute period, it's almost always a signal that the JVM is in the above "GC
death spiral". Even if it is not, extremely long GC pauses are very bad for
performance, and it makes sense to treat them in the same way as OOM, i.e. fail
the task and ask the user to increase their executors' heap size.
I ran some experiments where I replaced MapJoinMemoryExhaustionHandler with
checking GC time percentage reported by GcTimeMonitor, and it work well.
GcTimeMonitor will become available for other projects when Hadoop 3.0.0-GA is
released (which, according to Hadoop developers, should happen in a few weeks).
Currently Hive depends on Hadoop 3.0.0-beta1, so to use GcTimeMonitor in Hive,
we will need to change this dependency to Hadoop GA. Are there any objections
against:
(a) dependency change from Hadoop 3.0.0-beta1 to 3.0.0-GA
(b) replacing MapJoinMemoryExhaustionHandler with GcTimeMonitor ?
> HoS memory issues with MapJoinMemoryExhaustionHandler
> -
>
> Key: HIVE-17684
> URL: https://issues.apache.org/jira/browse/HIVE-17684
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>
> We have seen a number of memory issues due the {{HashSinkOperator}} use of
> the {{MapJoinMemoryExhaustionHandler}}. This handler is meant to detect
> scenarios where the small table is taking too much space in memory, in which
> case a {{MapJoinMemoryExhaustionError}} is thrown.
> The configs to control this logic are:
> {{hive.mapjoin.localtask.max.memory.usage}} (default 0.90)
> {{hive.mapjoin.followby.gby.localtask.max.memory.usage}} (default 0.55)
> The handler works by using the {{MemoryMXBean}} and uses the following logic
> to estimate how much memory the {{HashMap}} is consuming:
> {{MemoryMXBean#getHeapMemoryUsage().getUsed() /
> MemoryMXBean#getHeapMemoryUsage().getMax()}}
> The issue is that {{MemoryMXBean#getHeapMemoryUsage().getUsed()}} can be
> inaccurate. The value returned by this method returns all reachable and
> unreachable memory on the heap, so there may be a bunch of garbage data, and
> the JVM just hasn't taken the time to reclaim it all. This can lead to
> intermittent failures of this check even though a simple GC would have
> reclaimed enough space for the process to continue working.
> We should re-think the usage of {{MapJoinMemoryExhaustionHandler}} for HoS.
> In Hive-on-MR this probably made sense to use because every Hive task was run
> in a dedicated container, so a Hive Task could assume it created most of the
> data on the heap. However, in Hive-on-Spark there can be multiple Hive Tasks
> running in a single executor, each doing different things.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17898) Explain plan output enhancement
[ https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17898: --- Status: Patch Available (was: Open) > Explain plan output enhancement > --- > > Key: HIVE-17898 > URL: https://issues.apache.org/jira/browse/HIVE-17898 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17898.1.patch, HIVE-17898.2.patch, > HIVE-17898.3.patch, HIVE-17898.4.patch > > > We would like to enhance the explain plan output to display additional > information e.g.: > TableScan operator should have following additional info > * Actual table name (currently only alias name is displayed) > * Database name > * Column names being scanned -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17898) Explain plan output enhancement
[ https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17898: --- Attachment: HIVE-17898.4.patch > Explain plan output enhancement > --- > > Key: HIVE-17898 > URL: https://issues.apache.org/jira/browse/HIVE-17898 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17898.1.patch, HIVE-17898.2.patch, > HIVE-17898.3.patch, HIVE-17898.4.patch > > > We would like to enhance the explain plan output to display additional > information e.g.: > TableScan operator should have following additional info > * Actual table name (currently only alias name is displayed) > * Database name > * Column names being scanned -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17898) Explain plan output enhancement
[ https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17898: --- Status: Open (was: Patch Available) > Explain plan output enhancement > --- > > Key: HIVE-17898 > URL: https://issues.apache.org/jira/browse/HIVE-17898 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17898.1.patch, HIVE-17898.2.patch, > HIVE-17898.3.patch, HIVE-17898.4.patch > > > We would like to enhance the explain plan output to display additional > information e.g.: > TableScan operator should have following additional info > * Actual table name (currently only alias name is displayed) > * Database name > * Column names being scanned -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HIVE-18068) Replace LocalInterval by Interval in Druid storage handler
[ https://issues.apache.org/jira/browse/HIVE-18068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-18068 started by Jesus Camacho Rodriguez. -- > Replace LocalInterval by Interval in Druid storage handler > -- > > Key: HIVE-18068 > URL: https://issues.apache.org/jira/browse/HIVE-18068 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-18068.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18068) Replace LocalInterval by Interval in Druid storage handler
[ https://issues.apache.org/jira/browse/HIVE-18068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18068: --- Status: Patch Available (was: In Progress) > Replace LocalInterval by Interval in Druid storage handler > -- > > Key: HIVE-18068 > URL: https://issues.apache.org/jira/browse/HIVE-18068 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-18068.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18068) Replace LocalInterval by Interval in Druid storage handler
[ https://issues.apache.org/jira/browse/HIVE-18068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18068: --- Attachment: HIVE-18068.patch > Replace LocalInterval by Interval in Druid storage handler > -- > > Key: HIVE-18068 > URL: https://issues.apache.org/jira/browse/HIVE-18068 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-18068.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18068) Replace LocalInterval by Interval in Druid storage handler
[ https://issues.apache.org/jira/browse/HIVE-18068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-18068: -- > Replace LocalInterval by Interval in Druid storage handler > -- > > Key: HIVE-18068 > URL: https://issues.apache.org/jira/browse/HIVE-18068 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-14487) Add REBUILD statement for materialized views
[
https://issues.apache.org/jira/browse/HIVE-14487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-14487:
--
Assignee: Jesus Camacho Rodriguez
> Add REBUILD statement for materialized views
>
>
> Key: HIVE-14487
> URL: https://issues.apache.org/jira/browse/HIVE-14487
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>
> Support for rebuilding existing materialized views. The statement is the
> following:
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name REBUILD;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Work started] (HIVE-14487) Add REBUILD statement for materialized views
[
https://issues.apache.org/jira/browse/HIVE-14487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-14487 started by Jesus Camacho Rodriguez.
--
> Add REBUILD statement for materialized views
>
>
> Key: HIVE-14487
> URL: https://issues.apache.org/jira/browse/HIVE-14487
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>
> Support for rebuilding existing materialized views. The statement is the
> following:
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name REBUILD;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-15018) ALTER rewriting flag in materialized view
[
https://issues.apache.org/jira/browse/HIVE-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-15018:
---
Attachment: HIVE-15018.01.patch
> ALTER rewriting flag in materialized view
> --
>
> Key: HIVE-15018
> URL: https://issues.apache.org/jira/browse/HIVE-15018
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-15018.01.patch, HIVE-15018.patch
>
>
> We should extend the ALTER statement in case we want to change the rewriting
> behavior of the materialized view after we have created it.
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name DISABLE REWRITE;
> {code}
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name ENABLE REWRITE;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-15018) ALTER rewriting flag in materialized view
[
https://issues.apache.org/jira/browse/HIVE-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-15018:
---
Attachment: (was: HIVE-15018.01.patch)
> ALTER rewriting flag in materialized view
> --
>
> Key: HIVE-15018
> URL: https://issues.apache.org/jira/browse/HIVE-15018
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-15018.01.patch, HIVE-15018.patch
>
>
> We should extend the ALTER statement in case we want to change the rewriting
> behavior of the materialized view after we have created it.
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name DISABLE REWRITE;
> {code}
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name ENABLE REWRITE;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-15018) ALTER rewriting flag in materialized view
[
https://issues.apache.org/jira/browse/HIVE-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-15018:
---
Attachment: HIVE-15018.01.patch
[~ashutoshc], could you take a look? Thanks
https://reviews.apache.org/r/63845/
> ALTER rewriting flag in materialized view
> --
>
> Key: HIVE-15018
> URL: https://issues.apache.org/jira/browse/HIVE-15018
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-15018.01.patch, HIVE-15018.patch
>
>
> We should extend the ALTER statement in case we want to change the rewriting
> behavior of the materialized view after we have created it.
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name DISABLE REWRITE;
> {code}
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name ENABLE REWRITE;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[
https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253857#comment-16253857
]
Vihang Karajgaonkar commented on HIVE-16756:
{{org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=226)}} failure is caused because of
{{org.apache.thrift.transport.TTransportException:
java.net.SocketTimeoutException: Read timed out}}
{{org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=157)}} works for me. I don't think it is related to this patch either.
Rest of the tests have been failing for a while for other patches as well.
> Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException:
> / by zero"
>
>
> Key: HIVE-16756
> URL: https://issues.apache.org/jira/browse/HIVE-16756
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.3.0
>Reporter: Matt McCline
>Assignee: Vihang Karajgaonkar
>Priority: Critical
> Attachments: HIVE-16756.01.patch
>
>
> vectorization_div0.q needs to test the long data type testing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18056) CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming
[ https://issues.apache.org/jira/browse/HIVE-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-18056: Attachment: HIVE-18056.1.patch > CachedStore: Have a whitelist/blacklist config to allow selective caching of > tables/partitions and allow read while prewarming > -- > > Key: HIVE-18056 > URL: https://issues.apache.org/jira/browse/HIVE-18056 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-18056.1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18054) Make Lineage work with concurrent queries on a Session
[ https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18054: -- Attachment: HIVE-18054.3.patch > Make Lineage work with concurrent queries on a Session > --- > > Key: HIVE-18054 > URL: https://issues.apache.org/jira/browse/HIVE-18054 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Attachments: HIVE-18054.1.patch, HIVE-18054.2.patch, > HIVE-18054.3.patch > > > A Hive Session can contain multiple concurrent sql Operations. > Lineage is currently tracked in SessionState and is cleared when a query > completes. This results in Lineage for other running queries being lost. > To fix this, move LineageState from SessionState to QueryState. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18046) Metastore: default IS_REWRITE_ENABLED=false instead of NULL
[
https://issues.apache.org/jira/browse/HIVE-18046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18046:
---
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Pushed to master, thanks for reviewing [~sershe]!
> Metastore: default IS_REWRITE_ENABLED=false instead of NULL
> ---
>
> Key: HIVE-18046
> URL: https://issues.apache.org/jira/browse/HIVE-18046
> Project: Hive
> Issue Type: Bug
> Components: Materialized views, Metastore
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HIVE-18046.01.patch, HIVE-18046.patch
>
>
> The materialized view impl breaks old metastore sql write access, by
> complaining that the new table creation does not set this column up.
> {code}
> `IS_REWRITE_ENABLED` bit(1) NOT NULL,
> {code}
> {{NOT NULL DEFAULT 0}} would allow old metastore direct sql compatibility
> (not thrift).
> {code}
> 2017-11-09T07:11:58,331 ERROR [HiveServer2-Background-Pool: Thread-2354]
> metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
> 10) with error: javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.MTable@249dbf1" using statement
> "INSERT INTO `TBLS`
> (`TBL_ID`,`CREATE_TIME`,`DB_ID`,`LAST_ACCESS_TIME`,`OWNER`,`RETENTION`,`SD_ID`,`TBL_NAME`,`TBL_TYPE`,`VIEW_EXPANDED_TEXT`,`VIEW_ORIGINAL_TEXT`)
> VALUES (?,?,?,?,?,?,?,?,?,?,?)" failed : Field 'IS_REWRITE_ENABLED' doesn't
> have a default value
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:720)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:740)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:1038)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-14495) Add SHOW MATERIALIZED VIEWS statement
[
https://issues.apache.org/jira/browse/HIVE-14495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-14495:
---
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Pushed to master, thanks for reviewing [~ashutoshc]!
> Add SHOW MATERIALIZED VIEWS statement
> -
>
> Key: HIVE-14495
> URL: https://issues.apache.org/jira/browse/HIVE-14495
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Fix For: 3.0.0
>
> Attachments: HIVE-14495.01.patch, HIVE-14495.patch
>
>
> In the spirit of {{SHOW TABLES}}, we should support the following statement:
> {code:sql}
> SHOW MATERIALIZED VIEWS [IN database_name] ['identifier_with_wildcards'];
> {code}
> In contrast to {{SHOW TABLES}}, this command would only list the materialized
> views.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17651) TableScanOperator might miss vectorization on flag
[ https://issues.apache.org/jira/browse/HIVE-17651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253761#comment-16253761 ] Jesus Camacho Rodriguez commented on HIVE-17651: +1 > TableScanOperator might miss vectorization on flag > -- > > Key: HIVE-17651 > URL: https://issues.apache.org/jira/browse/HIVE-17651 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-17651.01ex.patch, HIVE-17651.02rc.patch, > HIVE-17651.02rc.patch, HIVE-17651.03.patch > > > https://github.com/apache/hive/blob/3bfcfdde0c0be2aab1afdf5b1bc71fdcc9e77360/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java#L259-L273 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17954) Implement pool, user, group and trigger to pool management API's.
[ https://issues.apache.org/jira/browse/HIVE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Jaiprakash updated HIVE-17954: - Attachment: HIVE-17954.07.patch Fixed test case failures. > Implement pool, user, group and trigger to pool management API's. > - > > Key: HIVE-17954 > URL: https://issues.apache.org/jira/browse/HIVE-17954 > Project: Hive > Issue Type: Sub-task >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash > Attachments: HIVE-17954.01.patch, HIVE-17954.02.patch, > HIVE-17954.03.patch, HIVE-17954.04.patch, HIVE-17954.05.patch, > HIVE-17954.06.patch, HIVE-17954.07.patch > > > Implement the following commands: > -- Pool management. > CREATE POOL `resource_plan`.`pool_path` WITH > ALLOC_FRACTION `fraction` > QUERY_PARALLELISM `parallelism` > SCHEDULING_POLICY `policy`; > ALTER POOL `resource_plan`.`pool_path` SET > PATH = `new_path`, > ALLOC_FRACTION = `fraction`, > QUERY_PARALLELISM = `parallelism`, > SCHEDULING_POLICY = `policy`; > DROP POOL `resource_plan`.`pool_path`; > -- Trigger to pool mappings. > ALTER RESOURCE PLAN `resource_plan` > ADD TRIGGER `trigger_name` TO `pool_path`; > ALTER RESOURCE PLAN `resource_plan` > DROP TRIGGER `trigger_name` TO `pool_path`; > -- User/Group to pool mappings. > CREATE USER|GROUP MAPPING `resource_plan`.`group_or_user_name` > TO `pool_path` WITH ORDERING `order_no`; > DROP USER|GROUP MAPPING `resource_plan`.`group_or_user_name`; -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17898) Explain plan output enhancement
[ https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253720#comment-16253720 ] Ashutosh Chauhan commented on HIVE-17898: - * For consistency, its better to use : database, table, columns, isTempTable as identifiers. * Some of failed tests above needs regeneration. +1 pending these changes. > Explain plan output enhancement > --- > > Key: HIVE-17898 > URL: https://issues.apache.org/jira/browse/HIVE-17898 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17898.1.patch, HIVE-17898.2.patch, > HIVE-17898.3.patch > > > We would like to enhance the explain plan output to display additional > information e.g.: > TableScan operator should have following additional info > * Actual table name (currently only alias name is displayed) > * Database name > * Column names being scanned -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17954) Implement pool, user, group and trigger to pool management API's.
[
https://issues.apache.org/jira/browse/HIVE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253600#comment-16253600
]
Hive QA commented on HIVE-17954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897752/HIVE-17954.06.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 11371 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testDestroyAndReturn
(batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testQueueName
(batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testReopen (batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testReuse (batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testReuseWithQueueing
(batchId=281)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.org.apache.hive.jdbc.TestTriggersMoveWorkloadManager
(batchId=233)
org.apache.hive.jdbc.TestTriggersWorkloadManager.org.apache.hive.jdbc.TestTriggersWorkloadManager
(batchId=230)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7830/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7830/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7830/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897752 - PreCommit-HIVE-Build
> Implement pool, user, group and trigger to pool management API's.
> -
>
> Key: HIVE-17954
> URL: https://issues.apache.org/jira/browse/HIVE-17954
> Project: Hive
> Issue Type: Sub-task
>Reporter: Harish Jaiprakash
>Assignee: Harish Jaiprakash
> Attachments: HIVE-17954.01.patch, HIVE-17954.02.patch,
> HIVE-17954.03.patch, HIVE-17954.04.patch, HIVE-17954.05.patch,
> HIVE-17954.06.patch
>
>
> Implement the following commands:
> -- Pool management.
> CREATE POOL `resource_plan`.`pool_path` WITH
> ALLOC_FRACTION `fraction`
> QUERY_PARALLELISM `parallelism`
> SCHEDULING_POLICY `policy`;
> ALTER POOL `resource_plan`.`pool_path` SET
> PATH = `new_path`,
> ALLOC_FRACTION = `fraction`,
> QUERY_PARALLELISM = `parallelism`,
> SCHEDULING_POLICY = `policy`;
> DROP POOL `resource_plan`.`pool_path`;
> -- Trigger to pool mappings.
> ALTER RESOURCE PLAN `resource_plan`
> ADD TRIGGER `trigger_name` TO `pool_path`;
> ALTER RESOURCE PLAN `resource_plan`
> DROP TRIGGER `trigger_name` TO `pool_path`;
> -- User/Group to pool mappings.
> CREATE USER|GROUP MAPPING `resource_plan`.`group_or_user_name`
> TO `pool_path` WITH ORDERING `order_no`;
> DROP USER|GROUP MAPPING `resource_plan`.`group_or_user_name`;
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18055) cache like pattern object using map object in like function
[
https://issues.apache.org/jira/browse/HIVE-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253491#comment-16253491
]
Hive QA commented on HIVE-18055:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897711/HIVE-18055.3-branch-1.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7829/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7829/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7829/
Messages:
{noformat}
This message was trimmed, see log for full details
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/util/Comparator.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/util/Iterator.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/util/List.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/util/Map.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/util/StringTokenizer.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/2.6.0/hadoop-common-2.6.0.jar(org/apache/hadoop/conf/Configuration.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/2.6.0/hadoop-common-2.6.0.jar(org/apache/hadoop/fs/Path.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/2.6.0/hadoop-common-2.6.0.jar(org/apache/hadoop/util/StringUtils.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/2.6.0/hadoop-common-2.6.0.jar(org/apache/hadoop/util/VersionInfo.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/lang/Iterable.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/2.6.0/hadoop-common-2.6.0.jar(org/apache/hadoop/io/Writable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/lang/String.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/aggregate/jetty-all-server/7.6.0.v20120127/jetty-all-server-7.6.0.v20120127.jar(org/eclipse/jetty/http/HttpStatus.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/util/HashMap.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/com/sun/jersey/jersey-core/1.14/jersey-core-1.14.jar(javax/ws/rs/core/MediaType.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/com/sun/jersey/jersey-core/1.14/jersey-core-1.14.jar(javax/ws/rs/core/Response.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-branch-1.2-source/ql/target/hive-exec-1.2.3-SNAPSHOT.jar(org/codehaus/jackson/map/ObjectMapper.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/lang/Exception.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/lang/Throwable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-7-openjdk-amd64/lib/ct.sym(META-INF/sym/rt.jar/java/io/Serializable.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/com/sun/jersey/jersey-server/1.14/jersey-server-1.14.jar(com/sun/jersey/api/core/PackagesResourceConfig.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/com/sun/jersey/jersey-servlet/1.14/jersey-servlet-1.14.jar(com/sun/jersey/spi/container/servlet/ServletContainer.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-branch-1.2-source/common/target/hive-common-1.2.3-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceStability.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-hdfs/2.6.0/hadoop-hdfs-2.6.0.jar(org/apache/hadoop/hdfs/web/AuthFilter.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/2.6.0/hadoop-common-2.6.0.jar(org/apache/hadoop/security/UserGroupInformation.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-auth/2.6.0/hadoop-auth-2.6.0.jar(org/apache/hadoop/security/authentication/client/PseudoAuthenticator.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-auth/2.6.0/hadoop-auth-2.6.0.jar(org/apache/hadoop/security/authentication/server/PseudoAuthenticationHandler.class)]]
[loading
[jira] [Commented] (HIVE-17954) Implement pool, user, group and trigger to pool management API's.
[
https://issues.apache.org/jira/browse/HIVE-17954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253485#comment-16253485
]
Hive QA commented on HIVE-17954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12897686/HIVE-17954.05.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 11371 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=174)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testApplyPlanQpChanges
(batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testDestroyAndReturn
(batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testQueueName
(batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testReopen (batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testReuse (batchId=281)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testReuseWithQueueing
(batchId=281)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hive.hcatalog.templeton.TestConcurrentJobRequestsThreadsAndTimeout.ConcurrentListJobsVerifyExceptions
(batchId=185)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.org.apache.hive.jdbc.TestTriggersMoveWorkloadManager
(batchId=233)
org.apache.hive.jdbc.TestTriggersWorkloadManager.org.apache.hive.jdbc.TestTriggersWorkloadManager
(batchId=230)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7828/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7828/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7828/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12897686 - PreCommit-HIVE-Build
> Implement pool, user, group and trigger to pool management API's.
> -
>
> Key: HIVE-17954
> URL: https://issues.apache.org/jira/browse/HIVE-17954
> Project: Hive
> Issue Type: Sub-task
>Reporter: Harish Jaiprakash
>Assignee: Harish Jaiprakash
> Attachments: HIVE-17954.01.patch, HIVE-17954.02.patch,
> HIVE-17954.03.patch, HIVE-17954.04.patch, HIVE-17954.05.patch,
> HIVE-17954.06.patch
>
>
> Implement the following commands:
> -- Pool management.
> CREATE POOL `resource_plan`.`pool_path` WITH
> ALLOC_FRACTION `fraction`
> QUERY_PARALLELISM `parallelism`
> SCHEDULING_POLICY `policy`;
> ALTER POOL `resource_plan`.`pool_path` SET
> PATH = `new_path`,
> ALLOC_FRACTION = `fraction`,
> QUERY_PARALLELISM = `parallelism`,
> SCHEDULING_POLICY = `policy`;
> DROP POOL `resource_plan`.`pool_path`;
> -- Trigger to pool mappings.
> ALTER RESOURCE PLAN `resource_plan`
> ADD TRIGGER `trigger_name` TO `pool_path`;
> ALTER RESOURCE PLAN `resource_plan`
> DROP TRIGGER `trigger_name` TO `pool_path`;
> -- User/Group to pool mappings.
> CREATE USER|GROUP MAPPING `resource_plan`.`group_or_user_name`
> TO `pool_path` WITH ORDERING `order_no`;
> DROP USER|GROUP MAPPING `resource_plan`.`group_or_user_name`;
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
