[jira] [Commented] (HIVE-13584) HBaseStorageHandler should support table pre-split
[
https://issues.apache.org/jira/browse/HIVE-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15942007#comment-15942007
]
Hive QA commented on HIVE-13584:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12817040/HIVE-13584.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 10512 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[drop_with_concurrency]
(batchId=231)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=141)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4371/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4371/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4371/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12817040 - PreCommit-HIVE-Build
> HBaseStorageHandler should support table pre-split
> --
>
> Key: HIVE-13584
> URL: https://issues.apache.org/jira/browse/HIVE-13584
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Affects Versions: 2.0.0
>Reporter: Svetozar Ivanov
>Assignee: Svetozar Ivanov
> Labels: HBase, PreSplit
> Attachments: HIVE-13584.1.patch, HIVE-13584.2.patch,
> HIVE-13584.3.patch, HIVE-13584.4.patch, HIVE-13584-branch-2.0.patch,
> HIVE-13584.patch
>
>
> Currently is not possible creation of HBase table via HBaseStorageHandler
> with specified pre-split strategy.
> I use it in my code with applied patch in that way:
> {code}
> public String buildCreateTblStatement(TableDescriptor tableDescriptor) {
> StringBuilder createTableStatement = ...;
> ...
> ...
> String tblProperties = buildTblProperties(tableDescriptor);
> if (StringUtils.isNotBlank(tblProperties)) {
> createTableStatement.append(" TBLPROPERTIES (" + tblProperties +
> ")");
> }
> ...
> return createTableStatement.toString();
> }
> private static String buildTblProperties(TableDescriptor tableDescriptor) {
> ...
> ...
> if (supportsPreSplit(tableDescriptor)) {
> sb.append(", ");
> sb.append(quoteInSingleQuotes(HBaseStorageHandler.SPLITS_NUMBER));
> sb.append(" = ");
>
> sb.append(quoteInSingleQuotes(String.valueOf(tableDescriptor.getNumRegions(;
> GeneratedPKDescriptor columnDescriptor = (GeneratedPKDescriptor)
> tableDescriptor.getPkDescriptors().get(0);
> sb.append(", ");
>
> sb.append(quoteInSingleQuotes(HBaseStorageHandler.SPLIT_ALGORITHM_FIRSTROW_KEY));
> sb.append(" = ");
>
> sb.append(quoteInSingleQuotes(columnDescriptor.getPkGenerator().firstRowKey()));
> sb.append(", ");
>
> sb.append(quoteInSingleQuotes(HBaseStorageHandler.SPLIT_ALGORITHM_LASTROW_KEY));
> sb.append(" = ");
>
> sb.append(quoteInSingleQuotes(columnDescriptor.getPkGenerator().lastRowKey()));
> }
> logger.trace("TBLPROPERTIES are [{}]", sb);
> return sb.toString();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-13584) HBaseStorageHandler should support table pre-split
[
https://issues.apache.org/jira/browse/HIVE-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15941958#comment-15941958
]
Pengcheng Xiong commented on HIVE-13584:
Hello, I am deferring this to Hive 3.0 as we are going to cut the first RC and
it is not marked as blocker. Please feel free to commit to the branch if this
can be resolved before the release.
> HBaseStorageHandler should support table pre-split
> --
>
> Key: HIVE-13584
> URL: https://issues.apache.org/jira/browse/HIVE-13584
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Affects Versions: 2.0.0
>Reporter: Svetozar Ivanov
>Assignee: Svetozar Ivanov
> Labels: HBase, PreSplit
> Attachments: HIVE-13584.1.patch, HIVE-13584.2.patch,
> HIVE-13584.3.patch, HIVE-13584.4.patch, HIVE-13584-branch-2.0.patch,
> HIVE-13584.patch
>
>
> Currently is not possible creation of HBase table via HBaseStorageHandler
> with specified pre-split strategy.
> I use it in my code with applied patch in that way:
> {code}
> public String buildCreateTblStatement(TableDescriptor tableDescriptor) {
> StringBuilder createTableStatement = ...;
> ...
> ...
> String tblProperties = buildTblProperties(tableDescriptor);
> if (StringUtils.isNotBlank(tblProperties)) {
> createTableStatement.append(" TBLPROPERTIES (" + tblProperties +
> ")");
> }
> ...
> return createTableStatement.toString();
> }
> private static String buildTblProperties(TableDescriptor tableDescriptor) {
> ...
> ...
> if (supportsPreSplit(tableDescriptor)) {
> sb.append(", ");
> sb.append(quoteInSingleQuotes(HBaseStorageHandler.SPLITS_NUMBER));
> sb.append(" = ");
>
> sb.append(quoteInSingleQuotes(String.valueOf(tableDescriptor.getNumRegions(;
> GeneratedPKDescriptor columnDescriptor = (GeneratedPKDescriptor)
> tableDescriptor.getPkDescriptors().get(0);
> sb.append(", ");
>
> sb.append(quoteInSingleQuotes(HBaseStorageHandler.SPLIT_ALGORITHM_FIRSTROW_KEY));
> sb.append(" = ");
>
> sb.append(quoteInSingleQuotes(columnDescriptor.getPkGenerator().firstRowKey()));
> sb.append(", ");
>
> sb.append(quoteInSingleQuotes(HBaseStorageHandler.SPLIT_ALGORITHM_LASTROW_KEY));
> sb.append(" = ");
>
> sb.append(quoteInSingleQuotes(columnDescriptor.getPkGenerator().lastRowKey()));
> }
> logger.trace("TBLPROPERTIES are [{}]", sb);
> return sb.toString();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-13584) HBaseStorageHandler should support table pre-split
[
https://issues.apache.org/jira/browse/HIVE-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369830#comment-15369830
]
Hive QA commented on HIVE-13584:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12817040/HIVE-13584.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10296 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestMinimrCliDriver.org.apache.hadoop.hive.cli.TestMinimrCliDriver
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/455/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/455/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-455/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12817040 - PreCommit-HIVE-MASTER-Build
> HBaseStorageHandler should support table pre-split
> --
>
> Key: HIVE-13584
> URL: https://issues.apache.org/jira/browse/HIVE-13584
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Affects Versions: 2.0.0
>Reporter: Svetozar Ivanov
>Assignee: Svetozar Ivanov
> Labels: HBase, PreSplit
> Attachments: HIVE-13584-branch-2.0.patch, HIVE-13584.1.patch,
> HIVE-13584.2.patch, HIVE-13584.3.patch, HIVE-13584.4.patch, HIVE-13584.patch
>
>
> Currently is not possible creation of HBase table via HBaseStorageHandler
> with specified pre-split strategy.
> I use it in my code with applied patch in that way:
> {code}
> public String buildCreateTblStatement(TableDescriptor tableDescriptor) {
> StringBuilder createTableStatement = ...;
> ...
> ...
> String tblProperties = buildTblProperties(tableDescriptor);
> if (StringUtils.isNotBlank(tblProperties)) {
> createTableStatement.append(" TBLPROPERTIES (" + tblProperties +
> ")");
> }
> ...
> return createTableStatement.toString();
> }
> private static String buildTblProperties(TableDescriptor tableDescriptor) {
> ...
> ...
> if (supportsPreSplit(tableDescriptor)) {
> sb.append(", ");
> sb.append(quoteInSingleQuotes(HBaseStorageHandler.SPLITS_NUMBER));
> sb.append(" = ");
>
> sb.append(quoteInSingleQuotes(String.valueOf(tableDescriptor.getNumRegions(;
> GeneratedPKDescriptor columnDescriptor = (GeneratedPKDescriptor)
> tableDescriptor.getPkDescriptors().get(0);
> sb.append(", ");
>
> sb.append(quoteInSingleQuotes(HBaseStorageHandler.SPLIT_ALGORITHM_FIRSTROW_KEY));
> sb.append(" = ");
>
> sb.append(quoteInSingleQuotes(columnDescriptor.getPkGenerator().firstRowKey()));
> sb.append(", ");
>
> sb.append(quoteInSingleQuotes(HBaseStorageHandler.SPLIT_ALGORITHM_LASTROW_KEY));
> sb.append(" = ");
>
> sb.append(quoteInSingleQuotes(columnDescriptor.getPkGenerator().lastRowKey()));
> }
> logger.trace("TBLPROPERTIES are [{}]", sb);
> return sb.toString();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13584) HBaseStorageHandler should support table pre-split
[ https://issues.apache.org/jira/browse/HIVE-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15336119#comment-15336119 ] Svetozar Ivanov commented on HIVE-13584: https://reviews.apache.org/r/48851/ > HBaseStorageHandler should support table pre-split > -- > > Key: HIVE-13584 > URL: https://issues.apache.org/jira/browse/HIVE-13584 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Affects Versions: 2.0.0 >Reporter: Svetozar Ivanov >Assignee: Svetozar Ivanov > Labels: HBase, PreSplit > Attachments: HIVE-13584-branch-2.0.patch, HIVE-13584.1.patch, > HIVE-13584.2.patch, HIVE-13584.3.patch, HIVE-13584.patch > > > Currently is not possible creation of HBase table via HBaseStorageHandler > with specified pre-split strategy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13584) HBaseStorageHandler should support table pre-split
[ https://issues.apache.org/jira/browse/HIVE-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308385#comment-15308385 ] Jesus Camacho Rodriguez commented on HIVE-13584: Removing 2.1.0 target as issue is not tagged as Critical/Blocker and the RC will be created tomorrow. Please feel free to commit to branch-2.1 anyway and fix for 2.1.0 if this happens before the release. > HBaseStorageHandler should support table pre-split > -- > > Key: HIVE-13584 > URL: https://issues.apache.org/jira/browse/HIVE-13584 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Affects Versions: 2.0.0 >Reporter: Svetozar Ivanov >Assignee: Svetozar Ivanov > Labels: HBase, PreSplit > Attachments: HIVE-13584-branch-2.0.patch, HIVE-13584.1.patch, > HIVE-13584.2.patch, HIVE-13584.3.patch, HIVE-13584.patch > > > Currently is not possible creation of HBase table via HBaseStorageHandler > with specified pre-split strategy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13584) HBaseStorageHandler should support table pre-split
[
https://issues.apache.org/jira/browse/HIVE-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307301#comment-15307301
]
Hive QA commented on HIVE-13584:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12806129/HIVE-13584.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 10156 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestJdbcWithMiniHA - did not produce a TEST-*.xml file
TestJdbcWithMiniMr - did not produce a TEST-*.xml file
TestMiniTezCliDriver-cte_4.q-vector_groupby_reduce.q-empty_join.q-and-12-more -
did not produce a TEST-*.xml file
TestMiniTezCliDriver-enforce_order.q-vector_partition_diff_num_cols.q-unionDistinct_1.q-and-12-more
- did not produce a TEST-*.xml file
TestOperationLoggingAPIWithTez - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.llap.tez.TestConverters.testFragmentSpecToTaskSpec
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLocksInSubquery
{noformat}
Test results:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/462/testReport
Console output:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/462/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-462/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12806129 - PreCommit-HIVE-MASTER-Build
> HBaseStorageHandler should support table pre-split
> --
>
> Key: HIVE-13584
> URL: https://issues.apache.org/jira/browse/HIVE-13584
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
>Affects Versions: 2.0.0
>Reporter: Svetozar Ivanov
>Assignee: Svetozar Ivanov
> Labels: HBase, PreSplit
> Attachments: HIVE-13584-branch-2.0.patch, HIVE-13584.1.patch,
> HIVE-13584.2.patch, HIVE-13584.3.patch, HIVE-13584.patch
>
>
> Currently is not possible creation of HBase table via HBaseStorageHandler
> with specified pre-split strategy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13584) HBaseStorageHandler should support table pre-split
[
https://issues.apache.org/jira/browse/HIVE-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15305196#comment-15305196
]
Hive QA commented on HIVE-13584:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12806129/HIVE-13584.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 251 failed/errored test(s), 6207 tests
executed
*Failed tests:*
{noformat}
TestCliDriver-alter_file_format.q-bucket_map_join_tez1.q-fetch_aggregation.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-alter_rename_table.q-udf_aes_decrypt.q-udf_add.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-alter_table_not_sorted.q-cbo_udf_max.q-exim_11_managed_external.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-ambiguitycheck.q-udf_bitwise_and.q-insert_overwrite_directory.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-authorization_1_sql_std.q-disallow_incompatible_type_change_off.q-schema_evol_text_nonvec_mapwork_table.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-authorization_cli_nonsql.q-orc_createas1.q-skewjoinopt19.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-authorization_cli_stdconfigauth.q-str_to_map.q-touch.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-authorization_create_table_owner_privs.q-cte_7.q-groupby_sort_7.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-authorization_create_temp_table.q-update_all_partitioned.q-lock4.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-authorization_parts.q-authorization_view_1.q-parquet_map_of_maps.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-authorization_role_grant2.q-union_remove_7.q-alter_view_as_select.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-auto_join18.q-dynamic_partition_pruning.q-date_comparison.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-auto_join30.q-udf_bitmap_empty.q-input17.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-auto_join_reordering_values.q-subquery_in.q-partition_timestamp2.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-auto_sortmerge_join_7.q-exim_04_evolved_parts.q-query_with_semi.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-autogen_colalias.q-skewjoinopt3.q-rcfile_merge1.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-avro_add_column.q-bucketmapjoin_negative.q-bucket_map_join_spark2.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-ba_table1.q-udf_notequal.q-input_part1.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-bucketcontext_4.q-orc_ends_with_nulls.q-correlationoptimizer9.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-bucketsortoptimize_insert_7.q-list_bucket_query_multiskew_1.q-skewjoin_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-cast_to_int.q-literal_double.q-skewjoinopt20.q-and-12-more - did
not produce a TEST-*.xml file
TestCliDriver-cbo_rp_gby_empty.q-load_nonpart_authsuccess.q-truncate_table.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-cbo_rp_join1.q-union_top_level.q-join18.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-cbo_rp_limit.q-load_exist_part_authsuccess.q-schema_evol_text_nonvec_mapwork_part_all_primitive.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-cbo_rp_lineage2.q-authorization_update.q-udf_pmod.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-cbo_rp_semijoin.q-custom_input_output_format.q-udf_reflect2.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-cbo_rp_udf_udaf.q-date_udf.q-limit0.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-cbo_rp_views.q-orc_split_elimination.q-authorization_view_disable_cbo_4.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-columnStatsUpdateForStatsOptimizer_2.q-cp_mj_rc.q-show_columns.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-columnstats_partlvl_dp.q-input31.q-udaf_percentile.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-compute_stats_string.q-noalias_subq1.q-vector_auto_smb_mapjoin_14.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-create_func1.q-bucketmapjoin3.q-enforce_order.q-and-12-more - did
not produce a TEST-*.xml file
TestCliDriver-create_genericudf.q-dynamic_partition_insert.q-skewjoinopt16.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-create_or_replace_view.q-compute_stats_double.q-reduce_deduplicate_exclude_gby.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-cte_mat_1.q-groupby_sort_6.q-update_after_multiple_inserts_special_characters.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-decimal_join.q
[jira] [Commented] (HIVE-13584) HBaseStorageHandler should support table pre-split
[
https://issues.apache.org/jira/browse/HIVE-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297634#comment-15297634
]
Hive QA commented on HIVE-13584:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12805526/HIVE-13584.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 82 failed/errored test(s), 9924 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniTezCliDriver-auto_join1.q-schema_evol_text_vec_mapwork_part_all_complex.q-vector_complex_join.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-auto_sortmerge_join_7.q-orc_merge9.q-tez_union_dynamic_partition.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-constprog_dpp.q-dynamic_partition_pruning.q-vectorization_10.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-explainuser_4.q-update_after_multiple_inserts.q-mapreduce2.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-order_null.q-vector_acid3.q-orc_merge10.q-and-12-more -
did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_coalesce.q-cbo_windowing.q-tez_join.q-and-12-more -
did not produce a TEST-*.xml file
TestMiniTezCliDriver-vectorization_13.q-auto_sortmerge_join_13.q-tez_bmj_schema_evolution.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vectorization_16.q-vector_decimal_round.q-orc_merge6.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-bucketsortoptimize_insert_7.q-smb_mapjoin_15.q-mapreduce1.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-groupby_grouping_id2.q-vectorization_13.q-auto_sortmerge_join_13.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-mapreduce2.q-groupby7_noskew.q-vectorization_5.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-union_top_level.q-join11.q-auto_join1.q-and-12-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_bucket4
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_bucket5
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_bucket6
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_disable_merge_for_bucketing
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_infer_bucket_sort_map_operators
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_infer_bucket_sort_num_buckets
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_list_bucket_dml_10
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge2
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge9
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge_diff_fs
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_reduce_deduplicate
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join1
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join2
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join3
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join4
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.org.apache.hadoop.hive.cli.TestMiniTezCliDriver
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_bucketpruning1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cbo_udf_udaf
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_column_names_with_leading_and_trailing_spaces
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_nonvec_mapwork_table
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_select_dummy_source
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_subquery_in
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_aggregate_without_gby
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_ve
[jira] [Commented] (HIVE-13584) HBaseStorageHandler should support table pre-split
[
https://issues.apache.org/jira/browse/HIVE-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295403#comment-15295403
]
Hive QA commented on HIVE-13584:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12805243/HIVE-13584.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 51 failed/errored test(s), 10004 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniTezCliDriver-auto_join1.q-schema_evol_text_vec_mapwork_part_all_complex.q-vector_complex_join.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-auto_sortmerge_join_7.q-orc_merge9.q-tez_union_dynamic_partition.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-dynpart_sort_optimization2.q-tez_dynpart_hashjoin_3.q-orc_vectorization_ppd.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-join1.q-mapjoin_decimal.q-union5.q-and-12-more - did not
produce a TEST-*.xml file
TestMiniTezCliDriver-script_pipe.q-vector_decimal_aggregate.q-vector_data_types.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-tez_union_group_by.q-vector_auto_smb_mapjoin_14.q-union_fast_stats.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_interval_2.q-schema_evol_text_nonvec_mapwork_part_all_primitive.q-tez_fsstat.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testPreemptionQueueComparator
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestFirstInFirstOutComparator.testWaitQueueComparatorWithinDagPriority
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestShortestJobFirstComparator.testWaitQueueComparator
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestShortestJobFirstComparator.testWaitQueueComparatorParallelism
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestShortestJobFirstComparator.testWaitQueueComparatorWithinDagPriority
org.apache.hadoop.hive.llap.tez.TestConverters.testFragmentSpecToTaskSpec
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskCommunicator.testFinishableStateUpdateFailure
org.apache.hadoop.hive.metastore.TestAuthzApiEmbedAuthorizerInRemote.org.apache.hadoop.hive.metastore.TestAuthzApiEmbedAuthorizerInRemote
org.apache.hadoop.hive.metastore.TestFilterHooks.org.apache.hadoop.hive.metastore.TestFilterHooks
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs
org.apache.hadoop.hive.metastore.TestHiveMetaStoreStatsMerge.testStatsMerge
org.apache.hadoop.hive.metastore.TestMetaStoreEndFunctionListener.testEndFunctionListener
org.apache.hadoop.hive.metastore.TestMetaStoreEventListenerOnlyOnCommit.testEventStatus
org.apache.hadoop.hive.metastore.TestMetaStoreInitListener.testMetaStoreInitListener
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.org.apache.hadoop.hive.metastore.TestMetaStoreMetrics
org.apache.hadoop.hive.metastore.TestPartitionNameWhitelistValidation.testAddPartitionWithCommas
org.apache.hadoop.hive.metastore.TestPartitionNameWhitelistValidation.testAddPartitionWithUnicode
org.apache.hadoop.hive.metastore.TestPartitionNameWhitelistValidation.testAddPartitionWithValidPartVal
org.apache.hadoop.hive.metastore.TestPartitionNameWhitelistValidation.testAppendPartitionWithCommas
org.apache.hadoop.hive.metastore.TestPartitionNameWhitelistValidation.testAppendPartitionWithUnicode
org.apache.hadoop.hive.metastore.TestPartitionNameWhitelistValidation.testAppendPartitionWithValidCharacters
org.apache.hadoop.hive.metastore.TestRetryingHMSHandler.testRetryingHMSHandler
org.apache.hadoop.hive.ql.security.TestClientSideAuthorizationProvider.testSimplePrivileges
org.apache.hadoop.hive.ql.security.TestExtendedAcls.org.apache.hadoop.hive.ql.security.TestExtendedAcls
org.apache.hadoop.hive.ql.security.TestFolderPermissions.org.apache.hadoop.hive.ql.security.TestFolderPermissions
org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.testSimplePrivileges
org.apache.hadoop.hive.ql.security.TestMultiAuthorizationPreEventListener.org.apache.hadoop.hive.ql.security.TestMultiAuthorizationPreEventListener
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationDrops.testDropPartition
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProvider.testSimplePrivileges
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProviderWithACL.testSimplePrivileges
org
[jira] [Commented] (HIVE-13584) HBaseStorageHandler should support table pre-split
[
https://issues.apache.org/jira/browse/HIVE-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15293010#comment-15293010
]
Hive QA commented on HIVE-13584:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12804953/HIVE-13584.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/334/testReport
Console output:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/334/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-334/
Messages:
{noformat}
This message was trimmed, see log for full details
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @
hive-cli ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/data/hive-ptest/working/apache-github-source-source/cli/src/test/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-cli ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/cli/target/tmp
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/cli/target/warehouse
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/cli/target/tmp/conf
[copy] Copying 15 files to
/data/hive-ptest/working/apache-github-source-source/cli/target/tmp/conf
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @
hive-cli ---
[INFO] Compiling 4 source files to
/data/hive-ptest/working/apache-github-source-source/cli/target/test-classes
[WARNING]
/data/hive-ptest/working/apache-github-source-source/cli/src/test/org/apache/hadoop/hive/cli/TestCliDriverMethods.java:
/data/hive-ptest/working/apache-github-source-source/cli/src/test/org/apache/hadoop/hive/cli/TestCliDriverMethods.java
uses unchecked or unsafe operations.
[WARNING]
/data/hive-ptest/working/apache-github-source-source/cli/src/test/org/apache/hadoop/hive/cli/TestCliDriverMethods.java:
Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-cli ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-cli ---
[INFO] Building jar:
/data/hive-ptest/working/apache-github-source-source/cli/target/hive-cli-2.1.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @
hive-cli ---
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-cli ---
[INFO] Installing
/data/hive-ptest/working/apache-github-source-source/cli/target/hive-cli-2.1.0-SNAPSHOT.jar
to
/home/hiveptest/.m2/repository/org/apache/hive/hive-cli/2.1.0-SNAPSHOT/hive-cli-2.1.0-SNAPSHOT.jar
[INFO] Installing
/data/hive-ptest/working/apache-github-source-source/cli/pom.xml to
/home/hiveptest/.m2/repository/org/apache/hive/hive-cli/2.1.0-SNAPSHOT/hive-cli-2.1.0-SNAPSHOT.pom
[INFO]
[INFO]
[INFO] Building Hive Contrib 2.1.0-SNAPSHOT
[INFO]
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-contrib ---
[INFO] Deleting
/data/hive-ptest/working/apache-github-source-source/contrib/target
[INFO] Deleting /data/hive-ptest/working/apache-github-source-source/contrib
(includes = [datanucleus.log, derby.log], excludes = [])
[INFO]
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-no-snapshots) @
hive-contrib ---
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-contrib
---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
hive-contrib ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/data/hive-ptest/working/apache-github-source-source/contrib/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-contrib ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-contrib
---
[INFO] Compiling 41 source files to
/data/hive-ptest/working/apache-github-source-source/contrib/target/classes
[WARNING]
/data/hive-ptest/working/apache-github-source-source/contrib/src/java/org/apache/hadoop/hive/contrib/udaf/example/UDAFExampleMax.java:
Some input files use or override a deprecated API.
[WARNING]
/data/hive-ptest/working/apache-github-source-source/contrib/src/java/org/apache/hadoop/hive/contrib/udaf/example/UDAFExampleMax
