[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988677#comment-14988677 ] Szehon Ho commented on HIVE-12330: -- The issue is that as part of HIVE-11489, I am cleaning the generated 'TestSparkCliDriver' before running the test. Turns out multiple tests can be running on the same machine, so this is not so good. I tuned the PTest server to run the drivers not in parallel, going to give a try. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Szehon Ho > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[
https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988842#comment-14988842
]
Hive QA commented on HIVE-12330:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12770487/HIVE-12229.3-spark.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 9650 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.initializationError
org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/990/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/990/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-990/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12770487 - PreCommit-HIVE-SPARK-Build
> Fix precommit Spark test part2
> --
>
> Key: HIVE-12330
> URL: https://issues.apache.org/jira/browse/HIVE-12330
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Attachments: HIVE-12229.3-spark.patch
>
>
> Regression because of HIVE-11489
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14994609#comment-14994609 ] Sergio Peña commented on HIVE-12330: I did more research about a solution for this. I will move with #3 where we have to rework in the generated testdriver file. My plan is to not use the QFileNames.txt file anymore. I will make some changes, and upload a patch for the test. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[
https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997172#comment-14997172
]
Hive QA commented on HIVE-12330:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12771135/HIVE-12330.4-spark.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 37 failed/errored test(s), 9159 tests
executed
*Failed tests:*
{noformat}
TestCliDriver-alter_table_not_sorted.q-groupby4_map.q-exim_11_managed_external.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-auto_join30.q-orc_ppd_decimal.q-input17.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-auto_join_reordering_values.q-vector_decimal_trailing.q-udf_concat_ws.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-bucket_map_join_tez1.q-fetch_aggregation.q-drop_database_removes_partition_dirs.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-bucketsortoptimize_insert_7.q-dynpart_sort_optimization2.q-decimal_3.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-cbo_rp_lineage2.q-authorization_update.q-udf_pmod.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-compute_stats_string.q-show_columns.q-vector_auto_smb_mapjoin_14.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-gby_star.q-decimal_join.q-lateral_view_cp.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-groupby3_map.q-current_date_timestamp.q-skewjoinopt8.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-groupby_grouping_sets5.q-auto_sortmerge_join_13.q-udf_concat.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-groupby_map_ppr_multi_distinct.q-vectorization_16.q-union_remove_15.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-index_compact.q-bucketcontext_3.q-merge_dynamic_partition2.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-insert_values_non_partitioned.q-union5.q-udf_lower.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-metadataonly1.q-union19.q-groupby_grouping_sets6.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-skewjoin_mapjoin7.q-optional_outer.q-ptf_rcfile.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-udf_bitwise_and.q-bucketcontext_4.q-orc_ends_with_nulls.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-udf_log2.q-cbo_rp_views.q-alter_merge_stats.q-and-12-more - did
not produce a TEST-*.xml file
TestCliDriver-unicode_notation.q-ppd_join4.q-union27.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-vectorization_10.q-list_bucket_dml_2.q-union26.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-vectorized_parquet.q-stats12.q-binarysortable_1.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-view.q-udf2.q-input16.q-and-12-more - did not produce a
TEST-*.xml file
TestMiniLlapCliDriver - did not produce a TEST-*.xml file
TestMiniTezCliDriver-auto_sortmerge_join_7.q-tez_union_group_by.q-orc_merge9.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-constprog_dpp.q-dynamic_partition_pruning.q-vectorization_10.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-delete_where_non_partitioned.q-auto_sortmerge_join_16.q-skewjoin.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-update_orig_table.q-union2.q-unionDistinct_2.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-avro_joins.q-join36.q-join4.q-and-12-more - did not produce
a TEST-*.xml file
TestSparkCliDriver-bucketmapjoin3.q-enforce_order.q-union11.q-and-12-more - did
not produce a TEST-*.xml file
TestSparkCliDriver-groupby_complex_types.q-auto_join9.q-groupby_map_ppr.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-groupby_grouping_id2.q-vectorization_13.q-auto_sortmerge_join_13.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-runtime_skewjoin_mapjoin_spark.q-union_remove_9.q-ppd_multi_insert.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-smb_mapjoin_15.q-mapreduce2.q-mapreduce1.q-and-12-more - did
not produce a TEST-*.xml file
TestSparkCliDriver-union22.q-union_remove_23.q-transform_ppr2.q-and-5-more -
did not produce a TEST-*.xml file
TestSparkCliDriver-union_top_level.q-auto_join1.q-join18.q-and-12-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic
org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/994/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIV
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[
https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997794#comment-14997794
]
Hive QA commented on HIVE-12330:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12771455/HIVE-12330.4-spark.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 41 failed/errored test(s), 9680 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_bucket_map_join_tez1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_bucket_map_join_tez2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_constprog_dpp
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_dynamic_partition_pruning
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_dynamic_partition_pruning_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_hybridgrace_hashjoin_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_hybridgrace_hashjoin_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llapdecider
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_lvj_mapjoin
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_mapjoin_decimal
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_mrr
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_ppd_basic
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_bmj_schema_evolution
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dml
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dynpart_hashjoin_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dynpart_hashjoin_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_fsstat
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_insert_overwrite_local_directory_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_result_complex
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_tests
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_joins_explain
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_multi_union
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_schema_evolution
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_self_join
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_smb_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_smb_main
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_decimal
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_dynamic_partition
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_group_by
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vectorized_dynamic_partition_pruning
org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/995/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/995/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-995/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 41 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12771455 - PreCommit-HIVE-SPARK-Build
> Fix precommit Spark test part2
> --
>
> Key: HIVE-12330
> URL: https://issues.apache.org/jira/browse/HIVE-12330
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Reporter: Szehon Ho
>Assignee
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[
https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999267#comment-14999267
]
Hive QA commented on HIVE-12330:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12771580/HIVE-12330.5-spark.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 42 failed/errored test(s), 9659 tests
executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver-tez_joins_explain.q-enforce_order.q-vector_partition_diff_num_cols.q-and-12-more
- did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_bucket_map_join_tez1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_bucket_map_join_tez2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_constprog_dpp
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_dynamic_partition_pruning
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_dynamic_partition_pruning_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_hybridgrace_hashjoin_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_hybridgrace_hashjoin_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llapdecider
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_lvj_mapjoin
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_mapjoin_decimal
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_mrr
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_ppd_basic
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_bmj_schema_evolution
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dml
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dynpart_hashjoin_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dynpart_hashjoin_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_fsstat
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_insert_overwrite_local_directory_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_hash
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_result_complex
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_tests
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_joins_explain
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_multi_union
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_schema_evolution
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_self_join
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_smb_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_smb_main
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_decimal
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_dynamic_partition
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_group_by
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_multiinsert
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vectorized_dynamic_partition_pruning
org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/997/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/997/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-997/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 42 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12771580 - PreCommit-HIVE-SPARK-Build
> Fix precommit Spark test part2
> --
>
> Key: HIVE-12330
> URL: https://issues.apache.org/jira/browse/HIVE-12330
> Pr
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999473#comment-14999473 ] Sergio Peña commented on HIVE-12330: [~xuefuz] Do you know if the MiniLlap driver is failing in you side? It is failing on mine, but I don't know why it did not fail on Jenkins before. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999516#comment-14999516 ] Xuefu Zhang commented on HIVE-12330: Yes, it's also failing on my local box too, but it passes in master. I guess this could be due to the fact that spark branch doesn't have all changes in the master. I can do another merge to resolve the issue. In the meantime, if those are the only failures, I think you patch is good. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999532#comment-14999532 ] Szehon Ho commented on HIVE-12330: -- Agree, +1, thanks for working on this Sergio! > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[
https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999572#comment-14999572
]
Sergio Peña commented on HIVE-12330:
I need to fix 2 more drivers {{TestHBaseCliDriver.vm}} and
{{TestHBaseNegativeCliDriver.vm}}. I left those at the end because they need
special initialization, and I did want to do a quick test about the current
changes on this patch.
[~xuefuz] [~szehon] If you think this patch is enough, then I can commit it.
> Fix precommit Spark test part2
> --
>
> Key: HIVE-12330
> URL: https://issues.apache.org/jira/browse/HIVE-12330
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Reporter: Szehon Ho
>Assignee: Sergio Peña
> Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch,
> HIVE-12330.5-spark.patch
>
>
> Regression because of HIVE-11489
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999578#comment-14999578 ] Szehon Ho commented on HIVE-12330: -- OK go ahead and fix those. Thanks > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[
https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15002617#comment-15002617
]
Hive QA commented on HIVE-12330:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12771990/HIVE-12330.6-spark.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 9783 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_annotate_stats_groupby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constantPropagateForSubQuery
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_memcheck
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver
org.apache.hive.jdbc.TestSSL.testSSLVersion
org.apache.tez.dag.app.rm.TestLlapTaskSchedulerService.testSimpleNoLocalityAllocation
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/999/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/999/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-999/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12771990 - PreCommit-HIVE-SPARK-Build
> Fix precommit Spark test part2
> --
>
> Key: HIVE-12330
> URL: https://issues.apache.org/jira/browse/HIVE-12330
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Reporter: Szehon Ho
>Assignee: Sergio Peña
> Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch,
> HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch
>
>
> Regression because of HIVE-11489
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15002632#comment-15002632 ] Sergio Peña commented on HIVE-12330: [~xuefuz] [~szehon] This is the final patch. The tests do not look related to the changes, are you ok with doing the commit? > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15002667#comment-15002667 ] Xuefu Zhang commented on HIVE-12330: [~spena], thanks for working on this. I'll leave the code review to Szehon, but I have two questions: 1. To confirm, these changes are applicable to master, right? That will happen the next time we merge spark branch to master. 2. Test run #998, which ran against latest spark branch w/ a dummy patch, seems having less failures than this one. I'd just to make sure that the env is indeed okay. Could you double check? We can reattach the patch to have another run. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15002684#comment-15002684 ] Sergio Peña commented on HIVE-12330: They're applicable to master. I'll re-attach the patch to see the tests. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15002843#comment-15002843 ] Szehon Ho commented on HIVE-12330: -- Thanks Sergio for the fix, +1 pending tests. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[
https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003035#comment-15003035
]
Hive QA commented on HIVE-12330:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12772029/HIVE-12330.7-spark.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 9783 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_annotate_stats_groupby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constantPropagateForSubQuery
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/1000/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/1000/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-1000/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12772029 - PreCommit-HIVE-SPARK-Build
> Fix precommit Spark test part2
> --
>
> Key: HIVE-12330
> URL: https://issues.apache.org/jira/browse/HIVE-12330
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Reporter: Szehon Ho
>Assignee: Sergio Peña
> Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch,
> HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch
>
>
> Regression because of HIVE-11489
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003093#comment-15003093 ] Sergio Peña commented on HIVE-12330: The errors are because output differences. The patch did not touch any of the output. Not sure why it is still failing here. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003097#comment-15003097 ] Xuefu Zhang commented on HIVE-12330: Yeah. It's a little unpredictable as well. Let's commit this and investigate those failures in a separate JIRA. Thanks. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003158#comment-15003158 ] Xuefu Zhang commented on HIVE-12330: spark-branch > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Fix For: 2.0.0 > > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003165#comment-15003165 ] Szehon Ho commented on HIVE-12330: -- It would be great to get this in master as well.. although I guess spark will plan to merge into master soon? > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Fix For: spark-branch > > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003171#comment-15003171 ] Sergio Peña commented on HIVE-12330: What about leaving it in Spark for a while so we can test everything is correct? until the above tests are fixed? > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Fix For: spark-branch > > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003188#comment-15003188 ] Szehon Ho commented on HIVE-12330: -- Hm those failures don't seem related (would have been initiailization errors), but sounds like a good plan. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Fix For: spark-branch > > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
