[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988677#comment-14988677 ] Szehon Ho commented on HIVE-12330: -- The issue is that as part of HIVE-11489, I am cleaning the generated 'TestSparkCliDriver' before running the test. Turns out multiple tests can be running on the same machine, so this is not so good. I tuned the PTest server to run the drivers not in parallel, going to give a try. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Szehon Ho > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988842#comment-14988842 ] Hive QA commented on HIVE-12330: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12770487/HIVE-12229.3-spark.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 9650 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.initializationError org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/990/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/990/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-990/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12770487 - PreCommit-HIVE-SPARK-Build > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Szehon Ho > Attachments: HIVE-12229.3-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14994609#comment-14994609 ] Sergio Peña commented on HIVE-12330: I did more research about a solution for this. I will move with #3 where we have to rework in the generated testdriver file. My plan is to not use the QFileNames.txt file anymore. I will make some changes, and upload a patch for the test. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997172#comment-14997172 ] Hive QA commented on HIVE-12330: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12771135/HIVE-12330.4-spark.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 37 failed/errored test(s), 9159 tests executed *Failed tests:* {noformat} TestCliDriver-alter_table_not_sorted.q-groupby4_map.q-exim_11_managed_external.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-auto_join30.q-orc_ppd_decimal.q-input17.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-auto_join_reordering_values.q-vector_decimal_trailing.q-udf_concat_ws.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-bucket_map_join_tez1.q-fetch_aggregation.q-drop_database_removes_partition_dirs.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-bucketsortoptimize_insert_7.q-dynpart_sort_optimization2.q-decimal_3.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-cbo_rp_lineage2.q-authorization_update.q-udf_pmod.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-compute_stats_string.q-show_columns.q-vector_auto_smb_mapjoin_14.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-gby_star.q-decimal_join.q-lateral_view_cp.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-groupby3_map.q-current_date_timestamp.q-skewjoinopt8.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-groupby_grouping_sets5.q-auto_sortmerge_join_13.q-udf_concat.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-groupby_map_ppr_multi_distinct.q-vectorization_16.q-union_remove_15.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-index_compact.q-bucketcontext_3.q-merge_dynamic_partition2.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-insert_values_non_partitioned.q-union5.q-udf_lower.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-metadataonly1.q-union19.q-groupby_grouping_sets6.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-skewjoin_mapjoin7.q-optional_outer.q-ptf_rcfile.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-udf_bitwise_and.q-bucketcontext_4.q-orc_ends_with_nulls.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-udf_log2.q-cbo_rp_views.q-alter_merge_stats.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-unicode_notation.q-ppd_join4.q-union27.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-vectorization_10.q-list_bucket_dml_2.q-union26.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-vectorized_parquet.q-stats12.q-binarysortable_1.q-and-12-more - did not produce a TEST-*.xml file TestCliDriver-view.q-udf2.q-input16.q-and-12-more - did not produce a TEST-*.xml file TestMiniLlapCliDriver - did not produce a TEST-*.xml file TestMiniTezCliDriver-auto_sortmerge_join_7.q-tez_union_group_by.q-orc_merge9.q-and-12-more - did not produce a TEST-*.xml file TestMiniTezCliDriver-constprog_dpp.q-dynamic_partition_pruning.q-vectorization_10.q-and-12-more - did not produce a TEST-*.xml file TestMiniTezCliDriver-delete_where_non_partitioned.q-auto_sortmerge_join_16.q-skewjoin.q-and-12-more - did not produce a TEST-*.xml file TestMiniTezCliDriver-update_orig_table.q-union2.q-unionDistinct_2.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-avro_joins.q-join36.q-join4.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-bucketmapjoin3.q-enforce_order.q-union11.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-groupby_complex_types.q-auto_join9.q-groupby_map_ppr.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-groupby_grouping_id2.q-vectorization_13.q-auto_sortmerge_join_13.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-runtime_skewjoin_mapjoin_spark.q-union_remove_9.q-ppd_multi_insert.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-smb_mapjoin_15.q-mapreduce2.q-mapreduce1.q-and-12-more - did not produce a TEST-*.xml file TestSparkCliDriver-union22.q-union_remove_23.q-transform_ppr2.q-and-5-more - did not produce a TEST-*.xml file TestSparkCliDriver-union_top_level.q-auto_join1.q-join18.q-and-12-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/994/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIV
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997794#comment-14997794 ] Hive QA commented on HIVE-12330: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12771455/HIVE-12330.4-spark.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 41 failed/errored test(s), 9680 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_bucket_map_join_tez1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_bucket_map_join_tez2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_constprog_dpp org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_dynamic_partition_pruning org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_dynamic_partition_pruning_2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_hybridgrace_hashjoin_1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_hybridgrace_hashjoin_2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llapdecider org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_lvj_mapjoin org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_mapjoin_decimal org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_mrr org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_ppd_basic org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_bmj_schema_evolution org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dml org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dynpart_hashjoin_1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dynpart_hashjoin_2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_fsstat org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_insert_overwrite_local_directory_1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_hash org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_result_complex org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_tests org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_joins_explain org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_multi_union org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_schema_evolution org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_self_join org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_smb_1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_smb_main org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_decimal org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_dynamic_partition org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_group_by org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_multiinsert org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vectorized_dynamic_partition_pruning org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/995/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/995/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-995/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 41 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12771455 - PreCommit-HIVE-SPARK-Build > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999267#comment-14999267 ] Hive QA commented on HIVE-12330: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12771580/HIVE-12330.5-spark.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 42 failed/errored test(s), 9659 tests executed *Failed tests:* {noformat} TestMiniTezCliDriver-tez_joins_explain.q-enforce_order.q-vector_partition_diff_num_cols.q-and-12-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_bucket_map_join_tez1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_bucket_map_join_tez2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_constprog_dpp org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_dynamic_partition_pruning org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_dynamic_partition_pruning_2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_hybridgrace_hashjoin_1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_hybridgrace_hashjoin_2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llapdecider org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_lvj_mapjoin org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_mapjoin_decimal org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_mrr org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_ppd_basic org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_bmj_schema_evolution org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dml org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dynpart_hashjoin_1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dynpart_hashjoin_2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_fsstat org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_insert_overwrite_local_directory_1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_hash org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_result_complex org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_tests org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_joins_explain org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_multi_union org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_schema_evolution org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_self_join org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_smb_1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_smb_main org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_decimal org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_dynamic_partition org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_group_by org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_multiinsert org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_1 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vectorized_dynamic_partition_pruning org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/997/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/997/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-997/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 42 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12771580 - PreCommit-HIVE-SPARK-Build > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Pr
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999473#comment-14999473 ] Sergio Peña commented on HIVE-12330: [~xuefuz] Do you know if the MiniLlap driver is failing in you side? It is failing on mine, but I don't know why it did not fail on Jenkins before. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999516#comment-14999516 ] Xuefu Zhang commented on HIVE-12330: Yes, it's also failing on my local box too, but it passes in master. I guess this could be due to the fact that spark branch doesn't have all changes in the master. I can do another merge to resolve the issue. In the meantime, if those are the only failures, I think you patch is good. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999532#comment-14999532 ] Szehon Ho commented on HIVE-12330: -- Agree, +1, thanks for working on this Sergio! > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999572#comment-14999572 ] Sergio Peña commented on HIVE-12330: I need to fix 2 more drivers {{TestHBaseCliDriver.vm}} and {{TestHBaseNegativeCliDriver.vm}}. I left those at the end because they need special initialization, and I did want to do a quick test about the current changes on this patch. [~xuefuz] [~szehon] If you think this patch is enough, then I can commit it. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999578#comment-14999578 ] Szehon Ho commented on HIVE-12330: -- OK go ahead and fix those. Thanks > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15002617#comment-15002617 ] Hive QA commented on HIVE-12330: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12771990/HIVE-12330.6-spark.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 9783 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_annotate_stats_groupby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constantPropagateForSubQuery org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_memcheck org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver org.apache.hive.jdbc.TestSSL.testSSLVersion org.apache.tez.dag.app.rm.TestLlapTaskSchedulerService.testSimpleNoLocalityAllocation {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/999/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/999/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-999/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12771990 - PreCommit-HIVE-SPARK-Build > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15002632#comment-15002632 ] Sergio Peña commented on HIVE-12330: [~xuefuz] [~szehon] This is the final patch. The tests do not look related to the changes, are you ok with doing the commit? > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15002667#comment-15002667 ] Xuefu Zhang commented on HIVE-12330: [~spena], thanks for working on this. I'll leave the code review to Szehon, but I have two questions: 1. To confirm, these changes are applicable to master, right? That will happen the next time we merge spark branch to master. 2. Test run #998, which ran against latest spark branch w/ a dummy patch, seems having less failures than this one. I'd just to make sure that the env is indeed okay. Could you double check? We can reattach the patch to have another run. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15002684#comment-15002684 ] Sergio Peña commented on HIVE-12330: They're applicable to master. I'll re-attach the patch to see the tests. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15002843#comment-15002843 ] Szehon Ho commented on HIVE-12330: -- Thanks Sergio for the fix, +1 pending tests. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003035#comment-15003035 ] Hive QA commented on HIVE-12330: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12772029/HIVE-12330.7-spark.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 9783 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_annotate_stats_groupby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constantPropagateForSubQuery org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/1000/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/1000/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-1000/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 8 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12772029 - PreCommit-HIVE-SPARK-Build > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003093#comment-15003093 ] Sergio Peña commented on HIVE-12330: The errors are because output differences. The patch did not touch any of the output. Not sure why it is still failing here. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003097#comment-15003097 ] Xuefu Zhang commented on HIVE-12330: Yeah. It's a little unpredictable as well. Let's commit this and investigate those failures in a separate JIRA. Thanks. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003158#comment-15003158 ] Xuefu Zhang commented on HIVE-12330: spark-branch > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Fix For: 2.0.0 > > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003165#comment-15003165 ] Szehon Ho commented on HIVE-12330: -- It would be great to get this in master as well.. although I guess spark will plan to merge into master soon? > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Fix For: spark-branch > > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003171#comment-15003171 ] Sergio Peña commented on HIVE-12330: What about leaving it in Spark for a while so we can test everything is correct? until the above tests are fixed? > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Fix For: spark-branch > > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12330) Fix precommit Spark test part2
[ https://issues.apache.org/jira/browse/HIVE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003188#comment-15003188 ] Szehon Ho commented on HIVE-12330: -- Hm those failures don't seem related (would have been initiailization errors), but sounds like a good plan. > Fix precommit Spark test part2 > -- > > Key: HIVE-12330 > URL: https://issues.apache.org/jira/browse/HIVE-12330 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Szehon Ho >Assignee: Sergio Peña > Fix For: spark-branch > > Attachments: HIVE-12229.3-spark.patch, HIVE-12330.4-spark.patch, > HIVE-12330.5-spark.patch, HIVE-12330.6-spark.patch, HIVE-12330.7-spark.patch > > > Regression because of HIVE-11489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)