[jira] [Commented] (HIVE-14329) fix flapping qtests - because of output string ordering
[ https://issues.apache.org/jira/browse/HIVE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403057#comment-15403057 ] Ashutosh Chauhan commented on HIVE-14329: - +1 > fix flapping qtests - because of output string ordering > --- > > Key: HIVE-14329 > URL: https://issues.apache.org/jira/browse/HIVE-14329 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Minor > Attachments: HIVE-14329.1.patch, HIVE-14329.2.patch, > HIVE-14329.3.patch, HIVE-14329.4.patch > > > it's a bit misleading to see some tests come and go in testresults; for > example: > https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/631/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_stats_list_bucket/history/ > These tests fail occasionally because of the ordering is different in the map. > The usual cause of these failures is a simple hashmap in > {{MetaDataFormatUtils}}: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java#L411 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14329) fix flapping qtests - because of output string ordering
[ https://issues.apache.org/jira/browse/HIVE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401515#comment-15401515 ] Hive QA commented on HIVE-14329: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12821172/HIVE-14329.3.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10421 tests executed *Failed tests:* {noformat} TestMsgBusConnection - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_nullable_union org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_counters org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_avro_non_nullable_union org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_list_bucket_dml_2 org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.stringifyValidTxns org.apache.hadoop.hive.metastore.TestHiveMetaStoreTxns.testTxnRange {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/716/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/716/console Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-716/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 8 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12821172 - PreCommit-HIVE-MASTER-Build > fix flapping qtests - because of output string ordering > --- > > Key: HIVE-14329 > URL: https://issues.apache.org/jira/browse/HIVE-14329 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Minor > Attachments: HIVE-14329.1.patch, HIVE-14329.2.patch, > HIVE-14329.3.patch > > > it's a bit annoying to see some tests come and go in testresults; for example: > https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/631/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_stats_list_bucket/history/ > These tests fail occasionally because of the ordering is different in the map. > The usual cause of these failures is a simple hashmap in > {{MetaDataFormatUtils}}: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java#L411 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14329) fix flapping qtests - because of output string ordering
[ https://issues.apache.org/jira/browse/HIVE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399565#comment-15399565 ] Hive QA commented on HIVE-14329: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12820719/HIVE-14329.2.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 60 failed/errored test(s), 10366 tests executed *Failed tests:* {noformat} TestMiniTezCliDriver-tez_self_join.q-filter_join_breaktask.q-vector_decimal_precision.q-and-12-more - did not produce a TEST-*.xml file TestMsgBusConnection - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_nullable_union org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnStatsUpdateForStatsOptimizer_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_alter_list_bucketing_table1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_skewed_table1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_extrapolate_part_stats_full org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_extrapolate_part_stats_partial org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_extrapolate_part_stats_partial_ndv org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_grouping_operators org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lb_fs_stats org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_multiskew_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_multiskew_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_multiskew_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_list_bucket_dml_10 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_alter_merge_stats_orc org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_column_names_with_leading_and_trailing_spaces org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_ctas org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_deleteAnalyze org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_analyze org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_nonvec_fetchwork_part org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_nonvec_fetchwork_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_nonvec_mapwork_part org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_nonvec_mapwork_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_vec_mapwork_part org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_orc_vec_mapwork_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_stats org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_nonvec_mapwork_part org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_nonvec_mapwork_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_vec_mapwork_part org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_vec_mapwork_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_schema_evol_text_vecrow_mapwork_part org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testC
[jira] [Commented] (HIVE-14329) fix flapping qtests - because of output string ordering
[ https://issues.apache.org/jira/browse/HIVE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398313#comment-15398313 ] Zoltan Haindrich commented on HIVE-14329: - https://reviews.apache.org/r/50586/ > fix flapping qtests - because of output string ordering > --- > > Key: HIVE-14329 > URL: https://issues.apache.org/jira/browse/HIVE-14329 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Minor > Attachments: HIVE-14329.1.patch, HIVE-14329.2.patch > > > it's a bit annoying to see some tests come and go in testresults; for example: > https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/631/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_stats_list_bucket/history/ > These tests fail occasionally because of the ordering is different in the map. > The usual cause of these failures is a simple hashmap in > {{MetaDataFormatUtils}}: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java#L411 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14329) fix flapping qtests - because of output string ordering
[ https://issues.apache.org/jira/browse/HIVE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15396265#comment-15396265 ] Zoltan Haindrich commented on HIVE-14329: - removed: inconsitency: {{StatsSetupConst#setColumnStatsState}} would put lowercase colnames if record is new; original colnames during update/addition choosen path: case sensitive > fix flapping qtests - because of output string ordering > --- > > Key: HIVE-14329 > URL: https://issues.apache.org/jira/browse/HIVE-14329 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Minor > > it's a bit annoying to see some tests come and go in testresults; for example: > https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/631/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_stats_list_bucket/history/ > These tests fail occasionally because of the ordering is different in the map. > The usual cause of these failures is a simple hashmap in > {{MetaDataFormatUtils}}: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java#L411 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14329) fix flapping qtests - because of output string ordering
[ https://issues.apache.org/jira/browse/HIVE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15396251#comment-15396251 ] Zoltan Haindrich commented on HIVE-14329: - removed: upgrade-while-read from: {{StatsSetupConst#areBasicStatsUptoDate}} > fix flapping qtests - because of output string ordering > --- > > Key: HIVE-14329 > URL: https://issues.apache.org/jira/browse/HIVE-14329 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Minor > > it's a bit annoying to see some tests come and go in testresults; for example: > https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/631/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_stats_list_bucket/history/ > These tests fail occasionally because of the ordering is different in the map. > The usual cause of these failures is a simple hashmap in > {{MetaDataFormatUtils}}: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java#L411 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14329) fix flapping qtests - because of output string ordering
[ https://issues.apache.org/jira/browse/HIVE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15396221#comment-15396221 ] Zoltan Haindrich commented on HIVE-14329: - contains: fix for {{StatsSetupConst#setBasicStatsState}}: failed to upgrade&set basic stats status > fix flapping qtests - because of output string ordering > --- > > Key: HIVE-14329 > URL: https://issues.apache.org/jira/browse/HIVE-14329 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Minor > > it's a bit annoying to see some tests come and go in testresults; for example: > https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/631/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_stats_list_bucket/history/ > These tests fail occasionally because of the ordering is different in the map. > The usual cause of these failures is a simple hashmap in > {{MetaDataFormatUtils}}: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java#L411 -- This message was sent by Atlassian JIRA (v6.3.4#6332)