[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14105327#comment-14105327 ] Rui Li commented on HIVE-7772: -- This patch adds some simple cases. Other cases require join or union to be ready. I also found some errors in the output file for some cases (e.g. enforce_order.q): {noformat} [Error 30017]: Skipping stats aggregation by error org.apache.hadoop.hive.ql.metadata.HiveException: [Error 30015]: Stats aggregator of type counter cannot be connected to {noformat} I think this is related to HIVE-7761, so I left these cases out as well. > Add tests for order/sort/distribute/cluster by query [Spark Branch] > --- > > Key: HIVE-7772 > URL: https://issues.apache.org/jira/browse/HIVE-7772 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Attachments: HIVE-7772-spark.patch > > > Now that these queries are supported, we should have tests to catch any > problems we may have. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14105352#comment-14105352 ] Hive QA commented on HIVE-7772: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12663387/HIVE-7772-spark.patch {color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 5983 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_fs_default_name2 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union_null {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/74/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/74/console Test logs: http://ec2-54-176-176-199.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-74/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12663387 > Add tests for order/sort/distribute/cluster by query [Spark Branch] > --- > > Key: HIVE-7772 > URL: https://issues.apache.org/jira/browse/HIVE-7772 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Attachments: HIVE-7772-spark.patch > > > Now that these queries are supported, we should have tests to catch any > problems we may have. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14105464#comment-14105464 ] Brock Noland commented on HIVE-7772: Nice work [~lirui]!! I think union_null failed since it's non-deterministic. I have a patch up on HIVE-7820 to fix this. > Add tests for order/sort/distribute/cluster by query [Spark Branch] > --- > > Key: HIVE-7772 > URL: https://issues.apache.org/jira/browse/HIVE-7772 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Attachments: HIVE-7772-spark.patch > > > Now that these queries are supported, we should have tests to catch any > problems we may have. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106297#comment-14106297 ] Rui Li commented on HIVE-7772: -- Thanks [~brocknoland], let me rebase my branch and see if I can add more tests. > Add tests for order/sort/distribute/cluster by query [Spark Branch] > --- > > Key: HIVE-7772 > URL: https://issues.apache.org/jira/browse/HIVE-7772 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Attachments: HIVE-7772-spark.patch > > > Now that these queries are supported, we should have tests to catch any > problems we may have. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106731#comment-14106731 ] Rui Li commented on HIVE-7772: -- Hi [~brocknoland], I tested other cases with latest code again but the error message I mentioned earlier still remains. These queries have a stage of {{Stats-Aggr Operator}}, which I think caused the error. It seems HIVE-7819 only avoids the exception but we still don't have a proper counter for spark task, so that {{CounterStatsAggregator.connect}} returns false and leads to the connection error I mentioned. I think we can include the tests in the patch for now and add more once spark counter is ready? > Add tests for order/sort/distribute/cluster by query [Spark Branch] > --- > > Key: HIVE-7772 > URL: https://issues.apache.org/jira/browse/HIVE-7772 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Attachments: HIVE-7772-spark.patch > > > Now that these queries are supported, we should have tests to catch any > problems we may have. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107290#comment-14107290 ] Brock Noland commented on HIVE-7772: bq. I think we can include the tests in the patch for now and add more once spark counter is ready? Agreed with your analysis and conclusion. > Add tests for order/sort/distribute/cluster by query [Spark Branch] > --- > > Key: HIVE-7772 > URL: https://issues.apache.org/jira/browse/HIVE-7772 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Attachments: HIVE-7772-spark.patch > > > Now that these queries are supported, we should have tests to catch any > problems we may have. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107302#comment-14107302 ] Brock Noland commented on HIVE-7772: +1 > Add tests for order/sort/distribute/cluster by query [Spark Branch] > --- > > Key: HIVE-7772 > URL: https://issues.apache.org/jira/browse/HIVE-7772 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Attachments: HIVE-7772-spark.patch > > > Now that these queries are supported, we should have tests to catch any > problems we may have. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14108704#comment-14108704 ] Rui Li commented on HIVE-7772: -- Add more tests when counter is ready. > Add tests for order/sort/distribute/cluster by query [Spark Branch] > --- > > Key: HIVE-7772 > URL: https://issues.apache.org/jira/browse/HIVE-7772 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Fix For: spark-branch > > Attachments: HIVE-7772-spark.patch > > > Now that these queries are supported, we should have tests to catch any > problems we may have. -- This message was sent by Atlassian JIRA (v6.2#6252)