[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14105327#comment-14105327
]
Rui Li commented on HIVE-7772:
--
This patch adds some simple cases. Other cases require join or union to be
ready.
I also found some errors in the output file for some cases (e.g.
enforce_order.q):
{noformat}
[Error 30017]: Skipping stats aggregation by error
org.apache.hadoop.hive.ql.metadata.HiveException: [Error 30015]: Stats
aggregator of type counter cannot be connected to
{noformat}
I think this is related to HIVE-7761, so I left these cases out as well.
> Add tests for order/sort/distribute/cluster by query [Spark Branch]
> ---
>
> Key: HIVE-7772
> URL: https://issues.apache.org/jira/browse/HIVE-7772
> Project: Hive
> Issue Type: Test
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-7772-spark.patch
>
>
> Now that these queries are supported, we should have tests to catch any
> problems we may have.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14105352#comment-14105352
]
Hive QA commented on HIVE-7772:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12663387/HIVE-7772-spark.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 5983 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_fs_default_name2
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union_null
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/74/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/74/console
Test logs:
http://ec2-54-176-176-199.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-74/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12663387
> Add tests for order/sort/distribute/cluster by query [Spark Branch]
> ---
>
> Key: HIVE-7772
> URL: https://issues.apache.org/jira/browse/HIVE-7772
> Project: Hive
> Issue Type: Test
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-7772-spark.patch
>
>
> Now that these queries are supported, we should have tests to catch any
> problems we may have.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14105464#comment-14105464 ] Brock Noland commented on HIVE-7772: Nice work [~lirui]!! I think union_null failed since it's non-deterministic. I have a patch up on HIVE-7820 to fix this. > Add tests for order/sort/distribute/cluster by query [Spark Branch] > --- > > Key: HIVE-7772 > URL: https://issues.apache.org/jira/browse/HIVE-7772 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Attachments: HIVE-7772-spark.patch > > > Now that these queries are supported, we should have tests to catch any > problems we may have. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106297#comment-14106297 ] Rui Li commented on HIVE-7772: -- Thanks [~brocknoland], let me rebase my branch and see if I can add more tests. > Add tests for order/sort/distribute/cluster by query [Spark Branch] > --- > > Key: HIVE-7772 > URL: https://issues.apache.org/jira/browse/HIVE-7772 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Attachments: HIVE-7772-spark.patch > > > Now that these queries are supported, we should have tests to catch any > problems we may have. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106731#comment-14106731
]
Rui Li commented on HIVE-7772:
--
Hi [~brocknoland], I tested other cases with latest code again but the error
message I mentioned earlier still remains. These queries have a stage of
{{Stats-Aggr Operator}}, which I think caused the error.
It seems HIVE-7819 only avoids the exception but we still don't have a proper
counter for spark task, so that {{CounterStatsAggregator.connect}} returns
false and leads to the connection error I mentioned.
I think we can include the tests in the patch for now and add more once spark
counter is ready?
> Add tests for order/sort/distribute/cluster by query [Spark Branch]
> ---
>
> Key: HIVE-7772
> URL: https://issues.apache.org/jira/browse/HIVE-7772
> Project: Hive
> Issue Type: Test
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-7772-spark.patch
>
>
> Now that these queries are supported, we should have tests to catch any
> problems we may have.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107290#comment-14107290 ] Brock Noland commented on HIVE-7772: bq. I think we can include the tests in the patch for now and add more once spark counter is ready? Agreed with your analysis and conclusion. > Add tests for order/sort/distribute/cluster by query [Spark Branch] > --- > > Key: HIVE-7772 > URL: https://issues.apache.org/jira/browse/HIVE-7772 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Attachments: HIVE-7772-spark.patch > > > Now that these queries are supported, we should have tests to catch any > problems we may have. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107302#comment-14107302 ] Brock Noland commented on HIVE-7772: +1 > Add tests for order/sort/distribute/cluster by query [Spark Branch] > --- > > Key: HIVE-7772 > URL: https://issues.apache.org/jira/browse/HIVE-7772 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Attachments: HIVE-7772-spark.patch > > > Now that these queries are supported, we should have tests to catch any > problems we may have. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7772) Add tests for order/sort/distribute/cluster by query [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14108704#comment-14108704 ] Rui Li commented on HIVE-7772: -- Add more tests when counter is ready. > Add tests for order/sort/distribute/cluster by query [Spark Branch] > --- > > Key: HIVE-7772 > URL: https://issues.apache.org/jira/browse/HIVE-7772 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Rui Li >Assignee: Rui Li > Fix For: spark-branch > > Attachments: HIVE-7772-spark.patch > > > Now that these queries are supported, we should have tests to catch any > problems we may have. -- This message was sent by Atlassian JIRA (v6.2#6252)
