[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148343731 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148343732 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43780/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148343576 [Test build #43780 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43780/console) for PR 8416 at commit [`4cd1796`](https://github.com/apache/spark/commit/4cd17966775b9f18c5d6a00b1671c6045f50a25a). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9123#issuecomment-148342508 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43778/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9123#issuecomment-148342506 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9123#issuecomment-148342359 [Test build #43778 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43778/console) for PR 9123 at commit [`6f97fb7`](https://github.com/apache/spark/commit/6f97fb7c9fce2920641e5b9dc63b2830e8037702). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9123#issuecomment-148335402 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43776/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9123#issuecomment-148335401 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9123#issuecomment-148335260 [Test build #43776 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43776/console) for PR 9123 at commit [`ac78af1`](https://github.com/apache/spark/commit/ac78af14508ac02a74202de564253b927f06c237). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4526][MLLIB]Gradient should be added ba...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3677#issuecomment-148333298 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4526][MLLIB]Gradient should be added ba...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3677#issuecomment-148333299 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43781/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4526][MLLIB]Gradient should be added ba...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3677#issuecomment-148332965 [Test build #43781 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43781/console) for PR 3677 at commit [`ad40bb5`](https://github.com/apache/spark/commit/ad40bb5210652a96f5bf48801f1e333aa937a2e9). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK][SPARK-10842]Eliminate creating duplica...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8923#issuecomment-148328330 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43772/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK][SPARK-10842]Eliminate creating duplica...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8923#issuecomment-148328328 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK][SPARK-10842]Eliminate creating duplica...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8923#issuecomment-148328074 [Test build #43772 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43772/console) for PR 8923 at commit [`6a68113`](https://github.com/apache/spark/commit/6a68113ff2bb5492aab8782840f767f999b6d11d). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148326579 [Test build #43782 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43782/console) for PR 9055 at commit [`7511f47`](https://github.com/apache/spark/commit/7511f47089ed58f913a81df2113cbe300903be63). * This patch **fails MiMa tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait SubQueryExpression extends Unevaluable ` * `case class Exists(subquery: LogicalPlan, positive: Boolean)` * `case class InSubquery(child: Expression, subquery: LogicalPlan, positive: Boolean)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148326617 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148326620 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43782/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11125] [SQL] Unreadable exception when ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9134#issuecomment-148326244 [Test build #43783 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43783/consoleFull) for PR 9134 at commit [`5b6e651`](https://github.com/apache/spark/commit/5b6e6510dd3825910659cac95784f56c3ae9df51). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11121][Core] Correct the TaskLocation t...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9096#issuecomment-148325204 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11121][Core] Correct the TaskLocation t...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9096#issuecomment-148325206 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43771/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11121][Core] Correct the TaskLocation t...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9096#issuecomment-148325066 [Test build #43771 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43771/console) for PR 9096 at commit [`8f77575`](https://github.com/apache/spark/commit/8f77575ad4e88e91864e03ab9c5549529e32c5f4). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11126][SQL]Fix a memory leak in SQLList...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9132#issuecomment-148323853 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11126][SQL]Fix a memory leak in SQLList...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9132#issuecomment-148323855 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43773/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11126][SQL]Fix a memory leak in SQLList...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9132#issuecomment-148323716 [Test build #43773 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43773/console) for PR 9132 at commit [`048cd5b`](https://github.com/apache/spark/commit/048cd5b5848c0f89213c897090035ec8cf6295df). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11125] [SQL] Unreadable exception when ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9134#issuecomment-148323656 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11125] [SQL] Unreadable exception when ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9134#issuecomment-148323628 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148322379 [Test build #43782 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43782/consoleFull) for PR 9055 at commit [`7511f47`](https://github.com/apache/spark/commit/7511f47089ed58f913a81df2113cbe300903be63). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11104][Streaming]Fix a deadlock in Stre...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9116#issuecomment-148321658 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Github user dbtsai commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-148321601
I'll be able to merge this PR once the following two issues are addressed,
1) Have a test to verify that `L1` with normal will throw
`IllegalArugmentExcpetion`. I prefer have the test inside `Seq("auto",
"l-bfgs") since I'll work on solving L1 with `normal` later, so what we need to
do it just remove the `Intercept`. Also, we can precise know what part is not
supported if we have the tests together.
2) Currently, we allow weighted LiR in `normal` mode, but we don't have a
test verifying they get the same solution. Please add a test for this. If they
are in fact returning different solution, then throw exception saying that
weighted LiR doesn't support `normal` mode, and in `auto` mode with weighted
LiR, we should do `l-bfgs`. Then we address the inconsistent issue in a
separated PR.
Thanks. Once those two issues are solved, I'll merge it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11104][Streaming]Fix a deadlock in Stre...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9116#issuecomment-148321661 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43777/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11104][Streaming]Fix a deadlock in Stre...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9116#issuecomment-148321535 [Test build #43777 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43777/console) for PR 9116 at commit [`7c44b8c`](https://github.com/apache/spark/commit/7c44b8c2db3b6b785705c9dccca92e2a8a8fce9d). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8652#issuecomment-148320815 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43779/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148320889 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4226][SQL]Add subquery (not) in/exists ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9055#issuecomment-148320902 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4526][MLLIB]Gradient should be added ba...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3677#issuecomment-148320850 [Test build #43781 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43781/consoleFull) for PR 3677 at commit [`ad40bb5`](https://github.com/apache/spark/commit/ad40bb5210652a96f5bf48801f1e333aa937a2e9). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8652#issuecomment-148320814 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8652#issuecomment-148320732 [Test build #43779 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43779/console) for PR 8652 at commit [`26818c7`](https://github.com/apache/spark/commit/26818c7178f3783b2fbac45f73752d8e27666703). * This patch **fails MiMa tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4526][MLLIB]Gradient should be added ba...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3677#issuecomment-148319006 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4526][MLLIB]Gradient should be added ba...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3677#issuecomment-148319023 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11124] JsonParser/Generator should be c...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9130#issuecomment-148317796 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43767/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11124] JsonParser/Generator should be c...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9130#issuecomment-148317795 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11124] JsonParser/Generator should be c...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9130#issuecomment-148317713 [Test build #43767 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43767/console) for PR 9130 at commit [`f1d212a`](https://github.com/apache/spark/commit/f1d212a32dd6b8174b0db97c4fbc8b9daa1f427c). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8652#issuecomment-148316845 [Test build #43779 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43779/consoleFull) for PR 8652 at commit [`26818c7`](https://github.com/apache/spark/commit/26818c7178f3783b2fbac45f73752d8e27666703). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148316840 [Test build #43780 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43780/consoleFull) for PR 8416 at commit [`4cd1796`](https://github.com/apache/spark/commit/4cd17966775b9f18c5d6a00b1671c6045f50a25a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148316257 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10185] [SQL] Feat sql comma separated p...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8416#issuecomment-148316285 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8652#issuecomment-148315435 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8652#issuecomment-148315405 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/8652#issuecomment-148315153 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9123#issuecomment-148314896 [Test build #43778 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43778/consoleFull) for PR 9123 at commit [`6f97fb7`](https://github.com/apache/spark/commit/6f97fb7c9fce2920641e5b9dc63b2830e8037702). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9123#issuecomment-148314571 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9123#issuecomment-148314559 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/9123#issuecomment-148314200 @cloud-fan Hi Wenchen, i have fixed the code based on your comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11104][Streaming]Fix a deadlock in Stre...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9116#issuecomment-148310651 [Test build #43777 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43777/consoleFull) for PR 9116 at commit [`7c44b8c`](https://github.com/apache/spark/commit/7c44b8c2db3b6b785705c9dccca92e2a8a8fce9d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11104][Streaming]Fix a deadlock in Stre...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9116#issuecomment-148310218 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11104][Streaming]Fix a deadlock in Stre...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9116#issuecomment-148310203 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8652#issuecomment-148310086 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/9123#discussion_r42092689
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
---
@@ -135,4 +135,21 @@ class AnalysisSuite extends AnalysisTest {
plan = testRelation.select(CreateStructUnsafe(Seq(a, (a +
1).as("a+1"))).as("col"))
checkAnalysis(plan, plan)
}
+
+ test("SPARK-10534: resolve attribute references in order by clause") {
+
+val a = testRelation2.output.head
+val c = testRelation2.output.toArray.apply(2)
+
+val sortProjected = Floor(Cast(Floor(c), DoubleType))
+val projected = Alias(a, "a")()
+val plan = testRelation2.select(a).orderBy(SortOrder(Floor(Floor(c)),
Ascending))
+
+val expected =
+ Project(Seq(a),
+Sort(Seq(SortOrder(sortProjected, Ascending)), true,
+ Project(Seq(a, c), testRelation2)))
+checkAnalysis(plan, expected)
+
+ }
--- End diff --
Thanks a LOT. I will make the change.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8652#issuecomment-148310064 [Test build #43775 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43775/console) for PR 8652 at commit [`26818c7`](https://github.com/apache/spark/commit/26818c7178f3783b2fbac45f73752d8e27666703). * This patch **fails MiMa tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8652#issuecomment-148310087 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43775/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/9123#discussion_r42092649
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -482,7 +482,12 @@ class Analyzer(
val newOrdering = resolveSortOrders(ordering, grandchild, throws =
true)
// Construct a set that contains all of the attributes that we need
to evaluate the
// ordering.
- val requiredAttributes = AttributeSet(newOrdering.filter(_.resolved))
+
+ val resolvedAttributes =
+newOrdering.flatMap(_.collect {case a : AttributeReference if
a.resolved => a})
+
+ val requiredAttributes = AttributeSet(resolvedAttributes)
--- End diff --
Thanks. Much simpler :-). Will make the change and test..
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11104][Streaming]Fix a deadlock in Stre...
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/9116#discussion_r42092604
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala ---
@@ -694,31 +694,39 @@ class StreamingContext private[streaming] (
* @param stopGracefully if true, stops gracefully by waiting for the
processing of all
* received data to be completed
*/
- def stop(stopSparkContext: Boolean, stopGracefully: Boolean): Unit =
synchronized {
-try {
- state match {
-case INITIALIZED =>
- logWarning("StreamingContext has not been started yet")
-case STOPPED =>
- logWarning("StreamingContext has already been stopped")
-case ACTIVE =>
- scheduler.stop(stopGracefully)
- // Removing the streamingSource to de-register the metrics on
stop()
- env.metricsSystem.removeSource(streamingSource)
- uiTab.foreach(_.detach())
- StreamingContext.setActiveContext(null)
- waiter.notifyStop()
- if (shutdownHookRef != null) {
-ShutdownHookManager.removeShutdownHook(shutdownHookRef)
- }
- logInfo("StreamingContext stopped successfully")
+ def stop(stopSparkContext: Boolean, stopGracefully: Boolean): Unit = {
+var shutdownHookRefToRemove: AnyRef = null
+synchronized {
+ try {
+state match {
+ case INITIALIZED =>
+logWarning("StreamingContext has not been started yet")
+ case STOPPED =>
+logWarning("StreamingContext has already been stopped")
+ case ACTIVE =>
+scheduler.stop(stopGracefully)
+// Removing the streamingSource to de-register the metrics on
stop()
+env.metricsSystem.removeSource(streamingSource)
+uiTab.foreach(_.detach())
+StreamingContext.setActiveContext(null)
+waiter.notifyStop()
+if (shutdownHookRef != null) {
+ shutdownHookRefToRemove = shutdownHookRef
+ shutdownHookRef = null
+}
+logInfo("StreamingContext stopped successfully")
+}
+// Even if we have already stopped, we still need to attempt to
stop the SparkContext
+// because a user might stop(stopSparkContext = false) and then
call
+// stop(stopSparkContext = true).
+if (stopSparkContext) sc.stop()
+ } finally {
+// The state should always be Stopped after calling `stop()`, even
if we haven't started yet
+state = STOPPED
}
- // Even if we have already stopped, we still need to attempt to stop
the SparkContext because
- // a user might stop(stopSparkContext = false) and then call
stop(stopSparkContext = true).
- if (stopSparkContext) sc.stop()
-} finally {
- // The state should always be Stopped after calling `stop()`, even
if we haven't started yet
- state = STOPPED
+}
+if (shutdownHookRefToRemove != null) {
--- End diff --
Yeah, I put them to `finally`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9123#issuecomment-148307180 [Test build #43776 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43776/consoleFull) for PR 9123 at commit [`ac78af1`](https://github.com/apache/spark/commit/ac78af14508ac02a74202de564253b927f06c237). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/9123#discussion_r42091740
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -482,7 +482,12 @@ class Analyzer(
val newOrdering = resolveSortOrders(ordering, grandchild, throws =
true)
// Construct a set that contains all of the attributes that we need
to evaluate the
// ordering.
- val requiredAttributes = AttributeSet(newOrdering.filter(_.resolved))
+
+ val resolvedAttributes =
+newOrdering.flatMap(_.collect {case a : AttributeReference if
a.resolved => a})
+
+ val requiredAttributes = AttributeSet(resolvedAttributes)
--- End diff --
how about
`val requiredAttributes = AttributeSet(newOrdering).filter(_.resolved)`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9123#issuecomment-148306682 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9123#issuecomment-148306706 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9741][SQL] Approximate Count Distinct u...
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/8362#issuecomment-148306575 we can create a hash expression, and codegen that. And then just use hyperloglog(hash(field)). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/9123#issuecomment-148306333
I wanna explain a bit more about this bug.
When we resolve sort ordering, we will use a [special
method](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala#L429-L447),
which only resolves `UnresolvedAttributes` and `UnresolvedExtractValue`.
However, for something like `Floor('a)`, even the `'a` is resolved, the floor
expression may still being unresolved as data type mismatch(for example, `'a`
is string type and `Floor` need double type), thus can't pass [this
filter](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala#L485),
and we can't push down this missing attribute `'a`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/9123#issuecomment-148306360 ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8652#issuecomment-148306268 [Test build #43775 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43775/consoleFull) for PR 8652 at commit [`26818c7`](https://github.com/apache/spark/commit/26818c7178f3783b2fbac45f73752d8e27666703). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-9741][SQL] Approximate Count Distinct u...
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/8362#issuecomment-148305817 Another thought on hashing. The ClearSpring hash is a generic hash function. We could used very specialized (hopefully fast) hashing functions, because we know the type of our input. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/8652#issuecomment-148305296 @yhuai, after some investigation, I think the we still can optimize the left/right/full outer join in broadcasting way if they don't have condition, but LeftSemi are not included, and it should be a bug in previous implementation. And I also add more unit test by comparing with Hive. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8652#issuecomment-148305081 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10484][SQL] Optimize the cartesian join...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8652#issuecomment-148305056 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11125] [SQL] Unreadable exception when ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9134#issuecomment-148302621 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11125] [SQL] Unreadable exception when ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9134#issuecomment-148302625 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43774/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11125] [SQL] Unreadable exception when ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9134#issuecomment-148302613 [Test build #43774 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43774/console) for PR 9134 at commit [`2bdf979`](https://github.com/apache/spark/commit/2bdf97974be7bd6abdf55792b95a568eb423817d). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11125] [SQL] Unreadable exception when ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9134#issuecomment-148301968 [Test build #43774 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43774/consoleFull) for PR 9134 at commit [`2bdf979`](https://github.com/apache/spark/commit/2bdf97974be7bd6abdf55792b95a568eb423817d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-10534][SQL] ORDER BY clause allows only...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/9123#discussion_r42089770
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
---
@@ -135,4 +135,21 @@ class AnalysisSuite extends AnalysisTest {
plan = testRelation.select(CreateStructUnsafe(Seq(a, (a +
1).as("a+1"))).as("col"))
checkAnalysis(plan, plan)
}
+
+ test("SPARK-10534: resolve attribute references in order by clause") {
+
+val a = testRelation2.output.head
+val c = testRelation2.output.toArray.apply(2)
+
+val sortProjected = Floor(Cast(Floor(c), DoubleType))
+val projected = Alias(a, "a")()
+val plan = testRelation2.select(a).orderBy(SortOrder(Floor(Floor(c)),
Ascending))
+
+val expected =
+ Project(Seq(a),
+Sort(Seq(SortOrder(sortProjected, Ascending)), true,
+ Project(Seq(a, c), testRelation2)))
+checkAnalysis(plan, expected)
+
+ }
--- End diff --
the test case can be simplified to:
```
val a = testRelation2.output(0)
val c = testRelation2.output(2)
val plan = testRelation2.select(c).orderBy(Floor(a).asc)
val expected = testRelation2.select(c,
a).orderBy(Floor(a.cast(DoubleType)).asc).select(c)
checkAnalysis(plan, expected)
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11125] [SQL] Unreadable exception when ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9134#issuecomment-148300064 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11125] [SQL] Unreadable exception when ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9134#issuecomment-148300028 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11125] [SQL] Unreadable exception when ...
GitHub user zjffdu opened a pull request: https://github.com/apache/spark/pull/9134 [SPARK-11125] [SQL] Unreadable exception when running spark-sql witho⦠â¦ut building with -Phive-thriftserver and SPARK_PREPEND_CLASSES is set This is the exception after this patch. ``` java.lang.NoClassDefFoundError: org/apache/hadoop/hive/cli/CliDriver at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:800) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:412) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:270) at org.apache.spark.util.Utils$.classForName(Utils.scala:173) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:647) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.cli.CliDriver at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 21 more Failed to load hive class. You need to build Spark with -Phive and -Phive-thriftserver. ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/zjffdu/spark SPARK-11125 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/9134.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #9134 commit 2bdf97974be7bd6abdf55792b95a568eb423817d Author: Jeff Zhang Date: 2015-10-15T07:02:05Z [SPARK-11125] [SQL] Unreadable exception when running spark-sql without building with -Phive-thriftserver and SPARK_PREPEND_CLASSES is set --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK][SPARK-10842]Eliminate creating duplica...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8923#issuecomment-148298768 [Test build #43772 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43772/consoleFull) for PR 8923 at commit [`6a68113`](https://github.com/apache/spark/commit/6a68113ff2bb5492aab8782840f767f999b6d11d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
