[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-126701745 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-126744686 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-126745928 [Test build #39223 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/39223/consoleFull) for PR 7515 at commit [`f0e1c21`](https://github.com/apache/spark/commit/f0e1c2168ce33a6637d86add3e0e0378c98933aa). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-126775700 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-126775576 [Test build #39223 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/39223/console) for PR 7515 at commit [`f0e1c21`](https://github.com/apache/spark/commit/f0e1c2168ce33a6637d86add3e0e0378c98933aa). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-126786929 LGTM. Merging! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/7515 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/7515#discussion_r36006029
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -850,9 +851,10 @@ class Analyzer(
var currentChild = child
var i = 0
while (i groupedWindowExpressions.size) {
-val (windowSpec, windowExpressions) = groupedWindowExpressions(i)
+val ((partitionSpec, orderSpec), windowExpressions) =
groupedWindowExpressions(i)
// Set currentChild to the newly created Window operator.
-currentChild = Window(currentChild.output, windowExpressions,
windowSpec, currentChild)
+currentChild = Window(currentChild.output, windowExpressions,
+ partitionSpec, orderSpec, currentChild)
--- End diff --
```
Window(
currentChild.output,
windowExpressions,
partitionSpec,
orderSpec,
currentChild)
```
I will fix it minot style thing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/7515#discussion_r35946255
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -818,7 +818,8 @@ class Analyzer(
failAnalysis(s$expr has multiple Window Specifications
($distinctWindowSpec). +
sPlease file a bug report with this error message, stack
trace, and the query.)
} else {
- distinctWindowSpec.head
+ val unbounded = SpecifiedWindowFrame(RowFrame,
UnboundedPreceding, UnboundedFollowing)
--- End diff --
Let's add a comment at here to explain what we are doing. Seems, we create
a dummy window frame, so we only use `PARTITION BY` clause and `ORDER BY`
clause to do grouping.
Actually, can we just extract partitioning expressions and ordering
expressions and explicitly use them as the grouping key? That will make the
code easier to understand. Also, we probably want to remove the argument
`windowSpec` from both logical and physical Window operator because we create
`WindowFunctionFrame` based on the frame definition of every WindowExpression's
frame definition.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/7515#discussion_r35946650
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -818,7 +818,8 @@ class Analyzer(
failAnalysis(s$expr has multiple Window Specifications
($distinctWindowSpec). +
sPlease file a bug report with this error message, stack
trace, and the query.)
} else {
- distinctWindowSpec.head
+ val unbounded = SpecifiedWindowFrame(RowFrame,
UnboundedPreceding, UnboundedFollowing)
--- End diff --
I think extracting the grouping by and order by clauses is better, and it
also conveys the message that any Window operator should be capable of
processing multiple frames. The impact of the PR will be a bit larger in that
case; I'll have a look at it tomorrow.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-125820410 [Test build #38795 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38795/consoleFull) for PR 7515 at commit [`300d9f6`](https://github.com/apache/spark/commit/300d9f63c041465054cb9a41180ee3bc53a047fe). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-125820217 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-125820225 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-125835274 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-125835228 [Test build #38795 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38795/console) for PR 7515 at commit [`300d9f6`](https://github.com/apache/spark/commit/300d9f63c041465054cb9a41180ee3bc53a047fe). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-125819862 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-122912351 cc @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-122683654 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-8640] [SQL] Enable Processing of Multip...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7515#issuecomment-122683987 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
