Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/19193
Thank you, @aokolnychyi . :)
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user aokolnychyi commented on the issue:
https://github.com/apache/spark/pull/19193
Hi @dongjoon-hyun. Yep, I'll close this one.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/19193
Hi, @aokolnychyi .
https://github.com/apache/spark/pull/21580 is merged. Do we have something
to do more? Or can we close this?
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19193
For 1), yes let's forbid it.
For 2), My feeling is, in `Dataset` API we don't need `having` because it's
easy to change the order of the operator, users can call `filter` first then
Github user aokolnychyi commented on the issue:
https://github.com/apache/spark/pull/19193
@cloud-fan @hvanhovell I created PR #21473 that fixes StackOverflow.
Apart from that, I think we might have other potential problems.
**1. Window functions inside WHERE and
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19193
How about we leave this PR open and create another PR to prohibit nested
window function? Let's fix the stack overflow bug first and then decide if we
want to officially support this feature.
Github user aokolnychyi commented on the issue:
https://github.com/apache/spark/pull/19193
@hvanhovell @cloud-fan I think it would be safer to be consistent with
other databases and what Spark does for nested aggregate functions. It is
really simple to write a subquery to work around
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/19193
@cloud-fan and @aokolnychyi I am ok with supporting this, especially since
most of the work is already done. WDYT?
---
-
To
Github user aokolnychyi commented on the issue:
https://github.com/apache/spark/pull/19193
I checked PostgreSQL(10.3), MySQL(8.0), Hive(2.1.0).
**1. PostgreSQL**
```
postgres=# CREATE TABLE t1 (c1 integer, c2 integer);
postgres=# INSERT INTO t1 VALUES (1, 2),
Github user aokolnychyi commented on the issue:
https://github.com/apache/spark/pull/19193
Let me check other databases and come up with a summary.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19193
I feel it's much simpler to just prohibit this case, as we don't allow
nested aggregate function. Users can always use subquery to work around it. Do
other databases support window function
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19193
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88910/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19193
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19193
**[Test build #88910 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88910/testReport)**
for PR 19193 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19193
**[Test build #88910 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88910/testReport)**
for PR 19193 at commit
Github user aokolnychyi commented on the issue:
https://github.com/apache/spark/pull/19193
@hvanhovell here is a summary of tried scenarios:
```
val df = Seq((1, 2), (1, 3), (2, 4), (5, 5)).toDF("a", "b")
val window1 = Window.orderBy('a)
val window2 =
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19193
cc @hvanhovell
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19193
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19193
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84655/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19193
**[Test build #84655 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84655/testReport)**
for PR 19193 at commit
Github user aokolnychyi commented on the issue:
https://github.com/apache/spark/pull/19193
@gatorsmile @cloud-fan could you provide any input?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19193
**[Test build #84655 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84655/testReport)**
for PR 19193 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19193
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19193
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81644/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19193
**[Test build #81644 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81644/testReport)**
for PR 19193 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19193
**[Test build #81644 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81644/testReport)**
for PR 19193 at commit
26 matches
Mail list logo
