Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/16044
LGTM - merging to master/2.1. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
en
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16044
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16044
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69341/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16044
**[Test build #69341 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69341/consoleFull)**
for PR 16044 at commit
[`d4002c7`](https://github.com/apache/spark/commit/
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/16044
LGTM - pending jenkins
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishe
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/16044
LGTM - pending jenkins
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishe
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16044
**[Test build #69341 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69341/consoleFull)**
for PR 16044 at commit
[`d4002c7`](https://github.com/apache/spark/commit/d
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16044
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69287/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16044
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16044
**[Test build #69287 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69287/consoleFull)**
for PR 16044 at commit
[`7c3bca0`](https://github.com/apache/spark/commit/
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/16044
@hvanhovell Thank you for the detailed instruction on the test case.
@gatorsmile Also thanks for the directive.
In addition to the white box testing on the Optimized Logical Plan, I also
a
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16044
**[Test build #69287 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69287/consoleFull)**
for PR 16044 at commit
[`7c3bca0`](https://github.com/apache/spark/commit/7
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16044
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69262/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16044
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16044
**[Test build #69262 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69262/consoleFull)**
for PR 16044 at commit
[`8e34c50`](https://github.com/apache/spark/commit/
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/16044
You can just write an optimizer test case. You can for example add this to
the `FilterPushdownSuite`:
```scala
test("joins: only push down join conditions to the right of an existence
j
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/16044
You just need to add test cases to `FilterPushdownSuite`, like what
@hvanhovell did:
https://github.com/apache/spark/pull/16039/files#diff-acd51a263163be1e40af0b8b41f2b073
---
If your project i
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/16044
Hmm. `ExistenceJoin` is not exposed via `DataFrame` APIs. Let me study the
way `org.apache.spark.sql.execution.joins.ExistenceJoinSuite` was written then
I will add a new test case.
---
If your proj
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/16044
I know it is currently not exposed due to the way we optimize plans.
However this may change, and it is better to be safe than sorry. Also note that
we typically only test the rule at hand in opt
Github user nsyca commented on the issue:
https://github.com/apache/spark/pull/16044
@hvanhovell I can add a new test case but I want to reiterate that this is
not an exposed case because `ExistenceJoin` does not exist in the course of the
`PushPredicateThroughJoin` rule. The `Existen
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/16044
@nsyca this was not uncovered because we didn't add tests, could you add
one? `org.apache.spark.sql.catalyst.optimizer.FilterPushdownSuite` seems a good
place (they could be similar to the ones f
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16044
**[Test build #69262 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69262/consoleFull)**
for PR 16044 at commit
[`8e34c50`](https://github.com/apache/spark/commit/8
22 matches
Mail list logo
