Github user KevinZwx commented on the issue:
https://github.com/apache/spark/pull/16970
I'm a little confused with the behavior of dropDuplicates with watermark.
According to my understanding of the guide documentation, if I have the
following code, I expect to deduplicate still
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16970
@zsxwing Thanks, I am missing it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lw-lin commented on the issue:
https://github.com/apache/spark/pull/16970
@uncleGen I think `requiredChildDistribution =
ClusteredDistribution(keyExpressions) :: Nil` (please see
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16970
One question: witout aggregation, how to drop duplication between
partitions?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73297/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73297 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73297/testReport)**
for PR 16970 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73297 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73297/testReport)**
for PR 16970 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73285/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73285 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73285/testReport)**
for PR 16970 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73285 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73285/testReport)**
for PR 16970 at commit
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/16970
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73265/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73265 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73265/testReport)**
for PR 16970 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73247/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73247 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73247/testReport)**
for PR 16970 at commit
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/16970
@tdas I created https://issues.apache.org/jira/browse/SPARK-19690 to track
the issue when joining a batch DataFrame with a streaming DataFrame. I will fix
it in a separate PR to unblock this one as
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73247 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73247/testReport)**
for PR 16970 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73236/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73236 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73236/testReport)**
for PR 16970 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73236 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73236/testReport)**
for PR 16970 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73225/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73225 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73225/testReport)**
for PR 16970 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73225 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73225/testReport)**
for PR 16970 at commit
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/16970
overall looks good. just a bunch of nits.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73076/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73076 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73076/testReport)**
for PR 16970 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73076 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73076/testReport)**
for PR 16970 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73064/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73064 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73064/testReport)**
for PR 16970 at commit
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/16970
aw man. I should always refresh before starting a review
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/16970
@brkyvz looks like you were looking at my old changes. I pushed a new
commit and updated the PR description to reflect the latest supported queries.
---
If your project is set up for it, you can
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73064 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73064/testReport)**
for PR 16970 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73028/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16970
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73028 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73028/testReport)**
for PR 16970 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16970
**[Test build #73028 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73028/testReport)**
for PR 16970 at commit
44 matches
Mail list logo
