Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13545
What do you think `dropDuplicates`?
1. ds.select("_1", "_2", "_3").dropDuplicates(Seq("_1",
"_2")).orderBy("_1", "_2").show()
2. ds.select("_1", "_2", "_3").dropDuplicates("_1",
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/13545
For API design it would be better to be very conservative, because we
cannot remove APIs. There is always value in adding something, but there is
also a cost to maintenance and user experience (too
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13545
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13545
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60133/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13545
**[Test build #60133 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60133/consoleFull)**
for PR 13545 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13545
**[Test build #60133 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60133/consoleFull)**
for PR 13545 at commit
