Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21699
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94123/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #94123 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94123/testReport)**
for PR 21699 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #94123 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94123/testReport)**
for PR 21699 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21699
Jenkins, retest this, please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94109/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #94109 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94109/testReport)**
for PR 21699 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #94109 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94109/testReport)**
for PR 21699 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21699
@HyukjinKwon I revert the last changes. Please, take a look at the PR again.
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21699
If this PR proposes a different API then an overloaded version of
`pivot(String, Seq[Any])`, it's a different issue though I guess.
I would prefer to have `pivot(Column, Seq[Any])` and
Github user maryannxue commented on the issue:
https://github.com/apache/spark/pull/21699
> Actually I am mostly worry of the pivotColumn. Specifying multiple
columns via struct is not intuitive I believe.
It depends on whether we'd like to add extra interfaces for multiple
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21699
> pivot(column: Column, values, Seq[Column]), so that we can construct
different types in "values".
Actually I am mostly worry of the `pivotColumn`. Specifying multiple
columns via
Github user maryannxue commented on the issue:
https://github.com/apache/spark/pull/21699
Thank you for the change, @MaxGekk!
@HyukjinKwon my idea was actually that the overloaded versions of pivot
would be `pivot(column: Column, values, Seq[Column])`, so that we can construct
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93848/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #93848 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93848/testReport)**
for PR 21699 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #93848 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93848/testReport)**
for PR 21699 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21699
> it would be nice to support it and test it in DataFrame pivot too.
@maryannxue Supported and tested. Please, have a look at:
Github user maryannxue commented on the issue:
https://github.com/apache/spark/pull/21699
@MaxGekk LGTM, but one more thing to consider:
Since we support column list in SQL, it would be nice to support it and
test it in DataFrame pivot too. The only thing that we need to enable is
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93832/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #93832 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93832/testReport)**
for PR 21699 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #93832 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93832/testReport)**
for PR 21699 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93826/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #93826 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93826/testReport)**
for PR 21699 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #93826 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93826/testReport)**
for PR 21699 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21699
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93824/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #93824 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93824/testReport)**
for PR 21699 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #93824 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93824/testReport)**
for PR 21699 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21699
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user maryannxue commented on the issue:
https://github.com/apache/spark/pull/21699
@MaxGekk Please take a look at https://github.com/apache/spark/pull/21926.
There was a bug in PivotFirst and this PR should fix your test here.
---
Github user maryannxue commented on the issue:
https://github.com/apache/spark/pull/21699
@MaxGekk Yes, it was caused by my previous PR. The change in my PR was a
walk-around for an existing problem in either Aggregate or PivotFirst (I
suspect it's Aggregate) with struct-type
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93746/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #93746 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93746/testReport)**
for PR 21699 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #93746 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93746/testReport)**
for PR 21699 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21699
I merged the `master` branch to my branch `pivot-column`, and the changes
break my test. It seems recent changes in pivoting introduced a correctness
bug. See the test
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93733/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #93733 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93733/testReport)**
for PR 21699 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #93733 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93733/testReport)**
for PR 21699 at commit
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/21699
I'm OK with it.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21699
> it's basically reverting Reynold's decision. is he okay with it?
As far as I know, yes. /cc @rxin
---
-
To
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21699
it's basically reverting Reynold's decision. is he okay with it?
---
-
To unsubscribe, e-mail:
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21699
@gatorsmile The PR https://github.com/apache/spark/pull/21753 has been
merged already. Can we continue with this PR? @maryannxue I added the tests you
asked for in
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92719/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #92719 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92719/testReport)**
for PR 21699 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21699
@maryannxue Please, have a look at the PR.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #92719 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92719/testReport)**
for PR 21699 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21699
@MaxGekk, I think you should talk with @rxin rather then @aray's comment.
---
-
To unsubscribe, e-mail:
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21699
> Considering you can just make a call to withColumn first I'm not really
convinced in the utility of this PR.
Purpose of the PR is to make pivot API consistent to `groupBy` and clear.
Our
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92577/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #92577 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92577/testReport)**
for PR 21699 at commit
Github user aray commented on the issue:
https://github.com/apache/spark/pull/21699
Using either `Column` or `String` type was actually in my original PR:
https://github.com/apache/spark/pull/7841
@rxin later modified the api to only take a `String` prior to the release
as part
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #92577 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92577/testReport)**
for PR 21699 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21699
> Were you planning to add a new overload for each existing String
version, e.g. pivot(Column) and pivot(Column, java.util.List[Any])?
The methods have been added already. @rednaxelafx
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92568/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #92568 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92568/testReport)**
for PR 21699 at commit
Github user rednaxelafx commented on the issue:
https://github.com/apache/spark/pull/21699
This mostly looks good, but I'd like to ask a few things first:
1. The new overloaded `pivot()` that takes `Column` only exist for
`pivot(Column, Seq[Any])`. Were you planning to add a
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21699
cc @aray
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #92568 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92568/testReport)**
for PR 21699 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21699
jenkins, retest this, please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92561/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #92561 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92561/testReport)**
for PR 21699 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #92561 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92561/testReport)**
for PR 21699 at commit
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21699
`def pivot(pivotColumn: String)`, too?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21699
cc: @rxin @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92541/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #92541 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92541/testReport)**
for PR 21699 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21699
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21699
**[Test build #92541 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92541/testReport)**
for PR 21699 at commit
82 matches
Mail list logo
