Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19575
BTW, Thanks for your great works! I will add all your names in the
contributors of this PR
---
-
To unsubscribe, e-mail:
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/20272
IIUC there was a issue in launching Thrift Server on YARN cluster mode, and
I'm not sure whether it has been fixed (maybe @jerryshao can kindly check
that?) Anyway that is not a problem on
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19575
Thanks! I will submit a follow-up PR to rename it.
Merged to 2.3 and master.
---
-
To unsubscribe, e-mail:
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19575
is it possible to decide on the names for groupBy()-apply() UDFs as a
followup? it sounds like there are still things that need discussion
---
Github user debugger87 commented on the issue:
https://github.com/apache/spark/pull/18649
https://github.com/apache/spark/pull/19721 Fixed the same issue, i will
close it.
---
-
To unsubscribe, e-mail:
Github user debugger87 closed the pull request at:
https://github.com/apache/spark/pull/18649
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19575
Thanks @gatorsmile , I made
https://issues.apache.org/jira/browse/SPARK-23258 to track changing the
`maxRecordsPerBatch` conf and I will externalize it in this PR.
> group map ->
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/20397
> I think the renaming is worth to remove future confusions.
What future confusion?
I understand that the difference isn't obvious, but making the names less
accurate isn't a good
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19575
Actually, aggregation can only be executed on grouped data, so
`SQL_PANDAS_GROUPED_AGG_UDF` doesn't seem to be very concise. How about
`SQL_PANDAS_UDAF`? My only concern is how to support partial
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20397
About your last point, it's mostly my fault that I didn't schedule the work
well and missed this one. Since the last RC failed and next RC is not started
yet, I think this is a good window to get
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20397
About the renaming, a lot of people complained to me about why the namings
are not consistent, including @rxin . I named it `ReadTask` at the beginning
because it really works like a task. But I
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19575#discussion_r164513778
--- Diff: docs/sql-programming-guide.md ---
@@ -1640,6 +1640,133 @@ Configuration of Hive is done by placing your
`hive-site.xml`, `core-site.xml` a
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86775/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86775 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86775/testReport)**
for PR 20386 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/342/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86775 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86775/testReport)**
for PR 20386 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20419
**[Test build #86774 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86774/testReport)**
for PR 20419 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/20419
I always leave the comment regardless of the
`spark.sql.codegen.useIdInClassName` for a unified way to access the ID from
the comment.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20419
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/341/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20419
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19575#discussion_r164509836
--- Diff: docs/sql-programming-guide.md ---
@@ -1640,6 +1640,133 @@ Configuration of Hive is done by placing your
`hive-site.xml`, `core-site.xml` a
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/20419#discussion_r164509380
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala
---
@@ -542,6 +542,7 @@ case class WholeStageCodegenExec(child:
Github user liufengdb commented on the issue:
https://github.com/apache/spark/pull/20420
LGTM! Thanks for doing this!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20350
Thanks for your contributions! Could you ping us again after 2.3 release?
---
-
To unsubscribe, e-mail:
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19575
I have two major comments.
- `group map` -> `grouped map` We need to also update `PythonEvalType`.
> SQL_PANDAS_GROUP_MAP_UDF -> SQL_PANDAS_GROUPED_MAP_UDF
>
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20422
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86768/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20422
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20422
**[Test build #86768 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86768/testReport)**
for PR 20422 at commit
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/20397
One last point: should significant changes to public APIs like this go in
just before or just after a release? 2.3.0 candidates have used ReadTask up to
now.
---
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/20397
@cloud-fan, thanks for pinging me on this.
-1: I don't think there's a compelling benefit to justify this change, and
I think it makes the API more confusing. I think we should revert this.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19575#discussion_r164496233
--- Diff: docs/sql-programming-guide.md ---
@@ -1640,6 +1640,133 @@ Configuration of Hive is done by placing your
`hive-site.xml`, `core-site.xml` a
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20402
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19575#discussion_r164495519
--- Diff: docs/sql-programming-guide.md ---
@@ -1640,6 +1640,133 @@ Configuration of Hive is done by placing your
`hive-site.xml`, `core-site.xml` a
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20402
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86769/
Test PASSed.
---
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/20402
Ok, merging this to master/4.0. Thanks for all the reviews!
---
-
To unsubscribe, e-mail:
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20250
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20402
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20375
Thanks! Merged to master/2.3
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20375
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20421
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86767/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20421
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20402
**[Test build #86769 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86769/testReport)**
for PR 20402 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20250
Thanks! Merged to master/2.3
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20421
**[Test build #86767 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86767/testReport)**
for PR 20421 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20397
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20397
The previous commit passed all test, and the last commit just changed some
comment and has nothing to do with the failed test, I'm merging it to
master/2.3, thanks!
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20420
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20420
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86771/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20420
**[Test build #86771 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86771/testReport)**
for PR 20420 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86770/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86770 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86770/testReport)**
for PR 20397 at commit
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/20383
That plan LGTM - we can merge into 2.3 after removing the non-existent
config, and getting a clean test run against the 2.3 branch.
Should be low risk.
---
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/20295
@HyukjinKwon @ueshin This is ready for review. I addressed the comments so
far.
@BryanCutler yeah I think kwargs is another option. But I think the API in
this PR is more consistent
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20295
**[Test build #86773 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86773/testReport)**
for PR 20295 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/340/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/20295#discussion_r164483676
--- Diff: python/pyspark/sql/udf.py ---
@@ -54,7 +54,7 @@ def _create_udf(f, returnType, evalType):
"Instead, create a 1-arg
Github user sethah commented on the issue:
https://github.com/apache/spark/pull/20332
Thanks a lot for your review, @MLnick!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/20332#discussion_r164479596
--- Diff: docs/ml-classification-regression.md ---
@@ -125,7 +123,8 @@ Continuing the earlier example:
Github user sethah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20332#discussion_r164476639
--- Diff: docs/ml-classification-regression.md ---
@@ -125,7 +123,8 @@ Continuing the earlier example:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86772/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86772 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86772/testReport)**
for PR 20397 at commit
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/20295
@HyukjinKwon Thanks for the comment. I will continue with the current
approach unless objection raises. I will work on comments and refinements in
the next day or two.
---
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/19575#discussion_r164473702
--- Diff: docs/sql-programming-guide.md ---
@@ -1640,6 +1640,133 @@ Configuration of Hive is done by placing your
`hive-site.xml`, `core-site.xml` a
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/19575#discussion_r164470776
--- Diff: docs/sql-programming-guide.md ---
@@ -1640,6 +1640,133 @@ Configuration of Hive is done by placing your
`hive-site.xml`, `core-site.xml` a
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20295
For https://github.com/apache/spark/pull/20295#issuecomment-360297123, I am
fine without new serialization protocol actually. I didn't have a strong
preference there because I wasn't sure if
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86766/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86766 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86766/testReport)**
for PR 20404 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20373
FYI, I manually tried to port cloudpickle#132 and cloudpickle#145 only with
corresponding test cases, and then check they were passed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86772 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86772/testReport)**
for PR 20397 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/339/
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20404
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user smurakozi commented on the issue:
https://github.com/apache/spark/pull/20045
Do you think I need to cover any other cases, @jiangxb1987 ?
---
-
To unsubscribe, e-mail:
Github user smurakozi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20235#discussion_r164427189
--- Diff: mllib/src/test/scala/org/apache/spark/ml/fpm/FPGrowthSuite.scala
---
@@ -34,86 +35,122 @@ class FPGrowthSuite extends SparkFunSuite with
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20420
**[Test build #86771 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86771/testReport)**
for PR 20420 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20420
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/338/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20420
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20420
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20402
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20404
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/337/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/336/
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/20397#discussion_r164425992
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/DataReaderFactory.java
---
@@ -22,21 +22,23 @@
import
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20402
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/335/
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/20397#discussion_r164425827
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/DataReaderFactory.java
---
@@ -22,21 +22,23 @@
import
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20422
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/334/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20422
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20421
**[Test build #86767 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86767/testReport)**
for PR 20421 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20421
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/333/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20422
**[Test build #86768 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86768/testReport)**
for PR 20422 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20404
**[Test build #86766 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86766/testReport)**
for PR 20404 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20421
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20402
**[Test build #86769 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86769/testReport)**
for PR 20402 at commit
301 - 400 of 487 matches
Mail list logo
