Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22690
cc @cloud-fan and @gatorsmile
Should we update migration guide as well?
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22503
@justinuang, this might affect existing users application. Although this
matches the behaviour to non-miltiline mode, can we explicitly mention it in
migration guide?
cc @cloud-fan and
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22747
This looks also external changes to existing application users. Shall we
update migration guide?
---
-
To unsubscribe,
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22773
Yup, will encourage to update the migration guide in that way.
---
-
To unsubscribe, e-mail:
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22773
Yes. The goal of migration guide is for helping end users upgrade their
Spark.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22754
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97996/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22754
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22754
**[Test build #97996 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97996/testReport)**
for PR 22754 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97998/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22512
**[Test build #97998 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97998/testReport)**
for PR 22512 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22728
(From https://github.com/apache/spark/pull/22773#issuecomment-432917994)
@gatorsmile and @cloud-fan, let's say this will break `DESCRIBE FUNCTION
EXTENDED`. Should we update migration guide as
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22815
BTW, should we update migration guide too?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user daviddingly commented on the issue:
https://github.com/apache/spark/pull/22803
> Thank you for your first contribution, @daviddingly
> Could you [create issues
ticket](https://issues.apache.org/jira/projects/SPARK) first and change the pr
title to
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22773
Sure, so for clarification, we will document everything that affects to
external users application, right?
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22514
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97995/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22514
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22773
All the things that could break the existing user applications should be
documented in the migration guide. This will simplify the system upgrade of our
end users.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22514
**[Test build #97995 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97995/testReport)**
for PR 22514 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22775
**[Test build #98001 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98001/testReport)**
for PR 22775 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22775
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22775
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22775
@cloud-fan, looks we are going to start another RC. Would you mind if I ask
to take a quick look before the new RC?
---
-
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22775
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22773
My impression so far was that we note things at migration notes when they
are improvements (not bugs), and non-trivial and related to backward
compatibility.
Shall we clarify what to
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22821
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22821
cc @dongjoon-hyun @gatorsmile @rxin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22821
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22821
**[Test build #98000 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98000/testReport)**
for PR 22821 at commit
GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/22821
[SPARK-25832][] remove newly added map related functions from
FunctionRegistry
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22773
I think the new names are better and expected, though it's safer to mention
it in the migration guide in case some users care about it.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22798
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97993/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22798
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22798
**[Test build #97993 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97993/testReport)**
for PR 22798 at commit
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22820#discussion_r228030940
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala
---
@@ -56,8 +56,10 @@
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22820#discussion_r228030819
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala
---
@@ -157,7 +157,10 @@ private[spark]
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/22810
Let me look on Friday during weekly review time.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22773
BTW, it's closer to bug rather then improvement tho. `from_json` should
have default name `from_json` rather then `jsontostructs` - end users would
have no idea why it's called `jsontostructs`.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22773
That's the exact issue I raised before and we ended up with not keeping the
compatibility in column names. @cloud-fan and @hvanhovell.
---
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22773
This is an external change. This needs a migration guide update.
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22809
**[Test build #97999 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97999/testReport)**
for PR 22809 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22809
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22809
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22749
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22749
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97991/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22749
**[Test build #97991 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97991/testReport)**
for PR 22749 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22819
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97990/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22819
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22803
**[Test build #4396 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4396/testReport)**
for PR 22803 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22819
**[Test build #97990 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97990/testReport)**
for PR 22819 at commit
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22803
Whoa, never seen a JVM crash like that!
```
[CodeBlob (0x7f6fc018add0)]
Framesize: 2
Runtime Stub (0x7f6fc018add0): handle_exception_from_callee Runtime1
stub
Could
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/22809#discussion_r228020366
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala ---
@@ -727,4 +728,67 @@ class DataFrameAggregateSuite extends
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22512
**[Test build #97998 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97998/testReport)**
for PR 22512 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user ouyangxiaochen commented on the issue:
https://github.com/apache/spark/pull/22813
Yes, it happened in our real environment.
The scenario as follows:
Some disk corruption in the production cluster which is normal.
SPARK_LOCAL_DIRS = /data1/bigdata/spark/tmp,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22803
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22803
**[Test build #97997 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97997/testReport)**
for PR 22803 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22803
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97997/
Test FAILed.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22755
+1
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22795
Thanks @viirya!!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22514#discussion_r228013784
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala
---
@@ -45,6 +46,11 @@ case class
Github user yucai commented on the issue:
https://github.com/apache/spark/pull/19788
@cloud-fan , exception screenshot. Let me know if you want any change.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22809
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22809#discussion_r228013503
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala ---
@@ -727,4 +728,67 @@ class DataFrameAggregateSuite extends
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22803
**[Test build #97997 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97997/testReport)**
for PR 22803 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/22795
Sorry for late. This LGTM.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user wangyum commented on the issue:
https://github.com/apache/spark/pull/22803
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22795#discussion_r228012874
--- Diff: python/pyspark/sql/functions.py ---
@@ -3023,6 +3023,42 @@ def pandas_udf(f=None, returnType=None,
functionType=None):
conversion on
Github user 10110346 commented on the issue:
https://github.com/apache/spark/pull/22723
What you say is reasonable.
But from the perspective of resource utilization, I think it is better to
replace `minPartitions` with `defaultParallelism`, we can see `BinaryFileRDD`
and
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22803
**[Test build #4395 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4395/testReport)**
for PR 22803 at commit
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22816#discussion_r228012237
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -114,7 +114,7 @@ private[spark] abstract class BasePythonRunner[IN,
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22816#discussion_r228012035
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -114,7 +114,7 @@ private[spark] abstract class BasePythonRunner[IN,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22754
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22754
**[Test build #97996 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97996/testReport)**
for PR 22754 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22754
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22237
Thanks all!!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22795
Thanks, @BryanCutler.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22815
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/22514
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user 10110346 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22754#discussion_r228009755
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -495,8 +495,8 @@ package object config {
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22815
**[Test build #97988 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97988/testReport)**
for PR 22815 at commit
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22514#discussion_r228009289
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala
---
@@ -45,6 +46,11 @@ case class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22514
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22514
**[Test build #97995 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97995/testReport)**
for PR 22514 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22514
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22815
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97988/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22820
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22820
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22798
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22798
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/4456/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22798
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/21933
Yeah this can be closed; we updated to 4.1.30
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22512
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97989/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22512
**[Test build #97989 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97989/testReport)**
for PR 22512 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22818
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22818
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97987/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22818
**[Test build #97987 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97987/testReport)**
for PR 22818 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22820
**[Test build #97994 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97994/testReport)**
for PR 22820 at commit
1 - 100 of 395 matches
Mail list logo
