Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21118#discussion_r204150008
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaContinuousReader.scala
---
@@ -53,7 +54,7 @@ class
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21118#discussion_r204149889
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala
---
@@ -125,16 +125,13 @@ object
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
Anyone know what's happening with this:
```
[error]
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19194
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19194
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93350/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19194
**[Test build #93350 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93350/testReport)**
for PR 19194 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93359/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
**[Test build #93359 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93359/testReport)**
for PR 21748 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
**[Test build #93361 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93361/testReport)**
for PR 21748 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21679
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93358/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
**[Test build #93358 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93358/testReport)**
for PR 21748 at commit
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21803
> Should we do schema.toDDL, or StructType.toDDL(schema)?
> schema.toDDL is more friendly.
For sure, `schema.toDDL` looks more natural. I put `toDDL` to the
`StructType object` only to
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21799
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
**[Test build #93359 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93359/testReport)**
for PR 21748 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
**[Test build #93360 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93360/testReport)**
for PR 21748 at commit
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/21799
LGTM
Merging with master
Thanks @MrBago !
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
@liyinan926 did some of my own edits on top of your suggestions for docs
wording on the latest patch.
---
-
To unsubscribe,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/1180/
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
**[Test build #93358 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93358/testReport)**
for PR 21748 at commit
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r204140632
--- Diff: docs/running-on-kubernetes.md ---
@@ -129,20 +129,27 @@ Spark executors must be able to connect to the Spark
driver over a hostname and
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/1180/
---
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r204139176
--- Diff: docs/running-on-kubernetes.md ---
@@ -129,20 +129,27 @@ Spark executors must be able to connect to the Spark
driver over a hostname and
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r204138506
--- Diff: docs/running-on-kubernetes.md ---
@@ -129,20 +129,27 @@ Spark executors must be able to connect to the Spark
driver over a hostname and
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r204139679
--- Diff: docs/running-on-kubernetes.md ---
@@ -129,20 +129,27 @@ Spark executors must be able to connect to the Spark
driver over a hostname and
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r204139065
--- Diff: docs/running-on-kubernetes.md ---
@@ -129,20 +129,27 @@ Spark executors must be able to connect to the Spark
driver over a hostname and
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r204139975
--- Diff: docs/running-on-kubernetes.md ---
@@ -129,20 +129,27 @@ Spark executors must be able to connect to the Spark
driver over a hostname and
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
**[Test build #93357 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93357/testReport)**
for PR 21748 at commit
Github user sujith71955 commented on the issue:
https://github.com/apache/spark/pull/21775
sure, i will update the PR based on the comments, Thanks for suggestions.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21818
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93349/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21818
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21818
**[Test build #93349 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93349/testReport)**
for PR 21818 at commit
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
We discussed this offline. After some experimentation, we concluded that
it's not actually straightforward to set up the headless service in the
Kubernetes scheduler code in client mode, which
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21830
Thanks @ueshin for this PR! Good to know about the re-assignments.
---
-
To unsubscribe, e-mail:
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21805#discussion_r204123221
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala
---
@@ -50,6 +50,8 @@ case class CachedRDDBuilder(
Github user Achuth17 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21608#discussion_r204121451
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/CommandUtils.scala
---
@@ -47,15 +48,27 @@ object CommandUtils extends Logging {
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21758
**[Test build #93356 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93356/testReport)**
for PR 21758 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21758
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93356/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21758
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21829
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21829
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93355/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21829
**[Test build #93355 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93355/testReport)**
for PR 21829 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/1
@maropu Can you take this over?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21758
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21758
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93354/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21758
**[Test build #93354 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93354/testReport)**
for PR 21758 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21758
**[Test build #93356 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93356/testReport)**
for PR 21758 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21829
**[Test build #93355 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93355/testReport)**
for PR 21829 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21758
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21829
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21758
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21829
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21758
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21758
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21758#discussion_r204109307
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -359,20 +368,56 @@ private[spark] class TaskSchedulerImpl(
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21758#discussion_r204108963
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -1386,29 +1418,90 @@ class DAGScheduler(
)
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21758
**[Test build #93354 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93354/testReport)**
for PR 21758 at commit
Github user jose-torres commented on a diff in the pull request:
https://github.com/apache/spark/pull/21118#discussion_r204108549
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala
---
@@ -125,16 +125,13 @@ object
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21830
**[Test build #93353 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93353/testReport)**
for PR 21830 at commit
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/21809#discussion_r204103981
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -182,6 +184,12 @@ private[ui] class StagePage(parent: StagesTab, store:
Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/21809#discussion_r204105954
--- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala
---
@@ -94,6 +94,13 @@ private[spark] class AppStatusStore(
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21830
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21830
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/21830
cc @mn-mikke @cloud-fan @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21803
schema.toDDL is more friendly.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
GitHub user ueshin opened a pull request:
https://github.com/apache/spark/pull/21830
[SPARK-24878][SQL] Fix reverse function for array type of primitive type
containing null.
## What changes were proposed in this pull request?
If we use `reverse` function for array type of
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/21803
Should we do schema.toDDL, or StructType.toDDL(schema)?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21775
This is an external change. Please add one more point in the migration
guide.
---
-
To unsubscribe, e-mail:
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/17631
cc @maropu How about taking this over? You can follow the solution we used
for partitioning columns?
---
-
To unsubscribe,
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21608
ping me next week. I will review this again. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21829#discussion_r204100563
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/AvroOutputWriterFactory.scala
---
@@ -17,21 +17,24 @@
package
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/20629
Thank you for reviewing @holdenk âº
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20629
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21774
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user jaceklaskowski commented on a diff in the pull request:
https://github.com/apache/spark/pull/21815#discussion_r204098057
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
---
@@ -270,7 +270,7 @@ case class FileSourceScanExec(
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/20629
Merged to master :) Thank you :)
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21774
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21822
**[Test build #93351 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93351/testReport)**
for PR 21822 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21608
**[Test build #93352 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93352/testReport)**
for PR 21608 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21821
We need a separate rule to eliminate barriers for the write path and CTAS,
since the input queries are not always children of these nodes. Thus, the
current EliminateBarriers does not work.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21822
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21822
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21774
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21774
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93345/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21774
**[Test build #93345 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93345/testReport)**
for PR 21774 at commit
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/21608
Jenkins retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cclauss commented on a diff in the pull request:
https://github.com/apache/spark/pull/20838#discussion_r204094593
--- Diff: dev/create-release/releaseutils.py ---
@@ -149,7 +152,11 @@ def get_commits(tag):
if not is_valid_author(author):
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/20838#discussion_r204093418
--- Diff: python/pyspark/streaming/dstream.py ---
@@ -23,6 +23,8 @@
if sys.version < "3":
from itertools import imap as map, ifilter as
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/20838#discussion_r204091905
--- Diff: dev/create-release/releaseutils.py ---
@@ -149,7 +152,11 @@ def get_commits(tag):
if not is_valid_author(author):
201 - 300 of 606 matches
Mail list logo
