Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93244/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
**[Test build #93244 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93244/testReport)**
for PR 21748 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21656
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/21748
> Can you point in the fork where the submission client is create the
headless service? (just to help me understand the internals).
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/1101/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user echarles commented on the issue:
https://github.com/apache/spark/pull/21748
> @mccheah agreed with @echarles that it would be great if the submission
client will still create a headless service for the driver if the driver is
running in a pod in client mode.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21748
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93243/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
**[Test build #93243 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93243/testReport)**
for PR 21748 at commit
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
Also make sure your driver can actually allocate the port specified by
`spark.driver.port`?
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21710
**[Test build #93245 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93245/testReport)**
for PR 21710 at commit
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
@echarles then you'd probably want more information such as the logs of the
executors, though I'd imagine that one would have trouble getting those given
that the executor exits so quickly. But
Github user huaxingao commented on a diff in the pull request:
https://github.com/apache/spark/pull/21710#discussion_r203481597
--- Diff: R/pkg/R/generics.R ---
@@ -1415,6 +1415,13 @@ setGeneric("spark.freqItemsets", function(object) {
standardGeneric("spark.freqI
#' @rdname
Github user echarles commented on the issue:
https://github.com/apache/spark/pull/21748
> Can you share on how you know that your executor pod has access to your
host set by spark.driver.host and spark.driver.port over the network?
spark.driver.host is set to the hostname of
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/21748
> The goal of this approach that specifically does not create a headless
service is so that the client mode implementation here is identical to the
client mode implementation of the other
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/1101/
---
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
> agreed with @echarles that it would be great if the submission client
will still create a headless service for the driver if the driver is running in
a pod in client mode.
The goal of
Github user squito commented on the issue:
https://github.com/apache/spark/pull/21729
lgtm
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
**[Test build #93244 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93244/testReport)**
for PR 21748 at commit
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/21748
@mccheah agreed with @echarles that it would be great if the submission
client will still create a headless service for the driver if the driver is
running in a pod in client mode.
---
Github user echarles commented on the issue:
https://github.com/apache/spark/pull/21748
@mccheah my pod has access to my host, so there is nothing to do on network
level. In other words, which are the steps to make this PR work in client mode
for Out-Cluster (assuming the network
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21440
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93234/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21440
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/21748
test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21440
**[Test build #93234 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93234/testReport)**
for PR 21440 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21748
**[Test build #93243 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93243/testReport)**
for PR 21748 at commit
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
@echarles it's highly likely your pods cannot communicate to your driver's
host because of some network partition or a firewall issue. The specific steps
you would have to take to fix that issue
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21803
**[Test build #93242 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93242/testReport)**
for PR 21803 at commit
Github user kyprifog commented on the issue:
https://github.com/apache/spark/pull/11005
@rxin Why was this pull request closed? Can you direct me to a new one
that could have replaced it? I was trying to track down the modification of
push downs for basic operations like "limit".
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21803
@maropu I added quoting of column names
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/21440#discussion_r203472192
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -659,6 +659,11 @@ private[spark] class BlockManager(
* Get block
Github user devaraj-kavali commented on the issue:
https://github.com/apache/spark/pull/21202
@srowen this issue still exists and needs to be merged.
---
-
To unsubscribe, e-mail:
Github user squito commented on the issue:
https://github.com/apache/spark/pull/21698
jumping in the middle on this discussion -- everybody has raised some great
points.
1) My first takeaway from this is just how hard it can be to reason about
this, because spark's exact
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19183
I thought this check also existed in the non-streaming code; the theory was
that if you have set a fixed number of executors but enabled dynamic
allocation, then that's probably a configuration
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21808
**[Test build #93241 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93241/testReport)**
for PR 21808 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21808
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21808
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18477
I took this over at https://github.com/apache/spark/pull/21808
I don't think this change is even right as it introduces scala-shell
examples.
---
GitHub user srowen opened a pull request:
https://github.com/apache/spark/pull/21808
[SPARK-21261][DOCS][SQL] SQL Regex document fix
## What changes were proposed in this pull request?
Fix regexes in spark-sql command examples.
This takes over
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21596
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93232/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21596
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21596
**[Test build #93232 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93232/testReport)**
for PR 21596 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21802
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93231/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21802
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21802
**[Test build #93231 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93231/testReport)**
for PR 21802 at commit
Github user echarles commented on the issue:
https://github.com/apache/spark/pull/21748
Thx @mccheah and @liyinan926. It now works with an headless service using
`spark.driver.host=spark-driver-service` and
`spark.kubernetes.driver.pod.name=spark-pod`
Two more questions:
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21800
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21807
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21807
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user Achuth17 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21608#discussion_r203456566
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/CommandUtils.scala
---
@@ -47,15 +48,27 @@ object CommandUtils extends Logging {
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21807
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21770
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/21202
@devaraj-kavali @skonto just checking this is still something that should
be merged, ready to go?
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21770
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93233/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21770
**[Test build #93233 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93233/testReport)**
for PR 21770 at commit
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r203455379
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala ---
@@ -160,11 +160,29 @@ case class
GitHub user mauropalsgraaf opened a pull request:
https://github.com/apache/spark/pull/21807
[SPARK-24536] Validate that limit clause cannot have a nullable expression
## What changes were proposed in this pull request?
It proposes a version in which nullable expressions
Github user pkuwm commented on a diff in the pull request:
https://github.com/apache/spark/pull/21802#discussion_r203453412
--- Diff: python/pyspark/sql/functions.py ---
@@ -2382,6 +2382,20 @@ def array_sort(col):
return
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21799
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93238/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21799
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21799
**[Test build #93238 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93238/testReport)**
for PR 21799 at commit
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/21748
As @mccheah mentioned, you probably need a headless service that exposes
port 7077 and 1 to connect to the driver using the driver pod name. I think
using the driver pod name should work
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21806
**[Test build #93240 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93240/testReport)**
for PR 21806 at commit
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/21806
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21806
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
My guess is that the executor is failing to contact the driver - I don't
think you can use the pod name as the hostname in minikube. Can you create a
headless service to bind your pod to a stable
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21806
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21806
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user gvr opened a pull request:
https://github.com/apache/spark/pull/21806
[SPARK-24846][SQL] Made hashCode ExprId independent of jvmId
## What changes were proposed in this pull request?
Made ExprId hashCode independent of jvmId to make canonicalization
independent of
Github user ifilonenko commented on a diff in the pull request:
https://github.com/apache/spark/pull/21652#discussion_r203446470
--- Diff:
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala
---
@@ -29,25
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21795
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21795
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21795
**[Test build #93239 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93239/testReport)**
for PR 21795 at commit
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/21754#discussion_r203437795
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/Exchange.scala
---
@@ -85,14 +85,20 @@ case class ReusedExchangeExec(override val
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21131#discussion_r203435933
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ---
@@ -764,6 +769,19 @@ private[spark] class TaskSetManager(
Github user yifeih commented on the issue:
https://github.com/apache/spark/pull/21743
hey @vanzin can you please take a look at this soon? thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21805
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21805
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21805
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user onursatici opened a pull request:
https://github.com/apache/spark/pull/21805
[SPARK-24850][SQL] fix str representation of CachedRDDBuilder
## What changes were proposed in this pull request?
As of https://github.com/apache/spark/pull/21018, InMemoryRelation includes
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/21734#discussion_r203427395
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala
---
@@ -193,8 +193,7 @@ object YarnSparkHadoopUtil
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21799
**[Test build #93238 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93238/testReport)**
for PR 21799 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21804
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93237/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21804
**[Test build #93237 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93237/testReport)**
for PR 21804 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21804
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21799
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21799
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21774#discussion_r203422909
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/CatalystDataToAvro.scala
---
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21774#discussion_r203424033
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/package.scala ---
@@ -36,4 +40,27 @@ package object avro {
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21774#discussion_r203418945
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/AvroDataToCatalyst.scala
---
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/21734#discussion_r203425909
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala
---
@@ -193,8 +193,7 @@ object YarnSparkHadoopUtil
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21774#discussion_r203416936
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/AvroDataToCatalyst.scala
---
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21774#discussion_r203422417
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/CatalystDataToAvro.scala
---
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21774#discussion_r203415956
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/AvroDataToCatalyst.scala
---
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21804
**[Test build #93237 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93237/testReport)**
for PR 21804 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21804
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21804
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/21804
cc @gatorsmile @HyukjinKwon
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
301 - 400 of 585 matches
Mail list logo
