Github user carsonwang commented on the issue:
https://github.com/apache/spark/pull/21754
This LGTM as a fix. However, ideally we should also support reusing an
exchange used in different joins. There is no need to shuffle write the same
table twice, we just need read it differently.
Github user jose-torres commented on the issue:
https://github.com/apache/spark/pull/21919
For file streams, the offsets are just indices into a log the source keeps
of which files it's seen. So a file sink doesn't have any access to those
offsets.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21403
**[Test build #93999 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93999/testReport)**
for PR 21403 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21403
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21403
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93999/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21923
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21754
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21754
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21972
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21923
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21199
**[Test build #94011 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94011/testReport)**
for PR 21199 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21970
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21948#discussion_r207293465
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataWriterFactory.java
---
@@ -50,4 +50,15 @@
*this ID will
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> These test failures are in Spark streaming. Is this just an intermittent
test failure or actually caused by this PR?
I was able to run the first failing test successfully. Can we get a
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21911
Generally, it looks good to me. Let us target this to the upcoming 2.4
release
We need to improve the test coverage:
- Add a test case when users specify multiple REPARTITION,
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/21889
> I was able to run the first failing test successfully. Can we get a
retest, please?
@ajacques I just rebased and pushed my branch off of master. Perhaps the
easiest thing to do would be
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21923
**[Test build #94032 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94032/testReport)**
for PR 21923 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21973
**[Test build #94033 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94033/testReport)**
for PR 21973 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21964
**[Test build #94007 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94007/testReport)**
for PR 21964 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21973
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/21976
@squito Note this is just to get people looking at this. I am working on
adding some scheduler integration tests but I have to extend those to support
multiple executors and allow for tasks
GitHub user tgravescs opened a pull request:
https://github.com/apache/spark/pull/21976
[SPARK-24909] Spark scheduler can hang when fetch failures, executor
â¦lost, task running on lost executor, and multiple stage attempts
## What changes were proposed in this pull
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21923
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21973
**[Test build #94033 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94033/testReport)**
for PR 21973 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21923
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20611
> Yes there is a change in the behavior, As i mentioned above in
descriptions now we will be able to support wildcard even in the folder level
for local file systems. Previous versions will
Github user jose-torres commented on the issue:
https://github.com/apache/spark/pull/21919
If the individual connectors aren't doing the counting, I don't see a good
reason to put the data inside WriterCommitMessage instead of just leaving
StreamWriterCommitProgress as its own
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21973
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21941
**[Test build #94028 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94028/testReport)**
for PR 21941 at commit
Github user arunmahadevan commented on the issue:
https://github.com/apache/spark/pull/21919
`numOutputRows` makes sense for all sinks, but I agree the counting should
be done at the framework and not by individual sinks. For metrics that does not
apply to all sinks, they could
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21608
**[Test build #94043 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94043/testReport)**
for PR 21608 at commit
Github user bersprockets commented on the issue:
https://github.com/apache/spark/pull/21950
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21935
**[Test build #94020 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94020/testReport)**
for PR 21935 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21754
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user hthuynh2 commented on the issue:
https://github.com/apache/spark/pull/21953
@tgravescs I updated it. Thanks.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21754
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94000/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21943
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21943
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93996/
Test PASSed.
---
Github user jose-torres commented on the issue:
https://github.com/apache/spark/pull/21919
I don't think so. The offsets for the file source need to be consumer
owned, because they need to work with files that were generated outside of
Spark.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21972
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user rvesse commented on a diff in the pull request:
https://github.com/apache/spark/pull/21669#discussion_r207299324
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala
---
@@ -107,7 +109,14 @@
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21955
**[Test build #94031 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94031/testReport)**
for PR 21955 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21721
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21976
**[Test build #94039 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94039/testReport)**
for PR 21976 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21754
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21923#discussion_r207323114
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -130,6 +130,12 @@ private[spark] class Executor(
private val
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21966
LGTM except some comments
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/21973
Thanks! merging to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13440
**[Test build #94042 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94042/testReport)**
for PR 13440 at commit
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/21895
retest this pelase
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cclauss closed the pull request at:
https://github.com/apache/spark/pull/21960
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21954
**[Test build #94019 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94019/testReport)**
for PR 21954 at commit
Github user cclauss closed the pull request at:
https://github.com/apache/spark/pull/21959
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AlbertPlaPlanas commented on the issue:
https://github.com/apache/spark/pull/16486
Was this ever implemented?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21935
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user ifilonenko commented on a diff in the pull request:
https://github.com/apache/spark/pull/21669#discussion_r207281949
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala
---
@@ -107,7 +109,14
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21969
**[Test build #93997 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93997/testReport)**
for PR 21969 at commit
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21948#discussion_r207294283
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataWriterFactory.java
---
@@ -50,4 +50,15 @@
*this ID will
Github user mengxr commented on the issue:
https://github.com/apache/spark/pull/21927
LGTM. Merged into master. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21948
I'm changing my +1 to -1 because read-side changes are mixed in and because
copies are the responsibility of data sources if they buffer and hold
references to earlier rows.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21972
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21972
**[Test build #94029 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94029/testReport)**
for PR 21972 at commit
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/21923#discussion_r207300158
--- Diff: core/src/main/java/org/apache/spark/AbstractExecutorPlugin.java
---
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21911
cc @maryannxue Please review it.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21923
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21923
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94034/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21923
**[Test build #94034 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94034/testReport)**
for PR 21923 at commit
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/21952
@viirya since you don't see the performance regression between 2.4 +
builtin reader and 2.4 + databricks reader, do you think the regression is
somewhere else in Spark?
Can you try 2.3
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20611#discussion_r207258477
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -303,94 +303,44 @@ case class LoadDataCommand(
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21972
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21972
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21972
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94018/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21972
**[Test build #94018 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94018/testReport)**
for PR 21972 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21972
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21935
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21911#discussion_r207285870
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/ResolveHintsSuite.scala
---
@@ -17,15 +17,25 @@
package
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20838
> I still count 10 undefined names in this repo.
Do you mean you found more instance to fix, right?
---
-
To
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21969
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21941
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user jiangxb1987 closed the pull request at:
https://github.com/apache/spark/pull/21918
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21948#discussion_r207295461
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousRateStreamSource.scala
---
@@ -89,8 +89,7 @@ class
Github user jzhuge commented on a diff in the pull request:
https://github.com/apache/spark/pull/21911#discussion_r207303282
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/ResolveHintsSuite.scala
---
@@ -17,15 +17,25 @@
package
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21923
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21923
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21955
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21955
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21915
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21915
**[Test build #94006 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94006/testReport)**
for PR 21915 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19449
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21975
**[Test build #94036 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94036/testReport)**
for PR 21975 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21721
**[Test build #94038 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94038/testReport)**
for PR 21721 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19449
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94014/
Test FAILed.
---
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/spark/pull/21955#discussion_r207316897
--- Diff:
external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaRDDSuite.scala
---
@@ -72,31 +72,37 @@ class KafkaRDDSuite extends
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21974
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21898
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21898
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/spark/pull/21955#discussion_r207317252
--- Diff:
external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaRDDSuite.scala
---
@@ -72,31 +72,37 @@ class KafkaRDDSuite extends
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21970
**[Test build #94025 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94025/testReport)**
for PR 21970 at commit
Github user sujith71955 commented on the issue:
https://github.com/apache/spark/pull/20611
No other changes in the load command behavior
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21952
@dbtsai I was thinking the same thing. I will do the test later after I
come back to my laptop.
---
-
To unsubscribe, e-mail:
1 - 100 of 1238 matches
Mail list logo
