Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22378
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22140
@BryanCutler @HyukjinKwon Thanks for your understanding. Normally, we are
very conservative to introduce any potential behavior change to the released
version.
I just reverted it from
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95887 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95887/testReport)**
for PR 22192 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/22192
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22341#discussion_r216406188
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -646,7 +647,47 @@ private[spark] class AppStatusListener(
}
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22341#discussion_r216404844
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -23,6 +23,7 @@ import java.util.function.Function
import
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22341#discussion_r216405553
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -646,7 +647,47 @@ private[spark] class AppStatusListener(
}
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22341#discussion_r216407616
--- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala ---
@@ -538,6 +538,14 @@ private class LiveRDD(val info: RDDInfo) extends
LiveEntity
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22341#discussion_r216406902
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -646,7 +647,47 @@ private[spark] class AppStatusListener(
}
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/22140
> Can we just simply take this out from branch-2.3?
Thanks @HyukjinKwon , that is fine with me. What do you think @gatorsmile ?
---
Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/22192
Logs suddenly cut off again without any exceptions, don't think this one is
a code error as well. Could someone retest this please?
---
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/22348#discussion_r216405506
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedParquetRecordReader.java
---
@@ -154,8 +159,6 @@ public void
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22366
I think that it is good to describe the flag is removed. This is because
the default behavior is not changed and users can know the flag is not
effective in the next release.
cc @HyukjinKwon
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22343
Hi, @seancxmao . Should we be consistent? IIRC, all the previous PR raises
Exception to prevent any potential issues. In this case, I have a feeling that
`convertMetastoreXXX` should be used
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22348
@SongYadong . `0.6%` seems to be too small to do this. Do you have another
benchmark result to show the benefit more clearly?
---
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21398
I didn't make the original change so I'm refraining from making a decision
about whether this should go in. Perhaps another fix is desired, perhaps the
current behavior is desired, but either way
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21308
@tigerquoll, I'm talking about the DataSourceV2 API in general. I'm not
sure if I think there is value in exposing partitions, but I'd be happy to hear
why you think they are valuable and think
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22381#discussion_r216392303
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -382,11 +392,37 @@ private[spark] class AppStatusListener(
}
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22381#discussion_r216391459
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -280,6 +284,12 @@ private[spark] class AppStatusListener(
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/20673
After benchmarking, observed that this didn't provide much benefit :(
Closing the PR
---
-
To unsubscribe, e-mail:
Github user brkyvz closed the pull request at:
https://github.com/apache/spark/pull/20673
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22375
**[Test build #95886 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95886/testReport)**
for PR 22375 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22375
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22375
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22375
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22371#discussion_r216389817
--- Diff:
core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala ---
@@ -138,13 +154,22 @@ private[spark] class
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/22371#discussion_r216387272
--- Diff:
core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala ---
@@ -138,13 +154,22 @@ private[spark] class
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21654#discussion_r216385714
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -375,6 +375,9 @@ def _truncate(self):
return int(self.sql_ctx.getConf(
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22371#discussion_r216383808
--- Diff:
core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala ---
@@ -138,13 +154,22 @@ private[spark] class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22380
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95878/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22380
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22380
**[Test build #95878 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95878/testReport)**
for PR 22380 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22364
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22364
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22376
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22376
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2979/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22376
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21308#discussion_r216382704
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/DeleteSupport.java ---
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software
Github user rgbkrk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21654#discussion_r216381942
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -375,6 +375,9 @@ def _truncate(self):
return int(self.sql_ctx.getConf(
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/22357
Btw, this PR isn't intended to address filter push down for schema pruning.
I do think it should be another one topic.
---
-
To
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22379
**[Test build #95885 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95885/testReport)**
for PR 22379 at commit
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22353#discussion_r216380605
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala ---
@@ -73,6 +73,11 @@ abstract class SparkPlan extends
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22376
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2979/
---
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/18600#discussion_r216380525
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlanInfo.scala ---
@@ -31,7 +31,6 @@ class SparkPlanInfo(
val
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/22357
I just read @mallman's comment. Thanks for that. Roughly, my two cents:
> IMO, we can get closer to settling the question of relative
performance/behavior by pushing down Parquet reader
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20999#discussion_r216380067
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -293,6 +293,28 @@ class AstBuilder(conf: SQLConf)
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22381#discussion_r216377882
--- Diff: core/src/main/scala/org/apache/spark/status/AppStatusSource.scala
---
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22381#discussion_r216377621
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -280,6 +284,12 @@ private[spark] class AppStatusListener(
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22381#discussion_r216378185
--- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala
---
@@ -503,9 +503,12 @@ private[spark] object AppStatusStore {
/**
Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/22381#discussion_r216377526
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -560,6 +561,7 @@ class SparkContext(config: SparkConf) extends Logging {
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22379
**[Test build #95874 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95874/testReport)**
for PR 22379 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22379
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95874/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22379
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22379
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95870/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22379
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22379
**[Test build #95870 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95870/testReport)**
for PR 22379 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22381
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22381
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22381
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22381
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22378
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22378
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22373
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22373
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22373
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22373
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22380
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22380
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22173
**[Test build #95876 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95876/testReport)**
for PR 22173 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21860
**[Test build #95873 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95873/testReport)**
for PR 21860 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22357
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22376
**[Test build #95883 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95883/testReport)**
for PR 22376 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22380
**[Test build #95878 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95878/testReport)**
for PR 22380 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22364
**[Test build #95869 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95869/testReport)**
for PR 22364 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22380
**[Test build #95875 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95875/testReport)**
for PR 22380 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22373
**[Test build #95877 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95877/testReport)**
for PR 22373 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/7
**[Test build #95872 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95872/testReport)**
for PR 7 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22378
**[Test build #95879 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95879/testReport)**
for PR 22378 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22357
**[Test build #95871 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95871/testReport)**
for PR 22357 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22379
**[Test build #95874 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95874/testReport)**
for PR 22379 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22379
**[Test build #95870 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95870/testReport)**
for PR 22379 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22381
**[Test build #95880 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95880/testReport)**
for PR 22381 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22357
**[Test build #95884 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95884/testReport)**
for PR 22357 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22382
**[Test build #95882 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95882/testReport)**
for PR 22382 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22381
**[Test build #95881 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95881/testReport)**
for PR 22381 at commit
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/22371#discussion_r216369925
--- Diff:
core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala ---
@@ -138,13 +154,22 @@ private[spark] class
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22364#discussion_r216369638
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/AttributeSet.scala
---
@@ -39,10 +41,15 @@ object AttributeSet {
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22383
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22383
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22383
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user mmolimar opened a pull request:
https://github.com/apache/spark/pull/22383
[SPARK-25395][JavaAPI] Removing Optional Spark Java API
## What changes were proposed in this pull request?
Previous Spark versions didn't require Java 8 and an ``Optional`` Spark
Java
Github user DaimonPl commented on the issue:
https://github.com/apache/spark/pull/22357
@mallman regarding
"But how do we know that pushing down IsNotNull(employer) does not negate
that instruction? "
isn't it pretty obvious that when you read 'employer.id' with
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/22366
> Is it better to add a description to docs/sql-programming-guide.md?
> Yea, let's note so that we can track what we change.
@kiszk @HyukjinKwon May I ask you to clarify this. Do you
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/18544
what's the status here?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/22357
> @mallman It will be great that we can have this fix in 2.4 release as
this can dramatically reduce the data being read in many applications which is
the purpose of the original work.
I
301 - 400 of 591 matches
Mail list logo
