Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r229013267
--- Diff: docs/running-on-kubernetes.md ---
@@ -799,4 +815,168 @@ specific to Spark on Kubernetes.
This sets the major Python version of the docker
Github user ifilonenko commented on a diff in the pull request:
https://github.com/apache/spark/pull/19045#discussion_r229012644
--- Diff:
resource-managers/kubernetes/integration-tests/tests/decomissioning_water.py ---
@@ -0,0 +1,38 @@
+#
+# Licensed to the Apache
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/22146
> Latest container selection w/ default to first LGTM
+1. LGTM on container selection.
---
-
To unsubscribe, e-mail:
GitHub user wangyum opened a pull request:
https://github.com/apache/spark/pull/22879
[SPARK-25872][SQL][TEST] Add an optimizer tracker for TPC-DS queries
## What changes were proposed in this pull request?
Add an optimizer tracker for TPC-DS queries. For example:
Github user ifilonenko commented on a diff in the pull request:
https://github.com/apache/spark/pull/19045#discussion_r229011769
--- Diff:
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala
---
@@
Github user yucai commented on a diff in the pull request:
https://github.com/apache/spark/pull/22845#discussion_r229011040
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVBenchmarks.scala
---
@@ -137,22 +124,15 @@ object CSVBenchmarks extends
Github user ifilonenko commented on a diff in the pull request:
https://github.com/apache/spark/pull/19045#discussion_r229011136
--- Diff:
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/DecommissionSuite.scala
---
@@ -0,0
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/22624#discussion_r229010495
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -230,7 +242,7 @@ class
Github user yucai commented on a diff in the pull request:
https://github.com/apache/spark/pull/22844#discussion_r229010476
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonBenchmarks.scala
---
@@ -195,23 +170,16 @@ object JSONBenchmarks
Github user erikerlandson commented on the issue:
https://github.com/apache/spark/pull/22146
Latest container selection w/ default to first LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22824
**[Test build #98221 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98221/testReport)**
for PR 22824 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22824
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98219/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22824
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22146
**[Test build #98219 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98219/testReport)**
for PR 22146 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22830
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22830
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22146
Kubernetes integration test status failure
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/4597/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22830
**[Test build #98220 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98220/testReport)**
for PR 22830 at commit
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/22824
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22146
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/4597/
---
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22830
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22146
**[Test build #98219 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98219/testReport)**
for PR 22146 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22830
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22830
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98194/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22830
**[Test build #98194 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98194/testReport)**
for PR 22830 at commit
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r229002442
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,944 @@
+/*
+ * Licensed to the Apache Software
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22824
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22824
**[Test build #98218 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98218/testReport)**
for PR 22824 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22824
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22309
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22309
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98200/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22309
**[Test build #98200 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98200/testReport)**
for PR 22309 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22861
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22861
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98207/
Test PASSed.
---
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/22824
test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22861
**[Test build #98207 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98207/testReport)**
for PR 22861 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22309
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22309
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98203/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22309
**[Test build #98203 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98203/testReport)**
for PR 22309 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22878
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22878
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98217/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22878
**[Test build #98217 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98217/testReport)**
for PR 22878 at commit
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r228994601
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,944 @@
+/*
+ * Licensed to the Apache Software
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/22789
Merged to master/2.4
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/22861#discussion_r228992409
--- Diff:
external/avro/src/test/scala/org/apache/spark/sql/execution/benchmark/AvroWriteBenchmark.scala
---
@@ -19,22 +19,17 @@ package
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/22861#discussion_r228991112
--- Diff:
external/avro/src/test/scala/org/apache/spark/sql/execution/benchmark/AvroWriteBenchmark.scala
---
@@ -19,22 +19,17 @@ package
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22878
**[Test build #98217 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98217/testReport)**
for PR 22878 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22878
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/19045
Hey @ifilonenko I'd appreciate your thoughts on the testing approach I took
here and if matches your suggestions.
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22878
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
GitHub user xuanyuanking opened a pull request:
https://github.com/apache/spark/pull/22878
[SPARK-25789][SQL] Support for Dataset of Avro
## What changes were proposed in this pull request?
Please credit to @bdrillard cause this mainly based on his previous work.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22146
Kubernetes integration test status failure
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/4594/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98216/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22146
**[Test build #98216 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98216/testReport)**
for PR 22146 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22789
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r228982925
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,944 @@
+/*
+ * Licensed to the Apache Software
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22146
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/4594/
---
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/22789
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22146
**[Test build #98216 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98216/testReport)**
for PR 22146 at commit
Github user yucai commented on the issue:
https://github.com/apache/spark/pull/22861
@dongjoon-hyun I used #22872 to make main args accessible for
`BenchmarkBase`'s subclass, this PR is mainly for refactoring
`DataSourceWriteBenchmark` and `BuiltInDataSourceWriteBenchmark`. But it
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/20999
@cloud-fan I checked the feasibility of doing what you suggested, but
unfortunately that method is very hard to be used (and I am not sure how safe
it is). The main issues are:
- it
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22861
**[Test build #98215 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98215/testReport)**
for PR 22861 at commit
Github user patrickbrownsync commented on a diff in the pull request:
https://github.com/apache/spark/pull/22855#discussion_r228970250
--- Diff:
core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala ---
@@ -298,30 +312,40 @@ class KryoDeserializationStream(
}
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/22830
Perhaps @jkbradley and @mengxr can comment on it. If the trait is
inheritable, then protected still means it is part of the API contract.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21654
Thanks, @holdenk for addressing my concern. I will try to join as well.
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22666#discussion_r228949792
--- Diff: sql/core/src/test/resources/sql-tests/inputs/csv-functions.sql ---
@@ -7,3 +7,11 @@ select from_csv('1', 'a InvalidType');
select
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22813
I don't know what else goes in the work dir. It isn't valid to reuse it for
anything else. Can you simply avoid using a work dir that is or has been used
by something else?
The argument for
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22877
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22877
**[Test build #98214 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98214/testReport)**
for PR 22877 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22877
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user peter-toth commented on a diff in the pull request:
https://github.com/apache/spark/pull/22617#discussion_r228944791
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArrayBenchmark.scala
---
@@ -68,9 +100,7 @@ object
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21588
Yes, that was what I was thinking at worst case. For clarification,
@wangyum made a try and all tests were passed at least -
https://github.com/apache/spark/pull/20659. Given this try, I think
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22873
**[Test build #98196 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98196/testReport)**
for PR 22873 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22873
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98196/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21654
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98211/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21654
**[Test build #98211 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98211/testReport)**
for PR 21654 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21654
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22146
**[Test build #98213 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98213/testReport)**
for PR 22146 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98213/
Test PASSed.
---
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22852
A quick pointer to security issues in other key places sounds good. As long
as it is increasing the chance users understand the specific issue and isn't
more general text to skip past, it is helping
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22146
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22146
Kubernetes integration test status failure
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/4592/
---
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22849#discussion_r228936286
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -240,7 +240,7 @@ class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22146
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/4592/
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22146
**[Test build #98213 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98213/testReport)**
for PR 22146 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22875
**[Test build #98212 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98212/testReport)**
for PR 22875 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22875
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22309
My only concern is, the value class handling is kind of spread out in
`ScalaReflection`. Maybe we need a better abstraction.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22875
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21654
**[Test build #98211 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98211/testReport)**
for PR 21654 at commit
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228925856
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -128,6 +128,15 @@ object ScalaReflection extends
301 - 400 of 585 matches
Mail list logo
