Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22325#discussion_r219236374
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -386,6 +386,7 @@ final class ShuffleBlockFetcherIterator(
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/19868
Sure, let me do it today.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user ash211 commented on the issue:
https://github.com/apache/spark/pull/22485
Sorry for not following through on getting this into Apache.
FWIW, it's been in the Palantir fork of Spark for over a year:
Github user fhoering closed the pull request at:
https://github.com/apache/spark/pull/22422
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user fhoering commented on the issue:
https://github.com/apache/spark/pull/22422
It turns out all this can be tailored by tuning the existing environment
variables. It is enough to generate the pex file with a generic entry point
that does the redirection to the custom module
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r219235188
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,926 @@
+/*
+ * Licensed to the Apache Software
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r219233070
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,926 @@
+/*
+ * Licensed to the Apache Software
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22501
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96369/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22501
**[Test build #96369 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96369/testReport)**
for PR 22501 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22501
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22494
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r219230810
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,926 @@
+/*
+ * Licensed to the Apache Software
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22494
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96360/
Test FAILed.
---
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r219230988
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,926 @@
+/*
+ * Licensed to the Apache Software
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22494
**[Test build #96360 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96360/testReport)**
for PR 22494 at commit
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r219229287
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,926 @@
+/*
+ * Licensed to the Apache Software
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22501
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22501
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22501
**[Test build #96369 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96369/testReport)**
for PR 22501 at commit
GitHub user wangyum opened a pull request:
https://github.com/apache/spark/pull/22501
[SPARK-25492][TEST] Refactor WideSchemaBenchmark to use main method
## What changes were proposed in this pull request?
Refactor `WideSchemaBenchmark` to use main method.
Generate
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r219223939
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,926 @@
+/*
+ * Licensed to the Apache Software
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22499
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22499
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96366/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22499
**[Test build #96366 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96366/testReport)**
for PR 22499 at commit
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r219222704
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,926 @@
+/*
+ * Licensed to the Apache Software
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r21974
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,926 @@
+/*
+ * Licensed to the Apache Software
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r219221688
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,926 @@
+/*
+ * Licensed to the Apache Software
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22450
totally agree, thanks for looking into it!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r219221157
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,926 @@
+/*
+ * Licensed to the Apache Software
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r219220733
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,926 @@
+/*
+ * Licensed to the Apache Software
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22462
ds v2 is pre-beta, so not a critical bug to me. I'm OK to fix it in 2.4 if
someone is willing to pay the effort for fixing the conflict.
---
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/22500#discussion_r219219972
--- Diff: sql/core/benchmarks/MiscBenchmark-results.txt ---
@@ -0,0 +1,132 @@
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22494
why do we care about breaking operations when turning off a behavior change
config? The config is prepared for this case: if a user hit a problem of the
new behavior, he can use this config to
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22500
**[Test build #96368 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96368/testReport)**
for PR 22500 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22500
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/22500#discussion_r219218036
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/MiscBenchmark.scala
---
@@ -17,251 +17,154 @@
package
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22500
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22450
I think there are 2 separate topics here:
- **Handling negative scale in decimal operations**
I am writing the design doc and I'll update this PR if needed (anyway
I'll add more test
GitHub user wangyum opened a pull request:
https://github.com/apache/spark/pull/22500
[SPARK-25488][TEST] Refactor MiscBenchmark to use main method
## What changes were proposed in this pull request?
Refactor `MiscBenchmark ` to use main method.
Generate benchmark
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22499
**[Test build #96366 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96366/testReport)**
for PR 22499 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22490
**[Test build #96367 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96367/testReport)**
for PR 22490 at commit
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r219216596
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,926 @@
+/*
+ * Licensed to the Apache Software
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22499
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22499
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user seancxmao opened a pull request:
https://github.com/apache/spark/pull/22499
[SPARK-25489][ML][TEST] Refactor UDTSerializationBenchmark
## What changes were proposed in this pull request?
Refactor `UDTSerializationBenchmark` to use main method and print the
output as
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22450
given how complex it is, I feel we can start the design at 2.5 and
implement it at 3.0, what do you think?
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22460
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96363/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22460
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22450
oh I see now what you mean, thanks. Yes, Hive does the same. We may have to
revisit completely our parsing of literals but since it is a breaking change I
am not sure it will be possible before
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22460
**[Test build #96363 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96363/testReport)**
for PR 22460 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22498
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22494
> So I don't see any harm of this PR.
If the user doesn't turn off the flag, of course nothing changes. If the
user does, then let's imagine this case. A user has this: `select 1234567891
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22498
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user pgandhi999 opened a pull request:
https://github.com/apache/spark/pull/22498
[SPARK-18364] : Expose metrics for YarnShuffleService
ExternalShuffleService exposes metrics as of SPARK-16405. However,
YarnShuffleService does not.
The work of instrumenting
Github user wangyum commented on the issue:
https://github.com/apache/spark/pull/22497
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22462
Thank you, @HyukjinKwon and @cloud-fan.
BTW, the bug exists in 2.4.0, doesnât it? Can we try another PR for Spark
2.4.0 at least?
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22497
**[Test build #96365 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96365/testReport)**
for PR 22497 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22497
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user rezasafi commented on the issue:
https://github.com/apache/spark/pull/22325
Looking at the unit-test.log and stacktrace the failed test doesn't seem to
be related to this change. There is errors like this
`18/09/19 20:47:53.776 shuffle-server-4270-1 ERROR
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22497
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user seancxmao opened a pull request:
https://github.com/apache/spark/pull/22497
[SPARK-25487][SQL][TEST] Refactor PrimitiveArrayBenchmark
## What changes were proposed in this pull request?
Refactor PrimitiveArrayBenchmark to use main method and print the output as
a
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22450
@dilipbiswal showed that DB2 and presto treat `1e100` as double instead of
decimal. We should consider this option and see what's the consequence of it.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22494
This PR is a no-op if users don't turn off #20023 . The benefit of this PR
is: before we fully fix that bug, if a user hit it, he can turn off #20023 to
temporarily work around it. So I don't see
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22496
**[Test build #96364 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96364/testReport)**
for PR 22496 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22496
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22496
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/22496
[SPARK-25422] DO NOT MERGE
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/squito/spark SPARK-25422
Alternatively you can review and apply
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22490
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96345/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22490
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22490
**[Test build #96345 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96345/testReport)**
for PR 22490 at commit
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22450
> seems there are more problems like the data type of literals
sorry, I haven't got what you mean here, may you please explain me?
> your long explanation makes me think we should
Github user squito commented on the issue:
https://github.com/apache/spark/pull/22483
lgtm
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22494
yes @cloud-fan, I see your point. I am neutral to this change honestly.
It is true that it avoids regressions form previous cases, but it is also
true that it doesn't make the behavior
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22460
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22460
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22376
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22376
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/3302/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22482
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96348/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22376
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22482
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22460
**[Test build #96363 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96363/testReport)**
for PR 22460 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22482
**[Test build #96348 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96348/testReport)**
for PR 22482 at commit
Github user abellina commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r219189730
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,807 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user abellina commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r219189328
--- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js
---
@@ -0,0 +1,807 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22494
2.3.2 and 2.4.0 are both in the RC stage, I don't want to spend a lot of
time to fix this long-standing bug and block 2 releases. cc @jerryshao too.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22494
yea, that's why I change the test to use `1e6 * 1000`, instead of `1000e6`.
The point is, we know there is a bug and it's hard to fix. What I'm trying
to do here is not fixing the bug,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22376
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/3302/
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22376
**[Test build #96362 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96362/testReport)**
for PR 22376 at commit
Github user ifilonenko commented on the issue:
https://github.com/apache/spark/pull/22376
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/22494
oh, I see now the problem. And it is much bigger than I thought, sorry.
Here we were returning a negative scale for `1e6` (it happens in the
`AstBuilder`, as we build a BigDecimal from it) also
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22495
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22495
**[Test build #96361 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96361/testReport)**
for PR 22495 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22495
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user yucai opened a pull request:
https://github.com/apache/spark/pull/22495
[SPARK-25486][TEST] Refactor SortBenchmark to use main method
## What changes were proposed in this pull request?
Refactor SortBenchmark to use main method.
Generate benchmark result:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22467
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22467
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96346/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22467
**[Test build #96346 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96346/testReport)**
for PR 22467 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22471
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96340/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22471
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22471
**[Test build #96340 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96340/testReport)**
for PR 22471 at commit
301 - 400 of 647 matches
Mail list logo
