Github user SaintBacchus closed the pull request at:
https://github.com/apache/spark/pull/5537
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature i
Github user aarondav commented on the pull request:
https://github.com/apache/spark/pull/5537#issuecomment-94187864
Ah, I realize now that the original change that brought this bug up (#4240)
was perhaps not the right solution. The issue that #4240 solved was actually
brought up befor
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5537#discussion_r28640429
--- Diff:
core/src/main/scala/org/apache/spark/network/netty/SparkTransportConf.scala ---
@@ -43,8 +43,12 @@ object SparkTransportConf {
* @param n
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/5537#discussion_r28640416
--- Diff:
core/src/main/scala/org/apache/spark/network/netty/SparkTransportConf.scala ---
@@ -43,8 +43,12 @@ object SparkTransportConf {
* @param n
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/5537#issuecomment-94114473
I see, it makes sense to remove it for services set up independently of the
Spark application, but not for Netty because the executors will still be able
to find the p
Github user aarondav commented on the pull request:
https://github.com/apache/spark/pull/5537#issuecomment-94062816
The changes as they currently stand LGTM, let's just fix up the style as
@vanzin pointed out.
---
If your project is set up for it, you can reply to this email and have
Github user aarondav commented on a diff in the pull request:
https://github.com/apache/spark/pull/5537#discussion_r28624950
--- Diff:
core/src/main/scala/org/apache/spark/network/netty/SparkTransportConf.scala ---
@@ -43,8 +43,12 @@ object SparkTransportConf {
* @param num
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/5537#discussion_r28622548
--- Diff:
core/src/main/scala/org/apache/spark/network/netty/SparkTransportConf.scala ---
@@ -43,8 +43,12 @@ object SparkTransportConf {
* @param numUs
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/5537#discussion_r28622322
--- Diff:
core/src/main/scala/org/apache/spark/network/netty/SparkTransportConf.scala ---
@@ -43,8 +43,12 @@ object SparkTransportConf {
* @param numUs
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/5537#discussion_r28622275
--- Diff:
core/src/main/scala/org/apache/spark/network/netty/SparkTransportConf.scala ---
@@ -43,8 +43,12 @@ object SparkTransportConf {
* @param numUs
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5537#issuecomment-93949861
[Test build #30473 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30473/consoleFull)
for PR 5537 at commit
[`962770c`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5537#issuecomment-93949870
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5537#issuecomment-93948083
[Test build #30473 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30473/consoleFull)
for PR 5537 at commit
[`962770c`](https://githu
Github user SaintBacchus commented on the pull request:
https://github.com/apache/spark/pull/5537#issuecomment-93947685
@andrewor14 `TransportServer#bindRightPort` will use in `Netty` network, in
that case have a retry mechanism is a better way.
@vanzin I have clone the configurat
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/5537#issuecomment-93836401
Given that the network-common library (where the retry logic is) is
supposed to be generic, and not just for shuffle, I'd rather keep the retry
logic there, so this change
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/5537#issuecomment-93799183
@SaintBacchus It's fine to set it to 0, but maybe it makes more sense to
just remove the logic that retries port in the first place since the shuffle
service is the on
Github user SaintBacchus commented on the pull request:
https://github.com/apache/spark/pull/5537#issuecomment-93691448
@srowen This code was used in `YarnShuffleService`, any spark application
can't get this property.
---
If your project is set up for it, you can reply to this email
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5537#issuecomment-93687641
CC @andrewor14 That makes some sense. I hope there aren't side effects to
setting this globally in this process.
---
If your project is set up for it, you can reply to th
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5537#issuecomment-93628986
[Test build #30391 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30391/consoleFull)
for PR 5537 at commit
[`7126547`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5537#issuecomment-93628990
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30
20 matches
Mail list logo
