Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-178141964
@zsxwing Other than the PR title - I think everything else says inflight in
code/documentation. Did you see anything to the contrary ?
---
If your project is set up
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-178129985
@tgravescs in general, this patch doesn't limit number of concurrent
outbound connections. Instead, it just limits number of in flight blocks. Could
you update the
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-178153656
Hm, I think `block` is also not `accurate`. Never mind, just update the
title should be enough.
---
If your project is set up for it, you can reply to this email and
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-178150124
> @zsxwing Other than the PR title - I think everything else says inflight
in code/documentation. Did you see anything to the contrary ?
I mean changing
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-174534386
ping @rxin @JoshRosen once more. I'll leave this open for a couple more
days.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-174773912
Actually cc @zsxwing who looked into this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user redsanket commented on a diff in the pull request:
https://github.com/apache/spark/pull/10838#discussion_r50576595
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -258,6 +268,9 @@ final class ShuffleBlockFetcherIterator(
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/10838#discussion_r50613356
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -258,6 +268,9 @@ final class ShuffleBlockFetcherIterator(
Github user holdenk commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-173682258
@wzhfy from looking at the linked original PR this is based on the large
number of requests happens with a very large # of nodes and smaller amounts of
data per node.
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/10838#discussion_r50449878
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -258,6 +268,8 @@ final class ShuffleBlockFetcherIterator(
Github user wzhfy commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-173437752
hi, @redsanket , in what situation will the number of requests become very
large?
---
If your project is set up for it, you can reply to this email and have your
reply
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/10838#discussion_r50269518
--- Diff: docs/configuration.md ---
@@ -392,6 +392,16 @@ Apart from these, the following properties are also
available, and may be useful
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-173242328
cc @rxin @JoshRosen
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-173242407
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/10838#discussion_r50269536
--- Diff: docs/configuration.md ---
@@ -392,6 +392,16 @@ Apart from these, the following properties are also
available, and may be useful
Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-173009800
LGTM - though would be better for someone else to also review this since I
might be too close to the code !
---
If your project is set up for it, you can reply to
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-172999704
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/10838#discussion_r50186875
--- Diff: docs/configuration.md ---
@@ -392,6 +392,17 @@ Apart from these, the following properties are also
available, and may be useful
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/10838#discussion_r50186855
--- Diff: docs/configuration.md ---
@@ -392,6 +392,17 @@ Apart from these, the following properties are also
available, and may be useful
Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/10838#discussion_r50186670
--- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala ---
@@ -81,7 +81,7 @@ import org.apache.spark.util.Utils
* - HTTP for broadcast
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-173008913
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
GitHub user redsanket opened a pull request:
https://github.com/apache/spark/pull/10838
[SPARK-6166] Limit number of concurrent outbound connections
This JIRA is related to
https://github.com/apache/spark/pull/5852
Had to do some minor rework and test to make sure it
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-173010832
**[Test build #49716 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49716/consoleFull)**
for PR 10838 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-173032887
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-173032736
**[Test build #49716 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49716/consoleFull)**
for PR 10838 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10838#issuecomment-173032886
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user redsanket commented on a diff in the pull request:
https://github.com/apache/spark/pull/10838#discussion_r50208257
--- Diff: docs/configuration.md ---
@@ -392,6 +392,17 @@ Apart from these, the following properties are also
available, and may be useful
Github user redsanket commented on a diff in the pull request:
https://github.com/apache/spark/pull/10838#discussion_r50208175
--- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala ---
@@ -81,7 +81,7 @@ import org.apache.spark.util.Utils
* - HTTP for broadcast
28 matches
Mail list logo
