Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15326
Hi @kayousterhout,
Many thanks again for your review.
Patch is ready to re-review.
---
-
To unsubscribe, e
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16992
Hi @squito,
Thanks for the review this patch. It is ready to re-review / merge.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/16992#discussion_r135415731
--- Diff: docs/job-scheduling.md ---
@@ -235,7 +235,7 @@ properties:
of the cluster. By default, each pool's `minShare`
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16992
Hi @kayousterhout @markhamstra @squito,
This PR is ready to review.
Thanks in advance for all feedbacks.
---
If your project is set up for it, you can reply to this email and
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15326
Hi @kayousterhout,
Many thanks again for your review and sorry about delay.
Last comments have just been addressed and patch is ready for re-review.
---
If your project is set up
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/15326#discussion_r120807562
--- Diff: core/src/test/scala/org/apache/spark/scheduler/PoolSuite.scala ---
@@ -292,7 +290,100 @@ class PoolSuite extends SparkFunSuite with
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15326
Hi @HyukjinKwon, thanks for the following this PR again. This looks
required but i am too busy for a while. Fix is already ready and will address
the last comments asap. Sorry for delay
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15326
Hi @HyukjinKwon, thanks to follow this PR. I am busy for a while.
Implementation is done and just last comments need to be addressed.
I will address them as well asap. Thanks
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15326
Hi @kayousterhout and @markhamstra,
- This PR is ready for review in the light of first-added-wins
(Schedulables: `Pool` / `TaskSetManager`) pattern. All feedback are welcome in
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15326
Jenkins test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15326
@kayousterhout, thanks for querying this PR.
It is not ready yet and needs a small change but it will be ready in a
couple of days. Will let you know ;)
---
If your project is set
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16905
Hi @squito,
I think jenkins can be triggered again. Last unrelated build - UT issue
looks fixed(https://issues.apache.org/jira/browse/SPARK-19988)
Thanks in advance
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16905
Thanks again @kayousterhout and @squito ;)
Jenkins needs to be retriggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16905
Thanks again @kayousterhout and @squito.
Merge conflict has been fixed and this PR is ready to build on jenkins and
merge (if there is no concern).
---
If your project is set up for
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16905
Thanks @kayousterhout and @squito for review this ;)
Last comment is also addressed via last commit: 18657d4.
---
If your project is set up for it, you can reply to this email and
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15604
Many thanks @kayousterhout for reviewing this again. Last comment has also
been addressed.
Also the following Jira has been created for previous second UT failure.
https
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/15604#discussion_r106280052
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala ---
@@ -191,7 +191,10 @@ private[spark] class
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15604
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15604
The following unit tests look failed due to timeout and _irrelevant_ so
triggered again.
**1-**
org.apache.spark.storage.BlockManagerProactiveReplicationSuite.proactive block
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16905
Hi @markhamstra, @kayousterhout and @squito
This PR is ready for final review and merge if there is no concern.
All feedbacks are welcome. Thanks in advance ;)
---
If your project
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15604
Hi @kayousterhout and @squito,
Last comments are addressed so this PR is ready to re-review / merge if it
is possible.
Thanks in advance ;)
---
If your project is set up for it
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/15604#discussion_r105567716
--- Diff: core/src/test/scala/org/apache/spark/scheduler/PoolSuite.scala ---
@@ -201,12 +202,96 @@ class PoolSuite extends SparkFunSuite with
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16905
Thanks @kayousterhout. However, it still looks not triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16905
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16905
Many thanks @kayousterhout and @markhamstra.
Also minor scala style issue is fixed with new patch.
---
If your project is set up for it, you can reply to this email and have your
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16905
Thanks @markhamstra.
I encounter the same problem in my every PR and tech-ops team helps to
trigger Jenkins. Problem may be related with Github Jenkins PR Builder plugin
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16905
Thanks @markhamstra.
I think jenkins needs to be triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15604
Hi @kayousterhout and @squito,
This PR is ready to re-review.
All feedbacks are welcome and thanks in advance ;)
---
If your project is set up for it, you can reply to this email
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16905
Hi @markhamstra,
Do you have chance to review this PR and my latest comment?
Thanks in advance ;)
---
If your project is set up for it, you can reply to this email and have your
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16992
cc @kayousterhout @markhamstra @squito
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
GitHub user erenavsarogullari opened a pull request:
https://github.com/apache/spark/pull/16992
[SPARK-19662][SCHEDULER][TEST] Add Fair Scheduler Unit Test coverage for
different build cases
## What changes were proposed in this pull request?
Fair Scheduler can be built via one
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15604
Hi @kayousterhout and @squito,
This PR is ready to re-review / merge if it is possible.
All feedbacks are welcome and thanks in advance ;)
---
If your project is set up for it
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16905
Hi All,
All comments are addressed and this is ready for re-review.
@markhamstra, if it is suitable, i plan to address _removing
SchedulingMode.NONE_ via separated PR. In the
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/16905#discussion_r101176246
--- Diff: core/src/main/scala/org/apache/spark/scheduler/Pool.scala ---
@@ -37,25 +37,22 @@ private[spark] class Pool(
val
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16905
Thanks @markhamstra and @srowen for the quick reviews.
I will address them asap by covering the additional patch suggested by Mark.
---
If your project is set up for it, you can reply
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/16905#discussion_r101173634
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -130,15 +130,17 @@ private[spark] class TaskSchedulerImpl
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/16905#discussion_r101164858
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -130,15 +130,17 @@ private[spark] class TaskSchedulerImpl
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16905
cc @kayousterhout @markhamstra @squito
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
GitHub user erenavsarogullari opened a pull request:
https://github.com/apache/spark/pull/16905
[SPARK-19567] [CORE] [SCHEDULER] Support some Schedulable variables
immutability and access
## What changes were proposed in this pull request?
Some `Schedulable` Entities(`Pool` and
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15604
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15604
build failure looks unrelated (due to timeout at KafkaSourceSuite level)
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15604
Hi @kayousterhout and @squito,
Firstly, many thanks for the review. All comments are addressed. This is
ready to re-review ;)
---
If your project is set up for it, you can reply to
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/15604#discussion_r100646612
--- Diff: core/src/test/scala/org/apache/spark/scheduler/PoolSuite.scala ---
@@ -178,4 +180,97 @@ class PoolSuite extends SparkFunSuite with
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15604
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16813
Thanks everyone for all your support :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16813
I think problem still continues. We can get techops support to trigger
jenkins.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16813
Thanks @kayousterhout.
I think jenkins needs to be triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/16813#discussion_r100422328
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala ---
@@ -83,15 +83,19 @@ private[spark] class
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/16813#discussion_r100409787
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala ---
@@ -69,19 +70,31 @@ private[spark] class
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/16813#discussion_r100398573
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala ---
@@ -69,19 +70,31 @@ private[spark] class
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15604
@kayousterhout thanks for query this PR. I will be submitting this and the
other one( #15326) once #16813 is merged.
---
If your project is set up for it, you can reply to this email
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16813
Hi @kayousterhout and @markhamstra,
All comments have been addressed via latest patch(790097e)
---
If your project is set up for it, you can reply to this email and have your
reply
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/16813#discussion_r99965179
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala ---
@@ -69,60 +72,81 @@ private[spark] class
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/16813#discussion_r99960547
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala ---
@@ -69,60 +72,81 @@ private[spark] class
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/16813#discussion_r99938138
--- Diff: core/src/test/scala/org/apache/spark/scheduler/PoolSuite.scala ---
@@ -201,6 +202,49 @@ class PoolSuite extends SparkFunSuite with
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16813
Hi @kayousterhout and @markhamstra,
Many thanks for the review and comments. All of them have just been
addressed via new patch.
It is ready for re-review.
---
If your project
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/16813
cc @squito @kayousterhout
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user erenavsarogullari opened a pull request:
https://github.com/apache/spark/pull/16813
[SPARK-19466][CORE][SCHEDULER] Improve Fair Scheduler Logging
Fair Scheduler Logging for the following cases can be useful for the user.
1. If **valid
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15237
Hi @squito,
Sorry, i am quite busy this week and happy to be merged this PR now if it
is also ok for you. I plan to address fileName logging via separated jira.
Also To
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15237
Hi @squito,
Firstly, thanks again for the review.
Sure, scheduler filename can be gotten via `buildPools` function by
creating a tuple / case class as `(Option[InputStream
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15237
Hi @squito,
Firstly, sorry for delay and thanks for review again.
All comments are addressed via new patch(a1b2924) and it is ready to
re-review / merge ;)
---
If your project is
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15604
Hi @kayousterhout @markhamstra @squito,
This PR aims to extend Pool usage policies unit test coverage and is also
ready for review.
Thanks and all feedbacks are welcome in advance
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/15237#discussion_r93819357
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala ---
@@ -102,38 +105,55 @@ private[spark] class
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/15237#discussion_r93819365
--- Diff: core/src/test/scala/org/apache/spark/scheduler/PoolSuite.scala ---
@@ -178,4 +177,36 @@ class PoolSuite extends SparkFunSuite with
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15237
Hi @squito,
Many thanks again for review. All comments have just been addressed via new
commit.
Also It is ready to rereview / merge ;)
---
If your project is set up for it, you
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15237
Many thanks @squito for review.
I will address them asap ;)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15237
cc @kayousterhout @markhamstra @squito
Thanks and all feedbacks are welcome in advance ;)
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15326
Hi @kayousterhout and @markhamstra,
This PR is ready for review if it is possible.
All feedbacks are welcome and thanks in advance ;)
---
If your project is set up for it, you
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15326
Kindly reminder :)
cc @kayousterhout @markhamstra
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15326
Hi @kayousterhout and @markhamstra,
Related PR #15463 is merged and this PR is ready for review. Also previous
comments have been addressed with
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15617
Thanks @kayousterhout and @zsxwing for the support. I have missed to build
with v2.10 and will help as a good experience for incoming PRs ;)
---
If your project is set up for it, you can
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15463
Thanks all. Patch looks passed on Jenkins so it is ready to merge if there
is no extra comments ;)
---
If your project is set up for it, you can reply to this email and have your
reply
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15604
cc @kayousterhout @markhamstra
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user erenavsarogullari opened a pull request:
https://github.com/apache/spark/pull/15604
Add Pool usage policies test coverage for FIFO & FAIR Schedulers
## What changes were proposed in this pull request?
The following FIFO & FAIR Schedulers Pool usage cases need
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15463
Thanks all for the support. Jenkins looks failed due to timeout.
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15463
Thanks @kayousterhout for support but I think problem still continues. Is
there any contact to get support?
---
If your project is set up for it, you can reply to this email and have
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15463
I think Jenkins can not be triggered ;) Is there any other way to trigger?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15463
Thanks @kayousterhout for review again. Jenkins did not start to build and
run the tests.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15463
Thanks @kayousterhout for the comments. They are addressed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/15463#discussion_r84172540
--- Diff: core/src/test/scala/org/apache/spark/scheduler/FakeTask.scala ---
@@ -37,12 +37,17 @@ object FakeTask {
}
def
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/15463#discussion_r84172514
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala ---
@@ -974,6 +974,22 @@ class TaskSetManagerSuite extends
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/15463#discussion_r84172485
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala ---
@@ -974,6 +974,22 @@ class TaskSetManagerSuite extends
Github user erenavsarogullari commented on a diff in the pull request:
https://github.com/apache/spark/pull/15463#discussion_r84172446
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala ---
@@ -974,6 +974,22 @@ class TaskSetManagerSuite extends
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15463
cc @kayousterhout @markhamstra
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user erenavsarogullari opened a pull request:
https://github.com/apache/spark/pull/15463
[SPARK-17894] [CORE] Ensure uniqueness of TaskSetManager name.
## What changes were proposed in this pull request?
`TaskSetManager` should have unique name to avoid adding
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15326
Hi @kayousterhout and @markhamstra,
In the light of our previous discussion, i committed second patch as
e126cd8. This PR' s changeset looks ok but it still needs u
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15326
Thanks @kayousterhout and @markhamstra.
In the light of the decision, i can happily work on the new
ticket(Uniqueness of TaskSetManager name) as well ;)
---
If your project is set up
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15326
Firstly, thanks @markhamstra and @kayousterhout for quick feedbacks.
I was aware of putting the check in `Pool.addSchedulable` to cover for all
`Schedulables` (`Pool` and
GitHub user erenavsarogullari opened a pull request:
https://github.com/apache/spark/pull/15326
[SPARK-17759] [CORE] SchedulableBuilder should avoid to create duplicate
fair scheduler-pools
## What changes were proposed in this pull request?
If `spark.scheduler.allocation.file
GitHub user erenavsarogullari opened a pull request:
https://github.com/apache/spark/pull/15237
[SPARK-17663] [CORE] SchedulableBuilder should handle invalid data access
via scheduler.alâ¦
## What changes were proposed in this pull request?
If `spark.scheduler.allocation.file
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15143
Thanks @srowen. PR Title is updated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15143
Thanks @srowen for the comment. The change-set looks ready.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user erenavsarogullari commented on the issue:
https://github.com/apache/spark/pull/15143
Hi @rxin,
Firstly, thanks for quick reply.
I was thinking for unit-test coverage perspective and a starter point to
contribute project but it is ok for me if PR is
GitHub user erenavsarogullari opened a pull request:
https://github.com/apache/spark/pull/15143
[SPARK-17584][Test] - Add unit test coverage for TaskState and ExecutorState
## What changes were proposed in this pull request?
- TaskState and ExecutorState expose isFailed and
94 matches
Mail list logo
