[jira] [Commented] (MESOS-2735) Change the interaction between the slave and the resource estimator from polling to pushing
[
https://issues.apache.org/jira/browse/MESOS-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14567860#comment-14567860
]
Benjamin Mahler commented on MESOS-2735:
{quote}
Finally, one of the advantages of the pull model is that it's easier to reason
about because we don't have "anonymous" lambdas that cause execution in some
other random place in the code (i.e., you can easily see in the slave where the
future that gets returned from `ResourceEstimator::estimate()` gets handled).
In addition, the ResourceEstimator remains "functional" in the sense that it
just has to return some value (or a future) from it's functions versus invoking
some callback that causes something to get run some other place (and in fact,
may also block, so isn't it safer for the ResourceEstimator to invoke the
callback in it's own `async`?).
The invocation of the `ResourceEstimator::estimate()` followed by the `.then`
is a nice pattern that let's us compose with other things as well, which is
harder to do with the lambda style callbacks and why we've avoided it where
we've been able (in fact, I'm curious which place in the code are you imitating
here?).
{quote}
The anonymous lambdas are because we do not have an abstraction to represent an
asychronous stream of results:
{code}
// Returns a stream of the unused resources that can be oversubscribed.
Stream unused();
{code}
This allows the caller to hold the stream and do composition on the next()
Future result, so we achieve push but we keep the ability to do composition.
{{http::Pipe}} is a string-specialized form of this and {{process::Queue}}
is an infinite-only version of {{process::Stream}}. An important additional
property of {{process::Stream}} is that it should provide "asynchronous
back-pressure" (e.g. unix pipes provide synchronous back-pressure by blocking
writes, however we need an asynchronous mechanism to back-pressure the writes),
to control the DoS concerns you guys have mentioned.
Take the allocator for example, it seems non-intuitive if the master had to
continually call a method to obtain the next allocation from
{{Future allocation()}}. At least, it requires more understanding
than what it expressed in the return type. Whereas, if we have a
{{Stream initialize()}} or {{Stream allocations()}},
we're clearly capturing the semantics within the return type. Back-pressure is
the key to ensure that the master's speed of consumption limits the rate at
which the allocator makes allocations. Today, {{process::Queue}} is an
equivalent replacement since allocations occur as an infinite stream and there
is no back-pressuring.
> Change the interaction between the slave and the resource estimator from
> polling to pushing
>
>
> Key: MESOS-2735
> URL: https://issues.apache.org/jira/browse/MESOS-2735
> Project: Mesos
> Issue Type: Bug
>Reporter: Jie Yu
>Assignee: Jie Yu
> Labels: twitter
>
> This will make the semantics more clear. The resource estimator can control
> the speed of sending resources estimation to the slave.
> To avoid cyclic dependency, slave will register a callback with the resource
> estimator and the resource estimator will simply invoke that callback when
> there's a new estimation ready. The callback will be a defer to the slave's
> main event queue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2735) Change the interaction between the slave and the resource estimator from polling to pushing
[ https://issues.apache.org/jira/browse/MESOS-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14554880#comment-14554880 ] Jie Yu commented on MESOS-2735: --- OK guys, I reverted the change here: https://reviews.apache.org/r/34559/ > Change the interaction between the slave and the resource estimator from > polling to pushing > > > Key: MESOS-2735 > URL: https://issues.apache.org/jira/browse/MESOS-2735 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > Labels: twitter > > This will make the semantics more clear. The resource estimator can control > the speed of sending resources estimation to the slave. > To avoid cyclic dependency, slave will register a callback with the resource > estimator and the resource estimator will simply invoke that callback when > there's a new estimation ready. The callback will be a defer to the slave's > main event queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2735) Change the interaction between the slave and the resource estimator from polling to pushing
[
https://issues.apache.org/jira/browse/MESOS-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14553007#comment-14553007
]
Jie Yu commented on MESOS-2735:
---
Thanks for all the comments!
tl;dr: I am still not fully convinced by the pull model, but I think we should
move on and not spending too much time arguing on this. So I'll do the change
as [~benjaminhindman] suggested (use 'Future oversubscribable()' in
ResourceEstimator). [~nnielsen], does this sound good to you?
The following are the reasons why I am still not convinced. Just write them
down for future references:
1) Regarding the DoS concern. I think the pull model is vulnerable to that as
well. In some sense, it's more vulnerable to that IMO. Why? Imagine that at
Twitter, we want to write a constant resource estimator (i.e., returns a fix
amount of cpus to the slave when the slave asks about how many resources can be
oversubscribed). Now, one natural implementation will be like the following:
{code}
Future oversubscribable()
{
return Resources::parse("cpus:2").get();
}
{code}
The estimator writer's thought process could be like: whenever the slave asks
me about how many resources can be oversubscribed, I'll tell it it's 2 cpus.
Well, this _natural_ implementation will cause a DoS problem for the slave
because its queue will be flooded, and it'll flood the logging file as well. I
would argue that a push model makes the programmer less likely to make such
mistake. Why? The programmer needs to write a tight loop (either synchronous or
asynchronous) to flood the slave, and we usually think twice when writing a
tight loop, don't we?
2) I liked the composability and and functional arguments Ben made in the
ticket. I think the push model mimics the actor style message passing model we
also constantly used in our code base. Just imagine replacing the lambda with
PID, and invoking the callback becomes calling dispatch on the pid. I
guess the message passing model is more suitable for entities across process
boundaries (e.g., master <-> slave, framework <-> master). Otherwise, we prefer
the funcitonal style in general. [~benjaminhindman], is that what you're trying
to convey? I think I agree with you on this. Just wondering if in the future,
we ever want to move resources estimator to a separate unix process?
> Change the interaction between the slave and the resource estimator from
> polling to pushing
>
>
> Key: MESOS-2735
> URL: https://issues.apache.org/jira/browse/MESOS-2735
> Project: Mesos
> Issue Type: Bug
>Reporter: Jie Yu
>Assignee: Jie Yu
> Labels: twitter
>
> This will make the semantics more clear. The resource estimator can control
> the speed of sending resources estimation to the slave.
> To avoid cyclic dependency, slave will register a callback with the resource
> estimator and the resource estimator will simply invoke that callback when
> there's a new estimation ready. The callback will be a defer to the slave's
> main event queue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2735) Change the interaction between the slave and the resource estimator from polling to pushing
[ https://issues.apache.org/jira/browse/MESOS-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551804#comment-14551804 ] Vinod Kone commented on MESOS-2735: --- Thanks Ben for the comments. I think the main motivations for the push model were to 1) make the writing of the slave logic (interfacing with estimator) simple and 2) make the writing of estimator module simple. Originally, with the pull model, it looked like we need to have 2 intervals within the slave: one for slave sending estimates to the master and one for slave getting estimates from the estimator. But if we assume that the estimators will be well behaved then we don't need an interval for the latter. The other issue, as you discussed in your comment, was about DoS. It *looked* like both the push and pull model had the same scope for DoS on the slave, so we didn't find a compelling a reason to go for pull because push was easier to implement on both sides of the interface. I said *looked*, because after processing your comments, I realized that the DoS behavior is different in push vs pull. In a push model a misbehaving estimator could do head of line blocking of other messages enqueued on the slave's queue, whereas in the pull model head of line blocking is not possible because the next (deferred) pull will be enqueued behind all the other messages. So, I'm ok going with pull for safety. Also, the composition argument can't be denied. Btw, the inspiration for the push model came from the allocator (and to a lesser extent Mesos class) which I think is very close to the estimator in terms of interactions. [~jieyu], ok with this? > Change the interaction between the slave and the resource estimator from > polling to pushing > > > Key: MESOS-2735 > URL: https://issues.apache.org/jira/browse/MESOS-2735 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > Labels: twitter > > This will make the semantics more clear. The resource estimator can control > the speed of sending resources estimation to the slave. > To avoid cyclic dependency, slave will register a callback with the resource > estimator and the resource estimator will simply invoke that callback when > there's a new estimation ready. The callback will be a defer to the slave's > main event queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2735) Change the interaction between the slave and the resource estimator from polling to pushing
[ https://issues.apache.org/jira/browse/MESOS-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551569#comment-14551569 ] Benjamin Hindman commented on MESOS-2735: - There are definitely differences in message queue behavior, one of which is significantly safer than the other. There are two safety concerns that I can think of, one of which [~jieyu] has addressed here but I'll repeat to be sure I properly understood. (1) Someone might write a ResourceEstimator that isn't asynchronous, causing the slave to "block" while the resource estimator estimates. (2) The ResourceEstimator might cause a denial of service attack on the slave. I understand the concern with (1) but I'm not too anxious about it. Why? It should be trivial to make a wrapper module which forces people to implement the ResourceEstimator to be asynchronous, either using `async` like you suggested or implementing a version of ResourceEstimator which wraps an actor (libprocess process). We'll only need to do this once and then other ResourceEstimator implementations can leverage this stuff. On the other hand, I don't like the behavior of push because of (2). Fundamentally, if the slave can't keep up with the rate at which the ResourceEstimator is pushing then we could create a denial of service issue with the slave, i.e., it takes a long time to process non-ResourceEstimator messages because it's queue is full of just ResourceEstimator messages. I'm more anxious about (2) than (1) because it's harder to find bugs in (2) than with (1) since once you fix (1) it stays fixed forever but any time you updated the algorithm you impact the potential to cause (2). Now, I acknowledge that implementing this as a pull versus push will make the implementation in the ResourceEstimator slightly more complicated, but not really. In particular, it should be trivial to always use a `Queue` to achieve the push semantics in any ResourceEstimator implementation, while still providing the pull semantics externally. Make sense? Finally, one of the advantages of the pull model is that it's easier to reason about because we don't have "anonymous" lambdas that cause execution in some other random place in the code (i.e., you can easily see in the slave where the future that gets returned from `ResourceEstimator::estimate()` gets handled). In addition, the ResourceEstimator remains "functional" in the sense that it just has to return some value (or a future) from it's functions versus invoking some callback that causes something to get run some other place (and in fact, may also block, so isn't it safer for the ResourceEstimator to invoke the callback in it's own `async`?). The invocation of the `ResourceEstimator::estimate()` followed by the `.then` is a nice pattern that let's us compose with other things as well, which is harder to do with the lambda style callbacks and why we've avoided it where we've been able (in fact, I'm curious which place in the code are you imitating here?). > Change the interaction between the slave and the resource estimator from > polling to pushing > > > Key: MESOS-2735 > URL: https://issues.apache.org/jira/browse/MESOS-2735 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > Labels: twitter > > This will make the semantics more clear. The resource estimator can control > the speed of sending resources estimation to the slave. > To avoid cyclic dependency, slave will register a callback with the resource > estimator and the resource estimator will simply invoke that callback when > there's a new estimation ready. The callback will be a defer to the slave's > main event queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2735) Change the interaction between the slave and the resource estimator from polling to pushing
[ https://issues.apache.org/jira/browse/MESOS-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551568#comment-14551568 ] Benjamin Hindman commented on MESOS-2735: - There are definitely differences in message queue behavior, one of which is significantly safer than the other. There are two safety concerns that I can think of, one of which [~jieyu] has addressed here but I'll repeat to be sure I properly understood. (1) Someone might write a ResourceEstimator that isn't asynchronous, causing the slave to "block" while the resource estimator estimates. (2) The ResourceEstimator might cause a denial of service attack on the slave. I understand the concern with (1) but I'm not too anxious about it. Why? It should be trivial to make a wrapper module which forces people to implement the ResourceEstimator to be asynchronous, either using `async` like you suggested or implementing a version of ResourceEstimator which wraps an actor (libprocess process). We'll only need to do this once and then other ResourceEstimator implementations can leverage this stuff. On the other hand, I don't like the behavior of push because of (2). Fundamentally, if the slave can't keep up with the rate at which the ResourceEstimator is pushing then we could create a denial of service issue with the slave, i.e., it takes a long time to process non-ResourceEstimator messages because it's queue is full of just ResourceEstimator messages. I'm more anxious about (2) than (1) because it's harder to find bugs in (2) than with (1) since once you fix (1) it stays fixed forever but any time you updated the algorithm you impact the potential to cause (2). Now, I acknowledge that implementing this as a pull versus push will make the implementation in the ResourceEstimator slightly more complicated, but not really. In particular, it should be trivial to always use a `Queue` to achieve the push semantics in any ResourceEstimator implementation, while still providing the pull semantics externally. Make sense? Finally, one of the advantages of the pull model is that it's easier to reason about because we don't have "anonymous" lambdas that cause execution in some other random place in the code (i.e., you can easily see in the slave where the future that gets returned from `ResourceEstimator::estimate()` gets handled). In addition, the ResourceEstimator remains "functional" in the sense that it just has to return some value (or a future) from it's functions versus invoking some callback that causes something to get run some other place (and in fact, may also block, so isn't it safer for the ResourceEstimator to invoke the callback in it's own `async`?). The invocation of the `ResourceEstimator::estimate()` followed by the `.then` is a nice pattern that let's us compose with other things as well, which is harder to do with the lambda style callbacks and why we've avoided it where we've been able (in fact, I'm curious which place in the code are you imitating here?). > Change the interaction between the slave and the resource estimator from > polling to pushing > > > Key: MESOS-2735 > URL: https://issues.apache.org/jira/browse/MESOS-2735 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > Labels: twitter > > This will make the semantics more clear. The resource estimator can control > the speed of sending resources estimation to the slave. > To avoid cyclic dependency, slave will register a callback with the resource > estimator and the resource estimator will simply invoke that callback when > there's a new estimation ready. The callback will be a defer to the slave's > main event queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2735) Change the interaction between the slave and the resource estimator from polling to pushing
[
https://issues.apache.org/jira/browse/MESOS-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550838#comment-14550838
]
Jie Yu commented on MESOS-2735:
---
{quote} One of the advantages that we had discussed in the past was that the
pull model enables us to move as fast as we possibly can, rather than just
getting a bunch of messages queued up in the slave that we have to process.
{quote}
I don't think there is a difference in terms of queueing messages. The pull
model also queues messages in the slave (e.g.,
'estimator->oversubscribed().then(defer(...))' also queues messages in slave's
queue).
{quote} Even if we want to collect more fine-grained resource estimations a
ResourceEstimator could do this and store this information until future polls.
{quote}
I think there's no fundamental difference between the pull and the push model.
The are only two subtle differences between the two: 1) the push model makes
less assumptions about the slave behavior. 2) the push model is safer in the
face of bad behaved resource estimator. Let me elaborate both of them below:
Regarding (1), let's use an example. Say we want to write a resource estimator
which sends constant number of cpus (say 2 cpus) every 10 seconds. If we use a
push model, we could just follow the
[NoopResourceEstimatorProcess|https://github.com/apache/mesos/blob/master/src/slave/resource_estimator.cpp#L52]
implementation in the code. Basically, we fork a libprocess and invoke the
registered callback every 10 seconds with 2 cpus.
Now, if we use a pull model, we first need to make an assumption that the slave
pull the resource estimator as fast as it can without any delay. If there's a
delay say 1 second, the resource estimator needs to adjust its internal delay
to be 9 seconds so that the total interval between two estimations is 10
seconds apart. When implementing the `Future oversubscribed()`
interface, the module writer needs to make another assumption about the slave
that the slave will not invoke the interface again if the previous estimation
is still pending. This is important because otherwise, the module writer needs
to maintain a list of Promises (instead of just one). I just feels that
there're so many implicit assumptions that the module writer needs to make in a
pull model.
Regarding (2), as I already stated in this ticket, since the slave invoked the
interface ('oversubscribed()') in its context, the module writer needs to make
sure the implementation of the interface does not block, otherwise the slave
will hang. An alternative is to use 'async' while invoking the interface in the
slave. I just feel this is rather not necessary if we use a push model.
> Change the interaction between the slave and the resource estimator from
> polling to pushing
>
>
> Key: MESOS-2735
> URL: https://issues.apache.org/jira/browse/MESOS-2735
> Project: Mesos
> Issue Type: Bug
>Reporter: Jie Yu
>Assignee: Jie Yu
> Labels: twitter
>
> This will make the semantics more clear. The resource estimator can control
> the speed of sending resources estimation to the slave.
> To avoid cyclic dependency, slave will register a callback with the resource
> estimator and the resource estimator will simply invoke that callback when
> there's a new estimation ready. The callback will be a defer to the slave's
> main event queue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2735) Change the interaction between the slave and the resource estimator from polling to pushing
[ https://issues.apache.org/jira/browse/MESOS-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550575#comment-14550575 ] Benjamin Hindman commented on MESOS-2735: - I'd also like to understand better why to go push instead of pull (poll). One of the advantages that we had discussed in the past was that the pull model enables us to move as fast as we possibly can, rather than just getting a bunch of messages queued up in the slave that we have to process. Even if we want to collect more fine-grained resource estimations a ResourceEstimator could do this and store this information until future polls. > Change the interaction between the slave and the resource estimator from > polling to pushing > > > Key: MESOS-2735 > URL: https://issues.apache.org/jira/browse/MESOS-2735 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > Labels: twitter > > This will make the semantics more clear. The resource estimator can control > the speed of sending resources estimation to the slave. > To avoid cyclic dependency, slave will register a callback with the resource > estimator and the resource estimator will simply invoke that callback when > there's a new estimation ready. The callback will be a defer to the slave's > main event queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2735) Change the interaction between the slave and the resource estimator from polling to pushing
[ https://issues.apache.org/jira/browse/MESOS-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14548606#comment-14548606 ] Jie Yu commented on MESOS-2735: --- Sorry that I just notice this reply here. I committed the patch already, but we can certainly revert it if you have strong opinion against it. > If the estimator never updates the "last estimate" in the slave, is the same > effect - no? Not the same effect. The slave won't be blocked at least in the push model, meaning that the slave will still be able to process all messages (e.g., runTask). In the polling model, a bad resource estimator can block slave's event queue. > Is the problem, that the current design doesn't support the "multiple firing" > problem, where the estimator updates while the callback is being executed? Could you please elaborate on the "multiple firing" problem? I am curious what example in your mind that makes you think that a push model is hard to use than the polling model. > Change the interaction between the slave and the resource estimator from > polling to pushing > > > Key: MESOS-2735 > URL: https://issues.apache.org/jira/browse/MESOS-2735 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > Labels: twitter > > This will make the semantics more clear. The resource estimator can control > the speed of sending resources estimation to the slave. > To avoid cyclic dependency, slave will register a callback with the resource > estimator and the resource estimator will simply invoke that callback when > there's a new estimation ready. The callback will be a defer to the slave's > main event queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2735) Change the interaction between the slave and the resource estimator from polling to pushing
[
https://issues.apache.org/jira/browse/MESOS-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546469#comment-14546469
]
Niklas Quarfot Nielsen commented on MESOS-2735:
---
Sorry that I am keep banging on this issue, but still not completely convinced
this makes a cleaner API.
Hope you can help me understand.
{quote}
1) This matches the existing allocator interface.
{quote}
But doesn't match many other APIs internally - don't feel this is a strong
argument.
{quote}
2) Slave no longer needs to configure the polling interval. It's up to the
resource estimator to decide when to send estimations.
{quote}
Didn't you agree that it could be done with
updates().then(lambda(&Slave::updateOverestimated, lambda::_1))?
{quote}
3) It avoid the potential issue that "estimator::update()" is blocking. Slave
will hang if "estimator::update()" blocks.
{quote}
If the estimator never updates the "last estimate" in the slave, is the same
effect - no?
Is the problem, that the current design doesn't support the "multiple firing"
problem, where the estimator updates while the callback is being executed?
[~benjaminhindman] Do you have an opinion on this?
> Change the interaction between the slave and the resource estimator from
> polling to pushing
>
>
> Key: MESOS-2735
> URL: https://issues.apache.org/jira/browse/MESOS-2735
> Project: Mesos
> Issue Type: Bug
>Reporter: Jie Yu
>Assignee: Jie Yu
> Labels: twitter
>
> This will make the semantics more clear. The resource estimator can control
> the speed of sending resources estimation to the slave.
> To avoid cyclic dependency, slave will register a callback with the resource
> estimator and the resource estimator will simply invoke that callback when
> there's a new estimation ready. The callback will be a defer to the slave's
> main event queue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2735) Change the interaction between the slave and the resource estimator from polling to pushing
[ https://issues.apache.org/jira/browse/MESOS-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546408#comment-14546408 ] Jie Yu commented on MESOS-2735: --- https://reviews.apache.org/r/34299/ > Change the interaction between the slave and the resource estimator from > polling to pushing > > > Key: MESOS-2735 > URL: https://issues.apache.org/jira/browse/MESOS-2735 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > Labels: twitter > > This will make the semantics more clear. The resource estimator can control > the speed of sending resources estimation to the slave. > To avoid cyclic dependency, slave will register a callback with the resource > estimator and the resource estimator will simply invoke that callback when > there's a new estimation ready. The callback will be a defer to the slave's > main event queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2735) Change the interaction between the slave and the resource estimator from polling to pushing
[ https://issues.apache.org/jira/browse/MESOS-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546193#comment-14546193 ] Jie Yu commented on MESOS-2735: --- 1) This matches the existing allocator interface. 2) Slave no longer needs to configure the polling interval. It's up to the resource estimator to decide when to send estimations. 3) It avoid the potential issue that "estimator::update()" is blocking. Slave will hang if "estimator::update()" blocks. > Change the interaction between the slave and the resource estimator from > polling to pushing > > > Key: MESOS-2735 > URL: https://issues.apache.org/jira/browse/MESOS-2735 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > Labels: twitter > > This will make the semantics more clear. The resource estimator can control > the speed of sending resources estimation to the slave. > To avoid cyclic dependency, slave will register a callback with the resource > estimator and the resource estimator will simply invoke that callback when > there's a new estimation ready. The callback will be a defer to the slave's > main event queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2735) Change the interaction between the slave and the resource estimator from polling to pushing
[ https://issues.apache.org/jira/browse/MESOS-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546167#comment-14546167 ] Niklas Quarfot Nielsen commented on MESOS-2735: --- Can you capture some of the recent discussions here? I wanted to understand how providing a callback to update the most recent value was different from having a callback hanging off the future on the estimator::update() > Change the interaction between the slave and the resource estimator from > polling to pushing > > > Key: MESOS-2735 > URL: https://issues.apache.org/jira/browse/MESOS-2735 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > Labels: twitter > > This will make the semantics more clear. The resource estimator can control > the speed of sending resources estimation to the slave. > To avoid cyclic dependency, slave will register a callback with the resource > estimator and the resource estimator will simply invoke that callback when > there's a new estimation ready. The callback will be a defer to the slave's > main event queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
