[jira] [Commented] (MESOS-5037) foreachkey behaviour is not expected in multimap
[
https://issues.apache.org/jira/browse/MESOS-5037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884055#comment-16884055
]
haosdent commented on MESOS-5037:
-
[~bmahler] Create at https://issues.apache.org/jira/browse/MESOS-9889
> foreachkey behaviour is not expected in multimap
>
>
> Key: MESOS-5037
> URL: https://issues.apache.org/jira/browse/MESOS-5037
> Project: Mesos
> Issue Type: Bug
> Components: stout
>Reporter: haosdent
>Priority: Major
> Labels: foundations, stout
>
> Currently the {{foreachkey}} implementation is
> {code}
> #define foreachkey(VAR, COL)\
> foreachpair (VAR, __foreach__::ignore, COL)
> {code}
> This works in most structures. But in multimap, one key may map to multi
> values. This means there are multi pairs which have same key. So when call
> {{foreachkey}}, the {{key}} would duplicated when iteration. My idea to solve
> this is we prefer call {{foreach}} on {{(COL).keys()}} if {{keys()}} method
> exists in {{COL}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (MESOS-5037) foreachkey behaviour is not expected in multimap
[
https://issues.apache.org/jira/browse/MESOS-5037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884051#comment-16884051
]
haosdent commented on MESOS-5037:
-
[~bmahler] No problem.
> foreachkey behaviour is not expected in multimap
>
>
> Key: MESOS-5037
> URL: https://issues.apache.org/jira/browse/MESOS-5037
> Project: Mesos
> Issue Type: Bug
> Components: stout
>Reporter: haosdent
>Priority: Major
> Labels: foundations, stout
>
> Currently the {{foreachkey}} implementation is
> {code}
> #define foreachkey(VAR, COL)\
> foreachpair (VAR, __foreach__::ignore, COL)
> {code}
> This works in most structures. But in multimap, one key may map to multi
> values. This means there are multi pairs which have same key. So when call
> {{foreachkey}}, the {{key}} would duplicated when iteration. My idea to solve
> this is we prefer call {{foreach}} on {{(COL).keys()}} if {{keys()}} method
> exists in {{COL}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (MESOS-5037) foreachkey behaviour is not expected in multimap
[
https://issues.apache.org/jira/browse/MESOS-5037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884048#comment-16884048
]
Benjamin Mahler commented on MESOS-5037:
[~haosd...@gmail.com] Can you file a separate ticket for the performance
problem? And we can keep this ticket as a foreachkey issue?
> foreachkey behaviour is not expected in multimap
>
>
> Key: MESOS-5037
> URL: https://issues.apache.org/jira/browse/MESOS-5037
> Project: Mesos
> Issue Type: Bug
> Components: stout
>Reporter: haosdent
>Priority: Major
> Labels: foundations, stout
>
> Currently the {{foreachkey}} implementation is
> {code}
> #define foreachkey(VAR, COL)\
> foreachpair (VAR, __foreach__::ignore, COL)
> {code}
> This works in most structures. But in multimap, one key may map to multi
> values. This means there are multi pairs which have same key. So when call
> {{foreachkey}}, the {{key}} would duplicated when iteration. My idea to solve
> this is we prefer call {{foreach}} on {{(COL).keys()}} if {{keys()}} method
> exists in {{COL}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (MESOS-5037) foreachkey behaviour is not expected in multimap
[
https://issues.apache.org/jira/browse/MESOS-5037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884045#comment-16884045
]
haosdent commented on MESOS-5037:
-
After change
{code}
-foreachkey (FrameworkID frameworkId,
- slaves.unreachableTasks.at(slaveInfo.id())) {
+foreach (FrameworkID frameworkId,
+ slaves.unreachableTasks.at(slaveInfo.id()).keys()) {
{code}
The problem gone.
> foreachkey behaviour is not expected in multimap
>
>
> Key: MESOS-5037
> URL: https://issues.apache.org/jira/browse/MESOS-5037
> Project: Mesos
> Issue Type: Bug
> Components: stout
>Reporter: haosdent
>Priority: Major
> Labels: foundations, stout
>
> Currently the {{foreachkey}} implementation is
> {code}
> #define foreachkey(VAR, COL)\
> foreachpair (VAR, __foreach__::ignore, COL)
> {code}
> This works in most structures. But in multimap, one key may map to multi
> values. This means there are multi pairs which have same key. So when call
> {{foreachkey}}, the {{key}} would duplicated when iteration. My idea to solve
> this is we prefer call {{foreach}} on {{(COL).keys()}} if {{keys()}} method
> exists in {{COL}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (MESOS-5037) foreachkey behaviour is not expected in multimap
[
https://issues.apache.org/jira/browse/MESOS-5037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884044#comment-16884044
]
haosdent commented on MESOS-5037:
-
[~bmahler] Sure, it is
https://github.com/apache/mesos/blob/master/src/master/master.cpp#L7707-L7708
{code}
void Master::__reregisterSlave(
...
foreachkey (FrameworkID frameworkId,
slaves.unreachableTasks.at(slaveInfo.id())) {
...
foreach (TaskID taskId,
slaves.unreachableTasks.at(slaveInfo.id()).get(frameworkId)) {
{code}
Our case is when network flapping, 3~4 agents reregister, then master would CPU
full and could not process any requests during that period.
> foreachkey behaviour is not expected in multimap
>
>
> Key: MESOS-5037
> URL: https://issues.apache.org/jira/browse/MESOS-5037
> Project: Mesos
> Issue Type: Bug
> Components: stout
>Reporter: haosdent
>Priority: Major
> Labels: foundations, stout
>
> Currently the {{foreachkey}} implementation is
> {code}
> #define foreachkey(VAR, COL)\
> foreachpair (VAR, __foreach__::ignore, COL)
> {code}
> This works in most structures. But in multimap, one key may map to multi
> values. This means there are multi pairs which have same key. So when call
> {{foreachkey}}, the {{key}} would duplicated when iteration. My idea to solve
> this is we prefer call {{foreach}} on {{(COL).keys()}} if {{keys()}} method
> exists in {{COL}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (MESOS-5037) foreachkey behaviour is not expected in multimap
[
https://issues.apache.org/jira/browse/MESOS-5037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884011#comment-16884011
]
Benjamin Mahler commented on MESOS-5037:
[~haosd...@gmail.com] can you post a link to the code in question?
> foreachkey behaviour is not expected in multimap
>
>
> Key: MESOS-5037
> URL: https://issues.apache.org/jira/browse/MESOS-5037
> Project: Mesos
> Issue Type: Bug
> Components: stout
>Reporter: haosdent
>Priority: Major
> Labels: foundations, stout
>
> Currently the {{foreachkey}} implementation is
> {code}
> #define foreachkey(VAR, COL)\
> foreachpair (VAR, __foreach__::ignore, COL)
> {code}
> This works in most structures. But in multimap, one key may map to multi
> values. This means there are multi pairs which have same key. So when call
> {{foreachkey}}, the {{key}} would duplicated when iteration. My idea to solve
> this is we prefer call {{foreach}} on {{(COL).keys()}} if {{keys()}} method
> exists in {{COL}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (MESOS-5037) foreachkey behaviour is not expected in multimap
[
https://issues.apache.org/jira/browse/MESOS-5037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16883939#comment-16883939
]
haosdent commented on MESOS-5037:
-
[~bmahler] Sry for the delay. The context is we have an agent which has 1k
tasks, and when the agent reregister, it would trigger 1000,000 rounds on this
loop. I have a flamegraph last week, but it is cleared by incident.
> foreachkey behaviour is not expected in multimap
>
>
> Key: MESOS-5037
> URL: https://issues.apache.org/jira/browse/MESOS-5037
> Project: Mesos
> Issue Type: Bug
> Components: stout
>Reporter: haosdent
>Priority: Major
> Labels: foundations, stout
>
> Currently the {{foreachkey}} implementation is
> {code}
> #define foreachkey(VAR, COL)\
> foreachpair (VAR, __foreach__::ignore, COL)
> {code}
> This works in most structures. But in multimap, one key may map to multi
> values. This means there are multi pairs which have same key. So when call
> {{foreachkey}}, the {{key}} would duplicated when iteration. My idea to solve
> this is we prefer call {{foreach}} on {{(COL).keys()}} if {{keys()}} method
> exists in {{COL}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (MESOS-5037) foreachkey behaviour is not expected in multimap
[
https://issues.apache.org/jira/browse/MESOS-5037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16883340#comment-16883340
]
Benjamin Mahler commented on MESOS-5037:
[~haosd...@gmail.com] foreachkey indeed sounds problematic for multimap.
I didn't follow the CPU load issue you found. Can you file a related ticket
explaining it? Be sure to show the code in question that is inducing the cpu
load, and attach perf data if possible.
> foreachkey behaviour is not expected in multimap
>
>
> Key: MESOS-5037
> URL: https://issues.apache.org/jira/browse/MESOS-5037
> Project: Mesos
> Issue Type: Bug
> Components: stout
>Reporter: haosdent
>Priority: Major
> Labels: foundations, stout
>
> Currently the {{foreachkey}} implementation is
> {code}
> #define foreachkey(VAR, COL)\
> foreachpair (VAR, __foreach__::ignore, COL)
> {code}
> This works in most structures. But in multimap, one key may map to multi
> values. This means there are multi pairs which have same key. So when call
> {{foreachkey}}, the {{key}} would duplicated when iteration. My idea to solve
> this is we prefer call {{foreach}} on {{(COL).keys()}} if {{keys()}} method
> exists in {{COL}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (MESOS-5037) foreachkey behaviour is not expected in multimap
[
https://issues.apache.org/jira/browse/MESOS-5037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16879902#comment-16879902
]
haosdent commented on MESOS-5037:
-
Recently we found a high CPU load case when mesos agent reregister. It is
caused by this bug as well.
{code}
-foreachkey (FrameworkID frameworkId,
- slaves.unreachableTasks.at(slaveInfo.id())) {
+foreach (FrameworkID frameworkId,
+ slaves.unreachableTasks.at(slaveInfo.id()).keys()) {
{code}
> foreachkey behaviour is not expected in multimap
>
>
> Key: MESOS-5037
> URL: https://issues.apache.org/jira/browse/MESOS-5037
> Project: Mesos
> Issue Type: Bug
> Components: stout
>Reporter: haosdent
>Priority: Major
> Labels: stout
>
> Currently the {{foreachkey}} implementation is
> {code}
> #define foreachkey(VAR, COL)\
> foreachpair (VAR, __foreach__::ignore, COL)
> {code}
> This works in most structures. But in multimap, one key may map to multi
> values. This means there are multi pairs which have same key. So when call
> {{foreachkey}}, the {{key}} would duplicated when iteration. My idea to solve
> this is we prefer call {{foreach}} on {{(COL).keys()}} if {{keys()}} method
> exists in {{COL}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
