[
https://issues.apache.org/jira/browse/MESOS-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14345481#comment-14345481
]
Alexander Rukletsov commented on MESOS-2353:
--------------------------------------------
{quote}
Could you be more concrete? Are you implying we copy all of the master's
in-memory data over to this http actor..?
{quote}
Yes, I was thinking about creating snapshots of (possibly all) master
bookkeeping structures for every {{state.json}} request, pass them into a
separate actor and do conversion and aggregation there. In order to prevent OOM
issues we can limit the number of http actor instances and therefore the number
of concurrent requests. However before start working on that, it would be nice
to come up with use cases we consider reasonable and wire up corresponding
scale tests to assess improvements (or degradation). Actually, we'd better do
the same for trivial fixes like adding {{.reserve()}}, but I'll follow up with
a patch later today.
> Improve performance of the master's state.json endpoint for large clusters.
> ---------------------------------------------------------------------------
>
> Key: MESOS-2353
> URL: https://issues.apache.org/jira/browse/MESOS-2353
> Project: Mesos
> Issue Type: Improvement
> Components: master
> Reporter: Benjamin Mahler
> Labels: newbie, twitter
>
> The master's state.json endpoint consistently takes a long time to compute
> the JSON result, for large clusters:
> {noformat}
> $ time curl -s -o /dev/null localhost:5050/master/state.json
> Mon Jan 26 22:38:50 UTC 2015
> real 0m13.174s
> user 0m0.003s
> sys 0m0.022s
> {noformat}
> This can cause the master to get backlogged if there are many state.json
> requests in flight.
> Looking at {{perf}} data, it seems most of the time is spent doing memory
> allocation / de-allocation. This ticket will try to capture any low hanging
> fruit to speed this up. Possibly we can leverage moves if they are not
> already being used by the compiler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
