[
https://issues.apache.org/jira/browse/STORM-636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287803#comment-14287803
]
ASF GitHub Bot commented on STORM-636:
--------------------------------------
Github user d2r commented on the pull request:
https://github.com/apache/storm/pull/392#issuecomment-71059798
For testing this I did the following:
I modified the ExclamationTopology as follows:
- bolts report a new RuntimeException via reportError on each call to
execute
- changed topology.max.error.report.per.interval: 100 (default 10), so each
executor/thread will write up to 100 errors to ZK every 10 seconds. This is a
quick-and-dirty way of increasing write load on ZK for testing.
- Replicated the Exclamation bolt an additional ~450 times, so there are
many more bolts in the topology.
Procedure:
- Launch the topology in Inactive state, wait for it to get assigned and
stable
- go to the topology page and activate the topology (at this point the load
time is on the order of seconds)
- Refresh the topology until I see fresh errors reported for each bolt
- Wait a a minute
- Refresh the page and record load times from the Firefox network inspector.
- Repeat 4 more times
Results:
Before patch (median about a minute):
- 57s
- 1m13s
- 55s
- 51s
- 1m34s
With patch (median about 10s)
- 12s
- 6s
- 8s
- 13s
- 10s
> UI/Monitor is slow for topologies with a large number of components
> -------------------------------------------------------------------
>
> Key: STORM-636
> URL: https://issues.apache.org/jira/browse/STORM-636
> Project: Apache Storm
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Derek Dagit
> Assignee: Derek Dagit
> Priority: Minor
>
> The getTopologyInfo method in nimbus fetches from ZK all errors reported by
> all components. This becomes too slow for topologies with a larger numbers
> of components (bolts/spouts).
> In one example, the UI consistently took over 5 minutes to load the topology
> page for a topology with nearly 500 components while ZK was under load.
> Errors are currently stored in ZooKeeper under individual znodes per
> component. This means that each call to getTopologyInfo needs to list
> children of each znode and then download the error znodes it finds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
