[
https://issues.apache.org/jira/browse/STORM-636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14293979#comment-14293979
]
ASF GitHub Bot commented on STORM-636:
--------------------------------------
Github user Parth-Brahmbhatt commented on the pull request:
https://github.com/apache/storm/pull/392#issuecomment-71706395
I did not know we only stored 10 errors, makes sense to not go with
pagination if its hardcoded to 10. I will take a look at the complete PR today.
Not to derail the discussion but personally, I would much rather not store
errors in zk at all if its just for rendering the errors in UI. If the
spouts/bolts could just store this in memory with some expiration that should
suffice and we could expose an API at worker layer to get this information
directly from it. If the host dies you lose some errors but that does not seem
like a big deal. The only downside will be ui would now have to make requests
against worker hosts to get erros but that seems ok to me, you would also get
parallelism as all these worker calls can be made in parallel. I haven't
thought this through completely and its probably much more work but I would
love to hear your opinion.
> UI/Monitor is slow for topologies with a large number of components
> -------------------------------------------------------------------
>
> Key: STORM-636
> URL: https://issues.apache.org/jira/browse/STORM-636
> Project: Apache Storm
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Derek Dagit
> Assignee: Derek Dagit
> Priority: Minor
>
> The getTopologyInfo method in nimbus fetches from ZK all errors reported by
> all components. This becomes too slow for topologies with a larger numbers
> of components (bolts/spouts).
> In one example, the UI consistently took over 5 minutes to load the topology
> page for a topology with nearly 500 components while ZK was under load.
> Errors are currently stored in ZooKeeper under individual znodes per
> component. This means that each call to getTopologyInfo needs to list
> children of each znode and then download the error znodes it finds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
