[ https://issues.apache.org/jira/browse/NIFI-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15178317#comment-15178317 ]
ASF subversion and git services commented on NIFI-1563: ------------------------------------------------------- Commit ad32cb8243cc20ce7b04cc1fce0263503322ce5b in nifi's branch refs/heads/NIFI-1563 from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ad32cb8 ] NIFI-1563: Federate requests and merge responses from nodes instead of storing bulletins and stats at NCM > Remove storage of components' stats and bulletins from NCM > ---------------------------------------------------------- > > Key: NIFI-1563 > URL: https://issues.apache.org/jira/browse/NIFI-1563 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework > Reporter: Mark Payne > Assignee: Mark Payne > Fix For: 1.0.0 > > Original Estimate: 168h > Remaining Estimate: 168h > > Currently, each node in a cluster sends a period heartbeat that contains the > stats for all components in the node. This happens every 5 seconds by > default. This results in quite a lot of chatter between the NCM and nodes. It > made sense to take this approach when the clustering concept was designed > because there were no Process Groups, and we had no notion of merging > responses from nodes for a web request. > However, now we should replicate the request to all nodes and then merge the > responses on demand, rather than storing this information on the NCM. This > requires far less bandwidth because we only need to pull the stats for a > particular Process Group and only on demand instead of every 5 seconds. > Additionally, this is laying groundwork for the Zero-Master clustering that > we want to have in place for 1.0.0. > In order to remove the stats from the NCM, we will also need to remove the > bulletins and stats history. These requests will all need to be federated and > the responses merged on-demand. -- This message was sent by Atlassian JIRA (v6.3.4#6332)