jira-importer opened a new issue, #4009: URL: https://github.com/apache/storm/issues/4009
<p><a href="https://github.com/nathanmarz/storm/issues/540"; class="external-link" target="_blank" rel="nofollow noopener">https://github.com/nathanmarz/storm/issues/540</a></p> <p>Now update topology code can only be done by kill it and re-submit a new one. During the kill and re-submit process some request may delay or fail. It is not so good for online service. So we consider to add topology online update recently.</p> <p>Mission</p> <p>update running topology code gracefully one worker after another without service total interrupted. Just update topology code, not update topology DAG structure including component, stream and task number.</p> <p>Proposal</p> <ul> <li>client use "storm update topology-name new-jar-file" to submit new-jar-file update request</li> <li>nimbus update stormdist dir, link topology-dir to new one</li> <li>nimbus update topology version on zk</li> <li>the supervisors that running this topology update it <ul> <li>check topology version on zk, if it is not the same as local version, a topology update begin</li> <li>each supervisor schedule the topology's worker update at a rand(expect-max-update-time) time point</li> <li>sync-supervisor download the latest code from nimbus</li> <li>sync-process check local worker heartbeat version(to be added), if it is not the same with sync-supervisor downloaded version, kill the worker</li> <li>sync-process restart killed worker</li> <li>new worker heartbeat to zk with version(to be added), it can be displayed on web ui to check update progress.</li> </ul> </li> </ul> <p>This feature is deployed in our production clusters. It's really useful for topologys handling online request waiting for response. Topology jar can be updated without entire service offline.</p> <p>We hope that this feature is useful for others too.</p> --- <details><summary><i>Originally reported by <a title="James Xu" href="https://issues.apache.org/jira/secure/ViewProfile.jspa?name=xumingming";>xumingming</a>, imported from: <a href="https://issues.apache.org/jira/browse/STORM-167"; target="_blank">proposal for storm topology online update</a></i></summary> <i><ul> <li><b>assignee</b>: <a title="Parth Brahmbhatt" href="https://issues.apache.org/jira/secure/ViewProfile.jspa?name=parth.brahmbhatt";>parth.brahmbhatt</a> <li><b>status</b>: Closed <li><b>priority</b>: Minor <li><b>resolution</b>: Abandoned <li><b>resolved</b>: 2024-07-02T10:54:46+01:00 <li><b>imported</b>: 2025-01-24 </ul></i> </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
