This is an automated email from the ASF dual-hosted git repository.
reiabreu pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/storm-site.git
The following commit(s) were added to refs/heads/main by this push:
new 496ecf83bc Retro fitting https://github.com/apache/storm/pull/8984
496ecf83bc is described below
commit 496ecf83bc2db64dcfd8c6d88caff5dc73fcce9c
Author: Rui Abreu <[email protected]>
AuthorDate: Sun Aug 23 21:14:45 2026 +0100
Retro fitting https://github.com/apache/storm/pull/8984
---
releases/3.0.0/Cluster-State-Serialization.md | 9 +++++++++
releases/3.0.0/Daemon-Fault-Tolerance.md | 4 +++-
releases/3.0.0/Lifecycle-of-a-topology.md | 1 +
3 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/releases/3.0.0/Cluster-State-Serialization.md
b/releases/3.0.0/Cluster-State-Serialization.md
index b59a0b2f40..8cc404ed07 100644
--- a/releases/3.0.0/Cluster-State-Serialization.md
+++ b/releases/3.0.0/Cluster-State-Serialization.md
@@ -9,6 +9,15 @@ ZooKeeper (and other configured state stores) such as topology
assignments, Nimb
summaries, `StormBase` records, log configs, credentials, worker heartbeats,
profile requests, errors, etc.
+> **Note on worker heartbeats.** Since 2.0
([STORM-2693](https://issues.apache.org/jira/browse/STORM-2693)),
+> worker liveness heartbeats are, by default, *not* persisted in ZooKeeper:
workers
+> write them to local disk, supervisors relay them to Nimbus over Thrift, and
Nimbus
+> keeps them in an in-memory heartbeat cache. Worker heartbeats are only
written to a
+> state store (the `WORKERBEATS_SUBTREE` path) when a heartbeat store such as
Pacemaker
+> is configured. The serialization described below still applies to those
stored
+> heartbeats, and to supervisor liveness (`SupervisorInfo`), which is always
kept as an
+> ephemeral ZooKeeper node.
+
It is distinct from
[tuple serialization](Serialization.html), which covers payloads exchanged
between spouts and bolts at runtime via Kryo.
diff --git a/releases/3.0.0/Daemon-Fault-Tolerance.md
b/releases/3.0.0/Daemon-Fault-Tolerance.md
index 8dce601a8b..b419e16cc5 100644
--- a/releases/3.0.0/Daemon-Fault-Tolerance.md
+++ b/releases/3.0.0/Daemon-Fault-Tolerance.md
@@ -7,7 +7,7 @@ Storm has several different daemon processes. Nimbus that
schedules workers, su
## What happens when a worker dies?
-When a worker dies, the supervisor will restart it. If it continuously fails
on startup and is unable to heartbeat to Nimbus, Nimbus will reschedule the
worker.
+When a worker dies, the supervisor will restart it. Worker liveness reaches
Nimbus indirectly: each worker writes heartbeats to local disk, and its
supervisor relays them to Nimbus over Thrift (this replaced the pre-2.0 model
in which workers heartbeat directly into ZooKeeper; see
[STORM-2693](https://issues.apache.org/jira/browse/STORM-2693)). If a worker
stops heartbeating for longer than `nimbus.task.timeout.secs`, Nimbus
reschedules it. A freshly launched worker is given a longer gra [...]
## What happens when a node dies?
@@ -19,6 +19,8 @@ The Nimbus and Supervisor daemons are designed to be
fail-fast (process self-des
Most notably, no worker processes are affected by the death of Nimbus or the
Supervisors. This is in contrast to Hadoop, where if the JobTracker dies, all
the running jobs are lost.
+Supervisor liveness is tracked differently from worker liveness. Each
supervisor registers itself as an ephemeral ZooKeeper node (its
`SupervisorInfo`, which also carries scheduling metadata such as ports and
resources). When a supervisor dies, its ZooKeeper session expires and the
ephemeral node disappears, so Nimbus detects the loss directly from ZooKeeper
rather than by timing out heartbeats. (This is why there is no active
Nimbus-side supervisor heartbeat-timeout setting.)
+
## Is Nimbus a single point of failure?
If you lose the Nimbus node, the workers will still continue to function.
Additionally, supervisors will continue to restart workers if they die.
However, without Nimbus, workers won't be reassigned to other machines when
necessary (like if you lose a worker machine).
diff --git a/releases/3.0.0/Lifecycle-of-a-topology.md
b/releases/3.0.0/Lifecycle-of-a-topology.md
index fe785f1e4c..83e8991c02 100644
--- a/releases/3.0.0/Lifecycle-of-a-topology.md
+++ b/releases/3.0.0/Lifecycle-of-a-topology.md
@@ -34,6 +34,7 @@ First a couple of important notes about topologies:
- Jars and configs are kept on local filesystem because they're too big
for Zookeeper. The jar and configs are copied into the path {nimbus local
dir}/stormdist/{topology id}
- `setup-storm-static` writes task -> component mapping into ZK
- `setup-heartbeats` creates a ZK "directory" in which tasks can heartbeat
+ - (**Since 2.0, STORM-2693**: workers no longer heartbeat directly
into ZooKeeper. A worker now writes liveness heartbeats to local disk, and its
supervisor relays them to Nimbus over Thrift. See [Daemon Fault
Tolerance](Daemon-Fault-Tolerance.html) for the current mechanism.)
- Nimbus calls `mk-assignment` to assign tasks to machines
[code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L458)
- Assignment record definition is here:
[code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/common.clj#L25)
- Assignment contains:
