To be clear, the critical analyzer will still log a WARN message and a thread dump if start-up exceeds the configured critical-analyzer-timeout, but it shouldn't *interfere* with the start-up process (i.e. it won't halt the JVM even when using the HALT critical-analyzer-policy).
Justin On Wed, Jan 7, 2026 at 3:40 PM Justin Bertram <[email protected]> wrote: > > I can get you standard analyzer logs with threaddump next time this > happens, if you are interested. > > I would definitely be interested in seeing that. > > > Justin > > On Wed, Jan 7, 2026 at 3:21 PM Vilius Šumskas via users < > [email protected]> wrote: > >> It still kicks-in at least here. I don‘t remember the exact version I saw >> it last time, but it was a recent 2.3x or 2.4x version. We have a pretty >> slow NFS shared storage for journal. If we gather something like 2GB of >> dead lettered messages in the journal (tens of thousands of small >> messages), we usually unable to restart the broker without increasing >> default critical analyzer timeout. >> >> >> >> I can get you standard analyzer logs with threaddump next time this >> happens, if you are interested. >> >> >> >> -- >> >> Vilius >> >> >> >> *From:* Justin Bertram <[email protected]> >> *Sent:* Wednesday, January 7, 2026 6:36 PM >> *To:* [email protected] >> *Subject:* Re: Artemis broker startup and pending messages impact >> >> >> >> To clarify, the critical analyzer should not interfere with broker >> start-up. This was an issue in the past, but it was resolved via >> ARTEMIS-3664 [1]. >> >> >> >> >> >> Justin >> >> >> >> [1] https://issues.apache.org/jira/browse/ARTEMIS-3664 >> >> >> >> On Wed, Jan 7, 2026 at 1:48 AM Vilius Šumskas via users < >> [email protected]> wrote: >> >> Hi, >> >> >> >> regarding #5, since you are talking about startup you mean readiness not >> liveness, right? Don’t do it with initialDelaySeconds. Better use startup >> probe with combination of failureThreshold and periodsSeconds of, let’s >> say, 10. This way you can define **maximum** delay startup time instead >> of **fixed** startup time. >> >> >> >> Also, startup probe should not be larger than critical-analyzer-timeout >> Artemis parameter as it comes in play during long startups when large data >> journal is loaded. Maybe even look at other parameters of critical analyzer >> and check if this doesn’t conflict with other Kubernetes probes. >> >> >> >> -- >> >> Vilius >> >> >> >> *From:* Shiv Kumar Dixit <[email protected]> >> *Sent:* Tuesday, January 6, 2026 5:59 PM >> *To:* [email protected] >> *Subject:* RE: Artemis broker startup and pending messages impact >> >> >> >> Thanks Justin for details. >> >> >> >> 1. So basically, broker only loads previously active messages (from >> ../data/journal) in memory on subsequent startups. >> 2. Messages paged to disk are loaded (from ../data/paging) when >> previous queues are empty, and consumers demand more. >> 3. We export queue size (artemis.persistent.size) per queue via >> metrics for monitoring purposes – I assume it is sum of all pending >> messages on a given queue (journal + paging + large-messages). Is this >> correct? >> 4. Is it correct to assume that total messages which are in >> ../data/journal folder and expected to be loaded into memory on subsequent >> startup is restricted by global-max-size param in heap. E.g. if my max >> heap >> is 6 GB and global-max-size is set to 1.2 GB then broker can hold max 1.2 >> GB of messages in memory (loaded via ../data/journal) and rest pending >> messages will be paged. >> 5. When we deploy broker in K8s, it also needs a liveness probe to >> indicate broker has started successfully. It means by this time all >> required messages etc. should be loaded from journal. Is there a guideline >> for the initialDelaySeconds that we should set for liveness probe in K8s? >> If it is too early then broker startup is not done fully and we risk >> restarts. If it is too late, we wait unnecessary for the first check to be >> triggered. >> >> >> >> Best Regards >> >> Shiv >> >> >> >> *From:* Justin Bertram <[email protected]> >> *Sent:* 06 January 2026 08:14 PM >> *To:* [email protected] >> *Subject:* Re: Artemis broker startup and pending messages impact >> >> >> >> >> >> *Unverified Sender: *The sender of this email has not been verified. >> Review the content of the message carefully and verify the identity of the >> sender before acting on this email: replying, opening attachments or >> clicking links. >> >> >> >> > When artemis broker starts up, does it need to first load all >> persistent messages in memory to be ready for normal broker operations? >> >> >> >> Among other things, the broker will parse and load messages persisted to >> the journal. However, it won't parse and load messages persisted to paging. >> Pages aren't parsed until the queue is empty and more messages are needed. >> >> >> >> In general, you can think of the journal as an up-to-date snapshot of >> what is in the heap. It includes address & queue definitions, messages, >> transaction meta-data, duplicate ID caches, etc. >> >> >> >> > if we stop broker with viz. 4 GB of persistent pending messages, does >> it need min 4 GB of free memory at startup to load these messages? >> >> >> >> If you stop the broker with 4GB of persistent pending messages then after >> you restart the broker and reload the messages you'll have that same 4GB of >> persistent pending messages. >> >> >> >> > If it uses heap memory for this work or non-heap memory is used for >> this? >> >> >> >> It uses heap. >> >> >> >> I think I answered #3 in the answer to #1 above. >> >> >> >> > If a message is of 10KB, does broker need exact 10KB to store the >> messages in memory and disk? Or additional space is needed for metadata or >> encoding etc.? >> >> >> >> There's always some additional space required for metadata, etc. >> >> >> >> >> >> Justin >> >> >> >> On Tue, Jan 6, 2026 at 5:09 AM Shiv Kumar Dixit < >> [email protected]> wrote: >> >> We are hosting Artemis broker in Kubernetes using operator-based >> solution. We deploy the broker as statefulset with 2 or 4 replicas. We >> assign for e.g. 6 GB for heap and 9 GB for pod, 1.2 GB (1/5 of max heap) >> for global-max-size. All addresses normally use -1 for max-size-bytes but >> some less frequently used queues are defined with 100KB for max-size-bytes >> to allow early paging. >> >> We need some information or documentation link to address following >> queries. >> >> >> >> 1. When artemis broker starts up, does it need to first load all >> persistent messages in memory to be ready for normal broker operations? >> E.g. if we stop broker with viz. 4 GB of persistent pending messages, does >> it need min 4 GB of free memory at startup to load these messages? >> 2. If it uses heap memory for this work or non-heap memory is used for >> this? >> 3. If we have defined global-max-size of 1.2 GB and address-full-policy >> is PAGE and broker has 4 GB of persistent pending messages, will broker >> first load 1.2 GB of messages in memory from all addresses for faster >> delivery and rest messages will remains on disk? Or first all messages must >> be loaded in memory and then paging starts afterwards by keeping 1.2 GB in >> memory and pushing rest to disk again? >> 4. If a message is of 10KB, does broker need exact 10KB to store the >> messages in memory and disk? Or additional space is needed for metadata or >> encoding etc.? >> >> >> >> >> >> Best Regards, >> >> Shiv >> >> >> >>
