Hi, it was first recommendation here. Second, I found some troubling posts about missing messages in newer versions, not sure if they're real problems or not. Third, 2.19 to 2.20 didn't require us to do any other configuration updates, which newer version might (?) require
On Wed, Nov 23, 2022 at 3:44 PM Clebert Suconic <clebert.suco...@gmail.com> wrote: > Why did you move to 2.20 and not the latest ? > > > On Wed, Nov 23, 2022 at 2:37 AM Thomas Wahlberg <thomas.wahlb...@nextup.fi > > > wrote: > > > We have now upgraded one environment to 2.20.0 and notice some difference > > in the logs. It might be pointing to a certain queue that receives over > 100 > > kb sized messages. 2.20.0 is still crashing in similar fashion. There's a > > huge IOWAIT reported from our monitoring tools during the crash/restart > and > > wrapper.log is filled with hundreds, if not thousands, of errors like > this, > > all during 1-2 second time period: > > > > INFO | jvm 2 | 2022/11/23 08:02:12 | WARN > > [org.apache.activemq.artemis.core.server] AMQ222061: Client connection > > failed, clearing up resources for session > > 90a07ca5-6a2e-11ed-9aa1-767acc7119a2 > > INFO | jvm 2 | 2022/11/23 08:02:12 | WARN > > [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up > resources > > for session 90a07ca5-6a2e-11ed-9aa1-767acc7119a2 > > INFO | jvm 2 | 2022/11/23 08:02:12 | WARN > > [org.apache.activemq.artemis.core.server] AMQ222061: Client connection > > failed, clearing up resources for session > > 90ce4367-6a2e-11ed-9aa1-767acc7119a2 > > INFO | jvm 2 | 2022/11/23 08:02:12 | WARN > > [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up > resources > > for session 90ce4367-6a2e-11ed-9aa1-767acc7119a2 > > > > > > > > > > On Tue, Nov 8, 2022 at 2:27 PM Thomas Wahlberg < > thomas.wahlb...@nextup.fi> > > wrote: > > > > > Thanks for your reply. We can indeed update to 2.20.0 and we're moving > > > forward with this now. Newer versions seemed to have new possible > issues > > so > > > we'll try with 2.20.0 for starters. > > > > > > On Tue, Nov 8, 2022 at 12:52 PM Roskvist Anton < > anton.roskv...@volvo.com > > > > > > wrote: > > > > > >> Hello Thomas, > > >> > > >> I might be mistaken but I believe this has been fixed in broker > version > > >> 2.20.0... it might be a good idea to update the broker version > > regardless > > >> since there has been quite a few bug fixes, added features and > > improvements > > >> made since 2.19.0. Be sure to follow the "Versions" documentation if > > >> upgrading: > > >> > > > https://activemq.apache.org/components/artemis/documentation/latest/versions.html > > >> > > >> If that's not an option you could try to set the > > >> "critical-analyzer-policy" to "LOG" instead to keep it from killing > the > > >> broker. This has the drawback that if there are any actual critical > > errors > > >> in the broker it will not self-terminate. Also note that I am just > > another > > >> user of the broker so any advise regarding this should be taken with a > > >> spoonful of "read the docs ". > > >> > > >> Br, > > >> Anton > > >> > > >> -----Original Message----- > > >> From: Thomas Wahlberg <thomas.wahlb...@nextup.fi> > > >> Sent: den 8 november 2022 11:27 > > >> To: users@activemq.apache.org > > >> Subject: Artemis: Artemis crashing and restarting randomly due to IO > > >> timeout > > >> > > >> [You don't often get email from thomas.wahlb...@nextup.fi. Learn why > > >> this is important at https://aka.ms/LearnAboutSenderIdentification ] > > >> > > >> CAUTION: This email originated from outside of the organization. If > > >> suspicious, please report it. > > >> > > >> Hi, > > >> we're using artemis 2.19.0 and we've noticed this is happening > randomly. > > >> Sometimes it happens twice per day, sometimes once a month. It doesn't > > >> seem to be related to any cron jobs or any server load. Sometimes > during > > >> the night, sometimes during the day. We're using Python with Artemis. > > The > > >> server monitoring tools don't report anything special regarding IO or > > >> IOWAIT during the time of the crash. Can you help us in any way? Any > > >> pointers on what to look for? > > >> > > >> Thank you in advance! > > >> > > >> Blow are some details: > > >> Artemis: 2.19.0 > > >> OS: Centos 7 > > >> openjdk version "1.8.0_131" > > >> OpenJDK Runtime Environment (build 1.8.0_131-b11) OpenJDK 64-Bit > Server > > >> VM (build 25.131-b11, mixed mode) > > >> > > >> INFO | jvm 6 | ***************** | WARN > > >> [org.apache.activemq.artemis.utils.critical.CriticalMeasure] > Component > > >> org.apache.activemq.artemis.core.io.buffer.TimedBuffer is expired on > > path 4 > > >> INFO | jvm 6 | ***************** | ERROR > > >> [org.apache.activemq.artemis.core.server] AMQ224079: The process for > the > > >> virtual machine will be killed, as component > > >> org.apache.activemq.artemis.core.io.buffer.TimedBuffer@726de316 is > not > > >> responsive > > >> INFO | jvm 6 | ***************** | WARN > > >> [org.apache.activemq.artemis.core.server] AMQ222199: Thread dump: > > >> > > >> > > > ******************************************************************************* > > >> INFO | jvm 6 | ***************** | Complete Thread dump > > >> INFO | jvm 6 | ***************** | "Reference Handler" Id=2 > > RUNNABLE > > >> INFO | jvm 6 | ***************** | at > > >> java.base@11.0.14 > > >> /java.lang.ref.Reference.waitForReferencePendingList(Native > > >> Method) > > >> INFO | jvm 6 | ***************** | at java.base@11.0.14 > > >> /java.lang.ref.Reference.processPendingReferences(Reference.java:241) > > >> INFO | jvm 6 | ***************** | at java.base@11.0.14 > > >> /java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213) > > >> INFO | jvm 6 | ***************** | > > >> > > >> ... > > >> > > >> INFO | jvm 6 | ***************** | > > >> INFO | jvm 6 | ***************** | > > >> > > >> > > > =============================================================================== > > >> INFO | jvm 6 | ***************** | End Thread dump > > >> INFO | jvm 6 | ***************** | > > >> > > >> > > > ******************************************************************************* > > >> INFO | jvm 6 | ***************** | > > >> ERROR | wrapper | ***************** | JVM exited unexpectedly. > > >> STATUS | wrapper | ***************** | Reloading Wrapper > > configuration... > > >> STATUS | wrapper | ***************** | Launching a JVM... > > >> > > >> -- > > >> > > >> Thomas Wahlberg > > >> Chief Technology Officer > > >> Quriiri / NextUp Oy > > >> Puh. +358407031990 > > >> thomas.wahlb...@quriiri.fi > > >> > > >> This email message (including its attachments) is confidential and may > > >> contain privileged information and is intended solely for the use of > the > > >> individual and/or entity to whom it is addressed. If you are not the > > >> intended recipient of this e-mail you may not disseminate, distribute > or > > >> copy this e-mail (including its attachments), or any part thereof. If > > this > > >> e-mail is received in error, please notify the sender immediately by > > return > > >> e-mail and make sure that this e-mail (including its attachments), and > > all > > >> copies thereof, are immediately deleted from your system. Please > further > > >> note that when you communicate with us via email or visit our website > we > > >> process your personal data. See our privacy policy for more > information > > >> about how we process it: > https://www.volvogroup.com/en-en/privacy.html > > >> > > > > > > > > > -- > > > > > > Thomas Wahlberg > > > Chief Technology Officer > > > Quriiri / NextUp Oy > > > Puh. +358407031990 > > > thomas.wahlb...@quriiri.fi > > > > > > > > > -- > > > > Thomas Wahlberg > > Chief Technology Officer > > Quriiri / NextUp Oy > > Puh. +358407031990 > > thomas.wahlb...@quriiri.fi > > > -- > Clebert Suconic > -- Thomas Wahlberg Chief Technology Officer Quriiri / NextUp Oy Puh. +358407031990 thomas.wahlb...@quriiri.fi
