Hello, Here are the memory and cpu stats.
[kxn2@amq-d02 ~]$ free -h total used free shared buff/cache available Mem: 7.6G 745M 267M 422M 6.6G 6.2G Swap: 0B 0B 0B [kxn2@amq-a02 ~]$ free -h total used free shared buff/cache available Mem: 7.6G 1.0G 2.8G 148M 3.8G 6.1G Swap: 0B 0B 0B [kxn2@amq-p02 ~]$ free -h total used free shared buff/cache available Mem: 7.6G 964M 2.6G 414M 4.1G 6.0G Swap: 0B 0B 0B I increased -Xmx from 1G to 2G, on development to match acceptance and production. [kxn2@amq-d02 activemq]$ grep ACTIVEMQ_OPTS_MEMORY= bin/env ACTIVEMQ_OPTS_MEMORY="-Xms64M -Xmx2G" [kxn2@amq-a02 activemq]$ grep ACTIVEMQ_OPTS_MEMORY= bin/env ACTIVEMQ_OPTS_MEMORY="-Xms64M -Xmx2G" [kxn2@amq-p02 activemq]$ grep ACTIVEMQ_OPTS_MEMORY= bin/env ACTIVEMQ_OPTS_MEMORY="-Xms64M -Xmx2G" --- Karl Nordström Systems Administrator Penn State IT | Application Platforms ________________________________ From: Tim Bain <tb...@alumni.duke.edu> Sent: Tuesday, February 8, 2022 8:08 AM To: ActiveMQ Users <users@activemq.apache.org> Subject: Re: GC overhead limit exceeded It's probably also worth considering whether you're giving the JVM (and the machine) enough RAM for the amount of data you're asking it to hold in memory. Tim On Mon, Feb 7, 2022, 7:47 PM Matt Pavlovich <mattr...@gmail.com> wrote: > Hello Karl- > > Those error messages are usually indicative of a garbage collection (GC) > tuning problem at the JVM level. On any busy system running Java, the > default GC settings may not force garbage collection frequently enough in > order to keep memory resources available. > > I suggest you look into garbage collection logging and GC tuning options > for your version of the JDK. > > Thanks, > Matt > > > On Feb 7, 2022, at 2:57 PM, Nordstrom, Karl <k...@psu.edu> wrote: > > > > Hello activemq users, > > > > We are running v5.16.3 on our amq dev nodes and hit these errors today. > Has anybody else encounter these? The heap size on dev is 1G while it's 4G > on acceptance. I wonder if 5.16.3 requires more space than 5.15.12 did? > > > > 2022-02-07 15:22:39,325 | ERROR | JobScheduler: JMS Caught exception in > mainloop | org.apache.activemq.store.kahadb.scheduler.JobSchedulerImpl | > JobScheduler:JMS > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > > > 2022-02-07 15:22:45,761 | ERROR | Checkpoint failed | > org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal > Checkpoint Worker > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > > > > > --- > > > > Karl Nordström > > > > Systems Administrator > > > > Penn State IT | Application Platforms > >