Is there any indication of what actually caused the container to be restarted? Are you sure it's related to memory?
Justin Justin On Fri, Jan 23, 2026 at 6:51 AM Shiv Kumar Dixit <[email protected]> wrote: > > We have an Artemis broker setup running as stateful set with 2 replicas in > K8s. Each replica runs with 6000 MB of heap and 9000 MB of container memory > limit. We have some 300K pending messages on a given address and total > persistent size of it is 1.5 GB on ss-0. Due to some unforeseen issue, we > need to purge these messages. Whenever we try to purge the messages using > broker GUI via operations with a flush limit of 10 or 100 or 1000, ss-0 > container gets restarted. While browsing and purging works fine but it has > limited utility as we can't delete 300k messages that way. But using > operations to purge is making ss-0 restart. > > > > We have captured following stat from ss-0 pod before starting purge via > operations. Any suggestion to fix this issue? Is it a known issue? If we need > to review or tweak any memory setting? > > > > Memory utilization from broker GUI JMX --> java.lang --> Memory > > Heap memory usage { "init": 6291456000, "committed": 6291456000, "max": > 6291456000, "used": 4304010768 } > > Non heap memory usage { "init": 7667712, "committed": 134938624, > "max": 1224736768, "used": 130441792 } > > Object name java.lang:type=Memory > > > > Direct memory utilization from broker GUI JMX --> java.nio --> BufferPool > > Count Memory used Name Total capacity Object name > > 0 0 mapped - 'non-volatile memory' 0 > java.nio:name=mapped - 'non-volatile memory',type=BufferPool > > 163 46903209 direct 46903208 > java.nio:name=direct,type=BufferPool > > 26 274632 mapped 274632 > java.nio:name=mapped,type=BufferPool > > > > Container memory capture via k8s top pod command > > > kubectl -n xxx top pod xxxx-ss-0 --containers > > POD NAME CPU(cores) MEMORY(bytes) > > xxxx-ss-0 abcd 2998m 8322Mi > > > > Native memory summary of broker pod > > > kubectl -n xxx exec xxxx-ss-0 -- jcmd 1 VM.native_memory summary > > Native Memory Tracking: > > (Omitting categories weighting less than 1KB) > > Total: reserved=7354797KB, committed=6646793KB > > malloc: 99469KB #456441 > > mmap: reserved=7255328KB, committed=6547324KB > > - Java Heap (reserved=6144000KB, committed=6144000KB) > > (mmap: reserved=6144000KB, committed=6144000KB) > > - Class (reserved=427709KB, committed=9213KB) > > (classes #12081) > > ( instance classes #11254, array classes #827) > > (malloc=1725KB #39470) (at peak) > > (mmap: reserved=425984KB, committed=7488KB) > > ( Metadata: ) > > ( reserved=65536KB, committed=59648KB) > > ( used=59166KB) > > ( waste=482KB =0.81%) > > ( Class space:) > > ( reserved=425984KB, committed=7488KB) > > ( used=7092KB) > > ( waste=396KB =5.29%) > > - Thread (reserved=95220KB, committed=18452KB) > > (thread #172) > > (stack: reserved=94724KB, committed=17956KB) > > (malloc=298KB #1032) (peak=319KB #1075) > > (arena=198KB #340) (peak=757KB #256) > > - Code (reserved=251264KB, committed=48728KB) > > (malloc=3576KB #16397) (peak=3576KB #16398) > > (mmap: reserved=247688KB, committed=45152KB) > > - GC (reserved=282161KB, committed=282161KB) > > (malloc=21193KB #31118) (peak=22768KB #31692) > > (mmap: reserved=260968KB, committed=260968KB) > > - Compiler (reserved=1470KB, committed=1470KB) > > (malloc=1306KB #1820) (peak=1321KB #1814) > > (arena=165KB #5) (peak=45072KB #14) > > - Internal (reserved=1059KB, committed=1059KB) > > (malloc=1023KB #23952) (at peak) > > (mmap: reserved=36KB, committed=36KB) > > - Other (reserved=44711KB, committed=44711KB) > > (malloc=44711KB #204) (peak=45370KB #195) > > - Symbol (reserved=12165KB, committed=12165KB) > > (malloc=10175KB #299383) (peak=10199KB #299791) > > (arena=1990KB #1) (at peak) > > - Native Memory Tracking (reserved=7234KB, committed=7234KB) > > (malloc=102KB #1764) (peak=103KB #1769) > > (tracking overhead=7132KB) > > - Shared class space (reserved=16384KB, committed=12068KB) > > (mmap: reserved=16384KB, committed=12068KB) > > - Arena Chunk (reserved=179KB, committed=179KB) > > (malloc=179KB #411) (peak=47415KB #1421) > > - Module (reserved=275KB, committed=275KB) > > (malloc=275KB #1894) (at peak) > > - Safepoint (reserved=8KB, committed=8KB) > > (mmap: reserved=8KB, committed=8KB) > > - Synchronization (reserved=167KB, committed=167KB) > > (malloc=167KB #1578) (peak=171KB #1588) > > - Serviceability (reserved=1KB, committed=1KB) > > (malloc=1KB #6) (at peak) > > - Metaspace (reserved=65911KB, committed=60023KB) > > (malloc=375KB #355) (at peak) > > (mmap: reserved=65536KB, committed=59648KB) > > - String Deduplication (reserved=4381KB, committed=4381KB) > > (malloc=4381KB #34592) (peak=6551KB #38671) > > - Object Monitors (reserved=497KB, committed=497KB) > > (malloc=497KB #2448) (peak=572KB #2814) > > > > Thanks > > Shiv --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
