GitHub user gowthamCopilot created a discussion: Memory not being released by
GC after exchange completion — heap keeps growing regardless of component used
### Ask a question
**Description:**
I'm experiencing a persistent memory issue where JVM heap usage keeps climbing
and GC never brings it back to baseline, eventually resulting in
`java.lang.OutOfMemoryError: Java heap space`. This happens regardless of which
source component is used — I've tested with SFTP, timer, and other consumers
and the behavior is the same.
The route is intentionally minimal to isolate the issue. It does nothing except
receive an exchange and log a static string. No `${body}` logging, no Groovy
scripts, no split/aggregate, no heavy transformations. Despite this, memory
grows with every exchange and is never reclaimed by GC.
**Minimal route to reproduce:**
```yaml
- route:
id: route-2ed3
nodePrefixId: route-334
from:
id: from-8c14
description: dev sftp
uri: sftp
parameters:
host: "{{sftp.url}}"
directoryName: "{{sftp.directory}}"
recursive: true
idempotent: true
filterDirectory: ${file:name} != 'Backup'
idempotentKey: ${file:onlyname}_${file:modified}
username: "{{sftp.username}}"
password: "{{sftp.password}}"
move: Backup/${file:name}
moveFailed: Error/${date:now:yyyy}/${date:now:ddMMM}/${file:name}.failed
readLock: changed
steps:
- log:
id: log-1a49
message: sftp file received
- routeConfiguration:
description: Default Exception Handler
errorHandler:
id: errorHandler-844f
defaultErrorHandler:I'm experiencing a persistent memory issue where JVM
heap usage keeps climbing and GC never brings it back to baseline, eventually
resulting in `java.lang.OutOfMemoryError: Java heap space`. This happens
regardless of which source component is used — I've tested with SFTP, timer,
and other consumers and the behavior is the same.
The route is intentionally minimal to isolate the issue. It does nothing except
receive an exchange and log a static string. No `${body}` logging, no Groovy
scripts, no split/aggregate, no heavy transformations. Despite this, memory
grows with every exchange and is never reclaimed by GC.
id: defaultErrorHandler-4e1a
redeliveryPolicy:
id: redeliveryPolicy-1b22
maximumRedeliveries: 3
retriesExhaustedLogLevel: ERROR
retryAttemptedLogLevel: ERROR
```
The same behavior occurs if I replace SFTP with a timer or any other source
component — it is not specific to SFTP.
**Observed behavior:**
- Heap memory keeps climbing with every exchange processed (visible in Grafana)
- GC runs but baseline never returns to the initial level — it keeps rising
- Eventually crashes with `OutOfMemoryError: Java heap space`
- This happens even with the simplest possible route (source + static log)
**Expected behavior:**
- After each exchange completes, all exchange objects (body, headers,
properties) should be dereferenced
- GC should reclaim this memory, returning heap usage to baseline
- Memory graph should show a healthy sawtooth pattern — rise during processing,
drop back to baseline after GC
- A simple route like this should be able to run indefinitely without memory
growth
**Error stack trace:**
```
java.lang.OutOfMemoryError: Java heap space
at java.base/java.util.Arrays.copyOf(Unknown Source)
at
java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source)
at java.base/java.lang.AbstractStringBuilder.append(Unknown Source)
at java.base/java.lang.StringBuilder.append(Unknown Source)
at
ch.qos.logback.core.pattern.FormattingConverter.write(FormattingConverter.java:39)
at
ch.qos.logback.core.pattern.PatternLayoutBase.writeLoopOnConverters(PatternLayoutBase.java:174)
at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:200)
at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:41)
at
ch.qos.logback.core.encoder.LayoutWrappingEncoder.encode(LayoutWrappingEncoder.java:114)
at
ch.qos.logback.core.OutputStreamAppender.writeOut(OutputStreamAppender.java:203)
at
ch.qos.logback.core.OutputStreamAppender.subAppend(OutputStreamAppender.java:257)
at
ch.qos.logback.core.OutputStreamAppender.append(OutputStreamAppender.java:111)
at
ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:85)
at
ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:51)
at ch.qos.logback.classic.Logger.appendLoopOnAppenders(Logger.java:272)
at ch.qos.logback.classic.Logger.callAppenders(Logger.java:259)
at
ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:426)
at ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:386)
at ch.qos.logback.classic.Logger.info(Logger.java:584)
at org.apache.camel.spi.CamelLogger.log(CamelLogger.java:166)
at org.apache.camel.spi.CamelLogger.doLog(CamelLogger.java:103)
at org.apache.camel.processor.LogProcessor.process(LogProcessor.java:81)
```
**What I've already investigated and ruled out:**
1. **`${body}` in log statements** — Removed. All log statements now use only
static strings or header metadata. The issue persists.
2. **Groovy / scripting languages** — None used. No Groovy, JavaScript, or
other script expressions in the route.
3. **Split / Aggregate / Multicast** — None used. Simplest possible route.
4. **Heavy transformations** — None. No marshal, unmarshal, JSLT, or data
transformation.
5. **Connection pool leaks** — Not applicable in this minimal route (no
ActiveMQ, JDBC, etc.).
6. **Large file body in memory** — Even with small files (a few KB), memory
keeps growing.
**What I suspect but need help confirming:**
1. **Karavan runtime overhead** — Is there something in Karavan's runtime layer
(beyond standard Camel) that holds references to completed exchanges or route
metadata?
2. **Default logging configuration** — The stack trace shows the OOM happening
inside logback's `FormattingConverter.write()`. Could there be a default log
pattern in the Karavan-generated project that logs the exchange body
automatically (e.g., Camel tracing enabled, or a logback pattern that includes
`%msg` resolving to the body)?
3. **Camel exchange lifecycle in Karavan** — Is `exchange.done()` and
`UnitOfWork.done()` being called properly to release all references after
exchange completion?
4. **In-memory idempotent repository** — When `idempotent: true` is set, does
the default in-memory repository grow without bound? Is there a way to
configure eviction or switch to a persistent store through Karavan?
5. **Camel internal caching** — Are there any internal caches (type converters,
expression caches, endpoint caches) that grow unbounded in the Karavan runtime?
**Questions for the maintainers:**
1. Is there a recommended JVM / GC configuration for Karavan-deployed routes to
ensure proper memory cleanup?
2. Are there any known memory-related issues in Karavan 4.10.2 or Camel 4.10.2
that could cause this behavior?
3. What is the recommended way to diagnose memory issues in Karavan-deployed
routes? Is there a way to attach profiling tools (jmap, jstat, VisualVM) to the
Karavan runtime?
4. Would a heap dump (`jmap -dump:live,format=b,file=heap.hprof`) help diagnose
this? Happy to provide one.
**Grafana observation:**
- Heap usage pattern: continuously rising baseline, never drops back
- This is NOT a healthy sawtooth pattern — it's a staircase going up
**Additional context:**
- File sizes being processed: (specify)
- Approximate number of exchanges before OOM: (specify)
- Container memory limits (if applicable): (specify)
Any guidance on what might be holding references and preventing GC from
reclaiming memory would be greatly appreciated. Happy to provide heap dumps,
`jmap -histo` output, or any other diagnostics.
Thanks!
### Variant
Web Application
### Container Management (if applicable)
Kubernetes
### Operating System (if applicable)
Linux
### Version
4.10.2
GitHub link: https://github.com/apache/camel-karavan/discussions/1616
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
