[
https://issues.apache.org/jira/browse/CAMEL-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977868#action_12977868
]
Claus Ibsen commented on CAMEL-3497:
------------------------------------
The issue is the splitter copies the exchange for each splitted message. And
the CompletionService keeps reference to all exchanges, which means we end up
with a lot of Exchange at once in memory which eats up memory.
Will have to come up with some way of discarding not needed exchanges during
processing. Maybe even using something else than the CompletionService if its
the culprint.
I added this to the known issues to Camel 2.5 release notes.
> Splitter Component: Setting 'streaming="true" parallelProcessing="true"'
> consumes large amounts > of heap space for big original messages
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-3497
> URL: https://issues.apache.org/jira/browse/CAMEL-3497
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.5.0
> Reporter: Ralf Steppacher
> Assignee: Claus Ibsen
> Fix For: 2.6.0
>
>
> Setting 'streaming="true" parallelProcessing="true"' consumes large amounts
> of heap space for big original messages. E.g. 1024m of heap is not enough to
> process an 80Mb with 500'000 lines, splitting it line by line.
> The problem seems to be the ArrayList in MulticastProcessor line 224. It
> contains a Future<Exchange> object for every token delivered by the
> java.util.Scanner. The list is only cleared (going out of scope) after all
> Future objects have been completed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
