I just found the code.
class JsonBodyAppender {
def append(existing: String, next: String) = existing + "\n" + next
}
There are too many String there, maybe you can just append the message into a
file.
--
Willem Jiang
Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.iteye.com (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On October 14, 2014 at 2:32:00 PM, Willem Jiang ([email protected]) wrote:
> Can you show us how did you do the aggregation? Just some code snippet is OK.
>
> BTW, you can always using write the stream into to file to avoid load the
> whole message
> into memory.
>
> --
> Willem Jiang
>
> Red Hat, Inc.
> Web: http://www.redhat.com
> Blog: http://willemjiang.blogspot.com (English)
> http://jnn.iteye.com (Chinese)
> Twitter: willemjiang
> Weibo: 姜宁willem
>
>
>
> On October 14, 2014 at 5:23:21 AM, Andreas C. Osowski
> ([email protected])
> wrote:
> > Hey there.
> > I'm running into "OOM: Java heap space" exceptions trying to aggregate
> > twitter messages before pushing them to s3. (Xmx/Xms = 1.5g; there's
> > definitely sufficient free memory available when the exceptions happen)
> > I've also tried to use camel-leveldb as the AggregationRepository but
> > without luck. Smaller batch sizes (i.e. 100) work just fine... 5000
> > aggregated messages should also take up just 60mb-ish (assuming an average
> > of 3000chars per message).
> >
> > The relevant code can be found here:
> > https://gist.github.com/th0br0/a1484ea1ad18b8b20c25
> >
> > Could somebody point me to what I'm doing wrong?
> >
> > Thanks a lot!
> >
>
>
