[ https://issues.apache.org/jira/browse/SLING-10108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276108#comment-17276108 ]
Timothee Maret commented on SLING-10108: ---------------------------------------- The packages are binary-less. When distributing resources one by one, it is not a problem. It can be a problem indeed when doing deep distribution with large tree. Chunked distributed in SLING-10067 will mitigate that issue but we should indeed stream the package to solve it in all cases. Would you provide a patch ? > PackageMessageFactory can allocate huge amount of memory > -------------------------------------------------------- > > Key: SLING-10108 > URL: https://issues.apache.org/jira/browse/SLING-10108 > Project: Sling > Issue Type: Improvement > Components: Content Distribution > Affects Versions: Content Distribution Journal Core 0.1.16 > Reporter: Jörg Hoh > Priority: Major > > The PackageMessageFactory consumes the complete binary stream of a > distribution package into a single byte array [1]; and depending on the size > of the package this can cause severe memory issues with a potential OOM of > the JVM. And even in cases where it's not running into OOM, it can cause > major work of the GC to provide a contingous block of heap at that size. > In logs of existing environments I have seen values up to 390MiB: > {noformat} > org.apache.sling.distribution.journal.impl.publisher.PackageMessageFactory > Creating package binary with id [8702eb49-cf1b-4353-9c3d-67d59f7414f7] for > package [dstrpck-1611876137443-cabe281b-4399-46a1-9943-a8c6358c6da2], length > [393722362] > {noformat} > The logic should be changed, so that the package is not stored within memory, > but rather just streamed (if necessary at all). > > [1] > [https://github.com/apache/sling-org-apache-sling-distribution-journal/blob/master/src/main/java/org/apache/sling/distribution/journal/impl/publisher/PackageMessageFactory.java#L96] -- This message was sent by Atlassian Jira (v8.3.4#803005)