[
https://issues.apache.org/jira/browse/SLING-10108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jörg Hoh updated SLING-10108:
-----------------------------
Description:
The PackageMessageFactory consumes the complete binary stream of a distribution
package into a single byte array [1]; and depending on the size of the package
this can cause severe memory issues with a potential OOM of the JVM. And even
in cases where it's not running into OOM, it can cause major work of the GC to
provide a contingous block of heap at that size.
In logs of existing environments I have seen values up to 390MiB:
{noformat}
org.apache.sling.distribution.journal.impl.publisher.PackageMessageFactory
Creating package binary with id [8702eb49-cf1b-4353-9c3d-67d59f7414f7] for
package [dstrpck-1611876137443-cabe281b-4399-46a1-9943-a8c6358c6da2], length
[393722362]
{noformat}
The logic should be changed, so that the package is not stored within memory,
but rather just streamed (if necessary at all).
[1]
[https://github.com/apache/sling-org-apache-sling-distribution-journal/blob/master/src/main/java/org/apache/sling/distribution/journal/impl/publisher/PackageMessageFactory.java#L96]
was:
The PackageMessageFactory consumes the complete binary stream of a distribution
package into a single byte array [1]; and depending on the size of the package
this can cause severe memory issues with a potential OOM of the JVM. And even
in cases where it's not running into OOM, it typically causes major work of the
GC to provide a contingous block of heap at that size.
In logs of existing environments I have seen values up to 390MiB:
{noformat}
org.apache.sling.distribution.journal.impl.publisher.PackageMessageFactory
Creating package binary with id [8702eb49-cf1b-4353-9c3d-67d59f7414f7] for
package [dstrpck-1611876137443-cabe281b-4399-46a1-9943-a8c6358c6da2], length
[393722362]
{noformat}
The logic should be changed, so that the package is not stored within memory,
but rather just streamed (if necessary at all).
[1]
https://github.com/apache/sling-org-apache-sling-distribution-journal/blob/master/src/main/java/org/apache/sling/distribution/journal/impl/publisher/PackageMessageFactory.java#L96
> PackageMessageFactory can allocate huge amount of memory
> --------------------------------------------------------
>
> Key: SLING-10108
> URL: https://issues.apache.org/jira/browse/SLING-10108
> Project: Sling
> Issue Type: Improvement
> Components: Content Distribution
> Affects Versions: Content Distribution Journal Core 0.1.16
> Reporter: Jörg Hoh
> Priority: Major
>
> The PackageMessageFactory consumes the complete binary stream of a
> distribution package into a single byte array [1]; and depending on the size
> of the package this can cause severe memory issues with a potential OOM of
> the JVM. And even in cases where it's not running into OOM, it can cause
> major work of the GC to provide a contingous block of heap at that size.
> In logs of existing environments I have seen values up to 390MiB:
> {noformat}
> org.apache.sling.distribution.journal.impl.publisher.PackageMessageFactory
> Creating package binary with id [8702eb49-cf1b-4353-9c3d-67d59f7414f7] for
> package [dstrpck-1611876137443-cabe281b-4399-46a1-9943-a8c6358c6da2], length
> [393722362]
> {noformat}
> The logic should be changed, so that the package is not stored within memory,
> but rather just streamed (if necessary at all).
>
> [1]
> [https://github.com/apache/sling-org-apache-sling-distribution-journal/blob/master/src/main/java/org/apache/sling/distribution/journal/impl/publisher/PackageMessageFactory.java#L96]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
