GitHub user mxm opened a pull request:

    https://github.com/apache/flink/pull/2499

    [FLINK-4485] close and remove user class loader after job completion

    Keeping the user class loader around after job completion may lead to
    excessive temp space usage because all user jars are kept until the
    class loader is garbage collected. Tests showed that garbage collection
    can be delayed for a long time after the class loader is not referenced
    anymore. Note that for the class loader to not be referenced anymore,
    its job has to be removed from the archive.
    
    The fastest way to minimize temp space usage is to close and remove the
    URLClassloader after job completion. This requires us to keep a
    serializable copy of all data which needs the user class loader after
    job completion, e.g. to display data on the web interface.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/mxm/flink FLINK-4485

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/2499.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2499
    
----
commit 6ed17b9f5b9c13c80200ccf3db82bbfe727830bb
Author: Maximilian Michels <m...@apache.org>
Date:   2016-09-15T09:00:58Z

    [FLINK-4485] close and remove user class loader after job completion
    
    Keeping the user class loader around after job completion may lead to
    excessive temp space usage because all user jars are kept until the
    class loader is garbage collected. Tests showed that garbage collection
    can be delayed for a long time after the class loader is not referenced
    anymore. Note that for the class loader to not be referenced anymore,
    its job has to be removed from the archive.
    
    The fastest way to minimize temp space usage is to close and remove the
    URLClassloader after job completion. This requires us to keep a
    serializable copy of all data which needs the user class loader after
    job completion, e.g. to display data on the web interface.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Reply via email to