** Description changed:
[SRU]
[Impact]
- The use of memory grows as a site conf file enlarges when using augeas.
+ The use of memory grows as a site conf file enlarges when using augeas.
[Test Plan]
- - Copy the attached conf file here to /etc/apache2/sites-available and make
the site available to apache (a2ensite memorybomb).
- - Mime module has to be enabled as well (check it with a2query -m | grep
mime). If not, enabling it with a2enmod mime.
- - Reload apache2 configuration ( systemctl reload apache2 ). If something
wrong here, maybe a non-graphical web browser like lynx is need to be
installed.
- - Open htop in a different terminal
- - Run augtool -I httpd.aug
- - In augtool prompt use: match
/files/etc/apache2/sites-available/*[label()=~regexp('.*\\.conf')]//*[self::directive=~regexp('([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee][Oo][Pp][Tt][Ii][Oo][Nn][Aa][Ll])')]
- - Check in htop that augtool is consuming:
- - Before fixing the issue: near 2238MB
- - After fixing the issue (updated packages needed are augeas-lenses,
augeas-tools ans libaugeas0): near 179 MB
-
- PPA with built package with fix for testing:
ppa:mirespace/sru-lp-1828074-bionic-augeas-large-http-files
+ - Copy the attached conf file here to /etc/apache2/sites-available and make
the site available to apache (a2ensite memorybomb).
+ - Mime module has to be enabled as well (check it with a2query -m | grep
mime). If not, enabling it with a2enmod mime.
+ - Reload apache2 configuration ( systemctl reload apache2 ). If something
wrong here, maybe a non-graphical web browser like lynx is need to be installed.
+ - Open htop in a different terminal
+ - Run augtool -I httpd.aug
+ - In augtool prompt use: match
/files/etc/apache2/sites-available/*[label()=~regexp('.*\\.conf')]//*[self::directive=~regexp('([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee][Oo][Pp][Tt][Ii][Oo][Nn][Aa][Ll])')]
+ - Check in htop that augtool is consuming:
+ - Before fixing the issue: near 2238MB
+ - After fixing the issue (updated packages needed are augeas-lenses,
augeas-tools and libaugeas0): near 179 MB
+
+ PPA with built package with fix for testing: ppa:mirespace/sru-
+ lp-1828074-bionic-augeas-large-http-files
[Where problems could occur]
- This patch is cherry picked from upstream, so the bug is corrected in
+ This patch is cherry picked from upstream, so the bug is corrected in
following augeas package versions.
[Other Info]
- Fixed upstream in version 1.11.0, commit id :
+ Fixed upstream in version 1.11.0, commit id :
bbf31f719db54916993be9042254f6d77b61cb13
- Commit description: Replace pure function invocations in path
+ Commit description: Replace pure function invocations in path
expressions with their result.
- In path expressions, we generally need to evaluate functions against
- every node that we consider for the result set. For example, in the path
- expression /files/etc/hosts/*[ipaddr =~ regexp('127\\.')], the regexp
- function was evaluated against every entry in /etc/hosts. Evaluating that
- function requires the construction and compilation of a new regexp.
+ In path expressions, we generally need to evaluate functions against
+ every node that we consider for the result set. For example, in the path
+ expression /files/etc/hosts/*[ipaddr =~ regexp('127\\.')], the regexp
+ function was evaluated against every entry in /etc/hosts. Evaluating that
+ function requires the construction and compilation of a new regexp.
- Because of how memory is managed during evaluation of path expressions, the
memory used by all these copies of the same regexp is only freed after we are
done evaluating the path expression. This causes unacceptable memory usage in
large files (see hercules-team#569).
-
- To avoid these issues, we now distinguish between pure and impure functions
in the path expression interpreter. When we encounter a pure function, we
change the AST for the path expression so that the function invocation is
replaced with the result of invoking the function. With the example above, that
means we only construct and compile the regexp '127\\.' once, regardless of how
many nodes it gets checked against. That leads to a dramatic reduction in the
memory required to evaluate path expressions with such constructs against large
files.
-
- Fixes hercules-team#569
+ Because of how memory is managed during evaluation of path expressions,
+ the memory used by all these copies of the same regexp is only freed
+ after we are done evaluating the path expression. This causes
+ unacceptable memory usage in large files (see hercules-team#569).
- Bug on upstream: https://github.com/hercules-team/augeas/issues/569
+ To avoid these issues, we now distinguish between pure and impure
+ functions in the path expression interpreter. When we encounter a pure
+ function, we change the AST for the path expression so that the function
+ invocation is replaced with the result of invoking the function. With
+ the example above, that means we only construct and compile the regexp
+ '127\\.' once, regardless of how many nodes it gets checked against.
+ That leads to a dramatic reduction in the memory required to evaluate
+ path expressions with such constructs against large files.
+
+ Fixes hercules-team#569
+
+ Bug on upstream: https://github.com/hercules-team/augeas/issues/569
[Original Report]
This bug in libaugeas was fixed in version 1.11.0, but is not in the 18.04.2
repo.
See: https://github.com/hercules-team/augeas/issues/569
I found it using certbot and started a bug report there.
See: https://github.com/certbot/certbot/issues/7035
Which led me here to see if the newer version could be back-ported.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1828074
Title:
Memory usage issues with large Apache httpd configuration files
To manage notifications about this bug go to:
https://bugs.launchpad.net/augeas/+bug/1828074/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
