bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Maxim Cournoyer writes: > Sorry for the delays -- just wanted to say: I have another set of > patches that will allow continuing to use the guix.d subdirectory, as > well as allowing to reload newly installed packages on the fly. Sounds good. Thank you for improving the Emacs integration in Guix! Best wishes, Arne -- Unpolitisch sein heißt politisch sein ohne es zu merken signature.asc Description: PGP signature
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hello everyone, Arne Babenhauserheide writes: > Ludovic Courtès writes: > >> I agree that a solution needs to be implemented now, it’s not cool to >> leave fellow GNOME users without Emacs for several days. :-) > > That did not leave me without Emacs. It left me without GNOME. > > Priorities! :-) Haha! > (I cannot work without Emacs. It has all my planning and time tracking) > > Best wishes, > Arne Sorry for the delays -- just wanted to say: I have another set of patches that will allow continuing to use the guix.d subdirectory, as well as allowing to reload newly installed packages on the fly. It uses a hybrid of techniques found in the previous and current site-start.el, and uses a custom GUIX_EMACSLOADPATH instead of EMASCLOADPATH for full control of the load path without interference from Emacs default behavior. I'll post the new patches to "guix-patches", after I'll have them rebased on what's in master. A particular thank you to Clément for spending time testing (and pushing) the reworked patches authored by the one who broke their setup ;-). Maxim
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Clément Lassieur writes: >> Clément, perhaps you can push the patches now on behalf of Maxim? >> If Maxim eventually comes up and disagrees, we can always adjust. >> At any rate, it’s better than leaving the thing broken. > > I pushed them, and I'm closing the bug. Thank you Maxim for those > patches, and I hope you'll be back soon to keep hacking on Guix and > Emacs! Thank you! Best wishes, Arne -- Unpolitisch sein heißt politisch sein ohne es zu merken signature.asc Description: PGP signature
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Ludovic Courtès writes: > I agree that a solution needs to be implemented now, it’s not cool to > leave fellow GNOME users without Emacs for several days. :-) That did not leave me without Emacs. It left me without GNOME. Priorities! :-) (I cannot work without Emacs. It has all my planning and time tracking) Best wishes, Arne -- Unpolitisch sein heißt politisch sein ohne es zu merken signature.asc Description: PGP signature
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hi Ludo and Maxim, Ludovic Courtès writes: > Hi! > > clem...@lassieur.org (Clément Lassieur) skribis: > >> Any update about this? Any plan to push a fix or a revert? I've been >> using your new patches without any issue for a few days already. > > I agree that a solution needs to be implemented now, it’s not cool to > leave fellow GNOME users without Emacs for several days. :-) > > Clément, perhaps you can push the patches now on behalf of Maxim? > If Maxim eventually comes up and disagrees, we can always adjust. > At any rate, it’s better than leaving the thing broken. I pushed them, and I'm closing the bug. Thank you Maxim for those patches, and I hope you'll be back soon to keep hacking on Guix and Emacs! Clément
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Clément Lassieur writes: > Any update about this? Any plan to push a fix or a revert? I've been > using your new patches without any issue for a few days already. This would also be important for me. I’m currently forced to use xfce due to this bug, and that hinders me quite a bit, for example because it catches key-combinations I need to pass to the IDE. Best wishes, Arne -- Unpolitisch sein heißt politisch sein ohne es zu merken signature.asc Description: PGP signature
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hello Maxim, Any update about this? Any plan to push a fix or a revert? I've been using your new patches without any issue for a few days already. Thanks, Clément
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hello, Jelle Licht writes: > Maxim Cournoyer writes: > >> [...] >> I've tested these changes with a Gnome VM and the EMACSLOADPATH is now >> reduced to just the Emacs' lisp directory as well as the >> share/emacs/site-lisp directory of any profile. Thanks for the great >> ideas :-). > > Would this still allow Emacs to load packages from Guix' environments as well? Yes, and directly handled by Guix search path machinery, as in: guix environment --pure --ad-hoc emacs emacs-magit -- emacs --> Magit available in that Emacs instance spawned from that environment, given that EMACSLOADPATH has been set correctly. Maxim
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Maxim Cournoyer writes: > I was ready to revert the changes when I saw a reply from Leo Prikler in > this thread. They had really good ideas that I believe fix the > annoyances you reported about the recent changes, while preserving the > new plus points (per profile management of Emacs packages, 'guix > environment --ad-hoc' that works for Emacs, simplified build system). > > Perhaps you could try it out and see if it indeed fixes the problems you > reported? I just tried it, and except a build error with emacs-magit-todos[1], it works very well! Things installed with 'guix package -i' get loaded on Emacs reboot, which is great in my opinion. Thanks again, Clément [1]: --8<---cut here---start->8--- Checking /gnu/store/81a9rjhsw7rh9pc5a121j65107vngyz8-emacs-magit-todos-1.4/share/emacs/site-lisp/... Compiling /gnu/store/81a9rjhsw7rh9pc5a121j65107vngyz8-emacs-magit-todos-1.4/share/emacs/site-lisp/magit-todos-autoloads.el... Compiling /gnu/store/81a9rjhsw7rh9pc5a121j65107vngyz8-emacs-magit-todos-1.4/share/emacs/site-lisp/magit-todos.el... Cannot open load file: No such file or directory, transient command "/gnu/store/2iaibkz1q8xmrqr1pr6ksijw5ifny0fn-emacs-minimal-26.3/bin/emacs" "--quick" "--batch" "--eval=(progn (setq byte-compile-debug t) (byte-recompile-directory (file-name-as-directory \"/gnu/store/81a9rjhsw7rh9pc5a121j65107vngyz8-emacs-magit-todos-1.4/share/emacs/site-lisp\") 0 1))" failed with status 255 builder for `/gnu/store/vadbvc79isvvy0wjyx5i0rbr2gr7xlab-emacs-magit-todos-1.4.drv' failed with exit code 1 build of /gnu/store/vadbvc79isvvy0wjyx5i0rbr2gr7xlab-emacs-magit-todos-1.4.drv failed --8<---cut here---end--->8---
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Maxim Cournoyer writes: > [...] > I've tested these changes with a Gnome VM and the EMACSLOADPATH is now > reduced to just the Emacs' lisp directory as well as the > share/emacs/site-lisp directory of any profile. Thanks for the great > ideas :-). Would this still allow Emacs to load packages from Guix' environments as well?
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hello Clément, clem...@lassieur.org (Clément Lassieur) writes: [...] >> If those affected judge the situation dire enough, I don't mind >> reverting the changes to the Emacs library loading mechanism for the >> time being. > > Please, do so :) > > Lots of users don't have that bug, but there's still a change in their > workflow: they have to restart their session after installing new Emacs > packages. Maybe when that bug is fixed and this set of patch is > re-applied, there will be an opportunity to communicate about this? On > info-guix maybe, or on 'guix pull'. It would explain the pros and cons > of this new way of dealing with Emacs. I don't know if there was such > an announcement already, I didn't see it. WDYT? I was ready to revert the changes when I saw a reply from Leo Prikler in this thread. They had really good ideas that I believe fix the annoyances you reported about the recent changes, while preserving the new plus points (per profile management of Emacs packages, 'guix environment --ad-hoc' that works for Emacs, simplified build system). Perhaps you could try it out and see if it indeed fixes the problems you reported? Thank you, Maxim
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hi Maxim, Maxim Cournoyer writes: > I was ready to revert the changes when I saw a reply from Leo Prikler in > this thread. They had really good ideas that I believe fix the > annoyances you reported about the recent changes, while preserving the > new plus points (per profile management of Emacs packages, 'guix > environment --ad-hoc' that works for Emacs, simplified build system). > > Perhaps you could try it out and see if it indeed fixes the problems you > reported? Sure! I'll try this as soon as possible and get back to you. Thank you! Clément
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hello Leo! Leo Prikler writes: > Hi everyone, > > Am Dienstag, den 26.11.2019, 09:56 +0100 schrieb Ludovic Courtès: >> are we not going overboard with that big a environment variable? :-) > I think I vaguely remember a related discussion about the Emacs build > system adding the guix.d directory, which further worsens this problem > [1]. Putting that aside however, $EMACSLOADPATH should not contain > more than > - $GUIX_PROFILE/share/emacs/$EMACS_VERSION/lisp > - $GUIX_PROFILE/share/emacs/$EMACS_VERSION/site-lisp > - $GUIX_PROFILE/share/emacs/site-lisp > If I read (elisp)Library Search correctly, these directories each > contain a file to add their subdirectories to the load-path variable. > This can be confirmed by searching in the store or through message- > debugging. It appears, however, that these files are not quite > sufficient. While the load-path is indeed modified, no autoloading > occurs for files inside guix.d -- indeed, I doubt it would occur for > any package, regardless of how we name it. That's a precious find! I could validate your findings. The only place we don't have a union of the Elisp directories (with a subdirs.el file) is at build time, but in the event we'd stop producing guix.d the search path would work natively there (as well as causing any newly installed libraries to be found without any rescanning of directories). > After further digging around, this appears to be a bug in guix-emacs. > Rather than using the load-path variable, it uses $EMACSLOADPATH > directly via getenv. I suggest either recursing into subdirectories as > Emacs itself would or using load-path instead of reverse engineering > it, preferring the latter if applicable. > > Now that this has been cleared up, a fix should be in reach. First we > would fix guix-emacs, then we can restrict $EMACSLOADPATH to the above > three -- perhaps two, as the versioned site-lisp appears unused. Neat! I find that this works best when guix.d is removed, as otherwise 1) relying on the load-path would mean we'd have to restart Emacs when installed new libraries under guix.d directories (to have subdirs.el to its magic and add them to the load-path) 2) the emacs-build-system simplifications that were made would need to be reverted because at build time we don't have a profile with subdirs.el readily available, and must manually hunt for the guix.d subdirectories. 3) Even if we scanned directories recursively for autoloads from EMACSLOADPATH ourselves in emacs-guix.el, a user would still need to call the guix-emacs-autoload-packages manually after installing new Elisp packages to have Emacs find them. I've tested these changes with a Gnome VM and the EMACSLOADPATH is now reduced to just the Emacs' lisp directory as well as the share/emacs/site-lisp directory of any profile. Thanks for the great ideas :-). Some packages would need to be adapted to finalize the move to a guix.d-less installation directory (some recipes refer to it), but this is trivial to do. The documentation would need to be adapted as well. I can take care of this if someones deems the attached patches fit to fix the problems mentioned in this ticket. From 141e7e8c45c39fbe2e6cfa879f1dc7b7f721bbfc Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Sat, 23 Nov 2019 12:04:50 +0900 Subject: [PATCH 1/6] build: emacs-build-system: Unify the installation directory. This change aims to reduce the length of the EMACSLOADPATH environment variable, which was found to cause issues such as bug \#38309 (https://bugs.gnu.org/38309). It should also enable discovery of newly installed packages without refreshing the session's EMACSLOADPATH of the user profile (e.g., when launching Emacs from the desktop manager application launcher), as discussed in bug \#38309 (https://bugs.gnu.org/38309). * guix/build/emacs-build-system.scm (%legacy-install-suffix): Rename to... (%install-dir): ...this. (%install-suffix): Remove variable. (build): Adjust installation target directory. (patch-el-files): Likewise. (install): Likewise. (move-doc): Likewise. (make-autoloads): Likewise. --- guix/build/emacs-build-system.scm | 39 +-- 1 file changed, 16 insertions(+), 23 deletions(-) diff --git a/guix/build/emacs-build-system.scm b/guix/build/emacs-build-system.scm index f0c41812f1..e2b792d3dc 100644 --- a/guix/build/emacs-build-system.scm +++ b/guix/build/emacs-build-system.scm @@ -40,11 +40,10 @@ ;; ;; Code: -;; Directory suffix where we install ELPA packages. We avoid ".../elpa" as -;; Emacs expects to find the ELPA repository 'archive-contents' file and the -;; archive signature. -(define %legacy-install-suffix "/share/emacs/site-lisp") -(define %install-suffix (string-append %legacy-install-suffix "/guix.d")) +;;; All the packages are installed directly under site-lisp, which means that +;;; having that directory in the EMACSLOADPATH is enough to have them found by +;;; Emacs. +(define %install-dir "/share/emacs/site-lisp")
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Clément Lassieur writes: (in https://lists.gnu.org/archive/html/bug-guix/2019-11/msg00363.html) > Thanks for taking the time to look into this. I've seen your other > email, you can install libpcre3-dbg to have PCRE's debug symbols. It > might help. I thought you were reproducing with Ubuntu. Maxim Cournoyer writes: > About (2), I was thinking about using grafts -- IIUC this is one use > case where they can be useful (to fix a bug and avoid rebuilding many > packages). This won't fix anything for users of foreign distros.
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hello Ludovic, Ludovic Courtès writes: > Hi Maxim, > > Maxim Cournoyer skribis: > >> I could reproduce the gnome-session crash by generating a Guix VM with >> the attached OS configuration (it has about 100 Emacs packages installed >> to its system profile, which gives an EMACSLOADPATH length of about >> 13000 characters), and got the following backtrace (no debugging >> symbols): >> >> #0 0x77837e27 in match () from >> /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 >> #1 0x778467de in match () from >> /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 >> #2 0x7783ac48 in match () from >> /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 >> [...] >> #17435 0x778467de in match () from >> /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 >> #17436 0x7783ac48 in match () from >> /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 >> #17437 0x778399c6 in match () from >> /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 >> #17438 0x7784a441 in pcre_exec () from >> /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 >> #17439 0x77ceca8b in g_match_info_next () from >> /gnu/store/b8pr2k0i2zd07zmb7kpffmcimqi337if-glib-2.60.6/lib/libglib-2.0.so.0 >> #17440 0x77cee21f in g_regex_match_full () from >> /gnu/store/b8pr2k0i2zd07zmb7kpffmcimqi337if-glib-2.60.6/lib/libglib-2.0.so.0 >> #17441 0x77cee36a in g_regex_match () from >> /gnu/store/b8pr2k0i2zd07zmb7kpffmcimqi337if-glib-2.60.6/lib/libglib-2.0.so.0 >> #17442 0x0042b779 in gsm_util_export_activation_environment () >> #17443 0x0040adde in main () >> >> This tells us that the problem originates from glib. Looking at the >> number of match calls, libpcre is probably blowing up its stack as >> described in this ticket [0]. According to this link, it seems glib >> should be making use of PCRE's facilities to limit the depth of search, >> e.g. by using "match limit" and "recursion limit" as documented here [1] >> (search for "int pcre_exec(" on that page). >> >> Parallel to this, perhaps the regexp used by glib could be rewritten to >> not rely as much on the stack. > > Oh great, thanks for investigating! > > I have a shallow understanding of the issues, but (1) are we not going > overboard with that big a environment variable? :-), and (2) fixing GLib > or PCRE would require a full rebuild, can you think of a way to work > around the issue? About (1); it's definitely bigger than others environment variables we set in Guix (but not that different from PYTHONPATH when it is used on the build side), but it hasn't posed a problem so far outside of glib. I think having a recursive EMACSLOADPATH could be useful here and more convenient for all Emacs users, so probably would be a welcome change in upstream Emacs. That'd reduce the length of our EMACSLOADPATH greatly. I'm also interested in studying if we could use package.el to do the load the autoloads files and put the packages present under a directory in the load-path of Emacs. It seems its variable `package-user-dir' could play a role there. About (2), I was thinking about using grafts -- IIUC this is one use case where they can be useful (to fix a bug and avoid rebuilding many packages). Maxim
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hi everyone, Am Dienstag, den 26.11.2019, 09:56 +0100 schrieb Ludovic Courtès: > are we not going overboard with that big a environment variable? :-) I think I vaguely remember a related discussion about the Emacs build system adding the guix.d directory, which further worsens this problem [1]. Putting that aside however, $EMACSLOADPATH should not contain more than - $GUIX_PROFILE/share/emacs/$EMACS_VERSION/lisp - $GUIX_PROFILE/share/emacs/$EMACS_VERSION/site-lisp - $GUIX_PROFILE/share/emacs/site-lisp If I read (elisp)Library Search correctly, these directories each contain a file to add their subdirectories to the load-path variable. This can be confirmed by searching in the store or through message- debugging. It appears, however, that these files are not quite sufficient. While the load-path is indeed modified, no autoloading occurs for files inside guix.d -- indeed, I doubt it would occur for any package, regardless of how we name it. After further digging around, this appears to be a bug in guix-emacs. Rather than using the load-path variable, it uses $EMACSLOADPATH directly via getenv. I suggest either recursing into subdirectories as Emacs itself would or using load-path instead of reverse engineering it, preferring the latter if applicable. Now that this has been cleared up, a fix should be in reach. First we would fix guix-emacs, then we can restrict $EMACSLOADPATH to the above three -- perhaps two, as the versioned site-lisp appears unused. WDYT? Regards, Leo [1] As I only vaguely remember it, I can not find a source for this. However, the process that I've laid out should work fine with or without guix.d
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hello, clem...@lassieur.org (Clément Lassieur) skribis: > Before the patches, restarting Emacs was enough to have new packages > installed. Now I have to reboot my computer every time I 'guix package > -i emacs-something'. Emacs is central to my workflow and I often change > things around (as do a lot of Guix users). It is inconvenient, really. I wasn’t aware of that, it sounds like a drawback. I wonder if Emacs-Guix knows how to deal with that (until now it was able to directly load Emacs packages you’d install with itself). I think I would lean towards a revert (that is, a single commit reverting the 3 (?) patches that implement this new approach). Then we can discuss the change with less pressure and make sure stakeholders weigh in (they could have done that before but apparently many, myself included, missed that opportunity :-)). Thoughts? Thanks, Ludo’.
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Maxim Cournoyer writes: > Hello Ludovic, > > Ludovic Courtès writes: > >> Hi Maxim, >> >> Maxim Cournoyer skribis: >> >>> There would be a couple more commits to include in the revert to undo >>> the changes (one to the build system, others to adapt the renaming of >>> the emacs-set-load-path phase for some packages: >> >> Oh indeed. >> >> I must say I haven’t looked closely at the changes nor at the reasons >> for the regression, but IIUC, the regression is serious enough that we >> should have a way to address it quickly. > > The regression only seems to affect the "restarting the session", > e.g. logout then login, not the first boot, which means there's an > (inconvenient) workaround available for single user systems. Also, this assumes the users know about the bug. But some (most?) of them won't know about it and will spend a lot of time (as I did) trying to understand why their gnome-session crashes. > I've been trying to reproduce in a VM to get a backtrace (if those > affected by the problem could produce one, that'd help pinpoint the > problematic call to PCRE and its origin), but that'll need some more > time. > > If those affected judge the situation dire enough, I don't mind > reverting the changes to the Emacs library loading mechanism for the > time being. > > Thanks, > > Maxim
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hello Maxim, Thanks for taking the time to look into this. I've seen your other email, you can install libpcre3-dbg to have PCRE's debug symbols. It might help. Maxim Cournoyer writes: > Hello Ludovic, > > Ludovic Courtès writes: > >> Hi Maxim, >> >> Maxim Cournoyer skribis: >> >>> There would be a couple more commits to include in the revert to undo >>> the changes (one to the build system, others to adapt the renaming of >>> the emacs-set-load-path phase for some packages: >> >> Oh indeed. Well, maybe it would make sense to squash them into one revert commit, that would be re-reverted when the bug is fixed? >> I must say I haven’t looked closely at the changes nor at the reasons >> for the regression, but IIUC, the regression is serious enough that we >> should have a way to address it quickly. > > The regression only seems to affect the "restarting the session", > e.g. logout then login, not the first boot, which means there's an > (inconvenient) workaround available for single user systems. Before the patches, restarting Emacs was enough to have new packages installed. Now I have to reboot my computer every time I 'guix package -i emacs-something'. Emacs is central to my workflow and I often change things around (as do a lot of Guix users). It is inconvenient, really. > I've been trying to reproduce in a VM to get a backtrace (if those > affected by the problem could produce one, that'd help pinpoint the > problematic call to PCRE and its origin), but that'll need some more > time. Even if you find a solution, the fix will take a lot of time to land onto an Ubuntu release. > If those affected judge the situation dire enough, I don't mind > reverting the changes to the Emacs library loading mechanism for the > time being. Please, do so :) Lots of users don't have that bug, but there's still a change in their workflow: they have to restart their session after installing new Emacs packages. Maybe when that bug is fixed and this set of patch is re-applied, there will be an opportunity to communicate about this? On info-guix maybe, or on 'guix pull'. It would explain the pros and cons of this new way of dealing with Emacs. I don't know if there was such an announcement already, I didn't see it. WDYT? Thanks again, Clément
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hi Maxim, Maxim Cournoyer skribis: > I could reproduce the gnome-session crash by generating a Guix VM with > the attached OS configuration (it has about 100 Emacs packages installed > to its system profile, which gives an EMACSLOADPATH length of about > 13000 characters), and got the following backtrace (no debugging > symbols): > > #0 0x77837e27 in match () from > /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 > #1 0x778467de in match () from > /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 > #2 0x7783ac48 in match () from > /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 > [...] > #17435 0x778467de in match () from > /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 > #17436 0x7783ac48 in match () from > /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 > #17437 0x778399c6 in match () from > /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 > #17438 0x7784a441 in pcre_exec () from > /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 > #17439 0x77ceca8b in g_match_info_next () from > /gnu/store/b8pr2k0i2zd07zmb7kpffmcimqi337if-glib-2.60.6/lib/libglib-2.0.so.0 > #17440 0x77cee21f in g_regex_match_full () from > /gnu/store/b8pr2k0i2zd07zmb7kpffmcimqi337if-glib-2.60.6/lib/libglib-2.0.so.0 > #17441 0x77cee36a in g_regex_match () from > /gnu/store/b8pr2k0i2zd07zmb7kpffmcimqi337if-glib-2.60.6/lib/libglib-2.0.so.0 > #17442 0x0042b779 in gsm_util_export_activation_environment () > #17443 0x0040adde in main () > > This tells us that the problem originates from glib. Looking at the > number of match calls, libpcre is probably blowing up its stack as > described in this ticket [0]. According to this link, it seems glib > should be making use of PCRE's facilities to limit the depth of search, > e.g. by using "match limit" and "recursion limit" as documented here [1] > (search for "int pcre_exec(" on that page). > > Parallel to this, perhaps the regexp used by glib could be rewritten to > not rely as much on the stack. Oh great, thanks for investigating! I have a shallow understanding of the issues, but (1) are we not going overboard with that big a environment variable? :-), and (2) fixing GLib or PCRE would require a full rebuild, can you think of a way to work around the issue? Thanks, Ludo’.
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hello again, Ludovic Courtès writes: > Hi Maxim, > > Maxim Cournoyer skribis: > >> There would be a couple more commits to include in the revert to undo >> the changes (one to the build system, others to adapt the renaming of >> the emacs-set-load-path phase for some packages: > > Oh indeed. > > I must say I haven’t looked closely at the changes nor at the reasons > for the regression, but IIUC, the regression is serious enough that we > should have a way to address it quickly. I could reproduce the gnome-session crash by generating a Guix VM with the attached OS configuration (it has about 100 Emacs packages installed to its system profile, which gives an EMACSLOADPATH length of about 13000 characters), and got the following backtrace (no debugging symbols): --8<---cut here---start->8--- #0 0x77837e27 in match () from /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 #1 0x778467de in match () from /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 #2 0x7783ac48 in match () from /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 [...] #17435 0x778467de in match () from /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 #17436 0x7783ac48 in match () from /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 #17437 0x778399c6 in match () from /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 #17438 0x7784a441 in pcre_exec () from /gnu/store/5j6w0x3aq0i5r9565w92lrh016vlmv2d-pcre-8.43/lib/libpcre.so.1 #17439 0x77ceca8b in g_match_info_next () from /gnu/store/b8pr2k0i2zd07zmb7kpffmcimqi337if-glib-2.60.6/lib/libglib-2.0.so.0 #17440 0x77cee21f in g_regex_match_full () from /gnu/store/b8pr2k0i2zd07zmb7kpffmcimqi337if-glib-2.60.6/lib/libglib-2.0.so.0 #17441 0x77cee36a in g_regex_match () from /gnu/store/b8pr2k0i2zd07zmb7kpffmcimqi337if-glib-2.60.6/lib/libglib-2.0.so.0 #17442 0x0042b779 in gsm_util_export_activation_environment () #17443 0x0040adde in main () --8<---cut here---end--->8--- This tells us that the problem originates from glib. Looking at the number of match calls, libpcre is probably blowing up its stack as described in this ticket [0]. According to this link, it seems glib should be making use of PCRE's facilities to limit the depth of search, e.g. by using "match limit" and "recursion limit" as documented here [1] (search for "int pcre_exec(" on that page). Parallel to this, perhaps the regexp used by glib could be rewritten to not rely as much on the stack. [0] https://bugs.exim.org/show_bug.cgi?id=2126 [1] https://pcre.org/original/doc/html/pcreapi.html
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hello Ludovic, Ludovic Courtès writes: > Hi Maxim, > > Maxim Cournoyer skribis: > >> There would be a couple more commits to include in the revert to undo >> the changes (one to the build system, others to adapt the renaming of >> the emacs-set-load-path phase for some packages: > > Oh indeed. > > I must say I haven’t looked closely at the changes nor at the reasons > for the regression, but IIUC, the regression is serious enough that we > should have a way to address it quickly. The regression only seems to affect the "restarting the session", e.g. logout then login, not the first boot, which means there's an (inconvenient) workaround available for single user systems. I've been trying to reproduce in a VM to get a backtrace (if those affected by the problem could produce one, that'd help pinpoint the problematic call to PCRE and its origin), but that'll need some more time. If those affected judge the situation dire enough, I don't mind reverting the changes to the Emacs library loading mechanism for the time being. Thanks, Maxim
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hi Maxim, Maxim Cournoyer skribis: > There would be a couple more commits to include in the revert to undo > the changes (one to the build system, others to adapt the renaming of > the emacs-set-load-path phase for some packages: Oh indeed. I must say I haven’t looked closely at the changes nor at the reasons for the regression, but IIUC, the regression is serious enough that we should have a way to address it quickly. WDYT? Thanks, Ludo’.
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hello, Ludovic Courtès writes: > Hi Maxim, > > Maxim Cournoyer skribis: > >> Really sorry about this ugly regression :-/. > > Should we revert 47b3b4c2aa49e21f4cc32c97ff7bbbd069bb849c so we can > address this without pressure in the meantime? > > I think I missed the discussions around this patch, but we should get > Alex Kost into the loop (Alex did the initial work in that area, if I’m > not mistaken.) > > Thanks, > Ludo’. There would be a couple more commits to include in the revert to undo the changes (one to the build system, others to adapt the renaming of the emacs-set-load-path phase for some packages: --8<---cut here---start->8--- 63edbb65e4 * origin/master gnu: emacs-protobuf-mode: Rename the set-emacs-load-path phase. ffb2316548 * gnu: emacs-erlang: Rename the set-emacs-load-path phase. ed94123667 * gnu: emacs-pdf-tools: Adapt phase name. 418febb54f * gnu: emacs-scel: Fix build. 1bb39982f1 * gnu: emacs-realgud: Fix build. c51d4c7746 * gnu: emacs-pdf-tools: Fix build. b44357d02a * gnu: emacs-forge: Fix build. 47b3b4c2aa * gnu: emacs: Adapt the autoloads auxiliary code to use EMACSLOADPATH. e1d31e6457 * build-system: emacs: Simplify the SET-EMACS-LOAD-PATH phase. 215a45d9b8 * gnu: emacs: Locate Elisp libraries via EMACSLOADPATH. --8<---cut here---end--->8--- The bug in questino seems to have to do with a regression in recent versions PCRE, that trigger a crash on large environment variables as reported in [0]. There's a new version of PCRE2 (10.34) available; but I'm unsure if it addresses this particular problem: https://pcre.org/changelog.txt. If it doesn't we should report the problem to them. This could happen with any other environment variable than EMACSLOADPATH; it's just a matter of having an environment variable grow sufficiently long to trigger it. I'm testing a patch that may workaround this problem successfully right now (that would reduce the length of EMACSLOADPATH) I'll send an incomplete patch shortly for discussing the idea or testing. In case it's not clear; I'd rather workaround the issue or have it fixed at its source, ideally. WDYT? Maxim [0] https://bugzilla.redhat.com/show_bug.cgi?id=1430103
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hi Maxim, Maxim Cournoyer skribis: > Really sorry about this ugly regression :-/. Should we revert 47b3b4c2aa49e21f4cc32c97ff7bbbd069bb849c so we can address this without pressure in the meantime? I think I missed the discussions around this patch, but we should get Alex Kost into the loop (Alex did the initial work in that area, if I’m not mistaken.) Thanks, Ludo’.
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hello all, Really sorry about this ugly regression :-/. I'm surprised that a 22000 characters in an environment variable would be an issue though, given that that's probably around 22 KiB of memory and some people tested huge variables (>18 MiB) without facing any hard limit other than RAM [0]. Which leads me to suspect that the problem might be Gnome specific? I haven't experienced it, but then my EMACSLOADPATH variable is only about 7000 characters and I'm not using Gnome (I use ratpoison as a WM). If someone could confirm that they can login using another desktop (XFCE, perhaps?) that'd tell us we need to look more carefully at what gnome-session does and why it crashes on larger than average environment variables. I believe the "enhancement" #1 (to deprecate guix.d subdirectories) in XFCE?) as explained in bug #38273 [1] could lead to a much leaner EMACSLOADPATH, by reducing the number of entries necessary to refer to all the Elisp packages. [0] https://aplawrence.com/Unixart/variable_limits.html [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=38273#34 Maxim
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
Hello, > In case it's not clear: Booting works, but restarting the session > crashes. Not much to add, but I have the same issue and once logged out, I need to restart my machine to login again to GNOME on Ubuntu 18.04. Mathieu
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
"Alex Griffin" writes: > After upgrading my packages today, gnome-session started segfaulting. I think > I finally tracked the problem down to the recent changes in how Emacs searches > for packages. Apparently Emacs now uses the search path > $EMACSLOADPATH. Unfortunately, this is a very long value on my system: > > $ echo $EMACSLOADPATH | wc -c > 22525 > > When I add `unset EMACSLOADPATH` to the end of `~/.profile`, GNOME works > again. Same issue here. Since restarting Gnome Session is not possible anymore, the only way to add an Emacs package is to reboot. In case it's not clear: Booting works, but restarting the session crashes. Log attached, with: --8<---cut here---start->8--- nov. 22 13:02:15 rodion kernel: gnome-session-b[3879]: segfault at 7ffd5d8e4f10 ip 7fa420f1ba6a sp 7ffd5d8e4f00 error 6 in libpcre.so.3.13.3[7fa420f03000+7] --8<---cut here---end--->8--- Guix version: --8<---cut here---start->8--- Generation 14 nov. 21 2019 12:51:33 (current) guix fe9b72c repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: fe9b72c5861fabdf7f37862de393812ff3e423ac --8<---cut here---end--->8--- With an up-to-date Ubuntu 18.04.3 LTS. Cheers, Clément log Description: Binary data
bug#38309: Recent $EMACSLOADPATH changes crash gnome-session
After upgrading my packages today, gnome-session started segfaulting. I think I finally tracked the problem down to the recent changes in how Emacs searches for packages. Apparently Emacs now uses the search path $EMACSLOADPATH. Unfortunately, this is a very long value on my system: $ echo $EMACSLOADPATH | wc -c 22525 When I add `unset EMACSLOADPATH` to the end of `~/.profile`, GNOME works again. -- Alex Griffin