Re: A new wip-emacs branch
Hi Leo! On Thu, Apr 08 2021, Leo Prikler wrote: guix-emacs should still be loaded by site-start.el, which also initially loads your autoloads. Now that I've had more of a chance to play with it, you're right about this. I'm not sure what I did earlier, but it loaded properly just now. What changes for "Guix in Emacs modifying Emacs", is that you'll probably have to reload the subdirs.el file before autoloading the packages. Ah, okay. I just played around with this, and it seems like the sequence I now need is: $ guix install emacs-magit # shell command ... $ load subdirs # emacs command t $ guix-emacs-autoload-packages # emacs command (... list of autoload files ...) It also sees like I'm able to require the packages in Emacs after the "load subdirs" step, as well, so in practice it's still only two commands to make new Emacs packages loadable, it's just that the second command has changed. đź‘Ť Obviously, there are exceptions to this, that we can argue on a case by case basis, but to summarize, I don't think hardcoding paths throughout Emacs is a good idea. I think there are two different cases which are more clear-cut, with a significant middle ground that's fuzzy, and using PATH just ignores this distinction entirely. There are program uses that are an implementation detail (e.g. the fact that dired uses ls), and there are programs that a user is directly interacting with (e.g. anything that I run in a shell). My thinking is that ideally the former should use hard-coded paths, and the latter should come from PATH. The tricky ones are things like geiser, or magit, where the Emacs package is a wrapper around a program's functionality. These feel like it's reasonable to go either way, but they are also the types of packages that provide ways to easily change which program they run (i.e. geiser-guile-binary and magit-git-executable for these specific examples), so they could default to a hard-coded store path because a user can easily change them if they want to use a different version. Although, as you mentioned in a previous email, TRAMP may make even those "clear-cut" cases a bit trickier. I'll admit I haven't considered TRAMP much in my thinking. As a more general comment, I feel like Guix's wrappers are often treated as "cheaper" than they are. It makes me sad that using awesome as a window manager means that I have to have LD_LIBRARY_PATH= GI_TYPELIB_PATH= before calls to external programs to stop things from crashing (and working out that I needed that was a pain in itself). I'll admit that this case with Emacs and PATH seems less dangerous than the awesome wrapper, but I'm wary of the unexpected problems that it might cause. Carlo
Re: A new wip-emacs branch
Hi Carlo, Am Donnerstag, den 08.04.2021, 13:17 +1000 schrieb Carlo Zancanaro: > Hi Leo! > > Thanks so much for working to improve Emacs packaging in Guix! I > have a question and a comment about your approach on the wip-emacs > branch. > > On Tue, Apr 06 2021, Leo Prikler wrote: > > Emacs now gets its core lisp path from the wrapper rather than > > the search path and there's a new profile hook adding all > > top-level subdirectories to a subdirs.el, that gets loaded at > > startup. > > This sounds great in terms of Emacs starting in an already > established profile, but one key use case for me is to be able to > install new packages without restarting Emacs. Usually I can do > this in eshell by running > > $ guix install emacs-magit # shell command > ... > $ guix-emacs-autoload-packages # emacs command > ... > > I just tried this in a fresh profile with a Guix built from > wip-emacs, but it didn't seem to work. It's possible that I've > done something wrong (I'm doing it with time-machine, which adds > its own complexities), but are you expecting this to work? It > looks like guix-emacs wasn't loaded, and it wasn't on the load > path, but I haven't had a chance to investigate further than that. guix-emacs should still be loaded by site-start.el, which also initially loads your autoloads. What changes for "Guix in Emacs modifying Emacs", is that you'll probably have to reload the subdirs.el file before autoloading the packages. The following snippet in (normal-top-level) seems to be responsible for setting up the load-path during init: ;; Look in each dir in load-path for a subdirs.el file. If we ;; find one, load it, which will add the appropriate subdirs of ;; that dir into load-path. This needs to be done before setting ;; the locale environment, because the latter might need to load ;; some support files. ;; Look for a leim-list.el file too. Loading it will register ;; available input methods. (let ((tail load-path) (lispdir (expand-file-name "../lisp" data-directory)) dir) (while tail (setq dir (car tail)) (let ((default-directory dir)) (load (expand-file-name "subdirs.el") t t t)) ;; Do not scan standard directories that won't contain a leim- list.el. ;; https://lists.gnu.org/r/emacs-devel/2009-10/msg00502.html ;; (Except the preloaded one in lisp/leim.) (or (string-prefix-p lispdir dir) (let ((default-directory dir)) (load (expand-file-name "leim-list.el") t t t))) ;; We don't use a dolist loop and we put this "setq-cdr" command at ;; the end, because the subdirs.el files may add elements to the end ;; of load-path and we want to take it into account. (setq tail (cdr tail Perhaps we should extract it as a whole/some bits of it into a guix- emacs procedure, that is normally not called? > > Extending PATH in the same wrapper as EMACSLOADPATH seems to be > > a fairly cheap option, however. > > I'm not supportive of this, because extending PATH would also > change the binaries that are available through Emacs' shells, > which I use a lot. This would mean that either (a) the Emacs > packages can shadow what I've explicitly installed in my profile, > potentially leading to me running unexpected versions of programs, > or (b) installing something else in my profile might break > something in Emacs because the version has changed. This isn't > likely to be a major problem for coreutils and gzip, assuming > they're stable enough, but it is a problem in general. In my view > either patching the Emacs libraries (to avoid the conflict) or > propagating inputs (to expose the potential conflict while > building the profile) are better options. I don't think I agree with you here. For one, I'd suffix PATH like EMACSLOADPATH, so (a) will not happen. Recall, that in (b) you're describing the status quo. Yes, you will be able to bork Emacs by installing a malicious version of coreutils into your PATH, but that'd also happen if you did so currently. The only difference, is that you currently also bork it, if you don't have any coreutils at all, which is typically only the case in pure environments or containers. As for Emacs libraries written in Elisp, I'm kinda split. I'm not even sure if they should have a fallback when your environment is borked tbh. Consider Geiser for example. To correctly set up Geiser+Guile, you would not only Guile to be installed, but also GUILE_LOAD_PATH to be set to a meaningful value. This can be done with Guix environments by adding Guile, but doing so without Guile and its search paths from elisp is somewhat difficult. I also enjoy being able to hack with Geiser in Guile 3, even though the package builds against Guile 2.2, without needing to install a different one. Obviously, there are exceptions to this, that we can argue on a case by case basis, but to
Re: A new wip-emacs branch
Hi Leo! Thanks so much for working to improve Emacs packaging in Guix! I have a question and a comment about your approach on the wip-emacs branch. On Tue, Apr 06 2021, Leo Prikler wrote: Emacs now gets its core lisp path from the wrapper rather than the search path and there's a new profile hook adding all top-level subdirectories to a subdirs.el, that gets loaded at startup. This sounds great in terms of Emacs starting in an already established profile, but one key use case for me is to be able to install new packages without restarting Emacs. Usually I can do this in eshell by running $ guix install emacs-magit # shell command ... $ guix-emacs-autoload-packages # emacs command ... I just tried this in a fresh profile with a Guix built from wip-emacs, but it didn't seem to work. It's possible that I've done something wrong (I'm doing it with time-machine, which adds its own complexities), but are you expecting this to work? It looks like guix-emacs wasn't loaded, and it wasn't on the load path, but I haven't had a chance to investigate further than that. Extending PATH in the same wrapper as EMACSLOADPATH seems to be a fairly cheap option, however. I'm not supportive of this, because extending PATH would also change the binaries that are available through Emacs' shells, which I use a lot. This would mean that either (a) the Emacs packages can shadow what I've explicitly installed in my profile, potentially leading to me running unexpected versions of programs, or (b) installing something else in my profile might break something in Emacs because the version has changed. This isn't likely to be a major problem for coreutils and gzip, assuming they're stable enough, but it is a problem in general. In my view either patching the Emacs libraries (to avoid the conflict) or propagating inputs (to expose the potential conflict while building the profile) are better options. Thanks again! Carlo
Re: A new wip-emacs branch
Am Dienstag, den 06.04.2021, 11:06 +0200 schrieb Leo Prikler: > Hello Guix, > > this is a small progress report on wip-emacs. Emacs now gets its > core > lisp path from the wrapper rather than the search path and there's a > new profile hook adding all top-level subdirectories to a subdirs.el, > that gets loaded at startup. Emacs' build system has been rewritten > to > use ELPA-style subdirectories. Packages, that cause build failures > in > themselves or others by not adhering to this practice, have been > adjusted. Small progress report part II, here's the packages, that still contain plain files: /gnu/store/0nrhnq7csbbgh79scc68yy3y1l621hy0-emacs-dvc-0.0.0-1.591/ /gnu/store/cyga8wwz77280xcwk6nma6gsh49k954h-emacs-subdirs/ /gnu/store/gwkma1hhyp7cmfdlp6g00zzg18sk2ql2-emacs-guix-0.5.2-4.8ce6d21/ /gnu/store/gyjz18k9gh0bsv3j122abbcbi2qzgz4z-emacs-shroud-1.105/ /gnu/store/kz1lhfyzb38n0q1jqw3fvnjyylk67z57-emacs-w3m-2018-11-11/ /gnu/store/m6ws0xhs0svb7zcbk733n1ddlcpqd5ip-emacs-rime-1.0.4/ /gnu/store/mw7lghj1vsgcd6gp0vqx8sczgbmynym6-emacs-ddskk-17.1/ /gnu/store/n2zi43s18y4i0z002yj88n2ipr1ms7dv-emacs-julia-snail-1.0.0rc4/ /gnu/store/p71rm7qvscxs8kf68n4vacaf23xwz14q-emacs-haskell-mode-17.2/ /gnu/store/qi7kmb9hh9m11m5vrjasf26ydpjcbb5c-emacs-geiser-gauche-0.0.2/ /gnu/store/xdlhj4r2bw76sdkqbnsg5p0kvczfmjia-emacs-27.2/ /gnu/store/z88282qbjx5nnj9syddid5v3dw3qsvva-emacs-wget-0.5.0/ /gnu/store/zqapdihhi09cdls7zwv1bcannh7xqrjs-mu-1.4.15/ Obviously emacs-27.2 and emacs-subdirs deserve special treatment here, so they don't count. In total, that's 11 packages, which don't (yet) follow the ELPA subdirectory style. Regards, Leo
Re: A new wip-emacs branch
On Tue, Apr 06 2021, Leo Prikler wrote: >> FYI Geiser has recently been slit up into multiple packages with one >> core Geiser package[1]. Should the Geiser package be updated in >> wip-emacs or directly on master? >> >> [1]: https://github.com/melpa/melpa/pull/7514 > That depends. If geiser now uses an unaltered emacs-build-system it > can go either way, but if it still has some special needs to take care > of, updating it on wip-emacs will mean less work overall. Looks like someone already sent a patch to update it[1]. :) [1]: https://yhetil.org/guix-patches/byapr05mb4023349403c379ca3db8ab7dc5...@byapr05mb4023.namprd05.prod.outlook.com
Re: A new wip-emacs branch
Am Dienstag, den 06.04.2021, 20:21 +0200 schrieb Xinglu Chen: > On Tue, Apr 06 2021, Leo Prikler wrote: > > > There are still some packages, that use the old convention, e.g. > > emacs- > > geiser. > > FYI Geiser has recently been slit up into multiple packages with one > core Geiser package[1]. Should the Geiser package be updated in > wip-emacs or directly on master? > > [1]: https://github.com/melpa/melpa/pull/7514 That depends. If geiser now uses an unaltered emacs-build-system it can go either way, but if it still has some special needs to take care of, updating it on wip-emacs will mean less work overall. Regards, Leo
Re: A new wip-emacs branch
On Tue, Apr 06 2021, Leo Prikler wrote: > There are still some packages, that use the old convention, e.g. emacs- > geiser. FYI Geiser has recently been slit up into multiple packages with one core Geiser package[1]. Should the Geiser package be updated in wip-emacs or directly on master? [1]: https://github.com/melpa/melpa/pull/7514
Re: A new wip-emacs branch
Hello Guix, this is a small progress report on wip-emacs. Emacs now gets its core lisp path from the wrapper rather than the search path and there's a new profile hook adding all top-level subdirectories to a subdirs.el, that gets loaded at startup. Emacs' build system has been rewritten to use ELPA-style subdirectories. Packages, that cause build failures in themselves or others by not adhering to this practice, have been adjusted. I have attached a manifest, that builds all packages from emacs-xyz known not to fail on master. If some Emacs-related package is not covered by this manifest, but still breaks, please do report it while those patches still live on wip-emacs, so that they can be fixed in time. There are still some packages, that use the old convention, e.g. emacs- geiser. While those can be fixed as well, it is a low priority. In terms of UX it would also be nice to tackle the issue of coreutils and gzip being required to have core functionality. I'm not sure, whether patching Elisp files is the correct solution here, since Emacs could (via tramp) connect to other machines, where those store paths don't exist and it's not clear (to me) on which machine those commands are executed. Extending PATH in the same wrapper as EMACSLOADPATH seems to be a fairly cheap option, however. Regards, Leo (use-modules (gnu)) (use-package-modules emacs) (packages->manifest (cons emacs (filter identity (hash-map->list (lambda (k v) (and (not (member k ;; blacklist packages, that don't build on master '(emacs-el-patch emacs-org-generate emacs-md4rd emacs-picpocket))) (variable-ref v))) (module-obarray (resolve-interface '(gnu packages emacs-xyz)))
Re: A new wip-emacs branch
Am Sonntag, den 04.04.2021, 11:32 +0200 schrieb Xinglu Chen: > On Sat, Apr 03 2021, Leo Prikler wrote: > > > Your patch LGTM in a vaccum (except that package-version this- > > package > > could be abbreviated to just "version" IIUC), but I went for a > > different fix, since emacsql tried to avoid redundancies by putting > > in > > other redundancies. > > > > I ran a small test with emacs-emacsql-sqlite on > > 2be3daa19d2fd47933c622995d2f2dde714075e6, but please try it for > > yourself and check whether everything works as you expect it to. > > Thanks for fixing the problem, but now I noticed that 'emacs-pdf- > tools' > doesn't build. > > --8<---cut here---start->8--- > starting phase `emacs-add-source-to-load-path' > Backtrace: >7 (primitive-load > "/gnu/store/g7vcnpqrs3p76qm9xrnm1sz65ss…") > In ice-9/eval.scm: >191:35 6 (_ _) > In guix/build/gnu-build-system.scm: > 838:2 5 (gnu-build #:source _ #:outputs _ #:inputs _ #:phases . > #) > In ice-9/boot-9.scm: > 1736:10 4 (with-exception-handler _ _ #:unwind? _ # _) > In srfi/srfi-1.scm: >857:16 3 (every1 # > …) > In guix/build/gnu-build-system.scm: >847:30 2 (_ _) >847:30 1 (_ _) > In ice-9/boot-9.scm: > 1669:16 0 (raise-exception _ #:continuable? _) > > ice-9/boot-9.scm:1669:16: In procedure raise-exception: > Wrong type to apply: #f > note: keeping build directory `/tmp/guix-build-emacs-pdf-tools-0.90- > 1.c510442.drv-0' > builder for `/gnu/store/d992q7cfb1wlfffg15g7bgsa32qjanrm-emacs-pdf- > tools-0.90-1.c510442.drv' failed with exit code 1 > build of /gnu/store/d992q7cfb1wlfffg15g7bgsa32qjanrm-emacs-pdf-tools- > 0.90-1.c510442.drv failed > View build log at > '/var/log/guix/drvs/d9/92q7cfb1wlfffg15g7bgsa32qjanrm-emacs-pdf- > tools-0.90-1.c510442.drv.bz2'. > guix build: error: build of > `/gnu/store/d992q7cfb1wlfffg15g7bgsa32qjanrm-emacs-pdf-tools-0.90- > 1.c510442.drv' failed > --8<---cut here---end--->8--- > > The attached patch fixed the problem for me. > I pushed your patch with some slight cosmetic changes. Thanks!
Re: A new wip-emacs branch
On Sat, Apr 03 2021, Leo Prikler wrote: > Your patch LGTM in a vaccum (except that package-version this-package > could be abbreviated to just "version" IIUC), but I went for a > different fix, since emacsql tried to avoid redundancies by putting in > other redundancies. > > I ran a small test with emacs-emacsql-sqlite on > 2be3daa19d2fd47933c622995d2f2dde714075e6, but please try it for > yourself and check whether everything works as you expect it to. Thanks for fixing the problem, but now I noticed that 'emacs-pdf-tools' doesn't build. --8<---cut here---start->8--- starting phase `emacs-add-source-to-load-path' Backtrace: 7 (primitive-load "/gnu/store/g7vcnpqrs3p76qm9xrnm1sz65ss…") In ice-9/eval.scm: 191:35 6 (_ _) In guix/build/gnu-build-system.scm: 838:2 5 (gnu-build #:source _ #:outputs _ #:inputs _ #:phases . #) In ice-9/boot-9.scm: 1736:10 4 (with-exception-handler _ _ #:unwind? _ # _) In srfi/srfi-1.scm: 857:16 3 (every1 # …) In guix/build/gnu-build-system.scm: 847:30 2 (_ _) 847:30 1 (_ _) In ice-9/boot-9.scm: 1669:16 0 (raise-exception _ #:continuable? _) ice-9/boot-9.scm:1669:16: In procedure raise-exception: Wrong type to apply: #f note: keeping build directory `/tmp/guix-build-emacs-pdf-tools-0.90-1.c510442.drv-0' builder for `/gnu/store/d992q7cfb1wlfffg15g7bgsa32qjanrm-emacs-pdf-tools-0.90-1.c510442.drv' failed with exit code 1 build of /gnu/store/d992q7cfb1wlfffg15g7bgsa32qjanrm-emacs-pdf-tools-0.90-1.c510442.drv failed View build log at '/var/log/guix/drvs/d9/92q7cfb1wlfffg15g7bgsa32qjanrm-emacs-pdf-tools-0.90-1.c510442.drv.bz2'. guix build: error: build of `/gnu/store/d992q7cfb1wlfffg15g7bgsa32qjanrm-emacs-pdf-tools-0.90-1.c510442.drv' failed --8<---cut here---end--->8--- The attached patch fixed the problem for me. >From b49dd5be93bb5ba9630dca8d5bb142f258cb0781 Mon Sep 17 00:00:00 2001 Message-Id: From: Xinglu Chen Date: Sun, 4 Apr 2021 11:28:04 +0200 Subject: [PATCH wip-emacs] gnu: emacs-pdf-tools: Adjust to changes in emacs-build-system. * gnu/packages/emacs-xyz.scm (emacs-pdf-tools)[#:phases]: Rename 'add-source-to-load-path' to 'expand-load-path'. --- gnu/packages/emacs-xyz.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gnu/packages/emacs-xyz.scm b/gnu/packages/emacs-xyz.scm index 0382cfdd17..9dbfce6f37 100644 --- a/gnu/packages/emacs-xyz.scm +++ b/gnu/packages/emacs-xyz.scm @@ -2920,7 +2920,7 @@ during idle time, while Emacs is doing nothing else.") (emacs-substitute-variables "pdf-tools.el" ("pdf-tools-handle-upgrades" '() (add-after 'emacs-patch-variables 'emacs-add-source-to-load-path - (assoc-ref emacs:%standard-phases 'add-source-to-load-path)) + (assoc-ref emacs:%standard-phases 'expand-load-path)) (add-after 'emacs-add-source-to-load-path 'emacs-install (assoc-ref emacs:%standard-phases 'install)) (add-after 'emacs-install 'emacs-build base-commit: 2be3daa19d2fd47933c622995d2f2dde714075e6 -- 2.31.1
Re: A new wip-emacs branch
Am Samstag, den 03.04.2021, 13:57 +0200 schrieb Xinglu Chen: > On Sat, Apr 03 2021, Xinglu Chen wrote: > > > I tried the 'wip-emacs' branch (commit > > b18d605fcb51bcce56a1114f82645658db9f5b14), and I noticed that > > 'emacs-emacsql' was failing to install. The 'make-autoloads' phase > > fails with: > > I was able to fix this with the attached patch. Your patch LGTM in a vaccum (except that package-version this-package could be abbreviated to just "version" IIUC), but I went for a different fix, since emacsql tried to avoid redundancies by putting in other redundancies. I ran a small test with emacs-emacsql-sqlite on 2be3daa19d2fd47933c622995d2f2dde714075e6, but please try it for yourself and check whether everything works as you expect it to. Regards, Leo
Re: A new wip-emacs branch
On Sat, Apr 03 2021, Xinglu Chen wrote: > I tried the 'wip-emacs' branch (commit > b18d605fcb51bcce56a1114f82645658db9f5b14), and I noticed that > 'emacs-emacsql' was failing to install. The 'make-autoloads' phase > fails with: I was able to fix this with the attached patch. >From 59e3acc0bebe97e33c5ea557f40eddf6467cfff7 Mon Sep 17 00:00:00 2001 Message-Id: <59e3acc0bebe97e33c5ea557f40eddf6467cfff7.1617450934.git.pub...@yoctocell.xyz> From: Xinglu Chen Date: Sat, 3 Apr 2021 13:52:05 +0200 Subject: [PATCH wip-emacs] gnu: emacs-emacsql: Adjust to changes in emacs-build-system. * gnu/packages/emacs-xyz.scm (emacs-emacsql)[#:phases]: Update install directory for bytecompiled Emacs Lisp files. --- gnu/packages/emacs-xyz.scm | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/gnu/packages/emacs-xyz.scm b/gnu/packages/emacs-xyz.scm index 0bd7fe8f27..cd0138663f 100644 --- a/gnu/packages/emacs-xyz.scm +++ b/gnu/packages/emacs-xyz.scm @@ -15529,7 +15529,9 @@ object has been freed.") (install-file "sqlite/emacsql-sqlite" (string-append out "/bin")) (for-each (cut install-file <> - (string-append out "/share/emacs/site-lisp")) + (string-append out + "/share/emacs/site-lisp/emacsql-" + ,(package-version this-package))) (find-files "." "\\.elc*$"))) #t) (inputs base-commit: b18d605fcb51bcce56a1114f82645658db9f5b14 -- 2.31.1
Re: A new wip-emacs branch
On Thu, Apr 01 2021, Leo Prikler wrote: Hi, I tried the 'wip-emacs' branch (commit b18d605fcb51bcce56a1114f82645658db9f5b14), and I noticed that 'emacs-emacsql' was failing to install. The 'make-autoloads' phase fails with: --8<---cut here---start->8--- starting phase `make-autoloads' Opening directory: No such file or directory, /gnu/store/1bylpqrx1da96z3zc7s81sx78c89xccs-emacs-emacsql-3.0.0/share/emacs/site-lisp/emacsql-3.0.0 command "/gnu/store/70n63hq48ycj7sch0b6wmr6928hw10ig-emacs-minimal-27.2/bin/emacs" "--quick" "--batch" "--eval=(eval '(let ((backup-inhibited t) (generated-autoload-file \"/gnu/store/1bylpqrx1da96z3zc7s81sx78c89xccs-emacs-emacsql-3.0.0/share/emacs/site-lisp/emacsql-3.0.0/emacsql-autoloads.el\")) (update-directory-autoloads \"/gnu/store/1bylpqrx1da96z3zc7s81sx78c89xccs-emacs-emacsql-3.0.0/share/emacs/site-lisp/emacsql-3.0.0\")) nil)" failed with status 255 builder for `/gnu/store/wfrm2qwyl0kj3ip0rfplbgx6f8nc39vs-emacs-emacsql-3.0.0.drv' failed with exit code 1 build of /gnu/store/wfrm2qwyl0kj3ip0rfplbgx6f8nc39vs-emacs-emacsql-3.0.0.drv failed View build log at '/var/log/guix/drvs/wf/rm2qwyl0kj3ip0rfplbgx6f8nc39vs-emacs-emacsql-3.0.0.drv.bz2'. guix build: error: build of `/gnu/store/wfrm2qwyl0kj3ip0rfplbgx6f8nc39vs-emacs-emacsql-3.0.0.drv' failed --8<---cut here---end--->8--- I can only find '/gnu/store/1bylpqrx1da96z3zc7s81sx78c89xccs-emacs-emacsql-3.0.0/share/emacs/site-lisp/emacs', there is no 'emacsql-3.0.0' directory. Any ideas?
Re: A new wip-emacs branch
Hi Leo, great that you're improving the emacs-on-guix experience! I'm not sure whether this is related, but a while ago, after an update of emacs, I had an issue where emacs would not start in graphical mode (it segfaulted) until I ran `fc-cache -rv`. Kind regards, pinoaffe