| Message ID | 87mudgb4v5.fsf@gmail.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [bug#38015] Emacs in multiple profiles | expand |
Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > For one, I think restarting my Emacs server (started at login) won't be > enough to discover newly installed packages to my profile. Hmmm, I don't understand why this would be an issue. With updated environment variables, a new instance of Emacs should successfully find everything, shouldn't it? > I'll let some time pass (2 weeks), and if nobody objects, I'll merge it > to master. Sounds great, thanks for this awesome fix!
Hello Pierre, Pierre Neidhardt <mail@ambrevar.xyz> writes: > Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > >> For one, I think restarting my Emacs server (started at login) won't be >> enough to discover newly installed packages to my profile. > > Hmmm, I don't understand why this would be an issue. With updated > environment variables, a new instance of Emacs should successfully find > everything, shouldn't it? Yes, when the instance is really *new*, forked from a session where EMACSLOADPATH is set right. I typically start Emacs in server mode using a shepherd user service, so rarely restart Emacs. And even if I was to do 'herd restart emacs', its variables would still be inherited from the login process that spawned its service. But that's not too big an issue for Emacs when using Emacs-Guix, as one can do `guix-set-emacs-environment' and choose a profile to re-read and set (in Emacs itself) its environment variables. Re-running 'guix-emacs-autoload-packages' would then refresh the autoloads corectly, given that EMACSLOADPATH would have been refreshed with an updated profile. >> I'll let some time pass (2 weeks), and if nobody objects, I'll merge it >> to master. > > Sounds great, thanks for this awesome fix! Thank you for your interested in it, you motivated me to dust off a year old (maybe more) patch and fix the remaining bits. Maxim
Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > But that's not too big an issue for Emacs when using Emacs-Guix, as one > can do `guix-set-emacs-environment' and choose a profile to re-read and > set (in Emacs itself) its environment variables. Re-running > 'guix-emacs-autoload-packages' would then refresh the autoloads > corectly, given that EMACSLOADPATH would have been refreshed with an > updated profile. Damn, didn't know about those 2 functions, it's awesome! I could have used them many times in the past, had I known about them :p
Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > [+ guix-patches] > > Hello! > > Pierre Neidhardt <mail@ambrevar.xyz> writes: > >> I've reviewed and tested the patch: >> >> - review: 5/5 stars :D >> - test: pass! >> >> I've tested the following: >> >> $ /home/ambrevar/projects/guix/pre-inst-env guix package -i emacs-2048-game emacs -p ~/temp/foo >> $ cd ~/temp >> $ env -i $(which bash) --login --noprofile --norc >> bash-5.0$ /run/current-system/profile/bin/env DISPLAY=:1 emacs >> >> >> Content of *Messages*: >> >> Loading /gnu/store/ghdkfqnvrxipp4ikakags5rl7flywb72-profile/share/emacs/site-lisp/guix.d/2048-game-20151026.1233/2048-game-autoloads.el (source)...done >> For information about GNU Emacs and the GNU system, type C-h C-a. >> >> >> And >> >> M-x 2048-game RET >> >> works! > > I'm glad it works and fixes your use case :-). > >> Can we merge this on master? We have some 700 emacs packages that are >> going to be rebuilt, but it's rather light for the build farm. What do >> you think? > > I'm not worried about the 700 Emacs packages that will need to be > rebuild, but I'd prefer to take a bit of time to let people comment on > this non-trivial change. > > The new behavior (of not always loading stuff from the user profile and > system profiles) is different and may break the flow of some people. > > For one, I think restarting my Emacs server (started at login) won't be > enough to discover newly installed packages to my profile. Emacs-Magit > may also be impacted, I haven't reviewed if it uses that autoload > function from site-start.el. > > Refreshing an environment variable value for a process (such as Emacs) > is not as convenient as re-scanning a directory. > > I'll let some time pass (2 weeks), and if nobody objects, I'll merge it > to master. > > Thanks for the review! > > Maxim I'd like to test this, but I can't build Guix, so I can't test it. I don't know when I'll be able to test it, since I don't know when I'll be able to build Guix: "configure: error: Guix requires zlib." https://debbugs.gnu.org/cgi/bugreport.cgi?bug=38175 Is there a specific commit that you can recommend I use when applying the patches? Perhaps one you built successfully yourself? If you could also give me your currently installed Guix version so I can "guix pull" to it, that would be helpful.
My guix describe:
guix 2ffae26
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: 2ffae2689aef8efb325f97ec54abcdeee8588aa0
Note that I'm using substitutes.
Good luck!
Hi Pierre and Maxim, Pierre Neidhardt <mail@ambrevar.xyz> writes: > Good luck! Thank you, Pierre - and Maxim! I was able to apply the 3 patches (using commit 18a69803e2eea4f7555d6eafb6497a645c2d094f, which is on master) cleanly. I built a profile using the following manifest: --8<---------------cut here---------------start------------->8--- (use-modules (gnu packages)) (specifications->manifest '( "emacs" "emacs-bbdb" ;; ... a whole lot of omitted packages ... "emacs-sr-speedbar" "emacs-terraform-mode" )) --8<---------------cut here---------------end--------------->8--- I installed it like this: ./pre-inst-env guix package -m /home/marusich/Documents/guix-manifests/emacs.scm -p ~/.guix-extra-profiles/emacs/emacs I then emptied my default profile and added this to ~/.bash_profile: --8<---------------cut here---------------start------------->8--- GUIX_EXTRA_PROFILES="$HOME/.guix-extra-profiles" for i in $GUIX_EXTRA_PROFILES/*; do profile=$i/$(basename "$i") if [ -f "$profile"/etc/profile ]; then GUIX_PROFILE="$profile" . "$profile"/etc/profile export INFOPATH="$profile/share/info${INFOPATH:+:}$INFOPATH" export MANPATH="$profile/share/man${MANPATH:+:}$MANPATH" fi unset profile done --8<---------------cut here---------------end--------------->8--- I logged out and logged back in. It seems to work so far. I started Emacs from a GNOME terminal bash shell. There were no errors in my Messages buffer after I started Emacs. There were no error messages from the Emacs process, either. I'm enjoying my new Emacs right now! So far, I've noticed three things: - The fonts were different than I remember. Not sure why it happened, but I'm guessing it had to do with the fact that I emptied my default profile, and thus got rid of a lot of fonts accidentally. At this time, I doubt it's because of your changes. - In the GNOME application picker, when I type "emacs" into the search bar, there are no longer any results. Previously, Emacs showed up in the search. This is true for IceCat, too, which I also moved into its own separate profile, so I think it's probably an issue with our GNOME package instead of your changes. - Aspell does not work: "ispell-init-process: Error: No word lists can be found for the language "en_US"." I think this happens because the Guix package for aspell refers to the default profile: (add-after 'install 'wrap-aspell (lambda* (#:key outputs #:allow-other-keys) (let ((bin/aspell (string-append (assoc-ref outputs "out") "/bin/aspell"))) (wrap-program bin/aspell '("ASPELL_CONF" "" = ("${ASPELL_CONF:-\"dict-dir ${GUIX_PROFILE:-$HOME/.guix-profile}/lib/aspell\"}"))) #t))) We should probably follow up on these issues in separate bug reports. Other than the above, it all just works. I don't see any reason not to merge it into master, since even with these issues it is a strict improvement over the current situation in which we cannot use Emacs nearly at all in non-default profiles. If I notice anything else, I'll let you know! Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > Subject: [PATCH 1/3] gnu: emacs: Locate Elisp libraries via EMACSLOADPATH. LGTM! > Subject: [PATCH 2/3] build-system: emacs: Simplify the SET-EMACS-LOAD-PATH > phase. > > +(define* (add-source-to-load-path #:key dummy #:allow-other-keys) > + "Augment the EMACSLOADPATH environment variable with the source directory." > (let* ((source-directory (getcwd)) > - (input-elisp-directories (input-directories->el-directories > - (inputs->directories inputs))) > - (emacs-load-path-value > - (string-join > - (append input-elisp-directories (list source-directory)) > - ":" 'suffix))) > + (emacs-load-path-value (string-append (getenv "EMACSLOADPATH") ":" > + source-directory))) > (setenv "EMACSLOADPATH" emacs-load-path-value) > - (format #t "environment variable `EMACSLOADPATH' set to ~a\n" > - emacs-load-path-value))) > + (format #t "source directory ~s appended to the `EMACSLOADPATH' \ > +environment variable\n" source-directory))) It looks like the change is as follows: previously, we added multiple directories, including the source-directory, to the load path; after this change, we append just the source-directory to the EMACSLOADPATH. Can you remind me why we don't need to add the other elisp directories to the EMACSLOADPATH, too? It seems to work, so I'm probably just missing something obvious. > Subject: [PATCH 3/3] gnu: emacs: Adapt the autoloads auxiliary code to use > EMACSLOADPATH. As far as I can tell, this looks good to me, too. Finally: do you anticipate that this will cause any user-visible or backwards-incompatible changes that we should mention in the Guix channel's news file (and in the Guix NEWS file)?
Hi Chris, Thanks for testing! > - The fonts were different than I remember. Not sure why it happened, > but I'm guessing it had to do with the fact that I emptied my default > profile, and thus got rid of a lot of fonts accidentally. At this > time, I doubt it's because of your changes. Indeed, Emacs will only load fonts from the default profile. Not sure why. But this is unrelated to this change (it also affects current Guix). > - Aspell does not work: "ispell-init-process: Error: No word lists can > be found for the language "en_US"." I think this happens because > the Guix package for aspell refers to the default profile: > > (add-after 'install 'wrap-aspell > (lambda* (#:key outputs #:allow-other-keys) > (let ((bin/aspell (string-append (assoc-ref outputs "out") > "/bin/aspell"))) > (wrap-program bin/aspell > '("ASPELL_CONF" "" = > ("${ASPELL_CONF:-\"dict-dir ${GUIX_PROFILE:-$HOME/.guix-profile}/lib/aspell\"}"))) > #t))) More precisely, it's because of the patch applied to ASPELL. See bug #29686 for a discussion. I have a clue how to patch this properly, just need to get down to it. > Finally: do you anticipate that this will cause any user-visible or > backwards-incompatible changes that we should mention in the Guix > channel's news file (and in the Guix NEWS file)? As far I can see, the only incompatibility would arise if the user explicitly referred to the guix-emacs.el variables that are now gone. This change is cool enough I'm all for a NEWS entry nonetheless!
Hello Chris, Chris Marusich <cmmarusich@gmail.com> writes: [...] I've skipped your previous questions, as I believe they were well answered by Pierre (thank you, Pierre!). >> Subject: [PATCH 2/3] build-system: emacs: Simplify the SET-EMACS-LOAD-PATH >> phase. >> >> +(define* (add-source-to-load-path #:key dummy #:allow-other-keys) >> + "Augment the EMACSLOADPATH environment variable with the source directory." >> (let* ((source-directory (getcwd)) >> - (input-elisp-directories (input-directories->el-directories >> - (inputs->directories inputs))) >> - (emacs-load-path-value >> - (string-join >> - (append input-elisp-directories (list source-directory)) >> - ":" 'suffix))) >> + (emacs-load-path-value (string-append (getenv "EMACSLOADPATH") ":" >> + source-directory))) >> (setenv "EMACSLOADPATH" emacs-load-path-value) >> - (format #t "environment variable `EMACSLOADPATH' set to ~a\n" >> - emacs-load-path-value))) >> + (format #t "source directory ~s appended to the `EMACSLOADPATH' \ >> +environment variable\n" source-directory))) > > It looks like the change is as follows: previously, we added multiple > directories, including the source-directory, to the load path; after > this change, we append just the source-directory to the EMACSLOADPATH. > Can you remind me why we don't need to add the other elisp directories > to the EMACSLOADPATH, too? It seems to work, so I'm probably just > missing something obvious. The magic is in the set-paths phase that is inherited from the gnu-build-system. It uses the search path specification now specified in the Emacs package to set the EMACSLOADPATH environment variable itself. When building Elisp packages, it is useful to add the current source directory to that environment variable, so that 'require' directives are able to find the package's own modules (say, when running its test suite). >> Subject: [PATCH 3/3] gnu: emacs: Adapt the autoloads auxiliary code to use >> EMACSLOADPATH. > > As far as I can tell, this looks good to me, too. > > Finally: do you anticipate that this will cause any user-visible or > backwards-incompatible changes that we should mention in the Guix > channel's news file (and in the Guix NEWS file)? Thanks for the review! I can't anticipate any user-visible changes except the current (incorrect) behavior. This change is now live on master, with commit 47b3b4c2aa49e21f4cc32c97ff7bbbd069bb849c. Thanks for testing :-). Maxim
From 39b31bb30b8ecab67bd6de315b5aa1bb59a3ee20 Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer <maxim.cournoyer@gmail.com> Date: Tue, 29 Oct 2019 22:07:55 -0400 Subject: [PATCH 3/3] gnu: emacs: Adapt the autoloads auxiliary code to use EMACSLOADPATH. The Elisp directories to scan for autoloads are now taken from EMACSLOADPATH instead of from the user profile, environment profile or system profile. Manually adding the Elisp directories to the `load-path' is no longer necessary, as this is covered by Emacs when they are in EMACSLOADPATH. The caching logic is also removed, as this code is not typically run often and the gain is marginal (loading autoloads files is cheap). * gnu/packages/aux-files/emacs/guix-emacs.el (guix-emacs-directory) (guix-emacs-subdirs, guix-emacs-directories): Remove procedures. (guix-emacs-find-autoloads): Filter the directory entries by passing the regexp to `directory-files' directly, which is ten times faster. Remove deduplication. (guix-emacs-autoloads-regexp): Remove the group, which used to filter out the file extension; it no longer works this way due to passing the regexp to the `directory-files' procedure directly, which doesn't care about groups. (guix-emacs-autoload-packages): Update doc. Search package directories from EMACSLOADPATH. Do not populate the load-path. Remove cache. --- gnu/packages/aux-files/emacs/guix-emacs.el | 98 ++++++---------------- 1 file changed, 24 insertions(+), 74 deletions(-) diff --git a/gnu/packages/aux-files/emacs/guix-emacs.el b/gnu/packages/aux-files/emacs/guix-emacs.el index 9a49e8861c..46ee557f20 100644 --- a/gnu/packages/aux-files/emacs/guix-emacs.el +++ b/gnu/packages/aux-files/emacs/guix-emacs.el @@ -2,6 +2,7 @@ ;; Copyright © 2014, 2015, 2016, 2017 Alex Kost <alezost@gmail.com> ;; Copyright © 2017 Kyle Meyer <kyle@kyleam.com> +;; Copyright © 2019 Maxim Cournoyer <maxim.cournoyer@gmail.com> ;; This file is part of GNU Guix. @@ -24,91 +25,40 @@ ;; installed with Guix. ;;; Code: - -(require 'cl-lib) - -(defvar guix-user-profile (expand-file-name "~/.guix-profile")) - -(defvar guix-emacs-autoloads nil - "List of the last loaded Emacs autoloads.") +(require 'seq) (defvar guix-emacs-autoloads-regexp - (rx (group (* any) "-autoloads") - ".el" (zero-or-one "c") string-end) + (rx (* any) "-autoloads.el" (zero-or-one "c") string-end) "Regexp to match Emacs 'autoloads' file.") -(defun guix-emacs-directory (&optional profile) - "Return directory with Emacs packages installed in PROFILE. -If PROFILE is nil, use `guix-user-profile'." - (expand-file-name "share/emacs/site-lisp" - (or profile guix-user-profile))) - (defun guix-emacs-find-autoloads (directory) "Return a list of Emacs 'autoloads' files in DIRECTORY. The files in the list do not have extensions (.el, .elc)." - (cl-remove-duplicates - (delq nil - (mapcar (lambda (file) - (when (string-match guix-emacs-autoloads-regexp file) - (match-string 1 file))) - (directory-files directory 'full-name nil 'no-sort))) - :test #'string=)) - -(defun guix-emacs-subdirs (directory) - "Return list of DIRECTORY subdirectories." - (cl-remove-if (lambda (file) - (or (string-match-p (rx "/." string-end) file) - (string-match-p (rx "/.." string-end) file) - (not (file-directory-p file)))) - (directory-files directory 'full-name nil 'no-sort))) - -(defun guix-emacs-directories (&optional profile) - "Return the list of directories under PROFILE that contain Emacs packages. -This includes both `share/emacs/site-lisp/guix.d/PACKAGE' -sub-directories and `share/emacs/site-lisp' itself. - -If PROFILE is nil, use `guix-user-profile'. -Return nil, if Emacs packages are not installed in PROFILE." - (let ((root-dir (guix-emacs-directory (or profile guix-user-profile)))) - (when (file-directory-p root-dir) - (let* ((pkgs-dir (expand-file-name "guix.d" root-dir)) - (pkgs-dirs (when (file-directory-p pkgs-dir) - (guix-emacs-subdirs pkgs-dir)))) - (cons root-dir pkgs-dirs))))) + ;; `directory-files' doesn't honor group in regexp. + (mapcar #'file-name-sans-extension + (directory-files directory 'full-name guix-emacs-autoloads-regexp))) ;;;###autoload -(defun guix-emacs-autoload-packages (&rest profiles) - "Autoload Emacs packages installed in PROFILES. -If PROFILES are not specified, use a default user and system -profiles. +(defun guix-emacs-autoload-packages () + "Autoload Emacs packages found in EMACSLOADPATH. -'Autoload' means add directories with Emacs packages to -`load-path' and load 'autoloads' files matching +'Autoload' means to load the 'autoloads' files matching `guix-emacs-autoloads-regexp'." - (interactive (list (if (fboundp 'guix-read-package-profile) - (funcall 'guix-read-package-profile) - guix-user-profile))) - (let* ((env (getenv "GUIX_ENVIRONMENT")) - (profiles (or profiles - (append (list "/run/current-system/profile" - guix-user-profile) - (and env (list env)))))) - (dolist (profile profiles) - (let ((dirs (guix-emacs-directories profile))) - (when dirs - (let* ((autoloads (cl-mapcan #'guix-emacs-find-autoloads - dirs)) - (new-autoloads (cl-nset-difference autoloads - guix-emacs-autoloads - :test #'string=))) - (dolist (dir dirs) - (cl-pushnew (directory-file-name dir) - load-path - :test #'string=)) - (dolist (file new-autoloads) - (load file 'noerror)) - (setq guix-emacs-autoloads - (append new-autoloads guix-emacs-autoloads)))))))) + ;; FIXME: The autoloads generated by the emacs-build-system are not byte + ;; compiled. + (interactive) + (let* ((emacs-load-path (getenv "EMACSLOADPATH")) + (emacs-non-core-load-path-directories + ;; Filter out core Elisp directories, which are already autoloaded + ;; by Emacs. + (seq-filter (lambda (dir) + (string-match-p "/share/emacs/site-lisp" dir)) + (split-string emacs-load-path ":"))) + (autoloads (mapcan #'guix-emacs-find-autoloads + emacs-non-core-load-path-directories))) + (mapc (lambda (f) + (load f 'noerror)) + autoloads))) (provide 'guix-emacs) -- 2.23.0