| Message ID | 20191022152238.12856-1-dannym@scratchpost.org |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [bug#37868] guix: Allow multiple packages to provide Linux modules in the system profile. | expand |
Hi, any comments about this patch? I don't want to just push this to guix master without any discussion since it establishes an interface that has to keep working for a long time. Rationale of the patch: * Make Linux more modular, allowing the user to specify a union of Guix packages to use as "the kernel" (especially kernel modules). Summary of the patch: * Add a profile hook "linux-module-database" which creates the union of all system packages that have a subdirectory "lib/modules" in their derivation, then invokes depmod on that union and then provides the result in the system profile. * Adapt modprobe to check "lib/modules" inside the system profile, if available. Fall back to "/run/booted-system/kernel/lib/modules" otherwise. For the case where a person has just reconfigured Guix but doesn't want to reboot, modprobe will still work, taking the modules of the old generation (which doesn't necessarily have Linux kernel modules inside the profile yet--because it doesn't necessarily have this patch yet. But maybe it does). * Adapt operating-system-profile to automatically add the Kernel's modules to the system profile (since the system profile would be the only place searched, not doing so would be very bad). * Adapt linux-build-system not to invoke depmod again. Also, its worldview would be incomplete anyway because it wouldn't have the entire system profile. Open questions: * Why doesn't operating-system-profile successfully add linux-libre ? It should. I don't think Guix ever gets there in the first place. (adding linux-libre to operating-system's "packages" field manually does work) * Do we want to have this stuff in the system profile or do we want to have a "kernel profile" instead or something? I don't think the latter would help us much, but if we want it, better do it now. * Do we want to be able to add kernel modules in this fashion without requiring a reboot? If so, that would make the situation a lot more complicated and I don't see a safe way to do that.
Hi Danny, Danny Milosavljevic <dannym@scratchpost.org> writes: [...] > any comments about this patch? I still don't understand the internals of Guix to be able to comment yout patch, anyway... [...] > Rationale of the patch: > > * Make Linux more modular, allowing the user to specify a union of Guix packages > to use as "the kernel" (especially kernel modules). this would be a nice to have feature! > > Summary of the patch: > > * Add a profile hook "linux-module-database" which creates the union of all > system packages that have a subdirectory "lib/modules" in their derivation, > then invokes depmod on that union and then provides the result in the system > profile. > > * Adapt modprobe to check "lib/modules" inside the system profile, if available. > Fall back to "/run/booted-system/kernel/lib/modules" otherwise. > > For the case where a person has just reconfigured Guix but doesn't want to reboot, > modprobe will still work, taking the modules of the old generation (which doesn't > necessarily have Linux kernel modules inside the profile yet--because it doesn't > necessarily have this patch yet. But maybe it does). > > * Adapt operating-system-profile to automatically add the Kernel's modules to > the system profile (since the system profile would be the only place searched, > not doing so would be very bad). > > * Adapt linux-build-system not to invoke depmod again. Also, its worldview > would be incomplete anyway because it wouldn't have the entire system profile. > > Open questions: > > * Why doesn't operating-system-profile successfully add linux-libre ? > It should. I don't think Guix ever gets there in the first place. (adding > linux-libre to operating-system's "packages" field manually does work) > > * Do we want to have this stuff in the system profile or do we want to have > a "kernel profile" instead or something? I don't think the latter would help > us much, but if we want it, better do it now. > > * Do we want to be able to add kernel modules in this fashion without requiring > a reboot? If so, that would make the situation a lot more complicated and I > don't see a safe way to do that.
[sorry for the double posting, I sent my previuos message incomplete] Hi Danny, Danny Milosavljevic <dannym@scratchpost.org> writes: [...] > any comments about this patch? I still don't understand the internals of Guix to be able to comment your patch, but... [...] > Rationale of the patch: > > * Make Linux more modular, allowing the user to specify a union of Guix packages > to use as "the kernel" (especially kernel modules). this would be a nice to have feature! [...] > * Do we want to be able to add kernel modules in this fashion without requiring > a reboot? If so, that would make the situation a lot more complicated and I > don't see a safe way to do that. maybe I'm asking too much... but would it be possible to load and boot into the new (or another) kernel from the currently running kernel without a reboot, via kexec? something like https://wiki.archlinux.org/index.php/Kexec#Manually but with a clean stop/restart of system services? I know it could take some time (and maybe other things to patch) to have this feature, but maybe it is worth thinking of it in connection with this design change Thanks! Gio'
Hi Danny, Danny Milosavljevic <dannym@scratchpost.org> skribis: > any comments about this patch? I commented on an earlier version of this patch at <https://lists.gnu.org/archive/html/guix-devel/2019-10/msg00514.html>. Let me know what you think! > I don't want to just push this to guix master without any discussion since it > establishes an interface that has to keep working for a long time. I agree, thanks for the heads-up. > Open questions: > > * Why doesn't operating-system-profile successfully add linux-libre ? What do you mean? Currently ‘linux-libre’ is not added to the global profile, and I think it’s nicer this way (we’re not clobbering the profile). > * Do we want to be able to add kernel modules in this fashion without requiring > a reboot? If so, that would make the situation a lot more complicated and I > don't see a safe way to do that. If we arrange for those kernel modules to show up in /run/current-system/kernel as I suggested in the message linked above, it should work (assuming the running kernel and the target kernel are the same, of course). Thanks, Ludo’.
Hi Ludo, On Wed, 13 Nov 2019 14:30:56 +0100 Ludovic Courtès <ludo@gnu.org> wrote: > > * Why doesn't operating-system-profile successfully add linux-libre ? > > What do you mean? Currently ‘linux-libre’ is not added to the global > profile, and I think it’s nicer this way (we’re not clobbering the > profile). I've modified it to automatically add linux-libre to the system profile but it doesn't work for some reason. > > * Do we want to be able to add kernel modules in this fashion without requiring > > a reboot? If so, that would make the situation a lot more complicated and I > > don't see a safe way to do that. > > If we arrange for those kernel modules to show up in > /run/current-system/kernel as I suggested in the message linked above, > it should work (assuming the running kernel and the target kernel are > the same, of course). Hmm... I'll read it now :)
Hi Danny, Danny Milosavljevic <dannym@scratchpost.org> wrote: > any comments about this patch? > > I don't want to just push this to guix master without any discussion since it > establishes an interface that has to keep working for a long time. Thanks very much for bringing this to my attention. Generally, it looks good to me, although I agree with the suggestions that Ludovic has made, both here and in the thread on guix-devel: https://lists.gnu.org/archive/html/guix-devel/2019-10/msg00514.html I'm overloaded with other tasks at the moment, so I might not comment on this thread again, but I expect that I'll be happy with whatever you and Ludovic can agree on. Thanks! Mark
Danny Milosavljevic (2): build-system/linux-module: Disable depmod. system: Add kernel-module-packages to operating-system. gnu/system.scm | 26 ++++++-- guix/build/linux-module-build-system.scm | 5 +- guix/profiles.scm | 76 +++++++++++++++++++++++- 3 files changed, 99 insertions(+), 8 deletions(-)
diff --git a/gnu/services.scm b/gnu/services.scm index 6ee05d4580..2a6d2bc464 100644 --- a/gnu/services.scm +++ b/gnu/services.scm @@ -491,7 +491,12 @@ ACTIVATION-SCRIPT-TYPE." (program-file "modprobe" #~(begin (setenv "LINUX_MODULE_DIRECTORY" - "/run/booted-system/kernel/lib/modules") + (if (file-exists? + "/run/booted-system/profile/lib/modules") + "/run/booted-system/profile/lib/modules" + ;; Provides compatibility with previous + ;; Guix generations. + "/run/booted-system/kernel/lib/modules")) (apply execl #$modprobe (cons #$modprobe (cdr (command-line)))))))) diff --git a/gnu/system.scm b/gnu/system.scm index a353b1a5c8..66270b38bb 100644 --- a/gnu/system.scm +++ b/gnu/system.scm @@ -887,12 +887,14 @@ we're running in the final root." (define* (operating-system-profile os) "Return a derivation that builds the system profile of OS." (mlet* %store-monad - ((services -> (operating-system-services os)) + ((kernel -> (operating-system-kernel os)) + (services -> (operating-system-services os)) (profile (fold-services services - #:target-type profile-service-type))) + #:target-type + profile-service-type))) (match profile (("profile" profile) - (return profile))))) + (return (cons kernel profile)))))) ; FIXME: Doesn't work for some reason. I don't think this place is ever reached. (define (operating-system-root-file-system os) "Return the root file system of OS." diff --git a/guix/build/linux-module-build-system.scm b/guix/build/linux-module-build-system.scm index cd76df2de7..e4e6993a49 100644 --- a/guix/build/linux-module-build-system.scm +++ b/guix/build/linux-module-build-system.scm @@ -60,15 +60,14 @@ ;; part. (define* (install #:key inputs native-inputs outputs #:allow-other-keys) (let* ((out (assoc-ref outputs "out")) - (moddir (string-append out "/lib/modules")) - (kmod (assoc-ref (or native-inputs inputs) "kmod"))) + (moddir (string-append out "/lib/modules"))) ;; Install kernel modules (mkdir-p moddir) (invoke "make" "-C" (string-append (assoc-ref inputs "linux-module-builder") "/lib/modules/build") (string-append "M=" (getcwd)) - (string-append "DEPMOD=" kmod "/bin/depmod") + "DEPMOD=true" ; disable depmod. (string-append "MODULE_DIR=" moddir) (string-append "INSTALL_PATH=" out) (string-append "INSTALL_MOD_PATH=" out) diff --git a/guix/profiles.scm b/guix/profiles.scm index cd3b21e390..fd77392588 100644 --- a/guix/profiles.scm +++ b/guix/profiles.scm @@ -9,6 +9,7 @@ ;;; Copyright © 2017 Huang Ying <huang.ying.caritas@gmail.com> ;;; Copyright © 2017 Maxim Cournoyer <maxim.cournoyer@gmail.com> ;;; Copyright © 2019 Kyle Meyer <kyle@kyleam.com> +;;; Copyright © 2019 Danny Milosavljevic <dannym@scratchpost.org> ;;; ;;; This file is part of GNU Guix. ;;; @@ -1125,6 +1126,77 @@ for both major versions of GTK+." (hook . gtk-im-modules))) (return #f))))) +(define (linux-module-database manifest) + (mlet %store-monad + ((kmod (manifest-lookup-package manifest "kmod"))) + (define build + (with-imported-modules '((guix build utils) + (guix build union)) + #~(begin + (use-modules (srfi srfi-1) + (srfi srfi-26) + (guix build utils) + (guix build union) + (ice-9 ftw) + (ice-9 match)) + (let* ((inputs '#$(manifest-inputs manifest)) + (input-files (lambda (path) + (filter file-exists? + (map (cut string-append <> path) inputs)))) + (module-directories (input-files "/lib/modules")) + (System.maps (input-files "/System.map")) + (Module.symverss (input-files "/Module.symvers")) + (directory-entries (lambda (directory-name) + (filter (lambda (basename) + (not (string-prefix? "." + basename))) + (scandir directory-name)))) + ;; Note: Should result in one entry. + (versions (append-map directory-entries module-directories))) + ;; TODO: if len(module-directories) == 1: return module-directories[0] + (mkdir-p (string-append #$output "/lib/modules")) + ;; Iterate over each kernel version directory (usually one). + (for-each (lambda (version) + (let ((destination-directory (string-append #$output "/lib/modules/" version))) + (when (not (file-exists? destination-directory)) ; unique + (union-build destination-directory + ;; All directories with the same version as us. + (filter-map (lambda (directory-name) + (if (member version + (directory-entries directory-name)) + (string-append directory-name "/" version) + #f)) + module-directories) + #:create-all-directories? #t) + ;; Delete generated files (they will be recreated shortly). + (for-each (lambda (basename) + (when (string-prefix? "modules." basename) + (false-if-file-not-found + (delete-file + (string-append + destination-directory "/" + basename))))) + (directory-entries destination-directory)) + (unless (zero? (system* (string-append #$kmod "/bin/depmod") + "-e" ; Report symbols that aren't supplied + "-w" ; Warn on duplicates + "-b" #$output ; destination-directory + "-F" (match System.maps + ((x) x)) + "-E" (match Module.symverss + ((x) x)) + version)) + (display "FAILED\n" (current-error-port)) + (exit #f))))) + versions) + (exit #t))))) + (gexp->derivation "linux-module-database" build + #:local-build? #t + #:substitutable? #f + #:properties + `((type . profile-hook) + (hook . linux-module-database))))) + (define (xdg-desktop-database manifest) "Return a derivation that builds the @file{mimeinfo.cache} database from desktop files. It's used to query what applications can handle a given @@ -1425,7 +1497,8 @@ MANIFEST." gtk-im-modules texlive-configuration xdg-desktop-database - xdg-mime-database)) + xdg-mime-database + linux-module-database)) (define* (profile-derivation manifest #:key