| Message ID | 20200519072302.9202-3-janneke@gnu.org |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [bug#41350,v2,1/3] utils: Move 'reset-timestamps' out of database. | expand |
| Context | Check | Description |
|---|---|---|
| cbaines/comparison | success | View comparision |
| cbaines/git branch | success | View Git branch |
| cbaines/applying patch | fail | View Laminar job |
Hello Janneke, > + #$(if (hurd-target?) > + #~#+(grub-root-search store-device font-file) > + (grub-root-search store-device font-file)) > + #$(if (hurd-target?) > + #~#+(setup-gfxterm config font-file) > + (setup-gfxterm config font-file)) I'm not certain that hurd-target? always return the expected answer here. Now that we have let-system, the safer way to check for the target system could be to write: --8<---------------cut here---------------start------------->8--- #$(let-system (system target) (hurd-target? system target)) --8<---------------cut here---------------end--------------->8--- Ludo, is this correct? Furthermore, if you make sure that "grub-root-search" and "setup-gfxterm" return #+, you could maybe drop this part? > + > + (loader #$(if (hurd-target?) #~#+loader loader)) > + (linux #$(if (hurd-target?) > + #~(string-append > + #+linux "/" > + #+(system-linux-image-file-name)) > + #~(string-append > + #$linux "/" > + #$(system-linux-image-file-name)))) Same concern as above about "hurd-target?". > + ;; (if #$(hurd-target?) > + ;; '#+(append (list parted e2fsprogs dosfstools) > + ;; (map canonical-package > + ;; (list sed grep coreutils findutils gawk))) > + ;; '#$(append (list parted e2fsprogs dosfstools) > + ;; (map canonical-package > + ;; (list sed grep coreutils findutils gawk)))) > + ) Is this needed? Thanks, Mathieu
Hi, Mathieu Othacehe <othacehe@gnu.org> skribis: >> + #$(if (hurd-target?) >> + #~#+(grub-root-search store-device font-file) >> + (grub-root-search store-device font-file)) >> + #$(if (hurd-target?) >> + #~#+(setup-gfxterm config font-file) >> + (setup-gfxterm config font-file)) > > I'm not certain that hurd-target? always return the expected answer > here. Now that we have let-system, the safer way to check for the target > system could be to write: > > #$(let-system (system target) > (hurd-target? system target)) > > Ludo, is this correct? Definitely! Well, modulo indentation. ;-) Thanks, Ludo’.
Howdy! "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org> skribis: > Cross-building a vm-image is usually done using a cross-qemu, e.g, qemu-ARM, > because, e.g., a native, x86_64 Grub cannot install an armhf-Grub. That > solution does not work for the Hurd, as there is no qemu-HURD. > > This patch enables cross building vm-images for the Hurd using a native qemu > vm. > > * gnu/system/vm.scm (expression->derivation-in-linux-vm): Run native > qemu-command; use native linux, initrd, bootloader-package and > bootloader-installer, for the Hurd. > [preserve-target]: New helper to install cross-packages into the native vm. > * gnu/bootloader/grub.scm (eye-candy): Use native font, for the Hurd. > > Co-authored-by: Ludovic Courtès <ludo@gnu.org> I think the big question here, not Hurd-specific, is about the abstractions: 1. When cross-compiling, can the ‘qemu-image’ procedure to its job by running exclusively native software (in particular using a native QEMU, native kernel, etc.)? 2. What should ‘expression->derivation-in-linux-vm’ do when cross-compiling? I think the answer to (1) is yes, right? That is, we can build a QEMU image for AArch64/EFI from an x86_64 box. That means we’d run the x86_64/EFI ‘grub-install’ in the final step, an x86_64 ‘mke2fs’ and ‘parted’ to create the partitions, and so on. Likewise, we can create a QEMU GNU/Hurd image without entirely on GNU/Linux. As for (2), I’d say that when cross-compiling, it should just run native software but simply preserve references to cross-compiled software, which is what janneke’s patch does. How does that sound? Mathieu? (I guess I’m rewording what janneke wrote, but my mind became fuzzy last time I looked at it, so I thought it’s best to double-check. :-)) Thanks! Ludo’.
Mathieu Othacehe writes: Hello Mathieu, >> + #$(if (hurd-target?) >> + #~#+(grub-root-search store-device font-file) >> + (grub-root-search store-device font-file)) >> + #$(if (hurd-target?) >> + #~#+(setup-gfxterm config font-file) >> + (setup-gfxterm config font-file)) > > I'm not certain that hurd-target? always return the expected answer > here. Now that we have let-system, the safer way to check for the target > system could be to write: > > #$(let-system (system target) > (hurd-target? system target)) Okay... > Ludo, is this correct? Furthermore, if you make sure that > "grub-root-search" and "setup-gfxterm" return #+, you could maybe drop > this part? ...used #+ and dropped the IFs here, but used this insight below. >> + >> + (loader #$(if (hurd-target?) #~#+loader loader)) >> + (linux #$(if (hurd-target?) >> + #~(string-append >> + #+linux "/" >> + #+(system-linux-image-file-name)) >> + #~(string-append >> + #$linux "/" >> + #$(system-linux-image-file-name)))) > > Same concern as above about "hurd-target?". ...Ah, that makes sense...that's of course why Ludo already introduced (define-syntax-rule (check predicate) (let-system (system target) (predicate (or target system)))) here. So, using "(check hurd-triplet?)", as Ludo was already doing in some other places here. >> + ;; (if #$(hurd-target?) >> + ;; '#+(append (list parted e2fsprogs dosfstools) >> + ;; (map canonical-package >> + ;; (list sed grep coreutils findutils gawk))) >> + ;; '#$(append (list parted e2fsprogs dosfstools) >> + ;; (map canonical-package >> + ;; (list sed grep coreutils findutils gawk)))) >> + ) > > Is this needed? Oops -- that was unfinished business; that IF did not work yet; possibly because of not going through LET-SYSTEM. This helps a lot, I'm now again able to do both ./pre-inst-env guix system vm-image --target=i586-pc-gnu --no-grafts gnu/system/examples/bare-bones.tmpl ./pre-inst-env guix system vm-image gnu/system/examples/bare-bones.tmpl again! Phew... And that's also because I learned to include the full list of exported symbols in the autoload of (gnu store database). Mathieu, if it turns out that you are dropping qemu-image next week altogether then great!, and I have no problem whatsoever dropping this patch set as well. For now, I have reset wip-hurd-vm and put your commits towards the bottom of the wip-hurd-vm stack too. Sending an updated v3 patch set just because I'm so pleased with this result. Greetings, janneke
Hello Jan, > Mathieu, if it turns out that you are dropping qemu-image next week > altogether then great!, and I have no problem whatsoever dropping this > patch set as well. > > For now, I have reset wip-hurd-vm and put your commits towards the > bottom of the wip-hurd-vm stack too. I just pushed a few commits to your branch. I'm now able to run: --8<---------------cut here---------------start------------->8--- guix system disk-image --target=i586-pc-gnu gnu/system/examples/bare-hurd.tmpl --8<---------------cut here---------------end--------------->8--- no VM inside. We now have to backport some of your work to (gnu system image), but the good news is that closure registration is now working, and the Hurd boots. > Sending an updated v3 patch set just because I'm so pleased with this > result. I'll have a look :) Thanks, Mathieu
Mathieu Othacehe writes: > Hello Jan, > >> Mathieu, if it turns out that you are dropping qemu-image next week >> altogether then great!, and I have no problem whatsoever dropping this >> patch set as well. >> >> For now, I have reset wip-hurd-vm and put your commits towards the >> bottom of the wip-hurd-vm stack too. > > I just pushed a few commits to your branch. I'm now able to run: > > guix system disk-image --target=i586-pc-gnu gnu/system/examples/bare-hurd.tmpl > > no VM inside. Whoa, that's 10 commits -- great! I do need to use --no-grafts, if I don't I get guix system: error: gnu/packages/glib.scm:406:2: gobject-introspection@1.62.0: build system `meson' does not support cross builds Do you see that too? > We now have to backport some of your work to (gnu system image), but Just build and started it -- beautiful. I'll have a stab later tomorrow, unless someone beats me to it ;) > the good news is that closure registration is now working, and the Hurd > boots. very nice, that's already one ahead! > I'll have a look :) Thanks :-) Greetings, Janneke
Jan Nieuwenhuizen writes: Hello Mathieu, > Mathieu Othacehe writes: >> We now have to backport some of your work to (gnu system image), but > > Just build and started it -- beautiful. I'll have a stab later > tomorrow, unless someone beats me to it ;) I have just pushed a few commits to wip-hurd-vm. b0df9b0c6b squash! bootloader: grub: Add support for '<hurd-menu-entry>'. We were using a cross-built gnumach, which does not work. This did not happen with "vm-image" because "qemu-image" of how qemu-image built grub. Best to squash that later. 6169cb6396 Revert "system: hurd: Remove bash from boot process." As mentioned in the commit, where we used to run patched linux in the VM, the "Remove bash from boot process" patch would now rely on the host's linux being patched. It's a bit early times/inconvenient to demand this, I suppose... c378a2beec linux-boot: Update 'make-hurd-device-nodes'. Moved a part of the Hurd stuff here, and the rest b605a36031 WIP hurd-directives here. It does not work and I don't understand that I had to add a call to make-device-nodes...that's probably not how you want it to work? Anyway...no real luck yet. I'm stuck at: start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1] exec startup proc auth /hurd/startup: Failed to bind to /servers/startup: Device or resource busy . /gnu/store/5m6m1ic2m4m43nv4wl7nfa5ab9p76b2g-hurd-0.9-1.91a5167/hurd/exec: main.c:358: S_exec_init: Unexpected error: (ipc/mig) bad request message ID. Ideas? ... Ludo perhaps? Greetings, janneke
Hello Jan, > here. It does not work and I don't understand that I had to add a call > to make-device-nodes...that's probably not how you want it to work? No its more an omission, and for Linux, it seems that it works without make-device-nodes call. > > Anyway...no real luck yet. I'm stuck at: > > start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1] exec startup proc auth > /hurd/startup: Failed to bind to /servers/startup: Device or resource busy > . > /gnu/store/5m6m1ic2m4m43nv4wl7nfa5ab9p76b2g-hurd-0.9-1.91a5167/hurd/exec: main.c:358: S_exec_init: Unexpected error: (ipc/mig) bad request message ID. > > Ideas? ... Ludo perhaps? Running the following commands: --8<---------------cut here---------------start------------->8--- ./pre-inst-env guix system disk-image --target=i586-pc-gnu gnu/system/examples/bare-hurd.tmpl cp /gnu/store/the-image /tmp/img_ko ./pre-inst-env guix system vm-image --target=i586-pc-gnu gnu/system/examples/bare-hurd.tmpl cp /gnu/store/the-vm-image /tmp/img_ok.qcow2 qemu-img convert -f qcow2 -O raw /tmp/img_ok.qcow2 /tmp/img_ok.raw sudo losetup -P /dev/loop1 /tmp/img_ko sudo losetup -P /dev/loop2 /tmp/img_ok.raw sudo mount /dev/loop1p1 /mnt/img/ sudo mount /dev/loop2p1 /mnt/img2/ diff -r /mnt/img /mnt/img --8<---------------cut here---------------end--------------->8--- I get: --8<---------------cut here---------------start------------->8--- Only in /mnt/img2/dev: cons Only in /mnt/img2/dev: console Only in /mnt/img2/dev: fd Only in /mnt/img2/dev: klog Only in /mnt/img2/dev: kmsg Only in /mnt/img2/dev: log Only in /mnt/img2/dev: mem Only in /mnt/img2/dev: ptyp0 Only in /mnt/img2/dev: ptyp1 Only in /mnt/img2/dev: ptyp2 Only in /mnt/img2/dev: shm Only in /mnt/img2/dev: stderr Only in /mnt/img2/dev: stdin Only in /mnt/img2/dev: stdout Only in /mnt/img2/dev: time Only in /mnt/img2/dev: tty Only in /mnt/img2/dev: tty1 Only in /mnt/img2/dev: tty12 Only in /mnt/img2/dev: tty2 Only in /mnt/img2/dev: tty3 Only in /mnt/img2/dev: tty4 Only in /mnt/img2/dev: tty5 Only in /mnt/img2/dev: tty6 Only in /mnt/img2/dev: ttyp0 Only in /mnt/img2/dev: ttyp1 Only in /mnt/img2/dev: ttyp2 diff: /mnt/img2/dev/urandom: Structure needs cleaning Only in /mnt/img2/dev: vcs Only in /mnt/img2/etc: fstab Only in /mnt/img2/etc: group Only in /mnt/img2/etc: hostname Only in /mnt/img2/etc: login Only in /mnt/img2/etc: motd Only in /mnt/img2/etc: pam.d Only in /mnt/img2/etc: passwd Only in /mnt/img2/etc: profile Only in /mnt/img2/etc: protocols Only in /mnt/img2/etc: .pwd.lock Only in /mnt/img2/etc: services Only in /mnt/img2/etc: shadow Only in /mnt/img2/etc: shells Only in /mnt/img2/etc: skel Only in /mnt/img2/etc: ssh Only in /mnt/img2/etc: ssl Only in /mnt/img2/etc: static --8<---------------cut here---------------end--------------->8--- It seems that at least etc and dev directories are not correctly populated using disk-image. To be continued! Mathieu
Hi! Jan Nieuwenhuizen <janneke@gnu.org> skribis: > Mathieu Othacehe writes: > >> Hello Jan, >> >>> Mathieu, if it turns out that you are dropping qemu-image next week >>> altogether then great!, and I have no problem whatsoever dropping this >>> patch set as well. >>> >>> For now, I have reset wip-hurd-vm and put your commits towards the >>> bottom of the wip-hurd-vm stack too. >> >> I just pushed a few commits to your branch. I'm now able to run: >> >> guix system disk-image --target=i586-pc-gnu gnu/system/examples/bare-hurd.tmpl >> >> no VM inside. > > Whoa, that's 10 commits -- great! Woow, excellent! Thumbs up! (I’ll take a look at the graft issue, I feel a repulsion for dynamic binding growing every day…) Ludo’.
diff --git a/gnu/bootloader/grub.scm b/gnu/bootloader/grub.scm index 8c5b5eac0c..cee4c9f6c5 100644 --- a/gnu/bootloader/grub.scm +++ b/gnu/bootloader/grub.scm @@ -31,6 +31,7 @@ #:use-module (gnu system file-systems) #:use-module (gnu system keyboard) #:use-module (gnu packages bootloaders) + #:use-module (gnu packages hurd) #:autoload (gnu packages gtk) (guile-cairo guile-rsvg) #:autoload (gnu packages xorg) (xkeyboard-config) #:use-module (ice-9 match) @@ -211,8 +212,12 @@ else set menu_color_highlight=white/blue fi~%" #$setup-gfxterm-body - #$(grub-root-search store-device font-file) - #$(setup-gfxterm config font-file) + #$(if (hurd-target?) + #~#+(grub-root-search store-device font-file) + (grub-root-search store-device font-file)) + #$(if (hurd-target?) + #~#+(setup-gfxterm config font-file) + (setup-gfxterm config font-file)) #$(grub-setup-io config) #$(strip-mount-point store-mount-point image) diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm index b343141c18..527596421d 100644 --- a/gnu/system/vm.scm +++ b/gnu/system/vm.scm @@ -5,6 +5,7 @@ ;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com> ;;; Copyright © 2017 Marius Bakke <mbakke@fastmail.com> ;;; Copyright © 2018 Chris Marusich <cmmarusich@gmail.com> +;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org> ;;; ;;; This file is part of GNU Guix. ;;; @@ -181,19 +182,36 @@ made available under the /xchg CIFS share. SUBSTITUTABLE? determines whether the returned derivation should be marked as substitutable." (define user-builder - (program-file "builder-in-linux-vm" exp)) + (scheme-file "builder-in-linux-vm" exp)) + + (define (preserve-target obj) + (if target + (with-parameters ((%current-target-system target)) + obj) + obj)) (define loader - ;; Invoke USER-BUILDER instead using 'primitive-load'. The reason for - ;; this is to allow USER-BUILDER to dlopen stuff by using a full-featured - ;; Guile, which it couldn't do using the statically-linked guile used in - ;; the initrd. See example at + ;; Instead of using 'primitive-load', evaluate USER-BUILDER in a + ;; full-featured Guile so it can use dlopen stuff, which it couldn't do + ;; using the statically-linked guile used in the initrd. See example at ;; <https://lists.gnu.org/archive/html/guix-devel/2017-10/msg00233.html>. (program-file "linux-vm-loader" - ;; Communicate USER-BUILDER's exit status via /xchg so that - ;; the host can distinguish between success, failure, and - ;; kernel panic. - #~(let ((status (system* #$user-builder))) + ;; When cross-compiling, USER-BUILDER refers to the target + ;; (cross-compiled) system. Preserve that, even though + ;; LOADER itself is executed as a native program. + #~(let* ((guile #$(if (hurd-target?) + #~#+(file-append (default-guile) + "/bin/guile") + (file-append (default-guile) + "/bin/guile"))) + (status (system* guile "--no-auto-compile" + #$(if (hurd-target?) + (preserve-target user-builder) + user-builder)))) + + ;; Communicate USER-BUILDER's exit status via /xchg so + ;; that the host can distinguish between success, + ;; failure, and kernel panic. (call-with-output-file "/xchg/.exit-status" (lambda (port) (write status port))) @@ -227,10 +245,16 @@ substitutable." (let* ((native-inputs '#+(list qemu (canonical-package coreutils))) - (linux (string-append #$linux "/" - #$(system-linux-image-file-name))) - (initrd #$initrd) - (loader #$loader) + + (loader #$(if (hurd-target?) #~#+loader loader)) + (linux #$(if (hurd-target?) + #~(string-append + #+linux "/" + #+(system-linux-image-file-name)) + #~(string-append + #$linux "/" + #$(system-linux-image-file-name)))) + (initrd #$(if (hurd-target?) #~#+initrd initrd)) (graphs '#$(match references-graphs (((graph-files . _) ...) graph-files) (_ #f))) @@ -246,7 +270,9 @@ substitutable." (load-in-linux-vm loader #:output #$output #:linux linux #:initrd initrd - #:qemu (qemu-command target) + #:qemu #$(if (hurd-target?) + (qemu-command) + (qemu-command target)) #:memory-size #$memory-size #:make-disk-image? #$make-disk-image? #:single-file-output? #$single-file-output? @@ -288,7 +314,9 @@ substitutable." bootcfg-drv bootloader (register-closures? (and (has-guix-service-type? os) - (not (hurd-target?)))) + (not target) + ;(not (hurd-target?)) + )) (inputs '()) copy-inputs? (substitutable? #t)) @@ -349,9 +377,17 @@ system that is passed to 'populate-root-file-system'." (setlocale LC_ALL "en_US.utf8") (let ((inputs - '#$(append (list parted e2fsprogs dosfstools) + '#+(append (list parted e2fsprogs dosfstools) (map canonical-package - (list sed grep coreutils findutils gawk)))) + (list sed grep coreutils findutils gawk))) + ;; (if #$(hurd-target?) + ;; '#+(append (list parted e2fsprogs dosfstools) + ;; (map canonical-package + ;; (list sed grep coreutils findutils gawk))) + ;; '#$(append (list parted e2fsprogs dosfstools) + ;; (map canonical-package + ;; (list sed grep coreutils findutils gawk)))) + ) ;; This variable is unused but allows us to add INPUTS-TO-COPY ;; as inputs. @@ -426,12 +462,16 @@ system that is passed to 'populate-root-file-system'." #:partitions partitions #:grub-efi grub-efi #:bootloader-package - #$(bootloader-package bootloader) + (if #$(hurd-target?) + #+(bootloader-package bootloader) + #$(bootloader-package bootloader)) #:bootcfg #$bootcfg-drv #:bootcfg-location #$(bootloader-configuration-file bootloader) #:bootloader-installer - #$(bootloader-installer bootloader))))))) + (if #$(hurd-target?) + #+(bootloader-installer bootloader) + #$(bootloader-installer bootloader)))))))) #:system system #:target target #:make-disk-image? #t