| Message ID | 87mu66q3rt.fsf@gnu.org |
|---|---|
| Headers | show |
| Series | Use native qemu to build vm-image. | expand |
Hello Jan, > Cross-building a vm-image used to be done using a cross-qemu, e.g, qemu-ARM. > That does not work for the Hurd, as there is no qemu-HURD. Yes that's an issue. > > This patch switches to cross building vm-images using a native qemu vm. If > there a reason for using qemu-TARGET we may want to make this switch > conditional for cross building to the Hurd? Well first this 'vm-image' thing should be done in (gnu system image) as soon as I have sorted out the bootloader issue. So the solution we will find to overcome this Hurd issue will be temporary I hope. About using qemu-TARGET, let say you are on an armhf machine and you want to cross-build an x86-64 image. I fear that using a native, armhf Grub to install an x86-64 Grub won't work. So maybe, it would be better to do that only for the architectures without an available qemu-TARGET (only the Hurd for now)? > ...the problem is now avoided by removing the sqlite dependency when > cross-building by not registering closures and postponing the loading of (gnu > store database) and thus (sqlite3). When I produce a system without register closures, once booted, "guix build" something does not work. I don't know if its possible for guix-daemon to work without its database. An option could be to generate this database if its absent, at first boot? > ./pre-inst-env guix system vm-image --target=i586-pc-gnu --no-grafts \ > gnu/system/examples/bare-hurd.tmpl > > now produces a pretty nice hurd VM :-) Great to see you so close :) Thanks, Mathieu
Mathieu Othacehe writes: Hello Mathieu, Thanks for chiming in, I meant to CC you... :-) >> Cross-building a vm-image used to be done using a cross-qemu, e.g, qemu-ARM. >> That does not work for the Hurd, as there is no qemu-HURD. > > Yes that's an issue. Yes...luckily I think we found a way to use qemu-NATIVE to build packages for the Hurd. Now to see if this can be put into digestible form. >> This patch switches to cross building vm-images using a native qemu vm. If >> there a reason for using qemu-TARGET we may want to make this switch >> conditional for cross building to the Hurd? > > Well first this 'vm-image' thing should be done in (gnu system image) as > soon as I have sorted out the bootloader issue. So the solution we will > find to overcome this Hurd issue will be temporary I hope. Can you elaborate a bit on that? Do you think it makes sense to continue this temporary path some more (as it starts to work right now), or can we better abandon it and work the permanent solution? How could I contribute? > About using qemu-TARGET, let say you are on an armhf machine and you > want to cross-build an x86-64 image. I fear that using a native, armhf > Grub to install an x86-64 Grub won't work. Ah...right, that's helpful information. > So maybe, it would be better to do that only for the architectures > without an available qemu-TARGET (only the Hurd for now)? OK -- I have changed patch 2 and 3, and am sending a new patch set. >> ...the problem is now avoided by removing the sqlite dependency when >> cross-building by not registering closures and postponing the loading of (gnu >> store database) and thus (sqlite3). > > When I produce a system without register closures, once booted, "guix > build" something does not work. I don't know if its possible for > guix-daemon to work without its database. An option could be to generate > this database if its absent, at first boot? Ah, OK. I'm using this "solution" to use #:register-closures? #f now for the Hurd only and we'll hit the problem that causes later. We can then, s discussed on IRC, sqlite3 databases are indeed platform independent and we could use something like guix/scripts/pack.scm's store-database. WDYT? >> ./pre-inst-env guix system vm-image --target=i586-pc-gnu --no-grafts \ >> gnu/system/examples/bare-hurd.tmpl >> >> now produces a pretty nice hurd VM :-) > > Great to see you so close :) Yes... Greetings, janneke
Hello Jan, >> Well first this 'vm-image' thing should be done in (gnu system image) as >> soon as I have sorted out the bootloader issue. So the solution we will >> find to overcome this Hurd issue will be temporary I hope. > > Can you elaborate a bit on that? Do you think it makes sense to continue > this temporary path some more (as it starts to work right now), or can we > better abandon it and work the permanent solution? How could I contribute? Yes. Besides offering a more modular image creation API, the vocation of (gnu system image) is to offer a way to generate all kind of Guix system images (raw disk-images, ISO9660 images, Qemu images) without resorting to VM for image creation. Dropping VM, means that the image need to be build on the host, without root permissions. This brings several limitations. If is no longer possible to use "mount" for instance, or to call "grub-install". To get around these limitations, I used the same strategy as Buildroot, Yocto and OpenWrt that do not require root permissions to generate disk-images. For ISO9660 images: * I first build the "image-root" derivation. It's a store directory that contains the root file-system. * Then, I call make-iso9660-image that, run GNU Xorriso on this directory. For raw disk-images: * For each partition, I build the "partition-image-root" derivation. This is very similar to "image-root" but for the specified partition. * For each partition root directory, I create the corresponding partition image, using tools such as mke2fs or mkdosfs depending on the partition file-system type. * Then, I need to "assemble" the partitions in a disk-image. For that, I use "genimage" that will roughly use 'dd' to create a final disk-image with the partitions copied at the right offsets. * The missing part here is the bootloader installation. As I mentioned, grub-install refuses to take a disk-image as argument (it requires a mounted partition). For EFI systems there's a work-around. The idea is to call grub-mkstandalone to create a Grub binary in the ESP partition. This Grub is configured to load the Grub configuration file located on the root file-system at /boot/grub/grub.cfg path. Now back to the Hurd. I see that Debian is producing Hurd ISO images. We could try to call `guix system disk-image --target=i586-pc-gnu --file-system-type=iso9660 hurd.scm` and see if it works. Regarding raw disk-image, I think we could try to produce EFI compatible Hurd images. We could set the bootloader to grub-efi-bootloader in %hurd-default-operating-system. Finally, to produce raw disk-images with grub-minimal-bootloader or grub-bootloader (what you are trying to do), we need to find a way to make grub-install work on disk-images (MBR installation and so on). That true for the Hurd but that's also true for Linux. Sorry for the long explanation. Please tell me if something is not clear. The conclusion here is that, I think that we very few adaptations to your branch, we should be able to produce Hurd ISO images or Hurd EFI compatible disk-images. Maybe it would be a first step. Then, we could find a way to create "MBR compatible" Hurd and Linux disk-images in (gnu system image). WDYT? Thanks, Mathieu
Hey, > The conclusion here is that, I think that we very few adaptations to > your branch, we should be able to produce Hurd ISO images or Hurd EFI > compatible disk-images. Maybe it would be a first step. > > Then, we could find a way to create "MBR compatible" Hurd and Linux > disk-images in (gnu system image). Ok, I made further progress. Turns out the Hurd EFI solution was a dead-end, because I have a "no console will be available to os" message in Qemu, instead of Hurd console output. I don't feel like debugging this. So, back to the MBR solution. I had another look to what OpenWrt is doing. They found a really nice work-around! As I stated, we cannot use "grub-install", but this command is in reality a wrapper around "grub-mkimage" and "grub-bios-setup". --8<---------------cut here---------------start------------->8--- ####################################### # # # # # MBR # MBR-GAP # FIRST PARTITION # # # # # ####################################### --8<---------------cut here---------------end--------------->8--- grub-mkimage generates a Grub image, small enough so that it can fit in the MBR-GAP (the space between the MBR and the first partition). It cannot contain all Grub modules, but that's not an issue, because Grub will be able to find missing modules from the first partition when started. They also patched grub-bios-setup[1] so that it can work on a raw disk-image, and install the previously generated "grub.img". So with the following commands: --8<---------------cut here---------------start------------->8--- grub-mkimage -O i386-pc -o core.img biosdisk part_msdos ext2 echo "(hd0) /tmp/my-disk-image" > device.map grub-bios-setup -m device.map -r "hd0,msdos1" -d tmp /tmp/qemu-image2 --8<---------------cut here---------------end--------------->8--- I'm able to make a Guix system image bootable, without root permissions. It then starts the Hurd kernel and crashes, but that's another story :p So, if it's ok for you, I can integrate this stuff cleanly, and we won't need the vm-image stuff anymore. WDYT? Thanks, Mathieu [1]: https://github.com/openwrt/openwrt/blob/master/package/boot/grub2/patches/100-grub_setup_root.patch
Hey Ludo! > 1. When cross-compiling, can the ‘qemu-image’ procedure to its job by > running exclusively native software (in particular using a native > QEMU, native kernel, etc.)? I think the answer is yes, but I raised a concern about being able to run grub-install from an ARM system to build a cross-compiled x86-64 system (for instance). For now, running this command shows: --8<---------------cut here---------------start------------->8--- ls $(guix build --system=aarch64-linux grub)/lib/grub/ arm64-efi --8<---------------cut here---------------end--------------->8--- So, the native aarch64-linux is only able to install itself on the same system. I think can be fixed though (same as for grub-hybrid package). > As for (2), I’d say that when cross-compiling, it should just run native > software but simply preserve references to cross-compiled software, > which is what janneke’s patch does. Yes, I agree. However, I think I found a way to install Grub, without root permissions, from the host system (see: https://lists.gnu.org/archive/html/guix-patches/2020-05/msg00988.html). This should allow to deprecate the whole (gnu system vm) module. Thanks for having a look :) Mathieu
> Ah, qemu-img convert -- nice. So, we get different type of images. > Wonder if that could play some role? Hope not! > The most interesting differences I see are wrt Grub, e.g. > > -/mnt/boot/grub/fonts > -/mnt/boot/grub/fonts/unicode.pf2 > > could that still play a role? I'm having a look why this could be missing. It's because I just copy "lib/grub" folder in (gnu bootloader grub). I should also do that for the fonts I guess. But I doubt the issue comes from here. > --- vm-image.lst-s 2020-05-24 15:41:02.051314009 +0200 > +++ disk-image.lst-s 2020-05-24 15:41:08.407414141 +0200 > @@ -3,293 +3,577 @@ > /mnt/boot > /mnt/boot/activation > /mnt/boot/grub > -/mnt/boot/grub/fonts At some point, if we have the same files inside, it should work :p I noticed that doing a cfdisk on the raw converted, vm-image root partition reports: Attributes: 80 which doesn't appear on the disk-image. However, tune2fs does not show any noticeable difference (both ext_attr and Hurd OS type are set). Mathieu
Mathieu Othacehe writes: >> Ah, qemu-img convert -- nice. So, we get different type of images. >> Wonder if that could play some role? > > Hope not! Okay...let's continue searching elsewhere then. >> The most interesting differences I see are wrt Grub, e.g. >> >> -/mnt/boot/grub/fonts >> -/mnt/boot/grub/fonts/unicode.pf2 >> >> could that still play a role? I'm having a look why this could be missing. > > It's because I just copy "lib/grub" folder in (gnu bootloader grub). I > should also do that for the fonts I guess. But I doubt the issue comes > from here. Meanwhile I tried adding that by hand; indeed that's not it. >> --- vm-image.lst-s 2020-05-24 15:41:02.051314009 +0200 >> +++ disk-image.lst-s 2020-05-24 15:41:08.407414141 +0200 >> @@ -3,293 +3,577 @@ >> /mnt/boot >> /mnt/boot/activation >> /mnt/boot/grub >> -/mnt/boot/grub/fonts > > At some point, if we have the same files inside, it should work :p I > noticed that doing a cfdisk on the raw converted, vm-image root > partition reports: > > Attributes: 80 > > which doesn't appear on the disk-image. I am not aware that we would be using attributes, at least I reverted the xattr trick on wip-hurd-vm for the /servers. Can we find out where and what they are? > However, tune2fs does not show any noticeable difference (both ext_attr > and Hurd OS type are set). Indeed, I looked at that too. This may be so obvious once we find it... Janneke
Hi! Cross-building a vm-image used to be done using a cross-qemu, e.g, qemu-ARM. That does not work for the Hurd, as there is no qemu-HURD. This patch switches to cross building vm-images using a native qemu vm. If there a reason for using qemu-TARGET we may want to make this switch conditional for cross building to the Hurd? There is a tricky aspect to this: the "user-builder" includes all the cross-built packages for TARGET, but we need to hard the native qemu a native #+LOADER, loading native #+LINUX and #+INITRD. To accomplish this NATIVE/CROSS mixt, the builder is then run with a native guile, like so --8<---------------cut here---------------start------------->8--- (system* #+(file-append (default-guile) "/bin/guile") "--no-auto-compile" #$(preserve-target user-builder)) --8<---------------cut here---------------end--------------->8--- while the packages in USER-BUILDER will still be cross-built. However, because the builder has (with-extensions gcrypt-sqlite3&co (use-modules (gnu store database)) ...) this makes the native guile "see" the cross-built .GO files. That would "only" be a secondary problem except that loading (sqlite3) throws an exception when dynamic-loading the .SO fails. Seeing it like this I am not so sure anymore this is a bug, WDYT? ...so instead ...the problem is now avoided by removing the sqlite dependency when cross-building by not registering closures and postponing the loading of (gnu store database) and thus (sqlite3). I have reset wip-hurd-vm onto these changes, so you can also look there. Doing ./pre-inst-env guix system vm-image --target=i586-pc-gnu --no-grafts \ gnu/system/examples/bare-hurd.tmpl now produces a pretty nice hurd VM :-) Greetings, Janneke Jan (janneke) Nieuwenhuizen (3): utils: Move 'reset-timestamps' out of database. system: vm: Do not register-closures when cross-building. system: vm: Build vm-image using native qemu. gnu/bootloader/grub.scm | 4 +-- gnu/build/vm.scm | 3 ++- gnu/system/vm.scm | 54 ++++++++++++++++++++++++++--------------- guix/store/database.scm | 41 +++---------------------------- guix/utils.scm | 41 ++++++++++++++++++++++++++++--- 5 files changed, 79 insertions(+), 64 deletions(-)