[bug#41350,v2,3/3] system: vm: Build vm-image using native qemu, for the Hurd.

Hi!

Mathieu Othacehe <othacehe@gnu.org> skribis:

>>   1. When cross-compiling, can the ‘qemu-image’ procedure to its job by
>>      running exclusively native software (in particular using a native
>>      QEMU, native kernel, etc.)?
>
> I think the answer is yes, but I raised a concern about being able to
> run grub-install from an ARM system to build a cross-compiled x86-64
> system (for instance).
>
> For now, running this command shows:
>
> ls $(guix build --system=aarch64-linux grub)/lib/grub/
> arm64-efi
>
> So, the native aarch64-linux is only able to install itself on the same
> system. I think can be fixed though (same as for grub-hybrid package).
>
>> As for (2), I’d say that when cross-compiling, it should just run native
>> software but simply preserve references to cross-compiled software,
>> which is what janneke’s patch does.
>
> Yes, I agree.

So it took me the whole day, but I ended up with the following patches
addressing these two points.  With that on master, I can do:

  guix system vm --target=arm-linux-gnueabihf --no-grafts \
    gnu/system/examples/bare-bones.tmpl

and get a usable script that spawns an ARM VM (I still need to add ‘-M
virt -nographic’ and remove ‘-enable-kvm’, but that’s a secondary
issue.)

Likewise, this produces what looks like a valid image, except probably
for the bootloader (I used GRUB in bare-bones, which didn’t complain,
but the result doesn’t work):

  guix system vm-image --target=arm-linux-gnueabihf …

Thoughts?

I’ll see if I can test it with ‘wip-hurd-vm’ on top but I’m not sure
I’ll do it before the week-end.

> However, I think I found a way to install Grub, without root
> permissions, from the host system (see:
> https://lists.gnu.org/archive/html/guix-patches/2020-05/msg00988.html).

Yay!

> This should allow to deprecate the whole (gnu system vm) module.

Yeah, using the new image API will be nicer.  We still need to keep some
of the API in (guix system vm), at least for the ‘guix system vm’
command.

Thanks,
Ludo’.

> So it took me the whole day, but I ended up with the following patches

Hey Ludo,

> addressing these two points.  With that on master, I can do:
>
>   guix system vm --target=arm-linux-gnueabihf --no-grafts \
>     gnu/system/examples/bare-bones.tmpl

Nice! Many thanks for helping us on that :)

> Likewise, this produces what looks like a valid image, except probably
> for the bootloader (I used GRUB in bare-bones, which didn’t complain,
> but the result doesn’t work):
>
>   guix system vm-image --target=arm-linux-gnueabihf …
>
> Thoughts?

qemu-system-arm supports a whole range of machine. You can type:

--8<---------------cut here---------------start------------->8---
qemu-system-arm -M machine help
--8<---------------cut here---------------end--------------->8---

for the complete list. Depending on the selected machine, the required
bootloader differs. For the "virt" machine, it seems that we could pass
the bootloader as Qemu "bios" argument[1], but I'm not sure we can do
much better.

> Yeah, using the new image API will be nicer.  We still need to keep some
> of the API in (guix system vm), at least for the ‘guix system vm’
> command.

Yes, you are right.

> From 6e936131ca85aba24f82d35c4616afe835ac7da5 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
> Date: Wed, 27 May 2020 23:57:41 +0200
> Subject: [PATCH 7/7] gnu: guile-static: Disable JIT on ARMv7.

The whole serie LGTM. It's nice to have cross-compilation working for
both disk-images and vm-images!

Mathieu

[1]:
https://github.com/ARM-software/u-boot/blob/master/doc/README.qemu-arm

Mathieu Othacehe writes:

Hello!

>> So it took me the whole day, but I ended up with the following patches
>
> Hey Ludo,

Thanks so much for helping out!

>> addressing these two points.  With that on master, I can do:
>>
>>   guix system vm --target=arm-linux-gnueabihf --no-grafts \
>>     gnu/system/examples/bare-bones.tmpl
>
> Nice! Many thanks for helping us on that :)

+1

>> Likewise, this produces what looks like a valid image, except probably
>> for the bootloader (I used GRUB in bare-bones, which didn’t complain,
>> but the result doesn’t work):
>>
>>   guix system vm-image --target=arm-linux-gnueabihf …
>>
>> Thoughts?

I have locally rebased wip-hurd-vm and wip-disk-image (working title for
wip-hurd-vm minus vm-image support) on it: it looks nice and works for
me building a disk-impage.

Building a vm-image still needs quite some work: I tried building a
vm-image without hurd-specific vm-image patches (bisecting a bit:
register-closures/grub/qemu-image preserve-target stuff)...but in the
end I still needed all these

--8<---------------cut here---------------start------------->8---
0547e4c251 * wip-hurd-vm system: vm: Activate the Hurd.
8c28ab91a5 * system: vm: Initial vm-image support for the Hurd.
d9b84d03bc * system: vm: Add defaults for the Hurd.
29281d053f * system: vm: Build vm-image using native qemu, for the Hurd.
86edea90c4 * system: vm: Do not register-closures when cross-building to the Hurd.
a29437ad82 * utils: Move 'reset-timestamps' out of database.
--8<---------------cut here---------------end--------------->8---

(especially the "using native qemu" patch was greatly reduced now)
as well as --no-grafts to get a vm-image.

So..as discussed on IRC yesterday I'm voting to drop/pause the vm-image
work for the Hurd and reset wip-hurd-vm to my current gitlab
wip-hurd-disk (or possibly push wip-hurd-disk...)  WDYT?

>> From 6e936131ca85aba24f82d35c4616afe835ac7da5 Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
>> Date: Wed, 27 May 2020 23:57:41 +0200
>> Subject: [PATCH 7/7] gnu: guile-static: Disable JIT on ARMv7.
>
> The whole serie LGTM. It's nice to have cross-compilation working for
> both disk-images and vm-images!

I agree...although as noted above: how much effort do we want to put in
for the Hurd?

Greetings,
Janneke

Hi!

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> Building a vm-image still needs quite some work: I tried building a
> vm-image without hurd-specific vm-image patches (bisecting a bit:
> register-closures/grub/qemu-image preserve-target stuff)...but in the
> end I still needed all these

How does it fail exactly?  (I’ll give it a spin as time permits.)

> So..as discussed on IRC yesterday I'm voting to drop/pause the vm-image
> work for the Hurd and reset wip-hurd-vm to my current gitlab
> wip-hurd-disk (or possibly push wip-hurd-disk...)  WDYT?

Sure, whatever works best!

>>> From 6e936131ca85aba24f82d35c4616afe835ac7da5 Mon Sep 17 00:00:00 2001
>>> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
>>> Date: Wed, 27 May 2020 23:57:41 +0200
>>> Subject: [PATCH 7/7] gnu: guile-static: Disable JIT on ARMv7.
>>
>> The whole serie LGTM. It's nice to have cross-compilation working for
>> both disk-images and vm-images!
>
> I agree...although as noted above: how much effort do we want to put in
> for the Hurd?

I was looking at what needs to be fixed there, independently of the
Hurd.

But maybe we can just rebase ‘system-qemu-image’ & co. on top of (gnu
image)?  What prevents us from doing that, Mathieu?

If we can do that, then indeed, there’s no point in insisting on fixing
cross-compilation support in (gnu system vm).

Thanks for your feedback!

Ludo’.

Ludovic Courtès writes:

Hello!

> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> Building a vm-image still needs quite some work: I tried building a
>> vm-image without hurd-specific vm-image patches (bisecting a bit:
>> register-closures/grub/qemu-image preserve-target stuff)...but in the
>> end I still needed all these
>
> How does it fail exactly?  (I’ll give it a spin as time permits.)

As discussed on IRC: same errors as before, mostly (sqlite loading,
meson cross build using --no-grafts, linux build for i586-pc-gnu).

>> So..as discussed on IRC yesterday I'm voting to drop/pause the vm-image
>> work for the Hurd and reset wip-hurd-vm to my current gitlab
>> wip-hurd-disk (or possibly push wip-hurd-disk...)  WDYT?
>
> Sure, whatever works best!

Done!  I have just reset wip-hurd-vm, let the review+merge begin \o/

>>>> From 6e936131ca85aba24f82d35c4616afe835ac7da5 Mon Sep 17 00:00:00 2001
>>>> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
>>>> Date: Wed, 27 May 2020 23:57:41 +0200
>>>> Subject: [PATCH 7/7] gnu: guile-static: Disable JIT on ARMv7.
>>>
>>> The whole serie LGTM. It's nice to have cross-compilation working for
>>> both disk-images and vm-images!
>>
>> I agree...although as noted above: how much effort do we want to put in
>> for the Hurd?
>
> I was looking at what needs to be fixed there, independently of the
> Hurd.

Yes, so I kept your commits at the bottom of the new wip-hurd-vm, they
are clean and there's some value in them, I think.

> But maybe we can just rebase ‘system-qemu-image’ & co. on top of (gnu
> image)?  What prevents us from doing that, Mathieu?

+1

> If we can do that, then indeed, there’s no point in insisting on fixing
> cross-compilation support in (gnu system vm).

possibly, modulo s/no/litte/ s/point/urgency/
but yeah.  A bit sad, between us we put quite some effort into that...

Janneke

Hey Ludo & Jan,

> But maybe we can just rebase ‘system-qemu-image’ & co. on top of (gnu
> image)?  What prevents us from doing that, Mathieu?
>
> If we can do that, then indeed, there’s no point in insisting on fixing
> cross-compilation support in (gnu system vm).

I think we could proceed that way:

* Merge Ludo's serie on master.

* Then we could review & merge Jan's wip-hurd-disk.

* Finally, there's the wip-hurd-vm. I believe that some of stuff there
is fixed in Ludo's serie. So we would need to rebase it on top of
everything & finally push it.

Do you think that makes sense?

Thanks,

Mathieu

Mathieu Othacehe writes:

Hi Mathieu, Ludo

> Hey Ludo & Jan,
>
>> But maybe we can just rebase ‘system-qemu-image’ & co. on top of (gnu
>> image)?  What prevents us from doing that, Mathieu?
>>
>> If we can do that, then indeed, there’s no point in insisting on fixing
>> cross-compilation support in (gnu system vm).
>
> I think we could proceed that way:
>
> * Merge Ludo's serie on master.
>
> * Then we could review & merge Jan's wip-hurd-disk.

Oops, you sent your mail just a few minutes before mine; just reading it
now :-)

I push a fresh wip-hurd-vm, which is the above: wip-hurd-disk rebase
onto Ludo's series...almost the same.

> * Finally, there's the wip-hurd-vm. I believe that some of stuff there
> is fixed in Ludo's serie. So we would need to rebase it on top of
> everything & finally push it.
>
> Do you think that makes sense?

Yes, I think this is more or less my idea too and what I did...except
for leaving my hurd-specific qemu-image/vm-image work rest.

Hope that's OK!
Janneke

Hi,

Mathieu Othacehe <othacehe@gnu.org> skribis:

>> But maybe we can just rebase ‘system-qemu-image’ & co. on top of (gnu
>> image)?  What prevents us from doing that, Mathieu?
>>
>> If we can do that, then indeed, there’s no point in insisting on fixing
>> cross-compilation support in (gnu system vm).
>
> I think we could proceed that way:
>
> * Merge Ludo's serie on master.

I think that can wait because on IRC Janneke explained that it doesn’t
fix anything for GNU/Hurd (to my surprise).  So I’ll maybe check again
once the relevant Hurd bits are on master, instead of checking ARM
cross-compilation.

Anyway it’s much less important now that (gnu image) can be used for the
task!

> * Then we could review & merge Jan's wip-hurd-disk.

Do I get it right that we first need
<https://issues.guix.gnu.org/41560>?

The ‘wip-hurd-vm’ branch contains many things:

  1. (gnu system hurd) with the Hurd services etc.

  2. The ‘hurd’ field of <operating-system>.

  3. <menu-entry> with multiboot support.

  4. Hacks to work around vm.scm defects: uses of ‘with-parameters’,
     ‘hurd-target?’, disabling sqlite3, and #~#$ tricks.

I think part of the reason this cycle has been so long is that it’s been
kind of a big bang; big bangs are great because they lead to something
new and exciting, but they’re also intimidating.  :-) For me personally,
looking at all these aspects at once was just too much.

For merging, I think it’d be great to see #1 and #2 as a first step, and
then #3.

I do not want any of #4 :-), because I really think it could lead to
maintenance headaches down the road, which would make the kind of
changes we’re making today practically impossible in the future.

Thoughts?

Ludo’.

Ludovic Courtès writes:

> Hi,
>
> Mathieu Othacehe <othacehe@gnu.org> skribis:
>
>>> But maybe we can just rebase ‘system-qemu-image’ & co. on top of (gnu
>>> image)?  What prevents us from doing that, Mathieu?
>>>
>>> If we can do that, then indeed, there’s no point in insisting on fixing
>>> cross-compilation support in (gnu system vm).
>>
>> I think we could proceed that way:
>>
>> * Merge Ludo's serie on master.
>
> I think that can wait because on IRC Janneke explained that it doesn’t
> fix anything for GNU/Hurd (to my surprise).

Well, that's too black/white.  Of course this:

-                     (linux   (string-append #$linux "/"
-                                             #$(system-linux-image-file-name)))
-                     (initrd  #$initrd)
-                     (loader  #$loader)
+                     (linux   (string-append
+                               #+linux "/"
+                               #+(system-linux-image-file-name system)))
+                     (initrd  #+initrd)
+                     (loader  #+loader)

is an essential fix.  Only, that used to be on wip-hurd-vm -- so from
that perspective this fix only got reviewed and moved.

Pretty similarly for this

+  (define preserve-target
+    (if target
+        (lambda (obj)
+          (with-parameters ((%current-target-system target))
+            obj))
+        identity))

also an essential bit.  So, seen from master it fixes quite some bits!
What I meant to say is that I haven't seen /additional/ fixes to what
we had on wip-hurd-vm...

Because your fixes look clean and don't break anything, I think we
should maybe apply them to master -- and that's why I kept them at
the bottom of "wip-hurd-vm" for now.

> So I’ll maybe check again once the relevant Hurd bits are on master,
> instead of checking ARM cross-compilation.
>
> Anyway it’s much less important now that (gnu image) can be used for the
> task!

Yes, so from the Hurd perspective this patch set has shifted from
top-priority to probably not going to be used for the Hurd ;)

>> * Then we could review & merge Jan's wip-hurd-disk.
>
> Do I get it right that we first need
> <https://issues.guix.gnu.org/41560>?
>
> The ‘wip-hurd-vm’ branch contains many things:
>
>   1. (gnu system hurd) with the Hurd services etc.
>
>   2. The ‘hurd’ field of <operating-system>.
>
>   3. <menu-entry> with multiboot support.

Yes.

>   4. Hacks to work around vm.scm defects: uses of ‘with-parameters’,

Together with dropping qemu-image support for the Hurd, I have removed
most of this; what remains is in

--8<---------------cut here---------------start------------->8---
system: Add 'multiboot-modules' field to <boot-parameters>.
+             (mach (if (%current-target-system)
+                       ;; A cross-built GNUmach does not work
+                       (with-parameters ((%current-system "i686-linux")
+                                         (%current-target-system #f))
+                         mach)
+                       mach)))
[..]
+         (libc (if target
+                   (with-parameters ((%current-target-system #f))
+                     (cross-libc target))
+                   glibc))
--8<---------------cut here---------------end--------------->8---

>      ‘hurd-target?’, disabling sqlite3, and #~#$ tricks.

I think this is mostly gone, two instances of hurd-target? remain.  I'll
look more sharply at that too!

> I think part of the reason this cycle has been so long is that it’s been
> kind of a big bang; big bangs are great because they lead to something
> new and exciting, but they’re also intimidating.  :-) For me personally,
> looking at all these aspects at once was just too much.

Yes...

> For merging, I think it’d be great to see #1 and #2 as a first step, and
> then #3.

Yes, that would be an amazing feat.

> I do not want any of #4 :-), because I really think it could lead to
> maintenance headaches down the road, which would make the kind of
> changes we’re making today practically impossible in the future.

> Thoughts?

I fully agree, thanks!

Greetings,
Janneke

Jan Nieuwenhuizen writes:

Hi!

A quick headsup for /boot/activation; I have found how to do without
/boot/activation, using the regular (I think) path using kernel
arguments --system=, --load=, and system/boot.

I'll be pushing a couple of squash commits onto wip-hurd-vm for
that.

Janneke

Hi,

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> A quick headsup for /boot/activation; I have found how to do without
> /boot/activation, using the regular (I think) path using kernel
> arguments --system=, --load=, and system/boot.

Yay!

Ludo’.

Hello,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> A quick headsup for /boot/activation; I have found how to do without
>> /boot/activation, using the regular (I think) path using kernel
>> arguments --system=, --load=, and system/boot.

I was trying to see if there were any pending changes not yet committed
to our tree in this interesting but long thread?  Please help me out by
closing it if not, or pointing which changes remain :-).

Thanks,

Maxim

Hey Maxim,

Building 'vm-image' does not involve QEMU anymore so we can close this
one.

Thanks,

Mathieu