[bug#63527,v5,00/11] Rumpdisk support for the Hurd, really, *really*!

Hi!

So, with lots of kind help from the folks at #hurd and bug-hurd, and many
thanks to Josselin Poiret for sending their amazing "Add libc specific to Hurd
and update components" patch set (https://issues.guix.gnu.org/63641, note that
these series depend on this!) we now have rumpdisk support for the Hurd.

To actually use it instead of the gnumach builtin IDE driver, use

    (kernel-arguments '("noide"))

To test this, I'm using this patch

--8<---------------cut here---------------start------------->8---

build it using

--8<---------------cut here---------------start------------->8---
./pre-inst-env guix system image -t hurd-raw gnu/system/examples/bare-hurd.tmpl --keep-failed 
--8<---------------cut here---------------end--------------->8---

and run it like this

--8<---------------cut here---------------start------------->8---
guix shell qemu -- qemu-system-i386                     \
-m 4096                                                 \
--enable-kvm                                            \
--device rtl8139,netdev=net0                            \
--netdev user,id=net0,hostfwd=tcp:0.0.0.0:11022-:2222   \
--snapshot                                              \
--no-reboot                                             \
--device virtio-serial-pci                              \
--nographic                                             \
--serial mon:stdio                                      \
--hda /gnu/store/l35gz57nrkngc923f9zp9awxmq7ha1hc-disk-image
--8<---------------cut here---------------end--------------->8---

(It also still works without using "noide", we're still using that for
childhurd service where rumpdisk does not really make sense).

Enjoy!

Greetings,
Janneke

Janneke Nieuwenhuizen (11):
  gnu: Add libpciaccess-0.17.
  gnu: hurd: Update libpciaccess to 0.17.
  gnu: Add rumpkernel.
  gnu: Add hurd-shouldbeinlibc.
  gnu: parted: Support building for the Hurd.
  gnu: hurd: Add rumpkernel.
  hurd-boot: Setup pci-arbiter and rumpdisk translators.
  services: childhurd: Bump default qemu memory to 2048MB.
  system: hurd: Boot with pci.arbiter and rumpdisk.
  bootloader: grub: Use rumpdisk-style root when booting with "noide".
  gnu: gnumach: Support "noide" argument.

 doc/guix.texi                                 |  21 ++
 gnu/bootloader/grub.scm                       |  17 +-
 gnu/build/hurd-boot.scm                       |  27 +-
 gnu/local.mk                                  |   3 +
 gnu/packages/disk.scm                         |  19 +-
 gnu/packages/hurd.scm                         | 298 ++++++++++++++++--
 .../patches/gnumach-support-noide.patch       |  25 ++
 .../patches/hurd-fix-rumpdisk-build.patch     |  41 +++
 .../patches/hurd-rumpdisk-no-hd.patch         |  26 ++
 gnu/packages/xorg.scm                         |  17 +
 gnu/services/virtualization.scm               |   4 +-
 gnu/system.scm                                |  25 +-
 gnu/system/examples/bare-hurd.tmpl            |   4 +-
 13 files changed, 480 insertions(+), 47 deletions(-)
 create mode 100644 gnu/packages/patches/gnumach-support-noide.patch
 create mode 100644 gnu/packages/patches/hurd-fix-rumpdisk-build.patch
 create mode 100644 gnu/packages/patches/hurd-rumpdisk-no-hd.patch


base-commit: 88da863e0048721cef17a5de294084be1f10e7e9
  

Hi,

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

> So, with lots of kind help from the folks at #hurd and bug-hurd, and many
> thanks to Josselin Poiret for sending their amazing "Add libc specific to Hurd
> and update components" patch set (https://issues.guix.gnu.org/63641, note that
> these series depend on this!) we now have rumpdisk support for the Hurd.

Yay!! 🎉

> To actually use it instead of the gnumach builtin IDE driver, use
>
>     (kernel-arguments '("noide"))

Should we build Mach without this driver, or otherwise make “noide” the
default?  (We can answer that later, no rush.)

> guix shell qemu -- qemu-system-i386                     \
> -m 4096                                                 \
> --enable-kvm                                            \
> --device rtl8139,netdev=net0                            \

Once we have netdde, we can probably drop that line and use the default
NIC (e1000, IIRC).

Is “-m 4096” still needed?  You mentioned this as a workaround
initially, IIRC?

Ludo’.
  

Ludovic Courtès writes:

Hi Ludo!

> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> So, with lots of kind help from the folks at #hurd and bug-hurd, and many
>> thanks to Josselin Poiret for sending their amazing "Add libc specific to Hurd
>> and update components" patch set (https://issues.guix.gnu.org/63641, note that
>> these series depend on this!) we now have rumpdisk support for the Hurd.
>
> Yay!! 🎉

Yes \o/

>> To actually use it instead of the gnumach builtin IDE driver, use
>>
>>     (kernel-arguments '("noide"))
>
> Should we build Mach without this driver, or otherwise make “noide” the
> default?  (We can answer that later, no rush.)

Possibly, I would suggest for now to follow upstream, IIUC, rumpdisk is
still somewhat experimental (and slooowww).

I expect upstream/bothstream to drop the "noide" patch at some point,
and possibly eradicate the IDE driver from gnumach altogether?

>> guix shell qemu -- qemu-system-i386                     \
>> -m 4096                                                 \
>> --enable-kvm                                            \
>> --device rtl8139,netdev=net0                            \
>
> Once we have netdde, we can probably drop that line and use the default
> NIC (e1000, IIRC).
>
> Is “-m 4096” still needed?  You mentioned this as a workaround
> initially, IIRC?

-m 2048 should be enoug, it seems boot with -m 1200; but yea, we do _do_
 need more than 1024 (or 512 even)...

Greetings,
Janneke
  

I made cosmetic suggestions (as I usually do :-)) but the whole patch
series looks great to me, thank you!

Ludo’.
  

Ludovic Courtès writes:

> I made cosmetic suggestions (as I usually do :-))

Yes, and spotted typos/buglets), so thank you for all that!

> but the whole patch
> series looks great to me, thank you!

\o/

So, good to go after https://issues.guix.gnu.org/63641 merges!
  

Hi Janneke,

Janneke Nieuwenhuizen <janneke@gnu.org> writes:

> So, good to go after https://issues.guix.gnu.org/63641 merges!

Finally got around to review and merge this + netdde as
b4fe82bd8af735000d0f2165a23cfb695c181f6f.  Sorry for the delay, and
thanks for this amazing packaging!

Best,
  

Josselin Poiret writes:

Hi hurd-team ;)

> Janneke Nieuwenhuizen <janneke@gnu.org> writes:
>
>> So, good to go after https://issues.guix.gnu.org/63641 merges!
>
> Finally got around to review and merge this + netdde as
> b4fe82bd8af735000d0f2165a23cfb695c181f6f.  Sorry for the delay, and
> thanks for this amazing packaging!

Thanks for your extensive review with patches!

Does anything have to happen for ci to pick this up?  It seems
awfully quiet here

    https://ci.guix.gnu.org/search?query=system%3Ai586-gnu

It would be nice to have hurd-team rebased, and added to ci too, but
we'll probably want to wait until i568-gnu on master is built.

Greetings,
Janneke
  

Hello fantastic hackers!  :-)

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

>> Janneke Nieuwenhuizen <janneke@gnu.org> writes:
>>
>>> So, good to go after https://issues.guix.gnu.org/63641 merges!
>>
>> Finally got around to review and merge this + netdde as
>> b4fe82bd8af735000d0f2165a23cfb695c181f6f.  Sorry for the delay, and
>> thanks for this amazing packaging!
>
> Thanks for your extensive review with patches!

Yeah, thanks to the two of you!!

> Does anything have to happen for ci to pick this up?  It seems
> awfully quiet here
>
>     https://ci.guix.gnu.org/search?query=system%3Ai586-gnu

It was apparently picked up:

  https://ci.guix.gnu.org/eval/595625

However, there seem to be regressions when cross-compiling to
i586-pc-gnu, such as <https://ci.guix.gnu.org/build/1595110/details>:

--8<---------------cut here---------------start------------->8---
gcc dlopenold.c -c -std=gnu11 -fgnu89-inline  -g -O2 -Wall -Wwrite-strings -Wundef -Werror -fmerge-all-constants -frounding-math -fno-stack-protector -fno-common -Wstrict-prototypes -Wold-style-definition -fmath-errno    -fPIC   -ftls-model=initial-exec      -I../include -I/tmp/guix-build-glibc-2.35.drv-0/build/dlfcn  -I/tmp/guix-build-glibc-2.35.drv-0/build  -I../sysdeps/unix/sysv/linux/x86_64/64  -I../sysdeps/unix/sysv/linux/x86_64  -I../sysdeps/unix/sysv/linux/x86/include -I../sysdeps/unix/sysv/linux/x86  -I../sysdeps/x86/nptl  -I../sysdeps/unix/sysv/linux/wordsize-64  -I../sysdeps/x86_64/nptl  -I../sysdeps/unix/sysv/linux/include -I../sysdeps/unix/sysv/linux  -I../sysdeps/nptl  -I../sysdeps/pthread  -I../sysdeps/gnu  -I../sysdeps/unix/inet  -I../sysdeps/unix/sysv  -I../sysdeps/unix/x86_64  -I../sysdeps/unix  -I../sysdeps/posix  -I../sysdeps/x86_64/64  -I../sysdeps/x86_64/fpu/multiarch  -I../sysdeps/x86_64/fpu  -I../sysdeps/x86/fpu  -I../sysdeps/x86_64/multiarch  -I../sysdeps/x86_64  -I../sysdeps/x86/include -I../sysdeps/x86  -I../sysdeps/ieee754/float128  -I../sysdeps/ieee754/ldbl-96/include -I../sysdeps/ieee754/ldbl-96  -I../sysdeps/ieee754/dbl-64  -I../sysdeps/ieee754/flt-32  -I../sysdeps/wordsize-64  -I../sysdeps/ieee754  -I../sysdeps/generic  -I.. -I../libio -I. -nostdinc -isystem /gnu/store/qzqq7za0mqvxgp7q8jmzd97c22xbqd18-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/include -isystem /gnu/store/qzqq7za0mqvxgp7q8jmzd97c22xbqd18-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/include-fixed -isystem /gnu/store/sjq77rvlryjgksni6ry1648xkj807ynv-linux-libre-headers-5.15.49/include -D_LIBC_REENTRANT -include /tmp/guix-build-glibc-2.35.drv-0/build/libc-modules.h -DMODULE_NAME=libc -include ../include/libc-symbols.h  -DPIC -DSHARED     -DTOP_NAMESPACE=glibc -o /tmp/guix-build-glibc-2.35.drv-0/build/dlfcn/dlopenold.os -MD -MP -MF /tmp/guix-build-glibc-2.35.drv-0/build/dlfcn/dlopenold.os.dt -MT /tmp/guix-build-glibc-2.35.drv-0/build/dlfcn/dlopenold.os
"/gnu/store/d90028hf869mppv8pspbpgbj7xmdj4h6-gnumach-headers-cross-i586-pc-gnu-1.8+git20221224/include/mach/mach_types.defs", line 60: syntax error
"/gnu/store/d90028hf869mppv8pspbpgbj7xmdj4h6-gnumach-headers-cross-i586-pc-gnu-1.8+git20221224/include/mach/mach_types.defs", line 62: syntax error
"/gnu/store/d90028hf869mppv8pspbpgbj7xmdj4h6-gnumach-headers-cross-i586-pc-gnu-1.8+git20221224/include/mach/mach_types.defs", line 63: syntax error
"/gnu/store/d90028hf869mppv8pspbpgbj7xmdj4h6-gnumach-headers-cross-i586-pc-gnu-1.8+git20221224/include/mach/mach_types.defs", line 64: syntax error
"/gnu/store/d90028hf869mppv8pspbpgbj7xmdj4h6-gnumach-headers-cross-i586-pc-gnu-1.8+git20221224/include/mach/mach_types.defs", line 65: syntax error
"/gnu/store/d90028hf869mppv8pspbpgbj7xmdj4h6-gnumach-headers-cross-i586-pc-gnu-1.8+git20221224/include/mach/mach_types.defs", line 66: syntax error
"/gnu/store/d90028hf869mppv8pspbpgbj7xmdj4h6-gnumach-headers-cross-i586-pc-gnu-1.8+git20221224/include/mach/mach_types.defs", line 67: syntax error
"/gnu/store/d90028hf869mppv8pspbpgbj7xmdj4h6-gnumach-headers-cross-i586-pc-gnu-1.8+git20221224/include/mach/mach_types.defs", line 68: syntax error
"/gnu/store/d90028hf869mppv8pspbpgbj7xmdj4h6-gnumach-headers-cross-i586-pc-gnu-1.8+git20221224/include/mach/mach_types.defs", line 69: syntax error
"/gnu/store/d90028hf869mppv8pspbpgbj7xmdj4h6-gnumach-headers-cross-i586-pc-gnu-1.8+git20221224/include/mach/mach_types.defs", line 70: syntax error
"/gnu/store/d90028hf869mppv8pspbpgbj7xmdj4h6-gnumach-headers-cross-i586-pc-gnu-1.8+git20221224/include/mach/mach_types.defs", line 263: syntax error
"/gnu/store/d90028hf869mppv8pspbpgbj7xmdj4h6-gnumach-headers-cross-i586-pc-gnu-1.8+git20221224/include/mach/mach_types.defs", line 265: syntax error
"/gnu/store/d90028hf869mppv8pspbpgbj7xmdj4h6-gnumach-headers-cross-i586-pc-gnu-1.8+git20221224/include/mach/mach_types.defs", line 266: syntax error
"/gnu/store/d90028hf869mppv8pspbpgbj7xmdj4h6-gnumach-headers-cross-i586-pc-gnu-1.8+git20221224/include/mach/mach_types.defs", line 267: type 'rpc_time_value_t' not defined
echo 'dlfcn/dladdr.o dlfcn/dladdr1.o dlfcn/dlclose.o dlfcn/dlerror.o dlfcn/dlinfo.o dlfcn/dlmopen.o dlfcn/dlopen.o dlfcn/dlsym.o dlfcn/dlvsym.o dlfcn/libc_dlerror_result.o' > /tmp/guix-build-glibc-2.35.drv-0/build/dlfcn/stamp.oT
mv -f /tmp/guix-build-glibc-2.35.drv-0/build/dlfcn/stamp.oT /tmp/guix-build-glibc-2.35.drv-0/build/dlfcn/stamp.o
make[2]: *** [Machrules:168: /tmp/guix-build-glibc-hurd-headers-cross-i586-pc-gnu-2.37.drv-0/build/mach/mach/mach_interface.__h] Error 1
make[2]: Leaving directory '/tmp/guix-build-glibc-hurd-headers-cross-i586-pc-gnu-2.37.drv-0/glibc-2.37/mach'
make[1]: *** [Makefile:484: mach/install-headers] Error 2
make[1]: Leaving directory '/tmp/guix-build-glibc-hurd-headers-cross-i586-pc-gnu-2.37.drv-0/glibc-2.37'
make: *** [Makefile:9: install-headers] Error 2
error: in phase 'install': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("install-headers") exit-status: 2 term-signal: #f stop-signal: #f> 
phase `install' failed after 8.6 seconds
command "make" "install-headers" failed with status 2
echo 'dlfcn/dladdr.os dlfcn/dladdr1.os dlfcn/dlclose.os dlfcn/dlerror.os dlfcn/dlinfo.os dlfcn/dlmopen.os dlfcn/dlopen.os dlfcn/dlsym.os dlfcn/dlvsym.os dlfcn/libc_dlerror_result.os dlfcn/dlopenold.os' > /tmp/guix-build-glibc-2.35.drv-0/build/dlfcn/stamp.osT
builder for `/gnu/store/vk634lqm1fr9bfq1w6n68x33cj1vvyv8-glibc-hurd-headers-cross-i586-pc-gnu-2.37.drv' failed with exit code 1
@ build-failed /gnu/store/vk634lqm1fr9bfq1w6n68x33cj1vvyv8-glibc-hurd-headers-cross-i586-pc-gnu-2.37.drv - 1 builder for `/gnu/store/vk634lqm1fr9bfq1w6n68x33cj1vvyv8-glibc-hurd-headers-cross-i586-pc-gnu-2.37.drv' failed with exit code 1
--8<---------------cut here---------------end--------------->8---

Thoughts?

Ludo’.
  

Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

> However, there seem to be regressions when cross-compiling to
> i586-pc-gnu, such as <https://ci.guix.gnu.org/build/1595110/details>:

That's because of the revert of the mig update in
a041bbb4bf98cce72b14c554369fc56eeacc2f5d, since it apparently caused
problems on the data service.  I don't expect anything to build without
the new mig version, but I have no idea what could be causing this
issue.  Janneke, have you ever run into such a thing when building
natively?  Maybe it's just the childhurds not having enough RAM?

Best,
  

Josselin Poiret writes:

> Hi Ludo,
>
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> However, there seem to be regressions when cross-compiling to
>> i586-pc-gnu, such as <https://ci.guix.gnu.org/build/1595110/details>:
>
> That's because of the revert of the mig update in
> a041bbb4bf98cce72b14c554369fc56eeacc2f5d, since it apparently caused
> problems on the data service.  I don't expect anything to build without
> the new mig version, but I have no idea what could be causing this
> issue.

Yes, I re-cherry-picked the mig update and put it at the bottom of
hurd-team and can confirm that

    /gnu/store/8syklh9kg8rlkalzmb2rzgj0z6fgabyr-guile-static-stripped-tarball-3.0.9

> Janneke, have you ever run into such a thing when building
> natively?  Maybe it's just the childhurds not having enough RAM?

I've got no idea; it was suggested on IRC that a circular dependency was
created and that may be true.  Like you remarked on IRC, we probably
need the commencement patches too.

We can see about the rest of hurd-team that's mostly about building guix
natively, later.

Greetings,
Janneke