mbox series

[bug#72457,00/15] Rewrite bootloader subsystem.

Message ID cover.1722741997.git.lilah@lunabee.space
Headers show
Series Rewrite bootloader subsystem. | expand

Message

Lilah Tascheter Aug. 4, 2024, 3:50 a.m. UTC
Months in the making! Requires patchset #69343 "Simplify bootloader data
structures and procedures", to be regarded less of a blocker and more of a
conjoined patchset.

This is a massive rewrite to the entire way Guix handles bootloaders. Guix
currently ad-hocs non-GRUB support on top of a layer that was designed for GRUB
and nothing else. Big features enabled by this patchset include:

* Multiple separate bootloaders! This takes over the previous multi-target
  functionality used in RAID systems, as well as the previous abuse of the GRUB
  bootloader on Raspberry Pis when trying to chainload GRUB through EFI provided
  by U-Boot.
* Multiple distinct bootloader targets! Some bootloaders need more than just a
  single target, such as p-boot, which requires both a data target and a
  configuration partition.
* Proper disk image support! Now every bootloader will be able to be dynamically
  installed to disk images, without any special support.
* Support for bootloaders without configuration-file semantics! The install
  process is now much more generalized and no longer assumes bootloaders have a
  configuration file to be installed to a static, unchangable location.
* Proper rollback support! No longer just guesses at a bootloader-configuration
  when doing rollbacks, which prevented significant configuration from being
  done.
* The ability to specify either no bootloader or a bootloader without any
  targets, for situations in which it doesn't make sense (eg qemu images and
  disk images, respectively).
* A more robust target system allowing any bootloader to be able to transform
  the user-specified target into a mount path, device-relative path, device
  file, label, or uuid.

In addition, these changes allow the uki-efi-bootloader to be added, providing
secure boot support and the ability to boot Guix from EFI without another
bootloader intermediary.

This should make adding new types of bootloaders way easier in the future as
well. Especially EFI bootloaders, since the entire EFI install process is now
abstracted away, so specific bootloaders don't have to worry about setting
efivars or the limited ESP space.



The big thing is that this definately needs to be tested. We support lots of
bootloaders for lots of hardware and I definately do not possess any of it.
Please test this and send back the results for me to revise any bugs. I've made
sure as many tests as I could run work, and that GRUB and uefi-uki-bootloader
work both in regular systems and disk images.

Lilah Tascheter (15):
  guix: scripts: Rewrite reinstall-bootloader to use provenance data.
  gnu: Add bootloader target infastructure.
  guix: scripts: Remove unused code.
  gnu: Core bootloader changes.
  gnu: system: Remove useless boot parameters.
  gnu: bootloader: Add raspberry pi bootloader.
  gnu: system: Fix bootloader crypto device recognition.
  gnu: packages: Add pesign.
  gnu: packages: Add ukify.
  gnu: packages: Add systemd-stub.
  gnu: bootloaders: Add uki-efi-bootloader.
  gnu: system: Update examples.
  doc: Update bootloader documentation.
  gnu: tests: Update tests to new targets system.
  teams: Add bootloading team.

 doc/guix.texi                                 |  458 +++---
 etc/teams.scm                                 |   10 +
 gnu/bootloader.scm                            |  644 ++++++---
 gnu/bootloader/depthcharge.scm                |  153 +-
 gnu/bootloader/extlinux.scm                   |  149 +-
 gnu/bootloader/grub.scm                       | 1279 +++++++----------
 gnu/bootloader/u-boot.scm                     |  505 +++----
 gnu/bootloader/uki.scm                        |   96 ++
 gnu/build/bootloader.scm                      |  157 +-
 gnu/build/image.scm                           |   40 +-
 gnu/build/install.scm                         |   16 +-
 gnu/image.scm                                 |    3 +
 gnu/installer/parted.scm                      |   12 +-
 gnu/local.mk                                  |    1 +
 gnu/machine/ssh.scm                           |   71 +-
 gnu/packages/bootloaders.scm                  |  277 ++--
 gnu/packages/efi.scm                          |   47 +
 gnu/packages/raspberry-pi.scm                 |   18 -
 gnu/services/virtualization.scm               |   11 +-
 gnu/system.scm                                |   61 +-
 gnu/system/boot.scm                           |   16 +-
 gnu/system/examples/asus-c201.tmpl            |    6 +-
 gnu/system/examples/bare-bones.tmpl           |    7 +-
 gnu/system/examples/bare-hurd.tmpl            |    4 +-
 gnu/system/examples/beaglebone-black.tmpl     |    6 +-
 gnu/system/examples/desktop.tmpl              |    4 +-
 gnu/system/examples/docker-image.tmpl         |    6 +-
 gnu/system/examples/lightweight-desktop.tmpl  |    4 +-
 gnu/system/examples/plasma.tmpl               |    4 +-
 .../examples/raspberry-pi-64-nfs-root.tmpl    |   23 +-
 gnu/system/examples/raspberry-pi-64.tmpl      |   18 +-
 gnu/system/examples/vm-image.tmpl             |    5 +-
 gnu/system/hurd.scm                           |    4 +-
 gnu/system/image.scm                          |  162 ++-
 gnu/system/images/hurd.scm                    |    4 +-
 gnu/system/images/novena.scm                  |    3 +-
 .../images/orangepi-r1-plus-lts-rk3328.scm    |    3 +-
 gnu/system/images/pine64.scm                  |    3 +-
 gnu/system/images/pinebook-pro.scm            |    3 +-
 gnu/system/images/rock64.scm                  |    3 +-
 gnu/system/images/unmatched.scm               |    3 +-
 gnu/system/images/visionfive2.scm             |    3 +-
 gnu/system/images/wsl2.scm                    |   14 +-
 gnu/system/install.scm                        |   43 +-
 gnu/system/vm.scm                             |   11 -
 gnu/tests.scm                                 |    4 +-
 gnu/tests/ganeti.scm                          |    4 +-
 gnu/tests/image.scm                           |    4 +-
 gnu/tests/install.scm                         |   80 +-
 gnu/tests/nfs.scm                             |    4 +-
 gnu/tests/reconfigure.scm                     |   86 +-
 gnu/tests/telephony.scm                       |    4 +-
 gnu/tests/vnc.scm                             |    4 +-
 guix/scripts/system.scm                       |  161 +--
 guix/scripts/system/reconfigure.scm           |  159 +-
 guix/ui.scm                                   |    8 +
 tests/boot-parameters.scm                     |   16 +-
 57 files changed, 2371 insertions(+), 2533 deletions(-)
 create mode 100644 gnu/bootloader/uki.scm


base-commit: 7d781027c78bdea5fdb3f1c9c9ec432b9606d2b5

Comments

Sergey Trofimov Aug. 4, 2024, 8:53 a.m. UTC | #1
Lilah Tascheter <lilah@lunabee.space> writes:

> The big thing is that this definately needs to be tested. We support lots of
> bootloaders for lots of hardware and I definately do not possess any of it.
> Please test this and send back the results for me to revise any bugs. I've made
> sure as many tests as I could run work, and that GRUB and uefi-uki-bootloader
> work both in regular systems and disk images.
>

I've built a new system generation, but it didn't apply properly,
showing this error:

--8<---------------cut here---------------start------------->8---
   196:43  4 (_ #(#(#(#(#(#(#(#(#(#(#(#(#<directory (gnu bootloader grub) 7fa9e0f99dc0>) install (#<<bootloader-target> type: vendir expected?: #f path: "EFI/Guix" offset: #<procedure offset (a)> device: #f file-system: #f label: #f uuid: #f> #<<bootloader-target> type: install expected?: #f path: "boot" offset: #<procedure offset (a)> device: "/dev/sda1" file-system: "vfat" label: "GNU-ESP" uuid: #vu8(77 160 163 107)> #<<bootloader-target> type: root expected?: #f path: "/" offset: #<procedure offset (a)> device: "/dev/sda2" file-system: "ext4" label: "Guix_image" uuid: #vu8(246 188 138 216 255 26 23 84 83 48 172 24 246 188 138 216)> #<<bootloader-target> type: esp expected?: #f path: "/boot" offset: #<procedure offset (a)> device: "/dev/sda1" file-system: "vfat" label: "GNU-ESP" uuid: #vu8(77 160 163 107)>) "root" #<gexp port /storage/devel/ext/guix/gnu/bootloader/grub.scm:221:54 7fa9e2a11e40>) #<variable 7fa9e4b2b3a0 value: #<procedure form (a b)>>) (…)) #) #) #) #) #) #) #) #))
In gnu/system/uuid.scm:
    89:19  3 (dce-uuid->string #vu8(77 160 163 107))
In unknown file:
           2 (bytevector-uint-ref #vu8(77 160 163 107) 4 big 2)
In ice-9/boot-9.scm:
  1685:16  1 (raise-exception _ #:continuable? _)
  1685:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
In procedure bytevector-uint-ref: Value out of range: 4
--8<---------------cut here---------------end--------------->8---


Relevant config part: 
--8<---------------cut here---------------start------------->8---
  (bootloader
   (bootloader-configuration
    (bootloader grub-efi-bootloader)
    (targets (list (bootloader-target
                    (type 'esp)
                    (path "/boot"))))))

  (file-systems
   (cons* (file-system
            (mount-point "/")
            (device (file-system-label "Guix_image"))
            (type "ext4"))
          (file-system
            (mount-point "/boot")
            (device (file-system-label "GNU-ESP"))
            (type "vfat"))
          %base-file-systems))
--8<---------------cut here---------------end--------------->8---

Filesystem info:
--8<---------------cut here---------------start------------->8---
$ lsblk -o name,fstype,uuid,label,partuuid,partlabel
NAME   FSTYPE UUID                                 LABEL      PARTUUID                             PARTLABEL
sda                                                                                                
├─sda1 vfat   6BA3-A04D                            GNU-ESP    1ec0ace3-d9b3-4254-b9f4-c4aa21e572cd GNU-ESP
└─sda2 ext4   f6bc8ad8-ff1a-1754-5330-ac18f6bc8ad8 Guix_image 6dfb79bf-9ec3-4722-91f8-b8e08b8c52a0 Guix_image
--8<---------------cut here---------------end--------------->8---

Should PARTUUID be used instead of fs UUID? I wasn't able to quickly locate the
place where to make this change.
Sergey Trofimov Aug. 4, 2024, 9:19 a.m. UTC | #2
Sergey Trofimov <sarg@sarg.org.ru> writes:

> Lilah Tascheter <lilah@lunabee.space> writes:
>
>> The big thing is that this definately needs to be tested. We support lots of
>> bootloaders for lots of hardware and I definately do not possess any of it.
>> Please test this and send back the results for me to revise any bugs. I've made
>> sure as many tests as I could run work, and that GRUB and uefi-uki-bootloader
>> work both in regular systems and disk images.
>>
>
> I've built a new system generation, but it didn't apply properly,
> showing this error:
>
I solved it by explicitly specifying uuid in bootloader-target: `(uuid "6BA3-A04D" 'fat)`.

Another issue is that `./pre-inst-env guix system list-generations` fails to print
older generations. Older guix prints fine even generations created with
the new bootloader code.

--8<---------------cut here---------------start------------->8---
    619:8  3 (_ #(#(#(#(#(#(#(#(#(#(#(#(#<directory (guix scripts system) 7fcc08817b40>) 120 "/var/guix/profiles/system" #f) #<variable 7fcbef
8cd0 value: #<procedure display-channel (a)>>) "/var/guix/profiles/system-120-link") #<<boot-parameters> label: "GNU with Linux 6.9.7" root-de
ce: #<file-system-label "Guix_image"> bootloader-name: (grub-efi) store-device: #<file-system-label "Guix_image"> store-mount-point: "/" store
irectory-prefix: #f store-crypto-devices: () locale: "en_GB.UTF-8" kernel: "/gnu/store/c2xsizqd0rw8jxmrfvvwrvzwmb75r028-linux-6.9.7/bzImage" k
nel-arguments: ("root=Guix_image" #<gexp (string-append (if #<gexp-input #t:out> "gnu.system=" "--system=") #<gexp-input "/var/guix/profiles/s
tem-120-link":out>) /storage/devel/ext/guix/gnu/system/boot.scm:299:11 7fcbee836150> #<gexp (string-append (if #<gexp-input #t:out> "gnu.load=
"--load=") #<gexp-input "/var/guix/profiles/system-120-link":out> "/boot") /storage/devel/ext/guix/gnu/syste…>) #) #) #) #) #) #) …))
    163:9  2 (_ #(#(#(#(#(#(#(#(#(#(#(#(#<directory (guix scripts system) 7fcc08817b40>) 120 "/var/guix/profiles/system" #f) #<variable 7fcbef
8cd0 value: #<procedure display-channel (a)>>) "/var/guix/profiles/system-120-link") #<<boot-parameters> label: "GNU with Linux 6.9.7" root-de
ce: #<file-system-label "Guix_image"> bootloader-name: (grub-efi) store-device: #<file-system-label "Guix_image"> store-mount-point: "/" store
irectory-prefix: #f store-crypto-devices: () locale: "en_GB.UTF-8" kernel: "/gnu/store/c2xsizqd0rw8jxmrfvvwrvzwmb75r028-linux-6.9.7/bzImage" k
nel-arguments: ("root=Guix_image" #<gexp (string-append (if #<gexp-input #t:out> "gnu.system=" "--system=") #<gexp-input "/var/guix/profiles/s
tem-120-link":out>) /storage/devel/ext/guix/gnu/system/boot.scm:299:11 7fcbee836150> #<gexp (string-append (if #<gexp-input #t:out> "gnu.load=
"--load=") #<gexp-input "/var/guix/profiles/system-120-link":out> "/boot") /storage/devel/ext/guix/gnu/syste…>) #) #) #) #) #) #) …))
In unknown file:
           1 (string-join (grub-efi) #<undefined> #<undefined>)
In ice-9/boot-9.scm:
  1685:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
In procedure string-append: Wrong type (expecting string): grub-efi
--8<---------------cut here---------------end--------------->8---
Herman Rimm Sept. 12, 2024, 6:08 p.m. UTC | #3
Hello all,

I revised [PATCH v5 01/15], see issue #73202.  I had already begun
making similar changes to the other patches, have a look at [1].  I
rebase it often, and in that case you may find 'git fetch [remote for
[1]] bootloader', 'git reset [remote for [1]]/bootloader' and 'git
restore -p' helpful to keep track with a local branch.  If you have any
(small) changes feel free to send me a patch or repository URL.

Also, a compatibility macro would be worth looking into, see for example
origin-compatibility-helper in (guix packages).  A macro for the targets
field of bootloader-configuration could use each string as a path for
bootloader-target and make an educated guess for the type field based on
that path, or the bootloader itself.  Would that leave your existing
configuration(s) intact?  Anyway, I don't plan to work on that this
week, so feel free to.

Cheers,
Herman

[1]: https://codeberg.org/herman_rimm/guix/src/branch/bootloader
Herman Rimm Sept. 13, 2024, 7:56 a.m. UTC | #4
Hello,

On Thu, Sep 12, 2024 at 08:08:50PM +0200, Herman Rimm wrote:
> Also, a compatibility macro would be worth looking into, see for example
> origin-compatibility-helper in (guix packages).  A macro for the targets
> field of bootloader-configuration could use each string as a path for
> bootloader-target and make an educated guess for the type field based on
> that path, or the bootloader itself.

So it would make more sense to use a field sanitizer to do that, and
looking at warn-update-targets in (gnu bootloader), it already does
that.  Sorry for the noise.

Cheers,
Herman
Herman Rimm Sept. 15, 2024, 9:11 a.m. UTC | #5
Hi,

I dislike that so few patches change from revision to revision,
according to the diffstat.  I like the ratio of discussion to patch
submission messages in #68524 more, and I think splitting the patches
over multiple series will result in fewer patch submissions overall and
discussion that is relevant to every patch in a series.

I would like to submit a rewritten patch series.  Basically, it would
consist of patches #4, #6, #12, #13, and #14.  I want to submit #1, #2,
#3, #5 and #15 to issue #73202, and #7, #8, #9, #10 (excl. efibootmgr)
and #11 to #68524.

Parts of patch #4 which fit better with #73202 or function standalone
would be submitted to #73202.  Finally, #4 will be split into seven to
ten patches, hopefully making referring to changes easier and review
less demanding.

Please let me know what you think about the proposed patch series.

By the way, assuming 'herman' is the remote for [1], you can see the
current changes of my rewrite by running: git diff herman/bootloader-v5
herman/uki-efi-bootloader.

Aside from that, %test-installed-os is broken for (at least) the v5
patch series.  It results in [2], after having fixed
guix/scripts/system.scm like so:

      (mlet* %store-monad
             ((alt -> (generation->boot-alternative profile 1))
              (inst (apply install-bootloader local-eval bootloaders
-                     (list alt) #:dry-run (not install-bootloader?)
+                     (list alt) #:dry-run? (not install-bootloader?)
                      #:root-offset target bootmeta)))
        (maybe-copy (derivation->output-path inst)))

The steps I took to get [2] are: create channels.scm for my bootloader
branch, git checkout the branch, append %test-installed-os to
gnu/tests/install.scm, and run guix time-machine -C channels.scm --
build -f gnu/tests/install.scm.

On master (or issue #69343) it seems grub.cfg is built together with the
system and provenance derivations, before 'initializing operating system
under /mnt'.  If anything, let me know if you fail to reproduce [2] with
the fixed v5 series or on my branch.  I will try fix [2] next weekend,
so feel free to look into it this week.

Cheers,
Herman

[1]: https://codeberg.org/herman_rimm/guix
[2]:
...
guix (GNU Guix) 1022e6330f17bd5fa2cddbc29c7f9bebb9c1fe0e
Copyright (C) 2024 the Guix authors
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
+ export GUIX_BUILD_OPTIONS=--no-grafts
+ GUIX_BUILD_OPTIONS=--no-grafts
+ guix build isc-dhcp
/gnu/store/xai38dx802al26bnxcb788y027r5y7m5-isc-dhcp-4.4.3-P1
[1m[0m+ parted --script /dev/vdb mklabel gpt mkpart primary ext2 1M 3M mkpart primary ext2 3M 1.6G set 1 boot on set 1 bios_grub on
[   62.769814]  vdb: vdb1 vdb2
+ mkfs.ext4 -L my-root /dev/vdb2
mke2fs 1.47.0 (5-Feb-2023)
Discarding device blocks:      0/389888             done                            
Creating filesystem with 389888 4k blocks and 97536 inodes
Filesystem UUID: 043cc525-a895-466f-9265-b5aeea921454
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912

Allocating group tables:  0/12     done                            
Writing inode tables:  0/12     done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information:  0/12     done

+ mount /dev/vdb2 /mnt
[   63.052091] EXT4-fs (vdb2): mounted filesystem 043cc525-a895-466f-9265-b5aeea921454 r/w with ordered data mode. Quota mode: none.
+ df -h /mnt
Filesystem      Size  Used Avail Use% Mounted on
/dev/vdb2       1.5G  404K  1.4G   1% /mnt
+ herd start cow-store /mnt
Service user-homes has been started.
Starting service cow-store...
Service cow-store has been started.
+ mkdir /mnt/etc
+ cp /etc/target-config.scm /mnt/etc/config.scm
+ guix system init /mnt/etc/config.scm /mnt --no-substitutes
[1;35mThe following derivations will be built:
[0m  [2m/gnu/store/gs0rca6bwf8q6qd3x1k01bsp55g2i3hr-[0msystem.drv
  [2m/gnu/store/xvzfxgrsjch583l28davaz2wi80nzvkj-[0mprovenance.drv

[1m[0m
[Kbuilding /gnu/store/xvzfxgrsjch583l28davaz2wi80nzvkj-provenance.drv...

[K
[Kbuilding /gnu/store/gs0rca6bwf8q6qd3x1k01bsp55g2i3hr-system.drv...

[K/gnu/store/77pr7kifgjypha2hy0r1yxwmcg8aq9jn-system

initializing operating system under '/mnt'...
populating '/mnt'...
[1;35mThe following derivations will be built:
[0m  [2m/gnu/store/zvn5lrkaxfv4nj60v8h1fqqy4g1akybh-[0mmodule-import.drv
  [2m/gnu/store/k93g3ifqka252zb72kl433wh7fw8nicq-[0mmodule-import-compiled.drv
  [2m/gnu/store/1sgmdnq6zpdsmycbmssd7980fs8y45s8-[0minstall-bootloader.scm.drv
  [2m/gnu/store/5r4h565d6i8kx1ym0w2ynnzidggk0yg1-[0mmodule-import.drv
  [2m/gnu/store/6pmgvmr1hwj7pz7k4nxl899rjpn5g259-[0mmodule-import-compiled.drv
  [2m/gnu/store/hdv9h1ncyadn4k3vl0ls5mlzh75d16ka-[0mcore.img.drv
  [2m/gnu/store/3ik7z6x0kd140rdzsp9g6kd8dpk22syp-[0mcore.cfg.drv
  [2m/gnu/store/ln9848cyw0dkvnihvxkxbs05sakqd8la-[0mgrub.dir.drv
  [2m/gnu/store/3vl0qggja79733r1dsc5mjfj888cj13y-[0mgrub.cfg.drv
  [2m/gnu/store/a6vnyj0s34w9ad962h8g2g98ihkql7zm-[0mgrub-image.png.drv
  [2m/gnu/store/53fady56y14p4y0a7aw8rypc3qwbb1xs-[0mguile-cairo-1.11.2.drv
  [2m/gnu/store/09cwfzjgg02zcf66361qhxsfcanim7zl-[0mmake-4.4.1.drv
  [2m/gnu/store/09kis2ig2xk8xrgwzh2y42ya0jwkmc4a-[0mbzip2-boot0-1.0.8.drv
  [2m/gnu/store/0vnv3n7vllwb4672rxsd6xkfx50lz2is-[0mxz-mesboot-5.4.5.drv

... 

[1m[0m
[Kbuilding /gnu/store/zvn5lrkaxfv4nj60v8h1fqqy4g1akybh-module-import.drv...

[K
[Kbuilding /gnu/store/hx1w3j5icxgrhjhvkvs8cc00iag88cay-CPAN-Meta-2.150010.tar.gz.drv...
[K\
[K|

...

[K/builder for `/gnu/store/hx1w3j5icxgrhjhvkvs8cc00iag88cay-CPAN-Meta-2.150010.tar.gz.drv' failed to produce output path `/gnu/store/vbcdsc51aypfxaw6nf2qcdn3kyfx4ixb-CPAN-Meta-2.150010.tar.gz'

[K[31;1mbuild of /gnu/store/hx1w3j5icxgrhjhvkvs8cc00iag88cay-CPAN-Meta-2.150010.tar.gz.drv failed[0m
[1mView build log at '/var/log/guix/drvs/hx/1w3j5icxgrhjhvkvs8cc00iag88cay-CPAN-Meta-2.150010.tar.gz.drv.gz'.[0m

[Kbuilding /gnu/store/1498qsnrk07npvl4sblvlrpc1nhssjj9-CPAN-Meta-Requirements-2.140.tar.gz.drv...
cannot build derivation `/gnu/store/6lxv0agdrs4xfg4kjw4fakmsdsy83l5i-perl-cpan-meta-2.150010.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/acys5h6lbbrf66jssw3081saxc9g1mn4-perl-parse-cpan-meta-2.150010.drv': 1 dependencies couldn't be built

[Kbuilding /gnu/store/qnrf3grnxbb3z9a05jd77lbjry7xk5mv-Cython-0.29.32.tar.gz.drv...
cannot build derivation `/gnu/store/5qnifhvpm2zlm7wic44z06qih5mw337h-perl-test-pod-1.52.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/wikxyp9h4a21j3k0wvpl8pq0wzfz4zmd-perl-unicode-utf8-0.62.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/j0bv34d7z1wi0w79ggvmrq71yjklv46x-perl-path-tiny-0.118.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/4064nhqy89x4va9dryq9mi5fprp8b61d-perl-xml-xpath-1.48.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/103sn6krasxcca0wm4s1w4lhvqhj8j44-docbook-xsl-1.79.2-0.fe16c90.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/c8pfxw2akqj6za5vs1xdhci1zvjfjiji-vala-0.56.16.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/68i1misqx2w581cylh3hzkr7v6a0hws7-librsvg-2.56.4.drv': 1 dependencies couldn't be built

[Kbuilding /gnu/store/83y4zf06iqv3xjz3pjzhmdim76bs53p2-Python-3.10.7.tar.xz.drv...
cannot build derivation `/gnu/store/6csl2268yhidg8vddy6nb32623fxkypk-guile-rsvg-2.18.1-0.05c6a2f.drv': 1 dependencies couldn't be built

[Kbuilding /gnu/store/3ik7z6x0kd140rdzsp9g6kd8dpk22syp-core.cfg.drv...
cannot build derivation `/gnu/store/a6vnyj0s34w9ad962h8g2g98ihkql7zm-grub-image.png.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/ln9848cyw0dkvnihvxkxbs05sakqd8la-grub.dir.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/1sgmdnq6zpdsmycbmssd7980fs8y45s8-install-bootloader.scm.drv': 1 dependencies couldn't be built
guix system: [1;31merror: [0mbuild of `/gnu/store/1sgmdnq6zpdsmycbmssd7980fs8y45s8-install-bootloader.scm.drv' failed
environment variable `PATH' set to `/gnu/store/h6n705ghnl1qi7p4xm7z796nx7cl7dv0-qemu-minimal-8.2.2/bin'
QEMU runs as PID 22
connected to QEMU's monitor
read QEMU monitor prompt
connected to guest REPL
marionette is ready

;;; (uname #("Linux" "gnu" "6.10.7-gnu" "#1 SMP PREEMPT_DYNAMIC 1" "x86_64"))
Lilah Tascheter Sept. 17, 2024, 10:20 p.m. UTC | #6
> I dislike that so few patches change from revision to revision,
> according to the diffstat.
yeah, didn't feel the best to have to re-send the whole series for each
small change.

> I would like to submit a rewritten patch series.  Basically, it would
> consist of patches #4, #6, #12, #13, and #14.
since efi's getting split out, would it make sense to split #6 out too?
though, that could pose issues if it gets forgotten and the others get
merged. your choice!

> I want to submit #1, #2, #3, #5 and #15 to issue #73202
so, #73202'd end up being a general cleanup of the current bootloader
system, right? I feel #2 wouldn't quite fit there, seeing as it just
adds the infastructure needed for #4.

> #7, #8, #9, #10 (excl. efibootmgr) and #11 to #68524.
I'll send an unmerge to #68524 then!

> Parts of patch #4 which fit better with #73202 or function standalone
> would be submitted to #73202.  Finally, #4 will be split into seven
> to ten patches, hopefully making referring to changes easier and
> review less demanding.
this sounds great! but,
good fucking luck splitting up #4. a ton of the changes are
interconnected, and it'll be a pain to do so if you don't want some
commits to just not compile. if you can pull it off, that'd be amazing!

> [1]: https://codeberg.org/herman_rimm/guix
typo in gnu/build/bootloader.scm "thtat", also
gnu/system/install.scm(embedded-installation-os) operating-system-
bootloader's default is '(), not #f. otherwise, this looks great!!!

I'm also thinking now, since you mentioned the operating-system-
bootloader sanitizer in a previous email, it'd probably be a good idea
to expand the sanitizer to detect for 'part type targets too. a simple
/dev/.*[0-9] regex should work well? I can write a quick patch up for
you, or you can just include that when making the new patch series if
you'd prefer?

> [2]: ...
looks like the failure here was caused by cpan failing to build, which
shouldn't be (hopefully isn't) a result of this patchset. what does the
build log mentioned say?

thanks so much for all the help, by the way :)
- lilah
Herman Rimm Sept. 19, 2024, 3:35 p.m. UTC | #7
On Tue, Sep 17, 2024 at 05:20:05PM -0500, Lilah Tascheter wrote:
> > I would like to submit a rewritten patch series.  Basically, it would
> > consist of patches #4, #6, #12, #13, and #14.
> since efi's getting split out, would it make sense to split #6 out too?
> though, that could pose issues if it gets forgotten and the others get
> merged. your choice!
I would rather not adapt the existing Raspberry Pi bootloader to the new
system.

> > I want to submit #1, #2, #3, #5 and #15 to issue #73202
> so, #73202'd end up being a general cleanup of the current bootloader
> system, right? I feel #2 wouldn't quite fit there, seeing as it just
> adds the infastructure needed for #4.
#73202 is also preparation for this issue.  I think #2 is big enough for
it to reviewed on its own outside of this issue, and that #73202 is
small enough to fit #2.

> > #7, #8, #9, #10 (excl. efibootmgr) and #11 to #68524.
> I'll send an unmerge to #68524 then!
Thanks. 

> > Parts of patch #4 which fit better with #73202 or function standalone
> > would be submitted to #73202.  Finally, #4 will be split into seven
> > to ten patches, hopefully making referring to changes easier and
> > review less demanding.
> this sounds great! but,
> good fucking luck splitting up #4. a ton of the changes are
> interconnected, and it'll be a pain to do so if you don't want some
> commits to just not compile. if you can pull it off, that'd be amazing!
I don't intend for the commits to compile.  It's to adequately describe
the changes while fitting each commit message on a monitor and to give
reviewers the choice of squashing them together into one working commit.

> > [1]: https://codeberg.org/herman_rimm/guix
> typo in gnu/build/bootloader.scm "thtat", also
> gnu/system/install.scm(embedded-installation-os) operating-system-
> bootloader's default is '(), not #f. otherwise, this looks great!!!
Fixed, thanks.

> I'm also thinking now, since you mentioned the operating-system-
> bootloader sanitizer in a previous email, it'd probably be a good idea
> to expand the sanitizer to detect for 'part type targets too. a simple
> /dev/.*[0-9] regex should work well? I can write a quick patch up for
> you, or you can just include that when making the new patch series if
> you'd prefer?
I had stashed the changes I made to warn-update-targets.  I do try to
create branches and fixup commits instead to better track changes.
Anyway, maybe you can send a diff based on/relative to:

;; Based on report-duplicate-field-specifier from (guix records).
(define (report-duplicate-type-field targets)
  "Report the first target with duplicate type among TARGETS."
  (let loop ((targets targets)
             (seen    '()))
    (match targets
      ((target rest ...)
       (let (type (bootloader-target-type target)))
         (when (memq type seen)
           (error loc (G_ "target with duplicate type~%") duplicate))
         (loop rest (cons type seen)))
      (() #t))))

(define-with-syntax-properties (warn-update-targets (targets properties))
  (let ((targets (if (list? targets) targets (list targets)))
        (loc (source-properties->location properties)))
    (define string->target
      (match-lambda
        ((? bootloader-target? target) target)
        ((? string? s) (if (string-prefix? "/dev" s)
                           (bootloader-target
                             (type 'disk)
                             (device s))
                           (bootloader-target
                             (type 'esp)
                             (offset 'root)
                             (path s))))
        (x (error loc (G_ "invalid target '~a'~%") x))))

    ;; XXX: Should this be an error?
    (when (any string? targets)
      (warning loc (G_ "the 'targets' field should now contain \
<bootloader-target> records, inferring a best guess, this might break!~%")))
    (let* ((targets (map string->target targets)))
      (report-duplicate-type-field targets)
      targets)))

> > [2]: ...
> looks like the failure here was caused by cpan failing to build, which
> shouldn't be (hopefully isn't) a result of this patchset. what does the
> build log mentioned say?
I think because the build log was on a VFS I couldn't access it
directly.  But why do these packages need to be built at all?

> thanks so much for all the help, by the way :)
No problem.

Cheers,
Herman
Lilah Tascheter Sept. 20, 2024, 4:56 a.m. UTC | #8
> But why do these packages need to be built at all?
they're dependencies of guile-rsvg, which is pulled in by grub in order
to convert the guix svg logo to a png for the configured screen
resolution.

it looks like there's been some discussion on guix-devel on the topic,
under "%base-packages and default grub theme depend on rust" in january
and february. in the meantime, the workaround vagrantc suggested should
still work.

...this issue needs more attention
amano.kenji Oct. 18, 2024, 10:36 a.m. UTC | #9
Does this allow me to run some services whenever there are changes to /boot?

For example, I want to record file size and modification time for each file in /boot whenever there are changes to /boot. Or, I can record hashes of /boot files.

During boot, guix can check whether /boot hasn't changed from the record.

Combined with encrypted root, it can detect tampering attempts.
amano.kenji Oct. 19, 2024, 1:38 a.m. UTC | #10
https://slimbootloader.github.io/security/verified-boot.html says

> A hash function is used to create a digest during build and saved as part of the image which is then used to compare against the digest computed during boot to make sure they are the same. The digest calculated during build and saved as part of the image is trusted as its part of the trust chain.

> This method is used to verify components for which the digest can be computed during SBL build time.

> Signature verification

> This method of verification is used for independently updateable components like configuration data, IP firmware blobs, OS images, etc.

I wish this rewrite of bootloader subsystem allows the possibility of verified boot which doesn't have to be implemented now. Just make it possible to run services whenever there are changes to /boot.
amano.kenji Oct. 21, 2024, 8:32 a.m. UTC | #11
After further research, I concluded that it is better to have different bootloaders for creating hashes or gpg signatures instead of exposing service hooks.

Heads firmware just parses grub.cfg for a list of kernel images with initrd and shows kernel images on boot.

Thus, gnu guix can have `heads` bootloader which produces grub.cfg and copies kernel images and initrd to /boot.

`heads-signed` bootloader can sign all files in /boot with a gpg private key.

Heads is an alternative to BIOS and UEFI. It is an open-source firwmare for (coreboot) motherboards.