From patchwork Wed Sep 7 12:46:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Othacehe X-Patchwork-Id: 42268 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 271CC27BBEA; Wed, 7 Sep 2022 13:49:23 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id AEEA027BBE9 for ; Wed, 7 Sep 2022 13:49:21 +0100 (BST) Received: from localhost ([::1]:39244 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oVuUa-00077L-RT for patchwork@mira.cbaines.net; Wed, 07 Sep 2022 08:49:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44856) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oVuTM-0005g8-Eh for guix-patches@gnu.org; Wed, 07 Sep 2022 08:48:05 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:36975) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oVuTM-000781-6G for guix-patches@gnu.org; Wed, 07 Sep 2022 08:48:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1oVuTM-0002YM-2C for guix-patches@gnu.org; Wed, 07 Sep 2022 08:48:04 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#57643] [PATCH 3/3] doc: Add a "System Images" chapter. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 07 Sep 2022 12:48:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 57643 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 57643@debbugs.gnu.org Cc: Mathieu Othacehe Received: via spool by 57643-submit@debbugs.gnu.org id=B57643.16625548479670 (code B ref 57643); Wed, 07 Sep 2022 12:48:03 +0000 Received: (at 57643) by debbugs.gnu.org; 7 Sep 2022 12:47:27 +0000 Received: from localhost ([127.0.0.1]:53890 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oVuSk-0002Vt-I1 for submit@debbugs.gnu.org; Wed, 07 Sep 2022 08:47:27 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49602) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oVuSh-0002VK-Vd for 57643@debbugs.gnu.org; Wed, 07 Sep 2022 08:47:25 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:59976) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oVuSc-000725-Bm for 57643@debbugs.gnu.org; Wed, 07 Sep 2022 08:47:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=N2k/QKmnLGt2x1CmCmEIfmq1in10BBZALhX4RcaUqkQ=; b=Z0lX6fsCINcoAnqvmmYf 5b8yUx73x5/Pl1b4xa6z4UHscj61Hp7umG/X3HBfyo+4xIfNTNxZutYABcwGD7rNZk/vPfqb5s4Vh XANriGYOtAmNy/Yq3DVOsqIbfG5+/3k1lNyeOSIu1eHVX2RXJKMPQ3CHzmgChCojpjVDIC2y/j06a DyBH7AwJDf2JhVe3ooZVjKcA6FUqTHBaSdXvfKXZavdT89YZ2b2n1ncr2dxAIPtxI+GB9kezYp1QB ui1/Kdz9CV8L5BbYmu1TCkA1p/F0vHWjAoSgKvEygqPauQyVXu4TTQXJzi9fbpTTu0Q0iw7MlfaPS Hoi3Al3yPYz1sw==; Received: from ppp079167215024.access.hol.gr ([79.167.215.24]:46736 helo=localhost.localdomain) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oVuSX-0002jZ-Py; Wed, 07 Sep 2022 08:47:14 -0400 From: Mathieu Othacehe Date: Wed, 7 Sep 2022 14:46:33 +0200 Message-Id: <20220907124633.17013-3-othacehe@gnu.org> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20220907124633.17013-1-othacehe@gnu.org> References: <20220907124633.17013-1-othacehe@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * doc/guix.texi ("System Images"): New chapter. --- doc/guix.texi | 498 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 498 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index a24278e431..6a5824d4ab 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -183,6 +183,7 @@ Weblate} (@pxref{Translating Guix}). * Home Configuration:: Configuring the home environment. * Documentation:: Browsing software user manuals. * Platforms:: Defining platforms. +* System Images:: Creating system images. * Installing Debugging Files:: Feeding the debugger. * Using TeX and LaTeX:: Typesetting. * Security Updates:: Deploying security fixes quickly. @@ -411,6 +412,13 @@ Platforms * platform Reference:: Detail of platform declarations. * Supported Platforms:: Description of the supported platforms. +System Images + +* image Reference:: Detail of image declarations. +* Instantiate an Image:: How to instantiate an image record. +* image-type Reference:: Detail of image types declaration. +* Image Modules:: Definition of image modules. + Installing Debugging Files * Separate Debug Info:: Installing 'debug' outputs. @@ -41241,6 +41249,496 @@ Platform targeting x86 64 bits CPUs running WIN32. Platform targeting x86 CPUs running GNU/Hurd. @end defvr +@node System Images +@chapter Creating System Images. + +@cindex system images +When it comes to installing Guix System for the first time on a new +machine, you can basically proceed in three different ways. The first +one is to use an existing operating system on the machine to run the +@command{guix system init} command (@pxref{Invoking guix system}). The +second one, is to produce an installation image (@pxref{Building the +Installation Image}). This is a bootable system which role is to +eventually run @command{guix system init}. Finally, the third option +would be to produce an image that is a direct instantiation of the +system you wish to run. That image can then be copied on a bootable +device such as an USB drive or a memory card. The target machine would +then directly boot from it, without any kind of installation procedure. + +The @command{guix system image} command is able to turn an operating +system definition into a bootable image. This command supports +different image types, such as @code{efi-raw}, @code{iso9660} and +@code{docker}. Any modern @code{x86_64} machine will probably be able +to boot from an @code{iso9660} image. However, there are a few machines +out there that require specific image types. Those machines, in general +using @code{ARM} processors, may expect specific partitions at specific +offsets. + +This chapter explains how to define customized system images and how to +turn them into actual bootable images. + +@menu +* image Reference:: Detail of image declarations. +* Instantiate an Image:: How to instantiate an image record. +* image-type Reference:: Detail of image types declaration. +* Image Modules:: Definition of image modules. +@end menu + +@node image Reference +@section @code{image} Reference + +The @code{image} record, described right after, allows you to define a +customized bootable system image. + +@deftp {Data Type} image +This is the data type representing a system image. + +@table @asis +@item @code{name} (default: @code{#false}) +The image name as a symbol, @code{'my-iso9660} for instance. The name +is optional and it defaults to @code{#false}. + +@item @code{format} +The image format as a symbol. The following formats are supported: + +@itemize +@item @code{disk-image}, a raw disk image composed of one or multiple +partitions. + +@item @code{compressed-qcow2}, a compressed qcow2 image composed of +one or multiple partitions. + +@item @code{docker}, a Docker image. + +@item @code{iso9660}, an ISO-9660 image. + +@end itemize + +@item @code{platform} (default: @code{#false}) +The @code{platform} record the image is targeting (@pxref{Platforms}), +@code{aarch64-linux} for instance. By default, this field is set to +@code{#false} and the image will target the host platform. + +@item @code{size} (default: @code{'guess}) +The image size in bytes or @code{'guess}. The @code{'guess} symbol, +which is the default, means that the image size will be inferred based +on the image content. + +@item @code{operating-system} +The image's @code{operating-system} record that is instanciated. + +@item @code{partition-table-type} (default: @code{'mbr}) +The image partition table type as a symbol. Possible values are +@code{'mbr} and @code{'gpt}. It default to @code{'mbr}. + +@item @code{partitions} (default: @code{'()}) +The image partitions as a list of @code{partition} records +(@pxref{partition Reference}). + +@item @code{compression?} (default: @code{#true}) +Whether the image content should be compressed, as a boolean. It +defaults to @code{#true} and only applies to @code{'iso9660} image +formats. + +@item @code{volatile-root?} (default: @code{#true}) +Whether the image root partition should be made volatile, as a boolean. + +This is achieved by using a RAM backed file system (overlayfs) that is +mounted on top of the root partition by the initrd. It defaults to +@code{#true}. When set to @code{#false}, the image root partition is +mounted as read-write partition by the initrd. + +@item @code{shared-store?} (default: @code{#false}) +Whether the image's store should be shared with the host system, as a +boolean. This can be useful when creating images dedicated to virtual +machines. When set to @code{#false}, which is the default, the image's +@code{operating-system} closure is copied to the image. Otherwise, when +set to @code{#true}, it is assumed that the host store will be made +available at boot, using a @code{9p} mount for instance. + +@item @code{shared-network?} (default: @code{#false}) +Whether to use the host network interfaces within the image, as a +boolean. This is only used for the @code{'docker} image format. It +defaults to @code{#false}. + +@item @code{substitutable?} (default: @code{#true}) +Whether the image derivation should be substitutable, as a boolean. It +defaults to @code{true}. + +@end table +@end deftp + +@node partition Reference +@subsection @code{partition} Reference + +In @code{image} record may contain some partitions. + +@deftp {Data Type} partition +This is the data type representing an image partition. + +@table @asis +@item @code{size} (default: @code{'guess}) +The partition size in bytes or @code{'guess}. The @code{'guess} symbol, +which is the default, means that the partition size will be inferred +based on the partition content. + +@item @code{offset} (default: @code{0}) +The partition's start offset in bytes, relative to the image start or +the previous partition end. It defaults to @code{0} which means that +there is no offset applied. + +@item @code{file-system} (default: @code{"ext4"}) +The partition file system as a string, defaulting to @code{"ext4"}. The +supported values are @code{"vfat"}, @code{"fat16"}, @code{"fat32"} and +@code{"ext4"}. + +@item @code{file-system-options} (default: @code{'()}) +The partition file system creation options that should be passed to the +partition creation tool, as a list of strings. This is only supported +when creating @code{"ext4"} partitions. + +See the @code{"extended-options"} man page section of the +@code{"mke2fs"} tool for a more complete reference. + +@item @code{label} +The partition label as a mandatory string, @code{"my-root"} for +instance. + +@item @code{uuid} (default: @code{#false}) +The partition UUID as an @code{uuid} record (@pxref{File Systems}). By +default it is @code{#false}, which means that the partition creation +tool will attribute a random UUID to the partition. + +@item @code{flags} (default: @code{'()}) +The partition flags as a list of symbols. Possible values are +@code{'boot} and @code{'esp}. The @code{'boot} flags should be set if +you want to boot from this partition. Exactly one partition should have +this flag set, usually the root one. The @code{'esp} flag identifies a +UEFI System Partition. + +@item @code{initializer} (default: @code{#false}) +The partition initializer procedure as a gexp. This procedure is called +to populate a partition. If no initializer is passed, the +@code{initialize-root-partition} procedure from the @code{(gnu build +image)} module is used. + +@end table +@end deftp + +@node Instantiate an Image +@section Instantiate an Image + +Let's say you would like to create an MBR image with three distinct +partitions: + +@itemize +@item ESP vfat partition of 40MiB at offset 1024KiB. +@item DATA ext4 partition of 50MiB containing a dummy data file. +@item ROOT ext4 bootable partition containing the @code{%simple-os} +operating-system. +@end itemize + +You would then write the following image definition in a +@code{my-image.scm} file for instance. + +@lisp +(use-modules (gnu) + (gnu image) + (gnu tests) + (gnu system image) + (guix gexp)) + +(define MiB (expt 2 20)) + +(image + (format 'disk-image) + (operating-system %simple-os) + (partitions + (list + (partition + (size (* 40 MiB)) + (offset (* 1024 1024)) + (label "GNU-ESP") + (file-system "vfat") + (flags '(esp)) + (initializer (gexp initialize-efi-partition))) + (partition + (size (* 50 MiB)) + (label "DATA") + (file-system "ext4") + (initializer #~(lambda* (root . rest) + (mkdir root) + (call-with-output-file + (string-append root "/data") + (lambda (port) + (format port "my-data")))))) + (partition + (size 'guess) + (label root-label) + (file-system "ext4") + (flags '(boot)) + (initializer (gexp initialize-root-partition)))))) +@end lisp + +Note that the first and third partitions use generic initializers +procedures, initialize-efi-partition and initialize-root-partition +respectively. The initialize-efi-partition installs a GRUB EFI loader +that is loading the GRUB bootloader located in the root partition. The +initialize-root-partition instantiates a complete system as defined by +the @code{%simple-os} operating-system. + +You can now run: + +@example +guix system image my-image.scm +@end example + +to instantiate the @code{image} definition. That produces a disk image +which has the expected structure: + +@example +$ parted $(guix system image my-image.scm) print +@dots{} +Model: (file) +Disk /gnu/store/yhylv1bp5b2ypb97pd3bbhz6jk5nbhxw-disk-image: 1714MB +Sector size (logical/physical): 512B/512B +Partition Table: msdos +Disk Flags: + +Number Start End Size Type File system Flags + 1 1049kB 43.0MB 41.9MB primary fat16 esp + 2 43.0MB 95.4MB 52.4MB primary ext4 + 3 95.4MB 1714MB 1619MB primary ext4 boot +@end example + +The size of the @code{boot} partition has been inferred to @code{1619MB} +so that it is large enough to host the @code{%simple-os} +operating-system. + +You can also use existing @code{image} record definitions and inherit +from them to simplify the @code{image} definition. The @code{(gnu +system image)} module provides the following @code{image} definitions +variables. + +@defvr {Scheme Variable} efi-disk-image +A MBR disk-image composed of two partitions: a 64 bits ESP partition and +a ROOT boot partition. This image can be used on most @code{x86_64} and +@code{i686} machines, supporting BIOS or UEFI booting. +@end defvr + +@defvr {Scheme Variable} efi32-disk-image +Same as @code{efi-disk-image} but with a 32 bits EFI partition. +@end defvr + +@defvr {Scheme Variable} iso9660-image +An ISO-9660 image composed of a single bootable partition. This image +can also be used on most @code{x86_64} and @code{i686} machines. +@end defvr + +@defvr {Scheme Variable} docker-image +A Docker image that can be used to spawn a Docker container. +@end defvr + +Using the @code{efi-disk-image} we can simplify our previous +@code{image} declaration this way: + +@example +(use-modules (gnu) + (gnu image) + (gnu tests) + (gnu system image) + (guix gexp) + (ice-9 match)) + +(define MiB (expt 2 20)) + +(define data + (partition + (size (* 50 MiB)) + (label "DATA") + (file-system "ext4") + (initializer #~(lambda* (root . rest) + (mkdir root) + (call-with-output-file + (string-append root "/data") + (lambda (port) + (format port "my-data"))))))) + +(image + (inherit efi-disk-image) + (operating-system %simple-os) + (partitions + (match (image-partitions efi-disk-image) + ((esp root) + (list esp data root))))) +@end example + +This will give the exact same @code{image} instantiation but the +@code{image} declaration is simpler. + +@node image-type Reference +@section image-type Reference + +The @command{guix system image} command can, as we saw above, take a +file containing an @code{image} declaration as argument and produce an +actual disk image from it. The same command can also handle a file +containing an @code{operating-system} declaration as argument. In that +case, how is the @code{operating-system} turned into an image? + +That's where the @code{image-type} record intervenes. This record +defines how to transform an @code{operating-system} record into an +@code{image} record. + +@deftp {Data Type} image-type +This is the data type representing an image-type. + +@table @asis +@item @code{name} +The image-type name as a mandatory symbol, @code{'efi32-raw} for +instance. + +@item @code{constructor} +The image-type constructor, as a mandatory procedure that takes an +@code{operating-system} record as argument and returns an @code{image} +record. + +@end table +@end deftp + +There are several @code{image-type} records provided by the @code{(gnu +system image)} and the @code{(gnu system images @dots{})} modules. + +@defvr {Scheme Variable} efi-raw-image-type +Build an image based on the @code{efi-disk-image} image. +@end defvr + +@defvr {Scheme Variable} efi32-raw-image-type +Build an image based on the @code{efi32-disk-image} image. +@end defvr + +@defvr {Scheme Variable} qcow2-image-type +Build an image based on the @code{efi-disk-image} image but with the +@code{compressed-qcow2} image format. +@end defvr + +@defvr {Scheme Variable} iso-image-type +Build a compressed image based on the @code{iso9660-image} image. +@end defvr + +@defvr {Scheme Variable} uncompressed-iso-image-type +Build an image based on the @code{iso9660-image} image but with the +@code{compression?} field set to @code{#false}. +@end defvr + +@defvr {Scheme Variable} docker-image-type +Build an image based on the @code{docker-image} image. +@end defvr + +@defvr {Scheme Variable} raw-with-offset-image-type +Build an MBR image with a single partition starting at a @code{1024KiB} +offset. This is useful to leave some room to install a bootloader in +the post-MBR gap. +@end defvr + +@defvr {Scheme Variable} pinebook-pro-image-type +Build an image that is targeting the Pinebook Pro machine. The MBR +image contains a single partition starting at a @code{9MiB} offset. The +@code{u-boot-pinebook-pro-rk3399-bootloader} bootloader will be +installed in this gap. +@end defvr + +@defvr {Scheme Variable} rock64-image-type +Build an image that is targeting the Rock64 machine. The MBR image +contains a single partition starting at a @code{16MiB} offset. The +@code{u-boot-rock64-rk3328-bootloader} bootloader will be installed in +this gap. +@end defvr + +@defvr {Scheme Variable} novena-image-type +Build an image that is targeting the Novena machine. It has the same +characteristics as @code{raw-with-offset-image-type}. +@end defvr + +@defvr {Scheme Variable} pine64-image-type +Build an image that is targeting the Pine64 machine. It has the same +characteristics as @code{raw-with-offset-image-type}. +@end defvr + +@defvr {Scheme Variable} hurd-image-type +Build an image that is targeting a @code{i386} machine running the Hurd +kernel. The MBR image contains a single ext2 partitions with specific +@code{file-system-options} flags. +@end defvr + +@defvr {Scheme Variable} hurd-qcow2-image-type +Build an image similar to the one built by the @code{hurd-image-type} +but with the @code{format} set to @code{'compressed-qcow2}. +@end defvr + +So, if we get back to the @code{guix system image} command taking an +@code{operating-system} declaration as argument. By default, the +@code{efi-raw-image-type} is used to turn the provided +@code{operating-system} into an actual bootable image. + +To use a different @code{image-type}, the @code{--image-type} option can +be used. The @code{--list-image-types} option will list all the +supported image types. It turns out to be a textual listing of all the +@code{image-types} variables described just above (@pxref{Invoking guix +system}). + +@node Image Modules +@section Image Modules + +Let's take the example of the Pine64, an ARM based machine. To be able +to produce an image targeting this board, we need the following +elements: + +@itemize +@item An @code{operating-system} record containing at least +an appropriate kernel (@code{linux-libre-arm64-generic}) and bootloader +@code{u-boot-pine64-lts-bootloader}) for the Pine64. + +@item Possibly, an @code{image-type} record providing a way to +turn an @code{operating-system} record to an @code{image} record +suitable for the Pine64. + +@item An actual @code{image} that can be instantiated with the +@command{guix system image} command. + +@end itemize + +The @code{(gnu system images pine64)} module provides all those +elements: @code{pine64-barebones-os}, @code{pine64-image-type} and +@code{pine64-barebones-raw-image} respectively. + +The module returns the @code{pine64-barebones-raw-image} in order for +users to be able to run: + +@example +guix system image gnu/system/images/pine64.scm +@end example + +Now, thanks to the @code{pine64-image-type} record declaring the +@code{'pine64-raw} @code{image-type}, one could also prepare a +@code{my-pine.scm} file with the following content: + +@example +(use-modules (gnu system images pine64)) +(operating-system + (inherit pine64-barebones-os) + (timezone "Europe/Athens")) +@end example + +to customize the @code{pine64-barebones-os}, and run: + +@example +$ guix system image --image-type=pine64-raw my-pine.scm +@end example + +Note that there are other modules in the @code{gnu/system/images} +directory targeting @code{Novena}, @code{Pine64}, @code{PinebookPro} and +@code{Rock64} machines. + @node Installing Debugging Files @chapter Installing Debugging Files