diff mbox series

[bug#45692,4/4] gnu: Add ZFS service.

Message ID 4XtBUBZvLlaRj3Qw8d44cVUm2QWePWKlxR4G7yH_bwXA62SoedCr5VHkch5VkOXfEd5frGaddsTLEEyiVQC0kQmearhfiiveI626gGbVjts=@protonmail.com
State Accepted
Headers show
Series None | expand

Commit Message

raid5atemyhomework Jan. 9, 2021, 8:31 a.m. UTC
New version again.  Change: Fix documentation `doc/guix.texi` to remove mentions of `options` and add documentation about the new `dependencies` features.


From dfe9ad7512d348933beb5b42041965ff604422b5 Mon Sep 17 00:00:00 2001
From: raid5atemyhomework <raid5atemyhomework@protonmail.com>
Date: Wed, 6 Jan 2021 09:24:20 +0800
Subject: [PATCH 1/2] gnu: Add service to install ZFS.

* gnu/services/file-systems.scm: New file.
(zfs-service-type): New variable.
(<zfs-configuration>): New type.
(%zfs-zvol-dependency): New variable.
* gnu/local.mk: Add gnu/services/file-systems.scm.
* gnu/services/base.scm (dependency->shepherd-service-name): Export.
* doc/guix.texi (ZFS file system): New subsection.
---
 doc/guix.texi                 | 129 +++++++++++++++++++++++++
 gnu/local.mk                  |   1 +
 gnu/services/base.scm         |   4 +-
 gnu/services/file-systems.scm | 174 ++++++++++++++++++++++++++++++++++
 4 files changed, 307 insertions(+), 1 deletion(-)
 create mode 100644 gnu/services/file-systems.scm

--
2.30.0

Comments

Danny Milosavljevic Feb. 8, 2021, 3:31 a.m. UTC | #1
Hello,

is it necessary to manually load the ZFS Linux kernel module?  Usually, Linux
should autoload its drivers when the device files are accessed--module or not.

If that fails, didn't you patch the zfs package so it refers to modprobe anyway?

(I wanna prevent having 543 places in Guix where modprobe is invoked)

If it doesn't autoload, we can totally modprobe it, though.
raid5atemyhomework Feb. 8, 2021, 6:25 a.m. UTC | #2
Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Monday, February 8, 2021 3:31 AM, Danny Milosavljevic <dannym@scratchpost.org> wrote:

> Hello,
>
> is it necessary to manually load the ZFS Linux kernel module? Usually, Linux
> should autoload its drivers when the device files are accessed--module or not.
>
> If that fails, didn't you patch the zfs package so it refers to modprobe anyway?

It does, for `udev`.  Not certain if it would trigger the load, though.  Presumably it would, but I have not tested.

A lot of this code was written in 0.8.5, where the ZFS tools will *not* attempt to `modprobe` the module first, they just error out.  I think in 2.0.1 upstream added code to load the module if the ZFS tool(s) were invoked while the module was not loaded, which is why additional patch was added by me for the 2.0.1 to patch in `modprobe`.  I have not tested if the `modprobe`-ing by the ZFS tools actually *works*, though.

Maybe when I find some amount of motivation I can go test it.

>
> (I wanna prevent having 543 places in Guix where modprobe is invoked)
>
> If it doesn't autoload, we can totally modprobe it, though.


Thanks
raid5atemyhomework
diff mbox series

Patch

diff --git a/doc/guix.texi b/doc/guix.texi
index a31d355780..7004efe3c4 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -13886,6 +13886,135 @@  a file system declaration such as:
 compress-force=zstd,space_cache=v2"))
 @end lisp

+@node ZFS file system
+@subsection ZFS file system
+
+The ZFS file system has a license incompatible with the Linux kernel,
+and thus cannot be distributed with the kernel. However, as a user
+you have the right to do whatever you want on your own hardware,
+including download the ZFS source code, compile it, link it to your
+own private copy of Linux, and run it. You can even use the Guix build
+system to automate this.
+
+As a large and complex kernel module, ZFS on Linux has to be compiled
+with a specific version of the kernel. Often even the latest ZFS
+package available cannot be compiled with the latest Linux kernel
+package provided by Guix System. Thus, installing the @code{zfs}
+package is likely to fail.
+
+Instead, you have to use an older long-term-support Linux kernel.
+Do not use @code{linux-libre-lts}, since the latest long-term-support
+kernel might be too new for the ZFS package; instead, explicitly
+select the version number, like @code{linux-libre-5.4}, and upgrade
+it manually later when you have verified that the ZFS version
+available on Guix can be compiled with a later LTS kernel.
+
+Then, you have to modify your system configuration file and use the
+selected older kernel, and add the @code{zfs-service-type} service.
+
+@lisp
+(use-modules (gnu))
+(use-package-modules
+   ; @dots{}
+   linux)
+(use-service-modules
+   ; @dots{}
+   file-systems)
+
+(define my-kernel linux-libre-5.4)
+
+(operating-system
+  (kernel my-kernel)
+  ;; @dots{}
+  (services
+    (cons* (service zfs-service-type
+             (zfs-configuration
+               (kernel my-kernel)))
+           ; @dots{}
+           %desktop-services))
+  ;; @dots{}
+  )
+@end lisp
+
+@defvr {Scheme Variable} zfs-service-type
+This is the type of the service to compile and install OpenZFS to
+your operating system. It loads the ZFS module at startup, imports
+pools, mounts automounted ZFS datasets, and installs the ZFS command
+line tools.  Its value must be a @code{zfs-configuration} record
+(see below).
+
+Here is an example use:
+
+@lisp
+(service zfs-service-type
+  (zfs-configuration
+    (kernel linux-libre-5.4)))
+@end lisp
+@end defvr
+
+@deftp {Data Type} zfs-configuration
+This data type represents the configuration of the ZFS service.
+The available fields are:
+
+@table @asis
+@item @code{kernel}
+The package of the Linux kernel to compile ZFS for. Required. It
+@emph{must} be the same kernel you use in your operating system.
+
+@item @code{base-zfs} (default: @code{zfs})
+The ZFS package to use. It will be modified to use the indicated
+kernel.
+
+@item @code{dependencies} (default: @code{'()})
+A list of @code{<file-system>}s or @code{<mapped-device>}s that
+must be mounted or opened before ZFS scans for pools to import.
+For example, you might have LUKS containers as the leaf VDEVs of
+a ZFS pool.
+@end table
+@end deftp
+
+Once your system has been configured to include the ZFS service
+and you have restarted the system, you can manage ZFS pools and
+datasets with @code{zpool} and @code{zfs} commands.
+
+ZFS datasets with an appropriate @code{mountpoint} property will
+be automounted at startup after the root file system is started.
+Encrypted datasets that are automounted will cause boot to pause
+and prompt for the password to be provided on the console.
+
+It's possible to have a ZFS dataset as your @code{/home} by simply
+setting the @code{mountpoint} ZFS property. However, note that ZFS
+will refuse to mount over a non-empty directory, so if your root
+filesystem already has a non-empty @code{/home}, you should remove
+it and its contents, then restart the system.
+
+ZFS features @dfn{ZVOL}s, which are block devices that ZFS exposes
+to the system. You can put any file system inside a ZVOL. In order
+to mount such a filesystem at boot, you can declare it as dependent
+on @code{%zfs-zvol-dependency}.
+
+@defvr {Scheme Variable} %zfs-zvol-dependency
+This is an artificial @code{<mapped-device>} object which tells
+the file mounting service to wait for ZFS to provide ZVOLs before
+mounting the file system that is dependent on it. For example:
+
+@lisp
+(file-system
+  (device "/dev/zvol/pool-name/zvol-name")
+  (mount-point "/ext4-on-zfs")
+  (type "ext4")
+  (dependencies (list %zfs-zvol-dependency)))
+@end lisp
+
+Do @emph{not} add @code{%zfs-zvol-dependency} to your
+@code{mapped-devices} declaration, In addition, only use it as a
+@code{file-system} dependency if you instantiate a
+@code{zfs-service-type} service in your operating system.
+@end defvr
+
+Having ZFS as a root filesystem or as @code{/boot} is not supported
+yet.
+
 @node Mapped Devices
 @section Mapped Devices

diff --git a/gnu/local.mk b/gnu/local.mk
index 1151d4642e..5aeb45c4c2 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -602,6 +602,7 @@  GNU_SYSTEM_MODULES =				\
   %D%/services/dict.scm				\
   %D%/services/dns.scm				\
   %D%/services/docker.scm			\
+  %D%/services/file-systems.scm			\
   %D%/services/authentication.scm		\
   %D%/services/games.scm			\
   %D%/services/ganeti.scm			\
diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index 13cfb6a8a2..ef3680583b 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -188,7 +188,9 @@ 

             references-file

-            %base-services))
+            %base-services
+
+            dependency->shepherd-service-name))

 ;;; Commentary:
 ;;;
diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scm
new file mode 100644
index 0000000000..9061ab9582
--- /dev/null
+++ b/gnu/services/file-systems.scm
@@ -0,0 +1,174 @@ 
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services file-systems)
+  #:use-module (gnu packages file-systems)
+  #:use-module (gnu services)
+  #:use-module (gnu services base)
+  #:use-module (gnu services linux)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system mapped-devices)
+  #:use-module (guix gexp)
+  #:use-module (guix packages)
+  #:use-module (guix records)
+  #:export (zfs-service-type
+
+            zfs-configuration
+            zfs-configuration?
+            zfs-configuration-kernel
+            zfs-configuration-base-zfs
+            zfs-configuration-dependencies
+
+            %zfs-zvol-dependency))
+
+(define-record-type* <zfs-configuration>
+  zfs-configuration make-zfs-configuration zfs-configuration?
+  ; kernel you want to compile the base-zfs module for.
+  (kernel         zfs-configuration-kernel)
+  ; base package that will be compiled for the kernel
+  (base-zfs       zfs-configuration-base-zfs
+                  (default zfs))
+  ; list of <mapped-device> | <file-system> we should wait for,
+  ; before scanning for ZFS pools.
+  (dependencies   zfs-configuration-dependencies
+                  (default '())))
+
+;; This is a synthetic and unusable MAPPED-DEVICE; its only use
+;; is to be added as a (dependency ...) of some FILE-SYSTEM.
+(define %zfs-zvol-dependency
+  (mapped-device
+    (source '())
+    ;; The /* prevents naming conflict with non-ZFS device mappings,
+    ;; since it is not a valid name for mapped devices, and also
+    ;; implies "all zvols" in terms of globs.
+    (targets '("zvol/*"))
+    (type #f)))
+
+(define (make-zfs-package conf)
+  (let ((base-zfs (zfs-configuration-base-zfs conf))
+        (kernel   (zfs-configuration-kernel conf)))
+    (package
+      (inherit base-zfs)
+      (name (string-join (list (package-name base-zfs)
+                               "for"
+                               (package-name kernel)
+                               (package-version kernel)
+                               "version")
+                         "-"))
+      (arguments (cons* #:linux kernel (package-arguments base-zfs))))))
+
+(define (zfs-loadable-module conf)
+  (list (list (make-zfs-package conf) "module")))
+
+(define (zfs-shepherd-services conf)
+  (let* ((zfs-package     (make-zfs-package conf))
+         (zpool           (file-append zfs-package "/sbin/zpool"))
+         (zfs             (file-append zfs-package "/sbin/zfs"))
+         (zvol_wait       (file-append zfs-package "/bin/zvol_wait"))
+         (scheme-modules  `((srfi srfi-1)
+                            (srfi srfi-34)
+                            (srfi srfi-35)
+                            (rnrs io ports)
+                            ,@%default-modules)))
+
+    (define zfs-scan
+      (shepherd-service
+        (provision '(zfs-scan))
+        (documentation "Scans for ZFS pools.")
+        (requirement `(kernel-module-loader
+                       root-file-system
+                       ,@(map dependency->shepherd-service-name
+                              (zfs-configuration-dependencies conf))))
+        (modules scheme-modules)
+        (start #~(lambda _
+                   (guard (c ((message-condition? c)
+                              (format (current-error-port)
+                                      "error importing zpools: ~a~%"
+                                      (condition-message c))
+                              #f))
+                     ; TODO: optionally use a cachefile, for systems with dozens or
+                     ; hundreds of devices.
+                     (invoke/quiet #$zpool "import" "-a" "-N"))))
+        (stop #~(const #t))))
+
+    (define device-mapping-zvol/*
+      (shepherd-service
+        (provision '(device-mapping-zvol/*))
+        (documentation "Waits for ZFS ZVOL devices to appear.")
+        (requirement '(zfs-scan))
+        (modules scheme-modules)
+        (start #~(lambda _
+                   (guard (c ((message-condition? c)
+                              (format (current-error-port)
+                                      "error waiting for zvols: ~a~%"
+                                      (condition-message c))
+                              #f))
+                     (invoke/quiet #$zvol_wait))))
+        (stop #~(const #t))))
+
+    (define zfs-automount
+      (shepherd-service
+        (provision '(zfs-automount))
+        (documentation "Automounts ZFS datasets.")
+        (requirement '(zfs-scan))
+        (modules scheme-modules)
+        (start #~(lambda _
+                   (guard (c ((message-condition? c)
+                              (format (current-error-port)
+                                      "error automounting zfs: ~a~$")
+                              #f))
+                     ; (current-output-port) is typically connected to /dev/klog,
+                     ; so redirect it to (current-error-port) so that user can see
+                     ; prompts for passphrases on console
+                     (with-output-to-port (current-error-port)
+                       (lambda ()
+                         (invoke #$zfs "mount" "-a" "-l"))))))
+        (stop #~(lambda _
+                  ;; make sure we don't keep any ZFS mountpoints busy.
+                  (chdir "/")
+                  ;; unmount everything.
+                  (invoke/quiet #$zfs "unmount" "-a" "-f")))))
+
+    (list zfs-scan
+          device-mapping-zvol/*
+          zfs-automount)))
+
+(define zfs-service-type
+  (service-type (name 'zfs)
+                (extensions
+                  (list
+                    ; install the kernel module
+                    (service-extension kernel-loadable-module-service-type
+                                       zfs-loadable-module)
+                    ; load the kernel module
+                    (service-extension kernel-module-loader-service-type
+                                       (const '("zfs")))
+                    ; scan ZFS pools, automount filesystem, wait for zvols.
+                    (service-extension shepherd-root-service-type
+                                       zfs-shepherd-services)
+                    ; make sure automount occurs before file-systems target is reached
+                    (service-extension file-systems-target-service-type
+                                       (const '(zfs-automount)))
+                    ; install ZFS management tools
+                    (service-extension profile-service-type
+                                       (compose list make-zfs-package))
+                    ; install ZFS udev rules
+                    (service-extension udev-service-type
+                                       (compose list make-zfs-package))))
+                (description
+                  "Install ZFS, an advanced filesystem and volume manager.")))