From patchwork Mon Jan 4 01:05:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: raid5atemyhomework X-Patchwork-Id: 26205 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id A408527BC0A; Mon, 4 Jan 2021 01:06:18 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_PASS,T_DKIM_INVALID,URIBL_BLOCKED,URIBL_SBL,URIBL_SBL_A autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 6B31127BC09 for ; Mon, 4 Jan 2021 01:06:17 +0000 (GMT) Received: from localhost ([::1]:56218 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kwEK8-0000V3-0S for patchwork@mira.cbaines.net; Sun, 03 Jan 2021 20:06:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43654) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kwEJv-0000Mg-47 for guix-patches@gnu.org; Sun, 03 Jan 2021 20:06:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:55687) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kwEJu-0000uF-Sl for guix-patches@gnu.org; Sun, 03 Jan 2021 20:06:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kwEJu-0003dp-OA for guix-patches@gnu.org; Sun, 03 Jan 2021 20:06:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45643] [PATCH 1/3] gnu, doc: Create and document procedure to compile ZFS for specific kernel. References: In-Reply-To: Resent-From: raid5atemyhomework Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 04 Jan 2021 01:06:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45643 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: "45643@debbugs.gnu.org" <45643@debbugs.gnu.org> Received: via spool by 45643-submit@debbugs.gnu.org id=B45643.160972234113962 (code B ref 45643); Mon, 04 Jan 2021 01:06:02 +0000 Received: (at 45643) by debbugs.gnu.org; 4 Jan 2021 01:05:41 +0000 Received: from localhost ([127.0.0.1]:38999 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kwEJY-0003d8-U7 for submit@debbugs.gnu.org; Sun, 03 Jan 2021 20:05:41 -0500 Received: from mail-40140.protonmail.ch ([185.70.40.140]:47487) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kwEJW-0003cu-Rs for 45643@debbugs.gnu.org; Sun, 03 Jan 2021 20:05:39 -0500 Date: Mon, 04 Jan 2021 01:05:25 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail; t=1609722331; bh=8uBPq24ZQAcju/6ftAX6kgwPoVVUnBL2FQolRQsnxwc=; h=Date:To:From:Reply-To:Subject:From; b=Q2yN18vW1esFyANKckGQNYY/uS7EEcdq6fZ3ILk737tOLqXwHJ1jjaPvOxm1LDz3t H0AcmB0LftyHR4qcq5TUbe84CtvYzQ3xT3V5XU6TPKrax5HYCb29qXMzipGSjEwlR/ PqK4jDT0R1O32kzGU475kaOCCZ0+wqDwJxMro+sU= Message-ID: <_yQ96oMMZ2qR6_Y7zRGocuGc7bzzQl5zF_6x-AEmsOFM4YnJvcpGzfcnHNBNhYl0LxPvPsac1rL6eHG63g5W58W9K76qXYwKa57-hu36AYI=@protonmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" Reply-to: raid5atemyhomework , raid5atemyhomework via Guix-patches X-Patchwork-Original-From: raid5atemyhomework via Guix-patches via From: raid5atemyhomework X-getmail-retrieved-from-mailbox: Patches From 004a84a38266eb8735f1e127f5794bf6d66b9272 Mon Sep 17 00:00:00 2001 From: raid5atemyhomework Date: Fri, 1 Jan 2021 20:26:42 +0800 Subject: [PATCH 1/3] gnu, doc: Create and document procedure to compile ZFS for specific kernel. --- doc/guix.texi | 100 ++++++++++++++++++++++++++++++++++ gnu/local.mk | 1 + gnu/packages/file-systems.scm | 25 ++++++++- gnu/services/file-systems.scm | 72 ++++++++++++++++++++++++ 4 files changed, 196 insertions(+), 2 deletions(-) create mode 100644 gnu/services/file-systems.scm -- 2.29.2 diff --git a/doc/guix.texi b/doc/guix.texi index 1081ed26a3..5ad3907dbe 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -13885,6 +13885,106 @@ a file system declaration such as: compress-force=zstd,space_cache=v2")) @end lisp +@node ZFS file system +@subsection ZFS file system + +The ZFS on Linux file system cannot legally be downloaded as part of the +Linux kernel, but you as a user can do anything on hardware you own, +including download ZFS as source code, compile ZFS as a kernel module, +and link it into Linux. + +As a large and complex kernel module, ZFS has to be compiled for a +specific version of Linux. Often the latest ZFS package available in Guix +cannot be compiled with the latest Linux kernel available in Guix, so +installing the @code{zfs} package in your system configuration file will +fail. + +Instead, you have to check the +@url{https://github.com/openzfs/zfs/releases,OpenZFS release notes} for +the specific version of ZFS that Guix has packaged to determine what +Linux kernels you can use, then check the @code{linux-libre} packages +that Guix has packaged, and select one you can use on your system. + +Then, you have to modify your system configuration file, and create a +ZFS package that compiles using the specific Linux version you chose. +Below is a sketch of how you would modify your @code{operating-system} +declaration in order to set up ZFS: + +@lisp +(use-modules (gnu)) +;; @dots{} +(use-service-modules + ; @dots{} + file-systems) +(use-package-modules + ; @dots{} + linux + file-systems) + +;; @dots{} + +;;; (1) Select a specific kernel. +(define system-kernel linux-libre-5.4) +;;; (2) Define a ZFS package for your kernel. +(define system-zfs (make-zfs-package system-kernel)) + +;; @dots{} + +(operating-system + ;;; (3) Specify your selected kernel. + (kernel system-kernel) + ;;; (4) Add the "module" output of the system ZFS package to + ;;; the kernel-loadable modules. + (kernel-loadable-modules (list (list system-zfs "module"))) + + ;; @dots{} + + (packages + ;;; (5) Add the system ZFS package to global packages so that + ;;; "zfs", "zpool" etc. commands are available. + (cons* system-zfs + ; @dots{} + %base-packages)) + + ;; @dots{} + + (services + ;;; (6) Add a service that loads ZFS and all ZFS pools at boot. + (cons* (service zfs-loader-service-type + system-zfs) + ; @dots{} + %base-services)) + ;; @dots{} + ) +@end lisp + +@deffn (Scheme Procedure) make-zfs-package @var{kernel} +This procedure creates a package which, when included as a package +in your system, can be loaded as a kernel module for the specified +@var{kernel}, a Linux kernel package. +It is intended to create a system-specific ZFS kernel module for +the Linux kernel you will use in your system. +@end deffn + +@deffn (Scheme Variable) zfs-loader-service-type +This is the service that loads the ZFS kernel module and imports all +ZFS pools. The value associated with @code{zfs-loader-service-type} +services must be the ZFS package you defined for your system, and +whose @code{"module"} output you have included as a loadable kernel +module. +@end deffn + +When you have modified your @code{operating-system} to include ZFS +as in the above and reconfigured and rebooted, you will now be able +to create, mount, and manage ZFS pools. At each boot, ZFS will +automatically open any ZFS pools it detects in your system, and will +mount ZFS pools that have a non-@code{legacy} mountpoint. If ZFS +finds mountable ZFS filesystems that are encrypted by passphrase, it +will prompt for passphrases on the console. + +ZFS as root filesystem is not supported yet. ZFS for @code{/home} is +also probably not easily doable yet. + @node Mapped Devices @section Mapped Devices diff --git a/gnu/local.mk b/gnu/local.mk index c03a8b9f51..e59a3bb9f7 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -601,6 +601,7 @@ GNU_SYSTEM_MODULES = \ %D%/services/dict.scm \ %D%/services/dns.scm \ %D%/services/docker.scm \ + %D%/services/file-systems.scm \ %D%/services/authentication.scm \ %D%/services/games.scm \ %D%/services/ganeti.scm \ diff --git a/gnu/packages/file-systems.scm b/gnu/packages/file-systems.scm index 895ad069c5..e8f49a59fa 100644 --- a/gnu/packages/file-systems.scm +++ b/gnu/packages/file-systems.scm @@ -944,16 +944,37 @@ APFS.") ("openssl" ,openssl) ("python" ,python) ("python-cffi" ,python-cffi) - ("util-linux" ,util-linux "lib") + ("util-linux" ,util-linux) + ("util-linux-lib" ,util-linux "lib") ("zlib" ,zlib))) (home-page "https://zfsonlinux.org/") (synopsis "Native ZFS on Linux") (description "ZFS on Linux is an advanced file system and volume manager which was originally developed for Solaris and is now maintained by the OpenZFS -community.") +community. + +DO NOT INSTALL THIS PACKAGE. Instead, refer to the 'ZFS file system' section +of the Guix info manual for how to install ZFS.") (license license:cddl1.0))) +(define-public (make-zfs-package kernel) + (package + (inherit zfs) + (name (string-append "zfs-for-" + (package-name kernel) + "-" + (package-version kernel) + "-version")) + (arguments + (cons* #:linux kernel (package-arguments zfs))) + (description + "ZFS on Linux is an advanced file system and volume manager which was +originally developed for Solaris and is now maintained by the OpenZFS +community. + +This package has been compiled for a specific Linux kernel."))) + (define-public mergerfs (package (name "mergerfs") diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scm new file mode 100644 index 0000000000..bdc33f4028 --- /dev/null +++ b/gnu/services/file-systems.scm @@ -0,0 +1,72 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2021 raid5atemyhomework +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu services file-systems) + #:use-module (guix gexp) + #:use-module (gnu services) + #:use-module (gnu services shepherd)) + +;;; ZFS +(define (zfs-loader-shepherd-service system-zfs) + (let* ((zpool (file-append system-zfs "/sbin/zpool"))) + (list + (shepherd-service + (documentation "Load ZFS kernel module and import ZFS pools.") + (provision '(zfs-loader)) + (requirement '(file-systems)) + (one-shot? #t) + (modules `((srfi srfi-1) + (srfi srfi-34) + (srfi srfi-35) + (rnrs io ports) + ,@%default-modules)) + (start #~(lambda _ + (if (not (file-exists? "/proc/sys/kernel/modprobe")) + (begin (format (current-error-port) "error: ~a~%" + "Kernel is missing loadable module support.") + #f) + (and + (let ((modprobe (call-with-input-file + "/proc/sys/kernel/modprobe" get-line))) + (guard (c ((message-condition? c) + (format (current-error-port) + "error loading 'zfs' kernel module: ~a~%" + (condition-message c)) + #f)) + (invoke/quiet modprobe "--" "zfs"))) + (guard (c ((message-condition? c) + (format (current-error-port) + "error importing zpools: ~a~%" + (condition-message c)) + #f)) + ;; 'current-output-port' is typically connected to /dev/klog + ;; in PID 1, so redirect it to the console so we are talking + ;; to the user if zpool sees an encrpyted fs that needs a + ;; passphrase. + (with-output-to-port (current-error-port) + (lambda () + (invoke #$zpool "import" "-a" "-l")))))))))))) + +(define-public zfs-loader-service-type + (service-type + (name 'zfs-loader) + (description "Load ZFS kernel module and import ZFS pools.") + (extensions + (list (service-extension shepherd-root-service-type zfs-loader-shepherd-service) + (service-extension user-processes-service-type (const '(zfs-loader))))))) +