From patchwork Sat Jan 2 18:32:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: raid5atemyhomework X-Patchwork-Id: 26172 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id BAC0827BC08; Sat, 2 Jan 2021 18:33:16 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_PASS,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 60D2427BC09 for ; Sat, 2 Jan 2021 18:33:14 +0000 (GMT) Received: from localhost ([::1]:50668 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kvliD-0002wh-J8 for patchwork@mira.cbaines.net; Sat, 02 Jan 2021 13:33:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50832) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kvli2-0002wX-6v for guix-patches@gnu.org; Sat, 02 Jan 2021 13:33:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:49329) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kvli1-00018w-W9 for guix-patches@gnu.org; Sat, 02 Jan 2021 13:33:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kvli1-0006T5-Rj for guix-patches@gnu.org; Sat, 02 Jan 2021 13:33:01 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45592] [PATCH] gnu, doc: Create and document procedure to compile ZFS for specific kernel. Resent-From: raid5atemyhomework Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 02 Jan 2021 18:33:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45592 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: "45592@debbugs.gnu.org" <45592@debbugs.gnu.org> Received: via spool by 45592-submit@debbugs.gnu.org id=B45592.160961237024849 (code B ref 45592); Sat, 02 Jan 2021 18:33:01 +0000 Received: (at 45592) by debbugs.gnu.org; 2 Jan 2021 18:32:50 +0000 Received: from localhost ([127.0.0.1]:60875 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kvlhp-0006Sh-BQ for submit@debbugs.gnu.org; Sat, 02 Jan 2021 13:32:49 -0500 Received: from mail-40141.protonmail.ch ([185.70.40.141]:64750) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kvlhm-0006ST-Lk for 45592@debbugs.gnu.org; Sat, 02 Jan 2021 13:32:48 -0500 Date: Sat, 02 Jan 2021 18:32:35 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail; t=1609612359; bh=90UHV7vpSQjhhJJ2VCOUXzoA4d6FMyAJabQdjP91xyQ=; h=Date:To:From:Reply-To:Subject:In-Reply-To:References:From; b=NxhDGHkPKRoB5FVBHretUYTFrXgQQrTz86bLyYLKF3MaALa90ytQleBSywVj9SEpR ICOK0Vy46KXmWnW83MBowB2qmFutg4kcAUJGLEmQk83ePCdimyCN/L76lLTzD/31Ur LHSlh0I1+CfMz+vnHzhbdhdFEDCByVh0a/LYtOqA= Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" Reply-to: raid5atemyhomework , raid5atemyhomework via Guix-patches X-Patchwork-Original-From: raid5atemyhomework via Guix-patches via From: raid5atemyhomework X-getmail-retrieved-from-mailbox: Patches Yet another version! This time we now prompt the user at bootup for passphrases to decrypt ZFS encrypted filesystems. IF there aren't any, we just import any pools and mount what we can. Hope somebody actually reviews this........................ From 3c3014e31375ee0a59c7ad67b0f4ce9875325e47 Mon Sep 17 00:00:00 2001 From: raid5atemyhomework Date: Fri, 1 Jan 2021 20:26:42 +0800 Subject: [PATCH] gnu, doc: Create and document procedure to compile ZFS for specific kernel. --- doc/guix.texi | 100 ++++++++++++++++++++++++++++++++++ gnu/packages/file-systems.scm | 25 ++++++++- gnu/services/file-systems.scm | 72 ++++++++++++++++++++++++ 3 files changed, 195 insertions(+), 2 deletions(-) create mode 100644 gnu/services/file-systems.scm -- 2.29.2 diff --git a/doc/guix.texi b/doc/guix.texi index 1081ed26a3..edf950c9b5 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -13885,6 +13885,106 @@ a file system declaration such as: compress-force=zstd,space_cache=v2")) @end lisp +@node ZFS file system +@subsection ZFS file system + +The ZFS on Linux file system cannot legally be downloaded as part of the +Linux kernel, but you as a user can do anything on hardware you own, +including download ZFS as source code, compile ZFS as a kernel module, +and link it into Linux. + +As a large and complex kernel module, ZFS has to be compiled for a +specific version of Linux. Often the latest ZFS package available in Guix +cannot be compiled with the latest Linux kernel available in Guix, so +installing the @code{zfs} package in your system configuration file will +fail. + +Instead, you have to check the +@url{https://github.com/openzfs/zfs/releases,OpenZFS release notes} for +the specific version of ZFS that Guix has packaged to determine what +Linux kernels you can use, then check the @code{linux-libre} packages +that Guix has packaged, and select one you can use on your system. + +Then, you have to modify your system configuration file, and create a +ZFS package that compiles using the specific Linux version you chose. +Below is a sketch of how you would modify your @code{operating-system} +declaration in order to set up ZFS: + +@lisp +(use-modules (gnu)) +;; @dots{} +(use-service-modules + ; @dots{} + file-systems) +(use-package-modules + ; @dots{} + linux + file-systems) + +;; @dots{} + +;;; (1) Select a specific kernel. +(define system-kernel linux-libre-5.4) +;;; (2) Define a ZFS package for your kernel. +(define system-zfs (make-zfs-package system-kernel)) + +;; @dots{} + +(operating-system + ;;; (3) Specify your selected kernel. + (kernel system-kernel) + ;;; (4) Add the "module" output of the system ZFS package to + ;;; the kernel-loadable modules. + (kernel-loadable-modules (list (list system-zfs "module"))) + + ;; @dots{} + + (packages + ;;; (5) Add the system ZFS package to global packages so that + ;;; "zfs", "zpool" etc. commands are available. + (cons* system-zfs + ; @dots{} + %base-packages)) + + ;; @dots{} + + (services + ;;; (6) Add a service that loads ZFS and all ZFS pools at boot. + (cons* (service zfs-loader-service-type + system-zfs) + ; @dots{} + %base-services)) + ;; @dots{} + ) +@end lisp + +@deffn (Scheme Procedure) make-zfs-package @var{kernel} +This procedure creates a package which, when included as a package +in your system, can be loaded as a kernel module for the specified +@var{kernel}, a Linux kernel package. +It is intended to create a system-specific ZFS kernel module for +the Linux kernel you will use in your system. +@end deffn + +@deffn (Scheme Variable) zfs-loader-service-type +This is the service that loads the ZFS kernel module and imports all +ZFS pools. The value.associated with @code{zfs-loader-servie-type} +services must be the ZFS package you defined for your system, and +whose @code{"module"} output you have included as a loadable kernel +module. +@end deffn + +When you have modified your @code{operating-system} to include ZFS +as in the above and reconfigured and rebooted, you will now be able +to create, mount, and manage ZFS pools. At each boot, ZFS will +automatically open any ZFS pools it detects in your system, and will +mount ZFS pools that have a non-@code{legacy} mountpoint. If ZFS +finds mountable ZFS filesystems that are encrypted by passphrase, it +will prompt for passphrases on the console. + +ZFS as root filesystem is not supported yet. ZFS for @code{/home} is +also probably not easily doable yet. + @node Mapped Devices @section Mapped Devices diff --git a/gnu/packages/file-systems.scm b/gnu/packages/file-systems.scm index 895ad069c5..e8f49a59fa 100644 --- a/gnu/packages/file-systems.scm +++ b/gnu/packages/file-systems.scm @@ -944,16 +944,37 @@ APFS.") ("openssl" ,openssl) ("python" ,python) ("python-cffi" ,python-cffi) - ("util-linux" ,util-linux "lib") + ("util-linux" ,util-linux) + ("util-linux-lib" ,util-linux "lib") ("zlib" ,zlib))) (home-page "https://zfsonlinux.org/") (synopsis "Native ZFS on Linux") (description "ZFS on Linux is an advanced file system and volume manager which was originally developed for Solaris and is now maintained by the OpenZFS -community.") +community. + +DO NOT INSTALL THIS PACKAGE. Instead, refer to the 'ZFS file system' section +of the Guix info manual for how to install ZFS.") (license license:cddl1.0))) +(define-public (make-zfs-package kernel) + (package + (inherit zfs) + (name (string-append "zfs-for-" + (package-name kernel) + "-" + (package-version kernel) + "-version")) + (arguments + (cons* #:linux kernel (package-arguments zfs))) + (description + "ZFS on Linux is an advanced file system and volume manager which was +originally developed for Solaris and is now maintained by the OpenZFS +community. + +This package has been compiled for a specific Linux kernel."))) + (define-public mergerfs (package (name "mergerfs") diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scm new file mode 100644 index 0000000000..bdc33f4028 --- /dev/null +++ b/gnu/services/file-systems.scm @@ -0,0 +1,72 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2021 raid5atemyhomework +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu services file-systems) + #:use-module (guix gexp) + #:use-module (gnu services) + #:use-module (gnu services shepherd)) + +;;; ZFS +(define (zfs-loader-shepherd-service system-zfs) + (let* ((zpool (file-append system-zfs "/sbin/zpool"))) + (list + (shepherd-service + (documentation "Load ZFS kernel module and import ZFS pools.") + (provision '(zfs-loader)) + (requirement '(file-systems)) + (one-shot? #t) + (modules `((srfi srfi-1) + (srfi srfi-34) + (srfi srfi-35) + (rnrs io ports) + ,@%default-modules)) + (start #~(lambda _ + (if (not (file-exists? "/proc/sys/kernel/modprobe")) + (begin (format (current-error-port) "error: ~a~%" + "Kernel is missing loadable module support.") + #f) + (and + (let ((modprobe (call-with-input-file + "/proc/sys/kernel/modprobe" get-line))) + (guard (c ((message-condition? c) + (format (current-error-port) + "error loading 'zfs' kernel module: ~a~%" + (condition-message c)) + #f)) + (invoke/quiet modprobe "--" "zfs"))) + (guard (c ((message-condition? c) + (format (current-error-port) + "error importing zpools: ~a~%" + (condition-message c)) + #f)) + ;; 'current-output-port' is typically connected to /dev/klog + ;; in PID 1, so redirect it to the console so we are talking + ;; to the user if zpool sees an encrpyted fs that needs a + ;; passphrase. + (with-output-to-port (current-error-port) + (lambda () + (invoke #$zpool "import" "-a" "-l")))))))))))) + +(define-public zfs-loader-service-type + (service-type + (name 'zfs-loader) + (description "Load ZFS kernel module and import ZFS pools.") + (extensions + (list (service-extension shepherd-root-service-type zfs-loader-shepherd-service) + (service-extension user-processes-service-type (const '(zfs-loader))))))) +