From patchwork Sat Nov 9 00:23:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Massimo Zaniboni X-Patchwork-Id: 70154 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id D70EB27BBE9; Sat, 9 Nov 2024 00:35:25 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-7.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED, RCVD_IN_VALIDITY_CERTIFIED,RCVD_IN_VALIDITY_RPBL,RCVD_IN_VALIDITY_SAFE, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 9310827BBE2 for ; Sat, 9 Nov 2024 00:35:24 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t9ZRY-0002jn-ND; Fri, 08 Nov 2024 19:35:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t9ZRW-0002iw-N8 for guix-patches@gnu.org; Fri, 08 Nov 2024 19:35:10 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t9ZRW-0007cw-DO; Fri, 08 Nov 2024 19:35:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:From:To:Subject; bh=8D0oHQZ7lu02GmInNeEyMWnh/ELa6111/moiCw82ftU=; b=pQcfQg8GxfYak5gX29YT2U4oCJnCiZNmqk81mu7nA6sHkN94+HzXh5RXyV8oLMY6nGjKb4cw6oWK2tACwlhhflnxgumKUiK0q6KntAqznZ3hzSjp1MB0o3U5UllOpv0PtL4MskhgG+cEQ1ynVLEMWx/9caHOgu5HXogeyhFJeVKVm2oljHCmpnLBks8RlYCXj2qwXRurnAhKRTOBfXKIP+D2nfppwo3H152oXDd8yhIHif8VGPLjia2K41epuCGzQYjKoNVk/k+FMqj23/TUkDcwm5YvQgYjAvsB2KoAjrjN/FJgwF6Xc9G7nOk5B95uD0j99yAAc16h2t0C6/a6HQ==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1t9ZRO-0003ME-4b; Fri, 08 Nov 2024 19:35:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#74273] [PATCH] Support for bcachefs-like multi-device file-systems. Resent-From: Massimo Zaniboni Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Sat, 09 Nov 2024 00:35:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 74273 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 74273@debbugs.gnu.org Cc: Massimo Zaniboni , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-To: guix-patches@gnu.org X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by submit@debbugs.gnu.org id=B.173111247412853 (code B ref -1); Sat, 09 Nov 2024 00:35:01 +0000 Received: (at submit) by debbugs.gnu.org; 9 Nov 2024 00:34:34 +0000 Received: from localhost ([127.0.0.1]:52655 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t9ZQv-0003LE-Lp for submit@debbugs.gnu.org; Fri, 08 Nov 2024 19:34:34 -0500 Received: from lists.gnu.org ([209.51.188.17]:33444) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t9ZQs-0003L5-3J for submit@debbugs.gnu.org; Fri, 08 Nov 2024 19:34:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t9ZQr-0002c2-MB for guix-patches@gnu.org; Fri, 08 Nov 2024 19:34:29 -0500 Received: from mail.asterisell.com ([193.30.121.134]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t9ZQn-0007OW-MY for guix-patches@gnu.org; Fri, 08 Nov 2024 19:34:28 -0500 From: Massimo Zaniboni DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dokmelody.org; s=mail; t=1731112431; bh=8D0oHQZ7lu02GmInNeEyMWnh/ELa6111/moiCw82ftU=; h=From:To:Cc:Subject:Date; b=lSZt/GQ+A9UF5UeNV7OSozHA1H2dfJVRYiX3Kj1B3h+FXHxZRY7P71tgirXkm/VUi kau+nA58SqwoSnGEJDS0QJWx9isRLe3CjeoSQhBLUFzvXZeBJ4atIqwOT4PXzbFte9 yhP0dyTeniiXwZWvWu2ES1NA548sGoMc5LR7fAaQ= Date: Sat, 9 Nov 2024 01:23:43 +0100 Message-ID: <2142f04036761f24a045a176098b1d0f958ce3bf.1731111823.git.mzan@dokmelody.org> MIME-Version: 1.0 Received-SPF: pass client-ip=193.30.121.134; envelope-from=mzan@dokmelody.org; helo=mail.asterisell.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches Support multi-device like "/dev/sda:/dev/sdb". Change-Id: Iddd9c31f8c083a55e7a1fb193e7bbfb396e2def6 --- I'm using this patch on my system. This is the first patch that I send using Stacked Git (`stg`). I hope that the email format is correct. gnu/build/file-systems.scm | 49 ++++++++++++++++++++++++++++--------- gnu/machine/ssh.scm | 23 ++++++++++++++++- gnu/system/file-systems.scm | 15 ++++++++++++ guix/scripts/system.scm | 25 ++++++++++++++++++- 4 files changed, 98 insertions(+), 14 deletions(-) base-commit: 2a6d96425eea57dc6dd48a2bec16743046e32e06 prerequisite-patch-id: 25d78fbfbd3268c16c93cd5d222386a7f421979b prerequisite-patch-id: 30bc9aa990c70c6c1c45c951a58cf9a532b388fb prerequisite-patch-id: 0000000000000000000000000000000000000000 prerequisite-patch-id: e22870a8d4b3ab67b12e05b6242b7f1bf5ac193b diff --git a/gnu/build/file-systems.scm b/gnu/build/file-systems.scm index 41e1c9e..7dba7e0 100644 --- a/gnu/build/file-systems.scm +++ b/gnu/build/file-systems.scm @@ -9,6 +9,7 @@ ;;; Copyright © 2022 Oleg Pykhalov ;;; Copyright © 2024 Nicolas Graves ;;; Copyright © 2024 Richard Sent +;;; Copyright © 2024 Massimo Zaniboni ;;; ;;; This file is part of GNU Guix. ;;; @@ -1138,9 +1139,9 @@ (define find-partition-by-luks-uuid (define (canonicalize-device-spec spec) - "Return the device name corresponding to SPEC, which can be a , a -, the string 'none' or another string (typically a /dev -file name or an nfs-root containing ':/')." + "Return the device name corresponding to SPEC, which can be a , +a , the string 'none' or another string like a device, +a multi-device, file name, nfs-root." (define max-trials ;; Number of times we retry partition label resolution, 1 second per ;; trial. Note: somebody reported a delay of 16 seconds (!) before their @@ -1162,20 +1163,44 @@ (define (canonicalize-device-spec spec) (sleep 1) (loop (+ 1 count)))))))) + (define (resolve-multi-device find-partition multi-device) + (let ((specs (string-split multi-device #\:))) + (let loop + ((count 0)) + (let ((nfp (find (lambda (d) (not (find-partition d))) specs))) + (if nfp + ;; Some devices take a bit of time to appear, most notably USB + ;; storage devices. Thus, wait for the device to appear. + (if (> count max-trials) + (error "failed to resolve partition" nfp) + (begin + (format #t "waiting for partition '~a' to appear...~%" nfp) + (sleep 1) + (loop (+ 1 count)))) + multi-device))))) + (match spec ((? string?) - (if (or (string-contains spec ":/") ;nfs - (and (>= (string-length spec) 2) - (equal? (string-take spec 2) "//")) ;cifs - (string=? spec "none")) - spec ; do not resolve NFS / CIFS / tmpfs devices - ;; Nothing to do, but wait until SPEC shows up. - (resolve identity spec identity))) + (cond + ((multi-device-spec? spec) + (resolve-multi-device identity spec)) + ((string-contains spec ":/") + ;NFS, something like 'server:/some/path' + spec) + ((and (>= (string-length spec) 2) + (equal? (string-take spec 2) "//")) + ;CIFS + spec) + ((string=? spec "none") + ;tmpfs + spec) + (else + ;; Nothing to do, but wait until SPEC shows up. + (resolve identity spec identity)))) ((? file-system-label?) ;; Resolve the label. (resolve find-partition-by-label - (file-system-label->string spec) - identity)) + (file-system-label->string spec) identity)) ((? uuid?) (resolve find-partition-by-uuid (uuid-bytevector spec) diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm index 3e10d98..0054adf 100644 --- a/gnu/machine/ssh.scm +++ b/gnu/machine/ssh.scm @@ -2,6 +2,7 @@ ;;; Copyright © 2019 Jakob L. Kreuze ;;; Copyright © 2020-2023 Ludovic Courtès ;;; Copyright © 2024 Ricardo +;;; Copyright © 2024 Massimo Zaniboni ;;; ;;; This file is part of GNU Guix. ;;; @@ -241,6 +242,22 @@ (define (machine-check-file-system-availability machine) (file-system-device fs) (strerror errno)))))) + (define (check-multi-device-file-system fs) + (define multi-device (file-system-device fs)) + (define devices (string-split multi-device #\:)) + (define (check-device device) + (remote-let ((errno #~(catch 'system-error + (lambda () + (stat #$device) + #t) + (lambda args + (system-error-errno args))))) + (when (number? errno) + (raise (formatted-message (G_ "device '~a' not found: ~a") + device + (strerror errno)))))) + (map check-device devices)) + (define (check-labeled-file-system fs) (define remote-exp (with-imported-modules (source-module-closure @@ -278,8 +295,12 @@ (define (machine-check-file-system-availability machine) (machine-configuration machine)) (append (map check-literal-file-system (filter (lambda (fs) - (string? (file-system-device fs))) + (single-device-spec? (file-system-device fs))) file-systems)) + (append-map check-multi-device-file-system + (filter (lambda (fs) + (multi-device-spec? (file-system-device fs))) + file-systems)) (map check-labeled-file-system (filter (lambda (fs) (file-system-label? (file-system-device fs))) diff --git a/gnu/system/file-systems.scm b/gnu/system/file-systems.scm index 4ea8237..9f91bd7 100644 --- a/gnu/system/file-systems.scm +++ b/gnu/system/file-systems.scm @@ -5,6 +5,7 @@ ;;; Copyright © 2020, 2021 Maxim Cournoyer ;;; Copyright © 2021 Tobias Geerinckx-Rice ;;; Copyright © 2022 Oleg Pykhalov +;;; Copyright © 2024 Massimo Zaniboni ;;; ;;; This file is part of GNU Guix. ;;; @@ -73,6 +74,9 @@ (define-module (gnu system file-systems) spec->file-system specification->file-system-mapping + multi-device-spec? + single-device-spec? + %pseudo-file-system-types %fuse-control-file-system %binary-format-file-system @@ -309,6 +313,17 @@ (define (file-system-needed-for-boot? fs) (and (file-prefix? (file-system-mount-point fs) (%store-prefix)) (not (memq 'bind-mount (file-system-flags fs)))))) +(define (multi-device-spec? spec) + "Return #t if the specification is like '/dev/sda:/dev/sdb'." + (and (string? spec) + (string-contains spec ":/") + (string-prefix? "/dev/" spec))) + +(define (single-device-spec? spec) + "Return #t if the specification is a string, but not a multi-device." + (and (string? spec) + (not (multi-device-spec? spec)))) + (define (file-system->spec fs) "Return a list corresponding to file-system FS that can be passed to the initrd code." diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm index 99c58f3..3459891 100644 --- a/guix/scripts/system.scm +++ b/guix/scripts/system.scm @@ -11,6 +11,7 @@ ;;; Copyright © 2021 Brice Waegeneire ;;; Copyright © 2021 Simon Tournier ;;; Copyright © 2022 Tobias Geerinckx-Rice +;;; Copyright © 2024 Massimo Zaniboni ;;; ;;; This file is part of GNU Guix. ;;; @@ -601,9 +602,16 @@ (define (check-file-system-availability file-systems) (file-system-label? (file-system-device fs))) relevant)) + (define multi-device + (filter (lambda (fs) + (and (string? (file-system-device fs)) + (multi-device-spec? (file-system-device fs)))) + relevant)) + (define literal (filter (lambda (fs) - (string? (file-system-device fs))) + (and (string? (file-system-device fs)) + (single-device-spec? (file-system-device fs)))) relevant)) (define uuid @@ -637,6 +645,21 @@ (define (check-file-system-availability file-systems) label, write @code{(file-system-label ~s)} in your @code{device} field.") device device)))))) literal) + (for-each + (lambda (fs) + (let* ((devices-str (file-system-device fs)) + (devices (string-split devices-str #\:))) + (for-each + (lambda (device) + (catch 'system-error + (lambda () (stat device)) + (lambda args + (let ((errno (system-error-errno args))) + (error (file-system-location* fs) + (G_ " #8605 device '~a' not found in multi-device '~a': ~a~%") + device devices-str (strerror errno)))))) + devices))) + multi-device) (for-each (lambda (fs) (let ((label (file-system-label->string (file-system-device fs))))