From patchwork Thu Aug 13 12:34:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Othacehe X-Patchwork-Id: 23607 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id CE23727BBE4; Thu, 13 Aug 2020 13:36:01 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 1572027BBE3 for ; Thu, 13 Aug 2020 13:36:01 +0100 (BST) Received: from localhost ([::1]:36190 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k6CSe-0007va-LH for patchwork@mira.cbaines.net; Thu, 13 Aug 2020 08:36:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47208) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k6CRi-00075P-Va for guix-patches@gnu.org; Thu, 13 Aug 2020 08:35:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:35814) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1k6CRi-00046H-MG for guix-patches@gnu.org; Thu, 13 Aug 2020 08:35:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1k6CRi-0000zE-Ie for guix-patches@gnu.org; Thu, 13 Aug 2020 08:35:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42849] [PATCH 1/3] install: Factorize cow-store procedure. References: <20200813122323.262805-1-othacehe@gnu.org> In-Reply-To: <20200813122323.262805-1-othacehe@gnu.org> Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 13 Aug 2020 12:35:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42849 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 42849@debbugs.gnu.org Cc: Mathieu Othacehe Received: via spool by 42849-submit@debbugs.gnu.org id=B42849.15973220793744 (code B ref 42849); Thu, 13 Aug 2020 12:35:02 +0000 Received: (at 42849) by debbugs.gnu.org; 13 Aug 2020 12:34:39 +0000 Received: from localhost ([127.0.0.1]:47356 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k6CRL-0000y9-6L for submit@debbugs.gnu.org; Thu, 13 Aug 2020 08:34:39 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37056) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k6CRI-0000xq-1g for 42849@debbugs.gnu.org; Thu, 13 Aug 2020 08:34:38 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:53336) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k6CRC-00043t-Q9 for 42849@debbugs.gnu.org; Thu, 13 Aug 2020 08:34:30 -0400 Received: from [2a01:e0a:19b:d9a0:3107:b202:556:bd51] (port=48336 helo=localhost.localdomain) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1k6CRB-0001L2-Qt; Thu, 13 Aug 2020 08:34:30 -0400 From: Mathieu Othacehe Date: Thu, 13 Aug 2020 14:34:17 +0200 Message-Id: <20200813123419.263639-1-othacehe@gnu.org> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Move the cow-store procedure from the service declaration in (gnu system install) to (gnu build install), so that it can be called from within a different context than Shepherd. * gnu/build/install.scm (mount-cow-store, umount-cow-store): New procedures. * gnu/system/install.scm (make-cow-store): Remove it, (cow-store-service-type): adapt it accordingly. --- gnu/build/install.scm | 44 ++++++++++++++++++++++++++++++++++- gnu/system/install.scm | 52 ++++++++++-------------------------------- 2 files changed, 55 insertions(+), 41 deletions(-) diff --git a/gnu/build/install.scm b/gnu/build/install.scm index 87aa5d68da..91c7225c87 100644 --- a/gnu/build/install.scm +++ b/gnu/build/install.scm @@ -18,6 +18,7 @@ ;;; along with GNU Guix. If not, see . (define-module (gnu build install) + #:use-module (guix build syscalls) #:use-module (guix build utils) #:use-module (guix build store-copy) #:use-module (srfi srfi-26) @@ -26,7 +27,9 @@ evaluate-populate-directive populate-root-file-system install-database-and-gc-roots - populate-single-profile-directory)) + populate-single-profile-directory + mount-cow-store + umount-cow-store)) ;;; Commentary: ;;; @@ -229,4 +232,43 @@ This is used to create the self-contained tarballs with 'guix pack'." (_ #t))) +(define (mount-cow-store target backing-directory) + "Make the store copy-on-write, using TARGET as the backing store. This is +useful when TARGET is on a hard disk, whereas the current store is on a RAM +disk." + (define (set-store-permissions directory) + "Set the right perms on DIRECTORY to use it as the store." + (chown directory 0 30000) ;use the fixed 'guixbuild' GID + (chmod directory #o1775)) + + (let ((tmpdir (string-append target "/tmp"))) + (mkdir-p tmpdir) + (mount tmpdir "/tmp" "none" MS_BIND)) + + (let* ((rw-dir (string-append target backing-directory)) + (work-dir (string-append rw-dir "/../.overlayfs-workdir"))) + (mkdir-p rw-dir) + (mkdir-p work-dir) + (mkdir-p "/.rw-store") + (set-store-permissions rw-dir) + (set-store-permissions "/.rw-store") + + ;; Mount the overlay, then atomically make it the store. + (mount "none" "/.rw-store" "overlay" 0 + (string-append "lowerdir=" (%store-directory) "," + "upperdir=" rw-dir "," + "workdir=" work-dir)) + (mount "/.rw-store" (%store-directory) "" MS_MOVE) + (rmdir "/.rw-store"))) + +(define (umount-cow-store target backing-directory) + "Umount copy-on-write store." + (let ((tmp-dir "/remove")) + (mkdir-p tmp-dir) + (mount (%store-directory) tmp-dir "" MS_MOVE) + (umount tmp-dir) + (rmdir tmp-dir) + (delete-file-recursively + (string-append target backing-directory)))) + ;;; install.scm ends here diff --git a/gnu/system/install.scm b/gnu/system/install.scm index a87c2f4207..be5a678cec 100644 --- a/gnu/system/install.scm +++ b/gnu/system/install.scm @@ -175,39 +175,6 @@ manual." ;; Sub-directory used as the backing store for copy-on-write. "/tmp/guix-inst") -(define (make-cow-store target) - "Return a gexp that makes the store copy-on-write, using TARGET as the -backing store. This is useful when TARGET is on a hard disk, whereas the -current store is on a RAM disk." - - (define (set-store-permissions directory) - ;; Set the right perms on DIRECTORY to use it as the store. - #~(begin - (chown #$directory 0 30000) ;use the fixed 'guixbuild' GID - (chmod #$directory #o1775))) - - #~(begin - ;; Bind-mount TARGET's /tmp in case we need space to build things. - (let ((tmpdir (string-append #$target "/tmp"))) - (mkdir-p tmpdir) - (mount tmpdir "/tmp" "none" MS_BIND)) - - (let* ((rw-dir (string-append target #$%backing-directory)) - (work-dir (string-append rw-dir "/../.overlayfs-workdir"))) - (mkdir-p rw-dir) - (mkdir-p work-dir) - (mkdir-p "/.rw-store") - #$(set-store-permissions #~rw-dir) - #$(set-store-permissions "/.rw-store") - - ;; Mount the overlay, then atomically make it the store. - (mount "none" "/.rw-store" "overlay" 0 - (string-append "lowerdir=" #$(%store-prefix) "," - "upperdir=" rw-dir "," - "workdir=" work-dir)) - (mount "/.rw-store" #$(%store-prefix) "" MS_MOVE) - (rmdir "/.rw-store")))) - (define cow-store-service-type (shepherd-service-type 'cow-store @@ -222,13 +189,18 @@ the given target.") ;; This is meant to be explicitly started by the user. (auto-start? #f) - (start #~(case-lambda - ((target) - #$(make-cow-store #~target) - target) - (else - ;; Do nothing, and mark the service as stopped. - #f))) + (modules `((gnu build install) + ,@%default-modules)) + (start + (with-imported-modules (source-module-closure + '((gnu build install))) + #~(case-lambda + ((target) + (mount-cow-store target #$%backing-directory) + target) + (else + ;; Do nothing, and mark the service as stopped. + #f)))) (stop #~(lambda (target) ;; Delete the temporary directory, but leave everything ;; mounted as there may still be processes using it since From patchwork Thu Aug 13 12:34:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Othacehe X-Patchwork-Id: 23605 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 9FF6527BBE5; Thu, 13 Aug 2020 13:35:14 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id E91A827BBE3 for ; Thu, 13 Aug 2020 13:35:13 +0100 (BST) Received: from localhost ([::1]:34440 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k6CRt-00076V-Bb for patchwork@mira.cbaines.net; Thu, 13 Aug 2020 08:35:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47212) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k6CRj-00075q-Ek for guix-patches@gnu.org; Thu, 13 Aug 2020 08:35:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:35815) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1k6CRj-00046L-4h for guix-patches@gnu.org; Thu, 13 Aug 2020 08:35:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1k6CRj-0000zM-03 for guix-patches@gnu.org; Thu, 13 Aug 2020 08:35:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42849] [PATCH 2/3] linux-container: Add a jail? argument. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 13 Aug 2020 12:35:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42849 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 42849@debbugs.gnu.org Cc: Mathieu Othacehe Received: via spool by 42849-submit@debbugs.gnu.org id=B42849.15973220833755 (code B ref 42849); Thu, 13 Aug 2020 12:35:02 +0000 Received: (at 42849) by debbugs.gnu.org; 13 Aug 2020 12:34:43 +0000 Received: from localhost ([127.0.0.1]:47358 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k6CRO-0000yT-Me for submit@debbugs.gnu.org; Thu, 13 Aug 2020 08:34:43 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37066) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k6CRJ-0000xs-7w for 42849@debbugs.gnu.org; Thu, 13 Aug 2020 08:34:39 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:53338) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k6CRE-000445-0h for 42849@debbugs.gnu.org; Thu, 13 Aug 2020 08:34:32 -0400 Received: from [2a01:e0a:19b:d9a0:3107:b202:556:bd51] (port=48336 helo=localhost.localdomain) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1k6CRC-0001L2-Ph; Thu, 13 Aug 2020 08:34:31 -0400 From: Mathieu Othacehe Date: Thu, 13 Aug 2020 14:34:18 +0200 Message-Id: <20200813123419.263639-2-othacehe@gnu.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200813123419.263639-1-othacehe@gnu.org> References: <20200813123419.263639-1-othacehe@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches We may want to run a container inside the MNT namespace, without jailing the container. Add a "jail?" argument to "run-container" and "call-with-container" methods. * gnu/build/linux-container.scm (run-container): Add a "jail?" argument and honor it, (call-with-container): ditto, and pass the argument to "run-container". --- gnu/build/linux-container.scm | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/gnu/build/linux-container.scm b/gnu/build/linux-container.scm index 87695c98fd..bb9fb0d799 100644 --- a/gnu/build/linux-container.scm +++ b/gnu/build/linux-container.scm @@ -218,12 +218,13 @@ corresponds to the symbols in NAMESPACES." namespaces))) (define* (run-container root mounts namespaces host-uids thunk - #:key (guest-uid 0) (guest-gid 0)) + #:key (guest-uid 0) (guest-gid 0) (jail? #t)) "Run THUNK in a new container process and return its PID. ROOT specifies the root directory for the container. MOUNTS is a list of objects that specify file systems to mount inside the container. NAMESPACES is a list of symbols that correspond to the possible Linux namespaces: mnt, -ipc, uts, user, and net. +ipc, uts, user, and net. If JAIL? is false, MOUNTS list is ignored and the +container is not jailed. HOST-UIDS specifies the number of host user identifiers to map into the user namespace. GUEST-UID and GUEST-GID specify the first UID (respectively GID) @@ -243,7 +244,7 @@ that host UIDs (respectively GIDs) map to in the namespace." (match (read child) ('ready (purify-environment) - (when (memq 'mnt namespaces) + (when (and jail? (memq 'mnt namespaces)) (catch #t (lambda () (mount-file-systems root mounts @@ -300,13 +301,15 @@ delete it when leaving the dynamic extent of this call." (define* (call-with-container mounts thunk #:key (namespaces %namespaces) (host-uids 1) (guest-uid 0) (guest-gid 0) - (process-spawned-hook (const #t))) + (process-spawned-hook (const #t)) + (jail? #f)) "Run THUNK in a new container process and return its exit status; call PROCESS-SPAWNED-HOOK with the PID of the new process that has been spawned. MOUNTS is a list of objects that specify file systems to mount -inside the container. NAMESPACES is a list of symbols corresponding to -the identifiers for Linux namespaces: mnt, ipc, uts, pid, user, and net. By -default, all namespaces are used. +inside the container. NAMESPACES is a list of symbols corresponding to the +identifiers for Linux namespaces: mnt, ipc, uts, pid, user, and net. By +default, all namespaces are used. If JAIL? is false, the MOUNTS list is +ignored and the container is not jailed. HOST-UIDS is the number of host user identifiers to map into the container's user namespace, if there is one. By default, only a single uid/gid, that of @@ -324,7 +327,8 @@ load path must be adjusted as needed." (lambda (root) (let ((pid (run-container root mounts namespaces host-uids thunk #:guest-uid guest-uid - #:guest-gid guest-gid))) + #:guest-gid guest-gid + #:jail? jail?))) ;; Catch SIGINT and kill the container process. (sigaction SIGINT (lambda (signum) From patchwork Thu Aug 13 12:34:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Othacehe X-Patchwork-Id: 23606 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id BB75427BBE4; Thu, 13 Aug 2020 13:35:16 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 9AD8327BBE3 for ; Thu, 13 Aug 2020 13:35:15 +0100 (BST) Received: from localhost ([::1]:34522 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k6CRv-0007BC-6z for patchwork@mira.cbaines.net; Thu, 13 Aug 2020 08:35:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47220) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k6CRj-00076J-RZ for guix-patches@gnu.org; Thu, 13 Aug 2020 08:35:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:35816) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1k6CRj-00046P-Hh for guix-patches@gnu.org; Thu, 13 Aug 2020 08:35:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1k6CRj-0000zT-F8 for guix-patches@gnu.org; Thu, 13 Aug 2020 08:35:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42849] [PATCH 3/3] installer: Run the installation inside a container. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 13 Aug 2020 12:35:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42849 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 42849@debbugs.gnu.org Cc: Mathieu Othacehe Received: via spool by 42849-submit@debbugs.gnu.org id=B42849.15973220833761 (code B ref 42849); Thu, 13 Aug 2020 12:35:03 +0000 Received: (at 42849) by debbugs.gnu.org; 13 Aug 2020 12:34:43 +0000 Received: from localhost ([127.0.0.1]:47360 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k6CRP-0000yW-1Y for submit@debbugs.gnu.org; Thu, 13 Aug 2020 08:34:43 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37078) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k6CRK-0000xu-OL for 42849@debbugs.gnu.org; Thu, 13 Aug 2020 08:34:39 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:53340) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k6CRF-00044F-H7 for 42849@debbugs.gnu.org; Thu, 13 Aug 2020 08:34:33 -0400 Received: from [2a01:e0a:19b:d9a0:3107:b202:556:bd51] (port=48336 helo=localhost.localdomain) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1k6CRD-0001L2-KH; Thu, 13 Aug 2020 08:34:32 -0400 From: Mathieu Othacehe Date: Thu, 13 Aug 2020 14:34:19 +0200 Message-Id: <20200813123419.263639-3-othacehe@gnu.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200813123419.263639-1-othacehe@gnu.org> References: <20200813123419.263639-1-othacehe@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches When the store overlay is mounted, other processes such as kmscon, udev and guix-daemon may open files from the store, preventing the underlying install support from being umounted. See: https://lists.gnu.org/archive/html/guix-devel/2018-12/msg00161.html. To avoid this situation, mount the store overlay inside a container, and run the installation from within that container. * gnu/services/base.scm (guix-shepherd-service): Support an optional PID argument passed to the "start" method. If that argument is passed, ensure that guix-daemon enters the given PID MNT namespace. * gnu/installer/final.scm (umount-cow-store): Remove it, (install-system): run the installation from within a container. * gnu/installer/newt/final.scm (run-install-shell): Remove the display hack. --- gnu/installer/final.scm | 125 +++++++++++++++++------------------ gnu/installer/newt/final.scm | 7 -- gnu/services/base.scm | 60 ++++++++++------- 3 files changed, 99 insertions(+), 93 deletions(-) diff --git a/gnu/installer/final.scm b/gnu/installer/final.scm index 685aa81d89..a19018dc85 100644 --- a/gnu/installer/final.scm +++ b/gnu/installer/final.scm @@ -26,6 +26,8 @@ #:use-module (guix build syscalls) #:use-module (guix build utils) #:use-module (gnu build accounts) + #:use-module (gnu build install) + #:use-module (gnu build linux-container) #:use-module ((gnu system shadow) #:prefix sys:) #:use-module (rnrs io ports) #:use-module (srfi srfi-1) @@ -133,49 +135,18 @@ USERS." (_ #f)))))) pids))) -(define (umount-cow-store) - "Remove the store overlay and the bind-mount on /tmp created by the -cow-store service. This procedure is very fragile and a better approach would -be much appreciated." - (catch #t - (lambda () - (let ((tmp-dir "/remove")) - (syslog "Unmounting cow-store.~%") - - (mkdir-p tmp-dir) - (mount (%store-directory) tmp-dir "" MS_MOVE) - - ;; The guix-daemon has possibly opened files from the cow-store, - ;; restart it. - (restart-service 'guix-daemon) - - (syslog "Killing cow users.") - - ;; Kill all processes started while the cow-store was active (logins - ;; on other TTYs for instance). - (kill-cow-users tmp-dir) - - ;; Try to umount the store overlay. Some process such as udevd - ;; workers might still be active, so do some retries. - (let loop ((try 5)) - (syslog "Umount try ~a~%" (- 5 try)) - (sleep 1) - (let ((umounted? (false-if-exception (umount tmp-dir)))) - (if (and (not umounted?) (> try 0)) - (loop (- try 1)) - (if umounted? - (syslog "Umounted ~a successfully.~%" tmp-dir) - (syslog "Failed to umount ~a.~%" tmp-dir))))) - - (umount "/tmp"))) - (lambda args - (syslog "~a~%" args)))) - (define* (install-system locale #:key (users '())) "Create /etc/shadow and /etc/passwd on the installation target for USERS. Start COW-STORE service on target directory and launch guix install command in a subshell. LOCALE must be the locale name under which that command will run, or #f. Return #t on success and #f on failure." + (define backing-directory + ;; Sub-directory used as the backing store for copy-on-write. + "/tmp/guix-inst") + + (define (assert-exit x) + (primitive-exit (if x 0 1))) + (let* ((options (catch 'system-error (lambda () ;; If this file exists, it can provide @@ -188,7 +159,11 @@ or #f. Return #t on success and #f on failure." "--fallback") options (list (%installer-configuration-file) - (%installer-target-dir))))) + (%installer-target-dir)))) + (database-dir "/var/guix/db") + (database-file (string-append database-dir "/db.sqlite")) + (saved-database (string-append database-dir "/db.save")) + (ret #f)) (mkdir-p (%installer-target-dir)) ;; We want to initialize user passwords but we don't want to store them in @@ -198,27 +173,51 @@ or #f. Return #t on success and #f on failure." ;; passwords that we've put in there. (create-user-database users (%installer-target-dir)) - (dynamic-wind - (lambda () - (start-service 'cow-store (list (%installer-target-dir)))) - (lambda () - ;; If there are any connected clients, assume that we are running - ;; installation tests. In that case, dump the standard and error - ;; outputs to syslog. - (if (not (null? (current-clients))) - (with-output-to-file "/dev/console" - (lambda () - (with-error-to-file "/dev/console" - (lambda () - (setvbuf (current-output-port) 'none) - (setvbuf (current-error-port) 'none) - (run-command install-command #:locale locale))))) - (run-command install-command #:locale locale))) - (lambda () - (stop-service 'cow-store) - ;; Remove the store overlay created at cow-store service start. - ;; Failing to do that will result in further umount calls to fail - ;; because the target device is seen as busy. See: - ;; https://lists.gnu.org/archive/html/guix-devel/2018-12/msg00161.html. - (umount-cow-store) - #f)))) + ;; When the store overlay is mounted, other processes such as kmscon, udev + ;; and guix-daemon may open files from the store, preventing the + ;; underlying install support from being umounted. See: + ;; https://lists.gnu.org/archive/html/guix-devel/2018-12/msg00161.html. + ;; + ;; To avoid this situation, mount the store overlay inside a container, + ;; and run the installation from within that container. + (zero? + (call-with-container '() + (lambda () + (dynamic-wind + (lambda () + ;; Save the database, so that it can be restored once the + ;; cow-store is umounted. + (copy-file database-file saved-database) + (mount-cow-store (%installer-target-dir) backing-directory)) + (lambda () + ;; We need to drag the guix-daemon to the container MNT + ;; namespace, so that it can operate on the cow-store. + (stop-service 'guix-daemon) + (start-service 'guix-daemon (list (number->string (getpid)))) + + (setvbuf (current-output-port) 'none) + (setvbuf (current-error-port) 'none) + + ;; If there are any connected clients, assume that we are running + ;; installation tests. In that case, dump the standard and error + ;; outputs to syslog. + (set! ret + (if (not (null? (current-clients))) + (with-output-to-file "/dev/console" + (lambda () + (with-error-to-file "/dev/console" + (lambda () + (run-command install-command + #:locale locale))))) + (run-command install-command #:locale locale)))) + (lambda () + ;; Restart guix-daemon so that it does no keep the MNT namespace + ;; alive. + (restart-service 'guix-daemon) + (copy-file saved-database database-file) + + ;; Finally umount the cow-store and exit the container. + (umount-cow-store (%installer-target-dir) backing-directory) + (assert-exit ret)))) + #:namespaces '(mnt) + #:jail? #f)))) diff --git a/gnu/installer/newt/final.scm b/gnu/installer/newt/final.scm index fa8d6fea71..89684c4d8a 100644 --- a/gnu/installer/newt/final.scm +++ b/gnu/installer/newt/final.scm @@ -102,13 +102,6 @@ a specific step, or restart the installer.")) #:key (users '())) (clear-screen) (newt-suspend) - ;; XXX: Force loading 'bold' font files before mouting the - ;; cow-store. Otherwise, if the file is loaded by kmscon after the cow-store - ;; in mounted, it will be necessary to kill kmscon to umount to cow-store. - (display - (colorize-string - (format #f (G_ "Installing Guix System ...~%")) - (color BOLD))) (let ((install-ok? (install-system locale #:users users))) (newt-resume) install-ok?)) diff --git a/gnu/services/base.scm b/gnu/services/base.scm index 491f35702a..f62fd861ca 100644 --- a/gnu/services/base.scm +++ b/gnu/services/base.scm @@ -1558,36 +1558,50 @@ proxy of 'guix-daemon'...~%") (provision '(guix-daemon)) (requirement '(user-processes)) (actions (list shepherd-set-http-proxy-action)) - (modules '((srfi srfi-1))) + (modules '((srfi srfi-1) + (ice-9 match))) (start - #~(lambda _ + #~(lambda args (define proxy ;; HTTP/HTTPS proxy. The 'http_proxy' variable is set by ;; the 'set-http-proxy' action. (or (getenv "http_proxy") #$http-proxy)) (fork+exec-command - (cons* #$(file-append guix "/bin/guix-daemon") - "--build-users-group" #$build-group - "--max-silent-time" #$(number->string max-silent-time) - "--timeout" #$(number->string timeout) - "--log-compression" #$(symbol->string log-compression) - #$@(if use-substitutes? - '() - '("--no-substitutes")) - "--substitute-urls" #$(string-join substitute-urls) - #$@extra-options - - ;; Add CHROOT-DIRECTORIES and all their dependencies - ;; (if these are store items) to the chroot. - (append-map (lambda (file) - (append-map (lambda (directory) - (list "--chroot-directory" - directory)) - (call-with-input-file file - read))) - '#$(map references-file - chroot-directories))) + ;; When running the installer, we need guix-daemon to operate + ;; from within the same MNT namespace as the installation + ;; container. In that case only, enter the namespace of the + ;; process PID passed as start argument. + (append + (match args + ((pid) + (list #$(file-append util-linux "/bin/nsenter") + "-t" pid "-m")) + (else '())) + (cons* #$(file-append guix "/bin/guix-daemon") + "--build-users-group" #$build-group + "--max-silent-time" + #$(number->string max-silent-time) + "--timeout" #$(number->string timeout) + "--log-compression" + #$(symbol->string log-compression) + #$@(if use-substitutes? + '() + '("--no-substitutes")) + "--substitute-urls" #$(string-join substitute-urls) + #$@extra-options + + ;; Add CHROOT-DIRECTORIES and all their dependencies + ;; (if these are store items) to the chroot. + (append-map + (lambda (file) + (append-map (lambda (directory) + (list "--chroot-directory" + directory)) + (call-with-input-file file + read))) + '#$(map references-file + chroot-directories)))) #:environment-variables (append (list #$@(if tmpdir