From patchwork Thu Jul 11 20:26:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hartmut Goebel X-Patchwork-Id: 14573 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 0182217203; Thu, 11 Jul 2019 21:27:13 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 7A86F17202 for ; Thu, 11 Jul 2019 21:27:11 +0100 (BST) Received: from localhost ([::1]:45060 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hlfep-0004z6-6p for patchwork@mira.cbaines.net; Thu, 11 Jul 2019 16:27:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60724) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hlfel-0004uD-BE for guix-patches@gnu.org; Thu, 11 Jul 2019 16:27:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hlfej-00055P-Q9 for guix-patches@gnu.org; Thu, 11 Jul 2019 16:27:07 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:58097) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hlfej-00055J-Lz for guix-patches@gnu.org; Thu, 11 Jul 2019 16:27:05 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hlfej-0006rl-FO for guix-patches@gnu.org; Thu, 11 Jul 2019 16:27:05 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#36605] [PATCH] gc: Add option --keep-going. Resent-From: Hartmut Goebel Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 11 Jul 2019 20:27:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36605 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 36605@debbugs.gnu.org Received: via spool by 36605-submit@debbugs.gnu.org id=B36605.156287681326289 (code B ref 36605); Thu, 11 Jul 2019 20:27:05 +0000 Received: (at 36605) by debbugs.gnu.org; 11 Jul 2019 20:26:53 +0000 Received: from localhost ([127.0.0.1]:38666 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hlfeX-0006pw-0I for submit@debbugs.gnu.org; Thu, 11 Jul 2019 16:26:53 -0400 Received: from mail-out.m-online.net ([212.18.0.10]:57167) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hlfeS-0006pO-Nb for 36605@debbugs.gnu.org; Thu, 11 Jul 2019 16:26:49 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 45l6zD1zlQz1rYX8 for <36605@debbugs.gnu.org>; Thu, 11 Jul 2019 22:26:48 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 45l6zD1tJzz1qqkC for <36605@debbugs.gnu.org>; Thu, 11 Jul 2019 22:26:48 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id EVLYyrFq6D_9 for <36605@debbugs.gnu.org>; Thu, 11 Jul 2019 22:26:47 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-188-174-60-166.dynamic.mnet-online.de [188.174.60.166]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS for <36605@debbugs.gnu.org>; Thu, 11 Jul 2019 22:26:46 +0200 (CEST) Received: from lenashee.goebel-consult.de (lenashee.goebel-consult.de [192.168.110.2]) by hermia.goebel-consult.de (Postfix) with ESMTP id CD331603F4 for <36605@debbugs.gnu.org>; Thu, 11 Jul 2019 22:26:48 +0200 (CEST) From: Hartmut Goebel Date: Thu, 11 Jul 2019 22:26:10 +0200 Message-Id: <20190711202644.32014-2-h.goebel@crazy-compilers.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190711202644.32014-1-h.goebel@crazy-compilers.com> References: <58494E16.9010909@crazy-compilers.com> <20190711202644.32014-1-h.goebel@crazy-compilers.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/scripts/gc.scm (show-help, %options): Add option -k/--keep-going. (guix-gc): Pass value off option --keep-going on to delete-paths. * guix/store.scm (%protocol-version): Increment to 17. (delete-paths) New key-word parameter `keep-going?`, pass it on to run-rc. (run-gc): New key-word parameter `keep-going?`, send value of keep-going? to the daemon. * nix/libstore/store-api.hh (GCOptions): Add boolean keepGoing. * nix/libstore/worker-protocol.hh (PROTOCOL_VERSION) Increment to 17. * nix/nix-daemon/nix-daemon.cc (performOp)[wopCollectGarbage] Read keepGoing. * nix/libstore/gc.cc (LocalStore::collectGarbage) If keepGoing is true print an error message instead of throwing an error. * doc/guix.texi (Invoking guix gc): Document option --keep-going. --- doc/guix.texi | 8 +++++++- guix/scripts/gc.scm | 9 ++++++++- guix/store.scm | 12 ++++++++---- nix/libstore/gc.cc | 7 ++++++- nix/libstore/store-api.hh | 3 +++ nix/libstore/worker-protocol.hh | 2 +- nix/nix-daemon/nix-daemon.cc | 2 ++ 7 files changed, 35 insertions(+), 8 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index a3eba58..b8362d6 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -2098,7 +2098,13 @@ nothing and exit immediately. @itemx -d Attempt to delete all the store files and directories specified as arguments. This fails if some of the files are not in the store, or if -they are still live. +they are still live (with behaviour can be changed with +@option{--keep-going}). + +@item --keep-going +@itemx -k +Keep going when @option{--delete} is given and some of the store files +and directories specified as arguments fail to re removed. @item --list-failures List store items corresponding to cached build failures. diff --git a/guix/scripts/gc.scm b/guix/scripts/gc.scm index bdfee43..778e9a7 100644 --- a/guix/scripts/gc.scm +++ b/guix/scripts/gc.scm @@ -72,6 +72,9 @@ Invoke the garbage collector.\n")) --clear-failures remove PATHS from the set of cached failures")) (newline) (display (_ " + -k, --keep-going keep going when some of th pathes can not be deleted")) + (newline) + (display (_ " -h, --help display this help and exit")) (display (_ " -V, --version display version information and exit")) @@ -107,6 +110,9 @@ Invoke the garbage collector.\n")) (lambda (opt name arg result) (alist-cons 'action 'delete (alist-delete 'action result)))) + (option '(#\k "keep-going") #f #f + (lambda (opt name arg result) + (alist-cons 'keep-going? #t result))) (option '("optimize") #f #f (lambda (opt name arg result) (alist-cons 'action 'optimize @@ -228,7 +234,8 @@ Invoke the garbage collector.\n")) (let-values (((paths freed) (collect-garbage store))) (info (_ "freed ~h bytes~%") freed)))))) ((delete) - (delete-paths store (map direct-store-path paths))) + (delete-paths store (map direct-store-path paths) + #:keep-going? (assoc-ref opts 'keep-going?))) ((list-references) (list-relatives references)) ((list-requisites) diff --git a/guix/store.scm b/guix/store.scm index 2023875..4276db4 100644 --- a/guix/store.scm +++ b/guix/store.scm @@ -137,7 +137,7 @@ direct-store-path log-file)) -(define %protocol-version #x110) +(define %protocol-version #x111) (define %worker-magic-1 #x6e697863) ; "nixc" (define %worker-magic-2 #x6478696f) ; "dxio" @@ -938,7 +938,7 @@ is not an atomic operation.) When CHECK-CONTENTS? is true, check the contents of store items; this can take a lot of time." (not (verify store check-contents? repair?))))) -(define (run-gc server action to-delete min-freed) +(define* (run-gc server action to-delete min-freed #:key (keep-going? #f)) "Perform the garbage-collector operation ACTION, one of the `gc-action' values. When ACTION is `delete-specific', the TO-DELETE is the list of store paths to delete. IGNORE-LIVENESS? should always be @@ -956,6 +956,8 @@ and the number of bytes freed." ;; Obsolete `use-atime' and `max-atime' parameters. (write-int 0 s) (write-int 0 s)) + (when (>= (nix-server-minor-version server) 17) + (write-arg boolean keep-going? s)) ;; Loop until the server is done sending error output. (let loop ((done? (process-stderr server))) @@ -993,12 +995,14 @@ then collect at least MIN-FREED bytes. Return the paths that were collected, and the number of bytes freed." (run-gc server (gc-action delete-dead) '() min-freed)) -(define* (delete-paths server paths #:optional (min-freed (%long-long-max))) +(define* (delete-paths server paths #:optional (min-freed (%long-long-max)) + #:key (keep-going? #f)) "Delete PATHS from the store at SERVER, if they are no longer referenced. If MIN-FREED is non-zero, then stop after at least MIN-FREED bytes have been collected. Return the paths that were collected, and the number of bytes freed." - (run-gc server (gc-action delete-specific) paths min-freed)) + (run-gc server (gc-action delete-specific) paths min-freed + #:keep-going? keep-going?)) (define (import-paths server port) "Import the set of store paths read from PORT into SERVER's store. An error diff --git a/nix/libstore/gc.cc b/nix/libstore/gc.cc index 72eff52..6f2d8f7 100644 --- a/nix/libstore/gc.cc +++ b/nix/libstore/gc.cc @@ -662,8 +662,13 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results) foreach (PathSet::iterator, i, options.pathsToDelete) { assertStorePath(*i); tryToDelete(state, *i); - if (state.dead.find(*i) == state.dead.end()) + if (state.dead.find(*i) == state.dead.end()) { + if (options.keepGoing) { + printMsg(lvlError, format("cannot delete path `%1%' since it is still alive") % *i); + } else { throw Error(format("cannot delete path `%1%' since it is still alive") % *i); + } + } } } else if (options.maxFreed > 0) { diff --git a/nix/libstore/store-api.hh b/nix/libstore/store-api.hh index fa78d59..8b0f521 100644 --- a/nix/libstore/store-api.hh +++ b/nix/libstore/store-api.hh @@ -50,6 +50,9 @@ struct GCOptions /* Stop after at least `maxFreed' bytes have been freed. */ unsigned long long maxFreed; + /* keep going even if some of the paths can not be collected */ + bool keepGoing; + GCOptions(); }; diff --git a/nix/libstore/worker-protocol.hh b/nix/libstore/worker-protocol.hh index 99c1ee2..8faf193 100644 --- a/nix/libstore/worker-protocol.hh +++ b/nix/libstore/worker-protocol.hh @@ -6,7 +6,7 @@ namespace nix { #define WORKER_MAGIC_1 0x6e697863 #define WORKER_MAGIC_2 0x6478696f -#define PROTOCOL_VERSION 0x110 +#define PROTOCOL_VERSION 0x111 #define GET_PROTOCOL_MAJOR(x) ((x) & 0xff00) #define GET_PROTOCOL_MINOR(x) ((x) & 0x00ff) diff --git a/nix/nix-daemon/nix-daemon.cc b/nix/nix-daemon/nix-daemon.cc index a1fce25..b04cece 100644 --- a/nix/nix-daemon/nix-daemon.cc +++ b/nix/nix-daemon/nix-daemon.cc @@ -526,6 +526,8 @@ static void performOp(bool trusted, unsigned int clientVersion, readInt(from); readInt(from); } + if (GET_PROTOCOL_MINOR(clientVersion) >= 17) + options.keepGoing = readInt(from) != 0; GCResults results;