From patchwork Sun Jan 13 15:47:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 728 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 7113B16A42; Sun, 13 Jan 2019 15:48:05 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 128A9169F7 for ; Sun, 13 Jan 2019 15:48:05 +0000 (GMT) Received: from localhost ([127.0.0.1]:41596 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihzY-0004wL-Mg for patchwork@mira.cbaines.net; Sun, 13 Jan 2019 10:48:04 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55491) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihzW-0004wC-TJ for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gihzW-0004lx-4V for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:58735) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gihzW-0004lo-1W for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gihzV-0008NX-Vl for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#34060] [PATCH 01/10] profiling: Add a "gc" profiling component. References: <20190113154532.29606-1-ludo@gnu.org> In-Reply-To: <20190113154532.29606-1-ludo@gnu.org> Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 13 Jan 2019 15:48:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 34060 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 34060@debbugs.gnu.org Received: via spool by 34060-submit@debbugs.gnu.org id=B34060.154739446432092 (code B ref 34060); Sun, 13 Jan 2019 15:48:01 +0000 Received: (at 34060) by debbugs.gnu.org; 13 Jan 2019 15:47:44 +0000 Received: from localhost ([127.0.0.1]:57993 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzE-0008LR-Bi for submit@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:44 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:49258) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzC-0008Kx-2b for 34060@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:42 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id DB9AC19C1; Sun, 13 Jan 2019 16:47:40 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sx6C_2NJ5-Yf; Sun, 13 Jan 2019 16:47:39 +0100 (CET) Received: from gnu.org (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 78C7F1410; Sun, 13 Jan 2019 16:47:39 +0100 (CET) From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 13 Jan 2019 16:47:24 +0100 Message-Id: <20190113154733.29737-1-ludo@gnu.org> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/profiling.scm (show-gc-stats): New procedure. : Call 'register-profiling-hook!'. --- guix/profiling.scm | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/guix/profiling.scm b/guix/profiling.scm index 753fc6c22e..e1c205a543 100644 --- a/guix/profiling.scm +++ b/guix/profiling.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2017 Ludovic Courtès +;;; Copyright © 2017, 2019 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -18,6 +18,7 @@ (define-module (guix profiling) #:use-module (ice-9 match) + #:autoload (ice-9 format) (format) #:export (profiled? register-profiling-hook!)) @@ -50,3 +51,25 @@ (for-each (lambda (hook) (add-hook! hook thunk)) %profiling-hooks))) + +(define (show-gc-stats) + "Display garbage collection statistics." + (define MiB (* 1024 1024.)) + (define stats (gc-stats)) + + (format (current-error-port) "Garbage collection statistics: + heap size: ~,2f MiB + allocated: ~,2f MiB + GC times: ~a + time spent in GC: ~,2f seconds (~d% of user time)~%" + (/ (assq-ref stats 'heap-size) MiB) + (/ (assq-ref stats 'heap-total-allocated) MiB) + (assq-ref stats 'gc-times) + (/ (assq-ref stats 'gc-time-taken) + internal-time-units-per-second 1.) + (inexact->exact + (round (* (/ (assq-ref stats 'gc-time-taken) + (tms:utime (times)) 1.) + 100))))) + +(register-profiling-hook! "gc" show-gc-stats) From patchwork Sun Jan 13 15:47:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 729 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 0789E16A42; Sun, 13 Jan 2019 15:48:08 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 826EC169F7 for ; Sun, 13 Jan 2019 15:48:07 +0000 (GMT) Received: from localhost ([127.0.0.1]:41610 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihzb-0004xo-3I for patchwork@mira.cbaines.net; Sun, 13 Jan 2019 10:48:07 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55506) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihzX-0004wE-JF for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gihzW-0004mM-Je for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:58736) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gihzW-0004mF-Gm for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gihzW-0008Ne-Dm for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#34060] [PATCH 02/10] guix package: Avoid 'find-newest-available-packages'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 13 Jan 2019 15:48:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 34060 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 34060@debbugs.gnu.org Received: via spool by 34060-submit@debbugs.gnu.org id=B34060.154739446532099 (code B ref 34060); Sun, 13 Jan 2019 15:48:02 +0000 Received: (at 34060) by debbugs.gnu.org; 13 Jan 2019 15:47:45 +0000 Received: from localhost ([127.0.0.1]:57995 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzE-0008LZ-PG for submit@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:45 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:49266) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzC-0008Kz-Jw for 34060@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:43 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id DA66F19D2; Sun, 13 Jan 2019 16:47:41 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id M9gHc20pL9Tf; Sun, 13 Jan 2019 16:47:40 +0100 (CET) Received: from gnu.org (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id D695B17CF; Sun, 13 Jan 2019 16:47:39 +0100 (CET) From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 13 Jan 2019 16:47:25 +0100 Message-Id: <20190113154733.29737-2-ludo@gnu.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190113154733.29737-1-ludo@gnu.org> References: <20190113154733.29737-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/scripts/package.scm (transaction-upgrade-entry): Use 'find-best-packages-by-name' instead of 'find-newest-available-packages'. * tests/packages.scm ("transaction-upgrade-entry, zero upgrades") ("transaction-upgrade-entry, one upgrade") ("transaction-upgrade-entry, superseded package"): Adjust accordingly. --- guix/scripts/package.scm | 51 ++++++++++++++++++++-------------------- tests/packages.scm | 14 +++++------ 2 files changed, 33 insertions(+), 32 deletions(-) diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm index 7ff6bfd6d8..872a7303fc 100644 --- a/guix/scripts/package.scm +++ b/guix/scripts/package.scm @@ -220,31 +220,32 @@ of relevance scores." ('dismiss transaction) (($ name version output (? string? path)) - (match (vhash-assoc name (find-newest-available-packages)) - ((_ candidate-version pkg . rest) - (match (package-superseded pkg) - ((? package? new) - (supersede entry new)) - (#f - (case (version-compare candidate-version version) - ((>) - (manifest-transaction-install-entry - (package->manifest-entry* pkg output) - transaction)) - ((<) - transaction) - ((=) - (let ((candidate-path (derivation->output-path - (package-derivation (%store) pkg)))) - ;; XXX: When there are propagated inputs, assume we need to - ;; upgrade the whole entry. - (if (and (string=? path candidate-path) - (null? (package-propagated-inputs pkg))) - transaction - (manifest-transaction-install-entry - (package->manifest-entry* pkg output) - transaction)))))))) - (#f + (match (find-best-packages-by-name name #f) + ((pkg . rest) + (let ((candidate-version (package-version pkg))) + (match (package-superseded pkg) + ((? package? new) + (supersede entry new)) + (#f + (case (version-compare candidate-version version) + ((>) + (manifest-transaction-install-entry + (package->manifest-entry* pkg output) + transaction)) + ((<) + transaction) + ((=) + (let ((candidate-path (derivation->output-path + (package-derivation (%store) pkg)))) + ;; XXX: When there are propagated inputs, assume we need to + ;; upgrade the whole entry. + (if (and (string=? path candidate-path) + (null? (package-propagated-inputs pkg))) + transaction + (manifest-transaction-install-entry + (package->manifest-entry* pkg output) + transaction))))))))) + (() (warning (G_ "package '~a' no longer exists~%") name) transaction))))) diff --git a/tests/packages.scm b/tests/packages.scm index 237feb7aba..eb8ede3207 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018 Ludovic Courtès +;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -96,8 +96,8 @@ (test-assert "transaction-upgrade-entry, zero upgrades" (let* ((old (dummy-package "foo" (version "1"))) - (tx (mock ((gnu packages) find-newest-available-packages - (const vlist-null)) + (tx (mock ((gnu packages) find-best-packages-by-name + (const '())) ((@@ (guix scripts package) transaction-upgrade-entry) (manifest-entry (inherit (package->manifest-entry old)) @@ -109,8 +109,8 @@ (test-assert "transaction-upgrade-entry, one upgrade" (let* ((old (dummy-package "foo" (version "1"))) (new (dummy-package "foo" (version "2"))) - (tx (mock ((gnu packages) find-newest-available-packages - (const (vhash-cons "foo" (list "2" new) vlist-null))) + (tx (mock ((gnu packages) find-best-packages-by-name + (const (list new))) ((@@ (guix scripts package) transaction-upgrade-entry) (manifest-entry (inherit (package->manifest-entry old)) @@ -126,8 +126,8 @@ (let* ((old (dummy-package "foo" (version "1"))) (new (dummy-package "bar" (version "2"))) (dep (deprecated-package "foo" new)) - (tx (mock ((gnu packages) find-newest-available-packages - (const (vhash-cons "foo" (list "2" dep) vlist-null))) + (tx (mock ((gnu packages) find-best-packages-by-name + (const (list dep))) ((@@ (guix scripts package) transaction-upgrade-entry) (manifest-entry (inherit (package->manifest-entry old)) From patchwork Sun Jan 13 15:47:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 735 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 7059816A42; Sun, 13 Jan 2019 15:48:35 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id D6A5E169F7 for ; Sun, 13 Jan 2019 15:48:32 +0000 (GMT) Received: from localhost ([127.0.0.1]:41677 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gii00-00059A-FE for patchwork@mira.cbaines.net; Sun, 13 Jan 2019 10:48:32 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55509) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihzX-0004wK-V0 for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gihzX-0004mr-3j for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:58737) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gihzX-0004mj-0i for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gihzW-0008Nm-UE for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#34060] [PATCH 03/10] packages: Remove 'find-newest-available-packages'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 13 Jan 2019 15:48:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 34060 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 34060@debbugs.gnu.org Received: via spool by 34060-submit@debbugs.gnu.org id=B34060.154739446832134 (code B ref 34060); Sun, 13 Jan 2019 15:48:02 +0000 Received: (at 34060) by debbugs.gnu.org; 13 Jan 2019 15:47:48 +0000 Received: from localhost ([127.0.0.1]:58001 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzH-0008M0-83 for submit@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:47 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:49272) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzD-0008L3-6D for 34060@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:43 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 8E2F31A2A; Sun, 13 Jan 2019 16:47:42 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Zl-gwi7WJxy2; Sun, 13 Jan 2019 16:47:41 +0100 (CET) Received: from gnu.org (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 3925618EF; Sun, 13 Jan 2019 16:47:40 +0100 (CET) From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 13 Jan 2019 16:47:26 +0100 Message-Id: <20190113154733.29737-3-ludo@gnu.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190113154733.29737-1-ludo@gnu.org> References: <20190113154733.29737-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Since commit 9ffc1c00e55eb7931846dbb3fafcf54716fff57c, 'find-newest-available-packages' and 'find-packages-by-name' were both building a vhash mapping package names to packages. This factorizes this bit, also reducing I/O, CPU, and memory usage. * gnu/packages.scm (find-best-packages-by-name): Remove. (find-best-packages-by-name): Use 'find-packages-by-name' instead of 'find-newest-available-packages'. --- gnu/packages.scm | 38 ++++++++++---------------------------- 1 file changed, 10 insertions(+), 28 deletions(-) diff --git a/gnu/packages.scm b/gnu/packages.scm index 532297239d..4a85cf4b87 100644 --- a/gnu/packages.scm +++ b/gnu/packages.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018 Ludovic Courtès +;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Courtès ;;; Copyright © 2013 Mark H Weaver ;;; Copyright © 2014 Eric Bavier ;;; Copyright © 2016, 2017 Alex Kost @@ -53,7 +53,6 @@ find-packages-by-name find-best-packages-by-name - find-newest-available-packages specification->package specification->package+output @@ -203,38 +202,21 @@ decreasing version order." matching) matching))))) -(define find-newest-available-packages - (mlambda () - "Return a vhash keyed by package names, and with -associated values of the form - - (newest-version newest-package ...) - -where the preferred package is listed first." - - ;; FIXME: Currently, the preferred package is whichever one - ;; was found last by 'fold-packages'. Find a better solution. - (fold-packages (lambda (p r) - (let ((name (package-name p)) - (version (package-version p))) - (match (vhash-assoc name r) - ((_ newest-so-far . pkgs) - (case (version-compare version newest-so-far) - ((>) (vhash-cons name `(,version ,p) r)) - ((=) (vhash-cons name `(,version ,p ,@pkgs) r)) - ((<) r))) - (#f (vhash-cons name `(,version ,p) r))))) - vlist-null))) - (define (find-best-packages-by-name name version) "If version is #f, return the list of packages named NAME with the highest version numbers; otherwise, return the list of packages named NAME and at VERSION." (if version (find-packages-by-name name version) - (match (vhash-assoc name (find-newest-available-packages)) - ((_ version pkgs ...) pkgs) - (#f '())))) + (match (find-packages-by-name name) + (() + '()) + ((matches ...) + ;; Return the subset of MATCHES with the higher version number. + (let ((highest (package-version (first matches)))) + (take-while (lambda (p) + (string=? (package-version p) highest)) + matches)))))) (define %sigint-prompt From patchwork Sun Jan 13 15:47:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 736 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 7344216A42; Sun, 13 Jan 2019 15:48:39 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 28A41169F7 for ; Sun, 13 Jan 2019 15:48:39 +0000 (GMT) Received: from localhost ([127.0.0.1]:41742 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gii06-0005JK-N0 for patchwork@mira.cbaines.net; Sun, 13 Jan 2019 10:48:38 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55540) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihzY-0004wf-QC for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gihzY-0004nc-01 for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:58738) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gihzX-0004nE-Ou for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gihzX-0008Nt-Do for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#34060] [PATCH 04/10] inferior: Add 'gexp->derivation-in-inferior'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 13 Jan 2019 15:48:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 34060 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 34060@debbugs.gnu.org Received: via spool by 34060-submit@debbugs.gnu.org id=B34060.154739447032145 (code B ref 34060); Sun, 13 Jan 2019 15:48:03 +0000 Received: (at 34060) by debbugs.gnu.org; 13 Jan 2019 15:47:50 +0000 Received: from localhost ([127.0.0.1]:58004 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzI-0008MC-06 for submit@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:48 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:49278) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzD-0008LE-Ps for 34060@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:44 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 3049A19C1; Sun, 13 Jan 2019 16:47:43 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c3WBrKZQPBuS; Sun, 13 Jan 2019 16:47:42 +0100 (CET) Received: from gnu.org (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 916EB1901; Sun, 13 Jan 2019 16:47:40 +0100 (CET) From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 13 Jan 2019 16:47:27 +0100 Message-Id: <20190113154733.29737-4-ludo@gnu.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190113154733.29737-1-ludo@gnu.org> References: <20190113154733.29737-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/inferior.scm (gexp->derivation-in-inferior): New procedure. --- guix/inferior.scm | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/guix/inferior.scm b/guix/inferior.scm index ba8d00866b..42b3545599 100644 --- a/guix/inferior.scm +++ b/guix/inferior.scm @@ -81,6 +81,8 @@ inferior-package->manifest-entry + gexp->derivation-in-inferior + %inferior-cache-directory inferior-for-channels)) @@ -484,6 +486,30 @@ PACKAGE must be live." ;; Compile PACKAGE for SYSTEM, optionally cross-building for TARGET. (inferior-package->derivation package system #:target target)) +(define* (gexp->derivation-in-inferior name exp guix + #:rest rest) + "Return a derivation that evaluates EXP with GUIX, an instance of Guix as +returned for example by 'channel-instances->derivation'. Other arguments are +passed as-is to 'gexp->derivation'." + (define trampoline + ;; This is a crude way to run EXP on GUIX. TODO: use 'raw-derivation' and + ;; make 'guix repl' the "builder"; this will require "opening up" the + ;; mechanisms behind 'gexp->derivation', and adding '-l' to 'guix repl'. + #~(begin + (use-modules (ice-9 popen)) + + (let ((pipe (open-pipe* OPEN_WRITE + #+(file-append guix "/bin/guix") + "repl"))) + ;; Unquote EXP right here so that its references to #$output + ;; propagate to the surrounding gexp. + (write '#$exp pipe) ;XXX: load path for EXP? + + (unless (zero? (close-pipe pipe)) + (error "inferior failed" #+guix))))) + + (apply gexp->derivation name trampoline rest)) + ;;; ;;; Manifest entries. From patchwork Sun Jan 13 15:47:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 732 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id C3BC5169F7; Sun, 13 Jan 2019 15:48:27 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 76A3E169F7 for ; Sun, 13 Jan 2019 15:48:27 +0000 (GMT) Received: from localhost ([127.0.0.1]:41692 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihzv-0005BT-1B for patchwork@mira.cbaines.net; Sun, 13 Jan 2019 10:48:27 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55552) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihzZ-0004x3-6K for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gihzY-0004nz-7N for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:05 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:58739) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gihzY-0004na-3q for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:04 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gihzX-0008O0-Ux for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#34060] [PATCH 05/10] discovery: Add 'fold-module-public-variables*'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 13 Jan 2019 15:48:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 34060 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 34060@debbugs.gnu.org Received: via spool by 34060-submit@debbugs.gnu.org id=B34060.154739447132156 (code B ref 34060); Sun, 13 Jan 2019 15:48:03 +0000 Received: (at 34060) by debbugs.gnu.org; 13 Jan 2019 15:47:51 +0000 Received: from localhost ([127.0.0.1]:58006 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzK-0008MQ-Lb for submit@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:50 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:49280) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzE-0008LF-5f for 34060@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:44 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 90C1F1901; Sun, 13 Jan 2019 16:47:43 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TiP7qIj5-SaG; Sun, 13 Jan 2019 16:47:42 +0100 (CET) Received: from gnu.org (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id ED6531410; Sun, 13 Jan 2019 16:47:40 +0100 (CET) From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 13 Jan 2019 16:47:28 +0100 Message-Id: <20190113154733.29737-5-ludo@gnu.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190113154733.29737-1-ludo@gnu.org> References: <20190113154733.29737-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/discovery.scm (fold-module-public-variables*): New procedure. --- guix/discovery.scm | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/guix/discovery.scm b/guix/discovery.scm index 3fc6e2c9e7..ef5ae73973 100644 --- a/guix/discovery.scm +++ b/guix/discovery.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018 Ludovic Courtès +;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -30,7 +30,8 @@ scheme-modules* fold-modules all-modules - fold-module-public-variables)) + fold-module-public-variables + fold-module-public-variables*)) ;;; Commentary: ;;; @@ -147,10 +148,33 @@ search. Entries in PATH can be directory names (strings) or (DIRECTORY SUB-DIRECTORY." (fold-modules cons '() path #:warn warn)) +(define (fold-module-public-variables* proc init modules) + "Call (PROC MODULE SYMBOL VARIABLE) for each variable exported by one of MODULES, +using INIT as the initial value of RESULT. It is guaranteed to never traverse +the same object twice." + ;; Here SEEN is populated by variables; if two different variables refer to + ;; the same object, we still let them through. + (identity ;discard second return value + (fold2 (lambda (module result seen) + (fold2 (lambda (sym+var result seen) + (match sym+var + ((sym . var) + (if (not (vhash-assq var seen)) + (values (proc module sym var result) + (vhash-consq var #t seen)) + (values result seen))))) + result + seen + (module-map cons module))) + init + vlist-null + modules))) + (define (fold-module-public-variables proc init modules) "Call (PROC OBJECT RESULT) for each variable exported by one of MODULES, using INIT as the initial value of RESULT. It is guaranteed to never traverse the same object twice." + ;; Note: here SEEN is populated by objects, not by variables. (identity ; discard second return value (fold2 (lambda (module result seen) (fold2 (lambda (var result seen) From patchwork Sun Jan 13 15:47:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 733 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 07D5B16A42; Sun, 13 Jan 2019 15:48:28 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id A915816A1F for ; Sun, 13 Jan 2019 15:48:27 +0000 (GMT) Received: from localhost ([127.0.0.1]:41655 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihzv-00055D-6m for patchwork@mira.cbaines.net; Sun, 13 Jan 2019 10:48:27 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55567) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihzZ-0004xI-JV for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gihzY-0004oO-HK for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:05 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:58740) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gihzY-0004oI-EK for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:04 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gihzY-0008O8-Bm for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:04 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#34060] [PATCH 06/10] pull: Build profile with 'channel-instances->derivation'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 13 Jan 2019 15:48:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 34060 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 34060@debbugs.gnu.org Received: via spool by 34060-submit@debbugs.gnu.org id=B34060.154739447132163 (code B ref 34060); Sun, 13 Jan 2019 15:48:04 +0000 Received: (at 34060) by debbugs.gnu.org; 13 Jan 2019 15:47:51 +0000 Received: from localhost ([127.0.0.1]:58008 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzL-0008Mb-2q for submit@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:51 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:49286) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzE-0008LS-My for 34060@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:45 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 221271410; Sun, 13 Jan 2019 16:47:44 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id M6OsGt4agd5p; Sun, 13 Jan 2019 16:47:43 +0100 (CET) Received: from gnu.org (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 505CA184B; Sun, 13 Jan 2019 16:47:41 +0100 (CET) From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 13 Jan 2019 16:47:29 +0100 Message-Id: <20190113154733.29737-6-ludo@gnu.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190113154733.29737-1-ludo@gnu.org> References: <20190113154733.29737-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/scripts/package.scm (build-and-use-profile): Rename 'manifest' to 'manifest-or-derivation' and allow it to be a derivation. * guix/scripts/pull.scm (build-and-install): Use 'channel-instances->derivation' instead of 'channel-instances->manifest'. --- guix/scripts/package.scm | 41 ++++++++++++++++++++++------------------ guix/scripts/pull.scm | 4 ++-- 2 files changed, 25 insertions(+), 20 deletions(-) diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm index 872a7303fc..4f483ac141 100644 --- a/guix/scripts/package.scm +++ b/guix/scripts/package.scm @@ -118,24 +118,27 @@ denote ranges as interpreted by 'matching-generations'." (else (leave (G_ "invalid syntax: ~a~%") pattern))))) -(define* (build-and-use-profile store profile manifest +(define* (build-and-use-profile store profile manifest-or-derivation #:key allow-collisions? bootstrap? use-substitutes? dry-run?) "Build a new generation of PROFILE, a file name, using the packages -specified in MANIFEST, a manifest object. When ALLOW-COLLISIONS? is true, -do not treat collisions in MANIFEST as an error." +specified in MANIFEST-OR-DERIVATION, a manifest object or a profile +derivation. When ALLOW-COLLISIONS? is true, do not treat collisions in +MANIFEST-OR-DERIVATION as an error." (when (equal? profile %current-profile) (ensure-default-profile)) - (let* ((prof-drv (run-with-store store - (profile-derivation manifest - #:allow-collisions? allow-collisions? - #:hooks (if bootstrap? - '() - %default-profile-hooks) - #:locales? (not bootstrap?)))) + (let* ((prof-drv (if (derivation? manifest-or-derivation) + manifest-or-derivation + (run-with-store store + (profile-derivation manifest-or-derivation + #:allow-collisions? allow-collisions? + #:hooks (if bootstrap? + '() + %default-profile-hooks) + #:locales? (not bootstrap?))))) (prof (derivation->output-path prof-drv))) (show-what-to-build store (list prof-drv) #:use-substitutes? use-substitutes? @@ -153,18 +156,20 @@ do not treat collisions in MANIFEST as an error." ;; overwriting a "previous future generation". (name (generation-file-name profile (+ 1 number)))) (and (build-derivations store (list prof-drv)) - (let* ((entries (manifest-entries manifest)) - (count (length entries))) + (let* ((entries (and (manifest? manifest-or-derivation) + (manifest-entries manifest-or-derivation))) + (count (and entries (length entries)))) (switch-symlinks name prof) (switch-symlinks profile (basename name)) (unless (string=? profile %current-profile) (register-gc-root store name)) - (format #t (N_ "~a package in profile~%" - "~a packages in profile~%" - count) - count) - (display-search-paths entries (list profile) - #:kind 'prefix))) + (when count + (format #t (N_ "~a package in profile~%" + "~a packages in profile~%" + count) + count) + (display-search-paths entries (list profile) + #:kind 'prefix)))) (warn-about-disk-space profile)))))) diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm index 6d1914f7c2..ce3d24a7f7 100644 --- a/guix/scripts/pull.scm +++ b/guix/scripts/pull.scm @@ -186,9 +186,9 @@ true, display what would be built without actually building it." (define update-profile (store-lift build-and-use-profile)) - (mlet %store-monad ((manifest (channel-instances->manifest instances))) + (mlet %store-monad ((drv (channel-instances->derivation instances))) (mbegin %store-monad - (update-profile profile manifest + (update-profile profile drv #:dry-run? dry-run?) (munless dry-run? (return (display-profile-news profile)))))) From patchwork Sun Jan 13 15:47:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 734 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id DA54516A43; Sun, 13 Jan 2019 15:48:34 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 3591D16A1F for ; Sun, 13 Jan 2019 15:48:34 +0000 (GMT) Received: from localhost ([127.0.0.1]:41725 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gii01-0005FA-LP for patchwork@mira.cbaines.net; Sun, 13 Jan 2019 10:48:33 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55620) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihzc-0004zV-0D for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:09 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gihza-0004qh-Ep for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:07 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:58744) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gihza-0004qW-BB for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:06 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gihza-0008Oc-8a for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:06 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#34060] [PATCH 07/10] channels: Compute a package cache and use it. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 13 Jan 2019 15:48:06 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 34060 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 34060@debbugs.gnu.org Received: via spool by 34060-submit@debbugs.gnu.org id=B34060.154739447632195 (code B ref 34060); Sun, 13 Jan 2019 15:48:06 +0000 Received: (at 34060) by debbugs.gnu.org; 13 Jan 2019 15:47:56 +0000 Received: from localhost ([127.0.0.1]:58016 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzP-0008N8-Hx for submit@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:56 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:49294) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzG-0008Ly-Uh for 34060@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:48 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 4DC4117CF; Sun, 13 Jan 2019 16:47:46 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2Us61FPvHH5q; Sun, 13 Jan 2019 16:47:43 +0100 (CET) Received: from gnu.org (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id A783F19FA; Sun, 13 Jan 2019 16:47:41 +0100 (CET) From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 13 Jan 2019 16:47:30 +0100 Message-Id: <20190113154733.29737-7-ludo@gnu.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190113154733.29737-1-ludo@gnu.org> References: <20190113154733.29737-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/packages.scm (cache-is-authoritative?, load-package-cache) (cache-lookup, generate-package-cache): New procedures. (%package-cache-file): New variable. (find-packages-by-name): Rename to... (find-packages-by-name/direct): ... this. (find-packages-by-name): Rewrite to use the package cache when 'cache-is-authoritative?' returns true. * tests/packages.scm ("find-packages-by-name + version, with cache") ("find-packages-by-name with cache"): New tests. * guix/channels.scm (package-cache-file): New procedure. (channel-instances->derivation): Use it in #:hooks. --- gnu/packages.scm | 127 +++++++++++++++++++++++++++++++++++++++++++-- guix/channels.scm | 32 +++++++++++- tests/packages.scm | 18 +++++++ 3 files changed, 172 insertions(+), 5 deletions(-) diff --git a/gnu/packages.scm b/gnu/packages.scm index 4a85cf4b87..6796db80a4 100644 --- a/gnu/packages.scm +++ b/gnu/packages.scm @@ -28,11 +28,14 @@ #:use-module (guix memoization) #:use-module ((guix build utils) #:select ((package-name->name+version - . hyphen-separated-name->name+version))) + . hyphen-separated-name->name+version) + mkdir-p)) #:autoload (guix profiles) (packages->manifest) #:use-module (guix describe) #:use-module (ice-9 vlist) #:use-module (ice-9 match) + #:autoload (ice-9 binary-ports) (put-bytevector) + #:autoload (system base compile) (compile) #:use-module (srfi srfi-1) #:use-module (srfi srfi-11) #:use-module (srfi srfi-26) @@ -56,7 +59,9 @@ specification->package specification->package+output - specifications->manifest)) + specifications->manifest + + generate-package-cache)) ;;; Commentary: ;;; @@ -135,6 +140,14 @@ for system '~a'") ;; Default search path for package modules. `((,%distro-root-directory . "gnu/packages"))) +(define (cache-is-authoritative?) + "Return true if the pre-computed package cache is authoritative. It is not +authoritative when entries have been added via GUIX_PACKAGE_PATH or '-L' +flags." + (equal? (%package-module-path) + (append %default-package-module-path + (package-path-entries)))) + (define %package-module-path ;; Search path for package modules. Each item must be either a directory ;; name or a pair whose car is a directory and whose cdr is a sub-directory @@ -183,7 +196,35 @@ is guaranteed to never traverse the same package twice." init modules)) -(define find-packages-by-name +(define %package-cache-file + ;; Location of the package cache. + "/lib/guix/package.cache") + +(define load-package-cache + (mlambda (profile) + "Attempt to load the package cache. On success return a vhash keyed by +package names. Return #f on failure." + (match profile + (#f #f) + (profile + (catch 'system-error + (lambda () + (define lst + (load-compiled (string-append profile %package-cache-file))) + (fold (lambda (item vhash) + (match item + (#(name version module symbol outputs + supported? deprecated? + file line column) + (vhash-cons name item vhash)))) + vlist-null + lst)) + (lambda args + (if (= ENOENT (system-error-errno args)) + #f + (apply throw args)))))))) + +(define find-packages-by-name/direct ;bypass the cache (let ((packages (delay (fold-packages (lambda (p r) (vhash-cons (package-name p) p r)) @@ -202,6 +243,37 @@ decreasing version order." matching) matching))))) +(define (cache-lookup cache name) + "Lookup package NAME in CACHE. Return a list sorted in increasing version +order." + (define (package-version? (vector-ref v2 1) (vector-ref v1 1))) + + (sort (vhash-fold* cons '() name cache) + package-versionbool (member (%current-system) + (package-supported-systems package))) + ,(->bool (package-superseded package)) + ,@(let ((loc (package-location package))) + (if loc + `(,(location-file loc) + ,(location-line loc) + ,(location-column loc)) + '(#f #f #f)))) + result))) + (_ + result))) + + (define exp + (fold-module-public-variables* expand-cache '() + (all-modules (%package-module-path) + #:warn + warn-about-load-error))) + + (mkdir-p (dirname cache-file)) + (call-with-output-file cache-file + (lambda (port) + ;; Store the cache as a '.go' file. This makes loading fast and reduces + ;; heap usage since some of the static data is directly mmapped. + (put-bytevector port + (compile `'(,@exp) + #:to 'bytecode + #:opts '(#:to-file? #t))))) + cache-file) + (define %sigint-prompt ;; The prompt to jump to upon SIGINT. diff --git a/guix/channels.scm b/guix/channels.scm index 6b860f3bd8..cf5edddf03 100644 --- a/guix/channels.scm +++ b/guix/channels.scm @@ -21,6 +21,7 @@ #:use-module (guix git) #:use-module (guix records) #:use-module (guix gexp) + #:use-module (guix modules) #:use-module (guix discovery) #:use-module (guix monads) #:use-module (guix profiles) @@ -31,7 +32,8 @@ #:use-module (srfi srfi-2) #:use-module (srfi srfi-9) #:use-module (srfi srfi-11) - #:autoload (guix self) (whole-package) + #:autoload (guix self) (whole-package make-config.scm) + #:autoload (guix inferior) (gexp->derivation-in-inferior) ;FIXME: circular dep #:use-module (ice-9 match) #:export (channel channel? @@ -416,11 +418,37 @@ channel instances." (zip instances derivations)))) (return (manifest entries)))) +(define (package-cache-file manifest) + "Build a package cache file for the instance in MANIFEST. This is meant to +be used as a profile hook." + (mlet %store-monad ((profile (profile-derivation manifest + #:hooks '()))) + + (define build + #~(begin + (use-modules (gnu packages)) + + (if (defined? 'generate-package-cache) + (begin + ;; Delegate package cache generation to the inferior. + (format (current-error-port) + "Generating package cache for '~a'...~%" + #$profile) + (generate-package-cache #$output)) + (mkdir #$output)))) + + (gexp->derivation-in-inferior "guix-package-cache" build + profile + #:properties '((type . profile-hook) + (hook . package-cache))))) + (define (channel-instances->derivation instances) "Return the derivation of the profile containing INSTANCES, a list of channel instances." (mlet %store-monad ((manifest (channel-instances->manifest instances))) - (profile-derivation manifest))) + (profile-derivation manifest + #:hooks (cons package-cache-file + %default-profile-hooks)))) (define latest-channel-instances* (store-lift latest-channel-instances)) diff --git a/tests/packages.scm b/tests/packages.scm index eb8ede3207..2720ba5a15 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -1005,6 +1005,24 @@ (((? (cut eq? hello <>))) #t) (wrong (pk 'find-packages-by-name wrong #f)))) +(test-equal "find-packages-by-name with cache" + (find-packages-by-name "guile") + (call-with-temporary-directory + (lambda (cache) + (generate-package-cache cache) + (mock ((guix describe) current-profile (const cache)) + (mock ((gnu packages) cache-is-authoritative? (const #t)) + (find-packages-by-name "guile")))))) + +(test-equal "find-packages-by-name + version, with cache" + (find-packages-by-name "guile" "2") + (call-with-temporary-directory + (lambda (cache) + (generate-package-cache cache) + (mock ((guix describe) current-profile (const cache)) + (mock ((gnu packages) cache-is-authoritative? (const #t)) + (find-packages-by-name "guile" "2")))))) + (test-assert "--search-paths with pattern" ;; Make sure 'guix package --search-paths' correctly reports environment ;; variables when file patterns are used (in particular, it must follow From patchwork Sun Jan 13 15:47:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 737 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 08B9916A42; Sun, 13 Jan 2019 15:48:52 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 8A6EE169F7 for ; Sun, 13 Jan 2019 15:48:51 +0000 (GMT) Received: from localhost ([127.0.0.1]:41784 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gii0J-0005ZJ-6R for patchwork@mira.cbaines.net; Sun, 13 Jan 2019 10:48:51 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55594) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihza-0004xj-B2 for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gihzZ-0004p0-2W for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:06 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:58741) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gihzY-0004ol-Tu for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:05 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gihzY-0008OF-R1 for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:04 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#34060] [PATCH 08/10] edit: Use 'specification->location' to read information from the cache. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 13 Jan 2019 15:48:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 34060 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 34060@debbugs.gnu.org Received: via spool by 34060-submit@debbugs.gnu.org id=B34060.154739447232170 (code B ref 34060); Sun, 13 Jan 2019 15:48:04 +0000 Received: (at 34060) by debbugs.gnu.org; 13 Jan 2019 15:47:52 +0000 Received: from localhost ([127.0.0.1]:58010 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzL-0008Mi-Dh for submit@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:51 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:49278) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzF-0008LE-Cy for 34060@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:45 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 1E1BC184B; Sun, 13 Jan 2019 16:47:45 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3Af6jyLNrkDq; Sun, 13 Jan 2019 16:47:44 +0100 (CET) Received: from gnu.org (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 1B72B17CF; Sun, 13 Jan 2019 16:47:42 +0100 (CET) From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 13 Jan 2019 16:47:31 +0100 Message-Id: <20190113154733.29737-8-ludo@gnu.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190113154733.29737-1-ludo@gnu.org> References: <20190113154733.29737-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches That way 'guix edit' doesn't need to load any package module. * gnu/packages.scm (find-package-locations, specification->location): New procedures. * guix/scripts/edit.scm (package->location-specification): Rename to... (location->location-specification): ... this. Expect a location object instead of a package. (guix-edit): Use 'specification->location' instead of 'specification->package'. * tests/packages.scm ("find-package-locations") ("find-package-locations with cache") ("specification->location"): New tests. --- gnu/packages.scm | 51 +++++++++++++++++++++++++++++++++++++++++++ guix/scripts/edit.scm | 29 ++++++++++-------------- tests/packages.scm | 23 +++++++++++++++++++ 3 files changed, 85 insertions(+), 18 deletions(-) diff --git a/gnu/packages.scm b/gnu/packages.scm index 6796db80a4..cf655e7448 100644 --- a/gnu/packages.scm +++ b/gnu/packages.scm @@ -55,10 +55,12 @@ fold-packages find-packages-by-name + find-package-locations find-best-packages-by-name specification->package specification->package+output + specification->location specifications->manifest generate-package-cache)) @@ -274,6 +276,31 @@ decreasing version order." versions modules symbols))) (find-packages-by-name/direct name version))) +(define* (find-package-locations name #:optional version) + "Return a list of version/location pairs corresponding to each package +matching NAME and VERSION." + (define cache + (load-package-cache (current-profile))) + + (if (and cache (cache-is-authoritative?)) + (match (cache-lookup cache name) + (#f '()) + ((#(name versions modules symbols outputs + supported? deprecated? + files lines columns) ...) + (fold (lambda (version* file line column result) + (if (and file + (or (not version) + (version-prefix? version version*))) + (alist-cons version* (location file line column) + result) + result)) + '() + versions files lines columns))) + (map (lambda (package) + (cons (package-version package) (package-location package))) + (find-packages-by-name/direct name version)))) + (define (find-best-packages-by-name name version) "If version is #f, return the list of packages named NAME with the highest version numbers; otherwise, return the list of packages named NAME and at @@ -393,6 +420,30 @@ present, return the preferred newest version." (let-values (((name version) (package-name->name+version spec))) (%find-package spec name version))) +(define (specification->location spec) + "Return the location of the highest-numbered package matching SPEC, a +specification such as \"guile@2\" or \"emacs\"." + (let-values (((name version) (package-name->name+version spec))) + (match (find-package-locations name version) + (() + (if version + (leave (G_ "~A: package not found for version ~a~%") name version) + (leave (G_ "~A: unknown package~%") name))) + (lst + (let* ((highest (match lst (((version . _) _ ...) version))) + (locations (take-while (match-lambda + ((version . location) + (string=? version highest))) + lst))) + (match locations + (((version . location) . rest) + (unless (null? rest) + (warning (G_ "ambiguous package specification `~a'~%") spec) + (warning (G_ "choosing ~a@~a from ~a~%") + name version + (location->string location))) + location))))))) + (define* (specification->package+output spec #:optional (output "out")) "Return the package and output specified by SPEC, or #f and #f; SPEC may optionally contain a version number and an output name, as in these examples: diff --git a/guix/scripts/edit.scm b/guix/scripts/edit.scm index 8b2b61d76a..da3d2775e8 100644 --- a/guix/scripts/edit.scm +++ b/guix/scripts/edit.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2015, 2016 Ludovic Courtès +;;; Copyright © 2015, 2016, 2019 Ludovic Courtès ;;; Copyright © 2015 Mathieu Lirzin ;;; ;;; This file is part of GNU Guix. @@ -21,7 +21,6 @@ #:use-module (guix ui) #:use-module (guix scripts) #:use-module (guix utils) - #:use-module (guix packages) #:use-module (gnu packages) #:use-module (srfi srfi-1) #:use-module (srfi srfi-37) @@ -63,14 +62,13 @@ Start $VISUAL or $EDITOR to edit the definitions of PACKAGE...\n")) file path)) absolute-file-name)) -(define (package->location-specification package) - "Return the location specification for PACKAGE for a typical editor command +(define (location->location-specification location) + "Return the location specification for LOCATION for a typical editor command line." - (let ((loc (package-location package))) - (list (string-append "+" - (number->string - (location-line loc))) - (search-path* %load-path (location-file loc))))) + (list (string-append "+" + (number->string + (location-line location))) + (search-path* %load-path (location-file location)))) (define (guix-edit . args) @@ -83,18 +81,13 @@ line." '())) (with-error-handling - (let* ((specs (reverse (parse-arguments))) - (packages (map specification->package specs))) - (for-each (lambda (package) - (unless (package-location package) - (leave (G_ "source location of package '~a' is unknown~%") - (package-full-name package)))) - packages) + (let* ((specs (reverse (parse-arguments))) + (locations (map specification->location specs))) (catch 'system-error (lambda () - (let ((file-names (append-map package->location-specification - packages))) + (let ((file-names (append-map location->location-specification + locations))) ;; Use `system' instead of `exec' in order to sanely handle ;; possible command line arguments in %EDITOR. (exit (system (string-join (cons (%editor) file-names)))))) diff --git a/tests/packages.scm b/tests/packages.scm index 2720ba5a15..8aa117a2e7 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -1131,6 +1131,29 @@ (lambda (key . args) key))) +(test-equal "find-package-locations" + (map (lambda (package) + (cons (package-version package) + (package-location package))) + (find-packages-by-name "guile")) + (find-package-locations "guile")) + +(test-equal "find-package-locations with cache" + (map (lambda (package) + (cons (package-version package) + (package-location package))) + (find-packages-by-name "guile")) + (call-with-temporary-directory + (lambda (cache) + (generate-package-cache cache) + (mock ((guix describe) current-profile (const cache)) + (mock ((gnu packages) cache-is-authoritative? (const #t)) + (find-package-locations "guile")))))) + +(test-equal "specification->location" + (package-location (specification->package "guile@2")) + (specification->location "guile@2")) + (test-end "packages") ;;; Local Variables: From patchwork Sun Jan 13 15:47:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 731 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 27C0816A42; Sun, 13 Jan 2019 15:48:11 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 9DA84169F7 for ; Sun, 13 Jan 2019 15:48:10 +0000 (GMT) Received: from localhost ([127.0.0.1]:41626 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihze-00050O-5B for patchwork@mira.cbaines.net; Sun, 13 Jan 2019 10:48:10 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55600) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihza-0004y2-Ks for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gihzZ-0004pQ-Gx for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:06 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:58742) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gihzZ-0004pI-DR for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:05 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gihzZ-0008OO-BL for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:05 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#34060] [PATCH 09/10] guix package: '--list-available' can use data from the cache. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 13 Jan 2019 15:48:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 34060 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 34060@debbugs.gnu.org Received: via spool by 34060-submit@debbugs.gnu.org id=B34060.154739447532182 (code B ref 34060); Sun, 13 Jan 2019 15:48:05 +0000 Received: (at 34060) by debbugs.gnu.org; 13 Jan 2019 15:47:55 +0000 Received: from localhost ([127.0.0.1]:58012 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzO-0008My-V1 for submit@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:55 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:49290) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzG-0008Lx-NA for 34060@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:47 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 22697156E; Sun, 13 Jan 2019 16:47:46 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id woEYQMFG9eTS; Sun, 13 Jan 2019 16:47:45 +0100 (CET) Received: from gnu.org (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 727471975; Sun, 13 Jan 2019 16:47:42 +0100 (CET) From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 13 Jan 2019 16:47:32 +0100 Message-Id: <20190113154733.29737-9-ludo@gnu.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190113154733.29737-1-ludo@gnu.org> References: <20190113154733.29737-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/packages.scm (fold-available-packages): New procedure. * guix/scripts/package.scm (process-query): Use it instead of 'fold-packages'. * tests/packages.scm ("fold-available-packages with/without cache"): New test. --- gnu/packages.scm | 45 ++++++++++++++++++++++++++++++++++++++++ guix/scripts/package.scm | 45 ++++++++++++++++++++++------------------ tests/packages.scm | 22 ++++++++++++++++++++ 3 files changed, 92 insertions(+), 20 deletions(-) diff --git a/gnu/packages.scm b/gnu/packages.scm index cf655e7448..a1814205f9 100644 --- a/gnu/packages.scm +++ b/gnu/packages.scm @@ -53,6 +53,7 @@ %default-package-module-path fold-packages + fold-available-packages find-packages-by-name find-package-locations @@ -182,6 +183,50 @@ flags." directory)) %load-path))) +(define (fold-available-packages proc init) + "Fold PROC over the list of available packages. For each available package, +PROC is called along these lines: + + (PROC NAME VERSION RESULT + #:outputs OUTPUTS + #:location LOCATION + …) + +PROC can use #:allow-other-keys to ignore the bits it's not interested in. +When a package cache is available, this procedure does not actually load any +package module." + (define cache + (load-package-cache (current-profile))) + + (if (and cache (cache-is-authoritative?)) + (vhash-fold (lambda (name vector result) + (match vector + (#(name version module symbol outputs + supported? deprecated? + file line column) + (proc name version result + #:outputs outputs + #:location (and file + (location file line column)) + #:supported? supported? + #:deprecated? deprecated?)))) + init + cache) + (fold-packages (lambda (package result) + (proc (package-name package) + (package-version package) + result + #:outputs (package-outputs package) + #:location (package-location package) + #:supported? + (->bool + (member (%current-system) + (package-supported-systems package))) + #:deprecated? + (->bool + (package-superseded package)))) + init))) + (define* (fold-packages proc init #:optional (modules (all-modules (%package-module-path) diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm index 4f483ac141..e6f633b630 100644 --- a/guix/scripts/package.scm +++ b/guix/scripts/package.scm @@ -741,29 +741,34 @@ processed, #f otherwise." (('list-available regexp) (let* ((regexp (and regexp (make-regexp* regexp))) - (available (fold-packages - (lambda (p r) - (let ((n (package-name p))) - (if (and (supported-package? p) - (not (package-superseded p))) - (if regexp - (if (regexp-exec regexp n) - (cons p r) - r) - (cons p r)) - r))) + (available (fold-available-packages + (lambda* (name version result + #:key outputs location + supported? superseded? + #:allow-other-keys) + (if (and supported? (not superseded?)) + (if regexp + (if (regexp-exec regexp name) + (cons `(,name ,version + ,outputs ,location) + result) + result) + (cons `(,name ,version + ,outputs ,location) + result)) + result)) '()))) (leave-on-EPIPE - (for-each (lambda (p) - (format #t "~a\t~a\t~a\t~a~%" - (package-name p) - (package-version p) - (string-join (package-outputs p) ",") - (location->string (package-location p)))) + (for-each (match-lambda + ((name version outputs location) + (format #t "~a\t~a\t~a\t~a~%" + name version + (string-join outputs ",") + (location->string location)))) (sort available - (lambda (p1 p2) - (string))) #t) From patchwork Sun Jan 13 15:47:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 730 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 2559816A42; Sun, 13 Jan 2019 15:48:10 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id DAC88169F7 for ; Sun, 13 Jan 2019 15:48:09 +0000 (GMT) Received: from localhost ([127.0.0.1]:41621 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihzd-0004zy-Bo for patchwork@mira.cbaines.net; Sun, 13 Jan 2019 10:48:09 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55599) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gihza-0004y1-KW for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gihzZ-0004q4-Uy for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:06 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:58743) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gihzZ-0004pu-Rg for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:05 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gihzZ-0008OV-Pk for guix-patches@gnu.org; Sun, 13 Jan 2019 10:48:05 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#34060] [PATCH 10/10] status: Distinguish 'package-cache' profile hook. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 13 Jan 2019 15:48:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 34060 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 34060@debbugs.gnu.org Received: via spool by 34060-submit@debbugs.gnu.org id=B34060.154739447532189 (code B ref 34060); Sun, 13 Jan 2019 15:48:05 +0000 Received: (at 34060) by debbugs.gnu.org; 13 Jan 2019 15:47:55 +0000 Received: from localhost ([127.0.0.1]:58014 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzP-0008N1-Ad for submit@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:55 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:49298) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gihzH-0008Lz-69 for 34060@debbugs.gnu.org; Sun, 13 Jan 2019 10:47:47 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 988AC184B; Sun, 13 Jan 2019 16:47:46 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3tqugQ4TwCbM; Sun, 13 Jan 2019 16:47:46 +0100 (CET) Received: from gnu.org (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id C952818EF; Sun, 13 Jan 2019 16:47:42 +0100 (CET) From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 13 Jan 2019 16:47:33 +0100 Message-Id: <20190113154733.29737-10-ludo@gnu.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190113154733.29737-1-ludo@gnu.org> References: <20190113154733.29737-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/status.scm (hook-message): Handle 'package-cache'. --- guix/status.scm | 2 ++ 1 file changed, 2 insertions(+) diff --git a/guix/status.scm b/guix/status.scm index 2928733257..5b339bdec7 100644 --- a/guix/status.scm +++ b/guix/status.scm @@ -314,6 +314,8 @@ on." (G_ "building fonts directory...")) ('manual-database (G_ "building database for manual pages...")) + ('package-cache ;package cache generated by 'guix pull' + (G_ "building package cache...")) (_ #f))) (define* (print-build-event event old-status status