From patchwork Thu Dec 26 18:01:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Baines X-Patchwork-Id: 19452 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 24E6017992; Thu, 26 Dec 2019 18:02:13 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id DA80017992 for ; Thu, 26 Dec 2019 18:02:12 +0000 (GMT) Received: from localhost ([::1]:55642 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ikXSe-0008Ri-43 for patchwork@mira.cbaines.net; Thu, 26 Dec 2019 13:02:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:60643) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ikXSV-0008NN-8H for guix-patches@gnu.org; Thu, 26 Dec 2019 13:02:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ikXSU-0004d0-3R for guix-patches@gnu.org; Thu, 26 Dec 2019 13:02:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:50242) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ikXST-0004cw-W0 for guix-patches@gnu.org; Thu, 26 Dec 2019 13:02:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ikXST-0007hH-UK for guix-patches@gnu.org; Thu, 26 Dec 2019 13:02:01 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#38754] [PATCH 1/2] guix: lint: Add an optional parameter for a store connection. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 26 Dec 2019 18:02:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 38754 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 38754@debbugs.gnu.org Received: via spool by 38754-submit@debbugs.gnu.org id=B38754.157738326929469 (code B ref 38754); Thu, 26 Dec 2019 18:02:01 +0000 Received: (at 38754) by debbugs.gnu.org; 26 Dec 2019 18:01:09 +0000 Received: from localhost ([127.0.0.1]:56213 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ikXRd-0007fF-40 for submit@debbugs.gnu.org; Thu, 26 Dec 2019 13:01:09 -0500 Received: from mira.cbaines.net ([212.71.252.8]:45644) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ikXRZ-0007f0-Uv for 38754@debbugs.gnu.org; Thu, 26 Dec 2019 13:01:07 -0500 Received: from localhost (140.126.9.51.dyn.plus.net [51.9.126.140]) by mira.cbaines.net (Postfix) with ESMTPSA id DE2F617928 for <38754@debbugs.gnu.org>; Thu, 26 Dec 2019 18:01:04 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id c15c81ca for <38754@debbugs.gnu.org>; Thu, 26 Dec 2019 18:01:04 +0000 (UTC) From: Christopher Baines Date: Thu, 26 Dec 2019 18:01:03 +0000 Message-Id: <20191226180104.10888-1-mail@cbaines.net> X-Mailer: git-send-email 2.24.1 In-Reply-To: <8736d7rnzg.fsf@cbaines.net> References: <8736d7rnzg.fsf@cbaines.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Previously, the derivation lint checker establishes a connection to the store for each supported system of each package. This change uses the same store connection for all supported systems, with the option of setting a parameter for a store connection which will be used instead of establishing a new connection. Previously, running the derivation linter for all packages would take around 6 and a half minutes, with this change, without setting the %lint-checker-store-connection parameter, the time is reduced to around 4 minutes. * guix/lint.scm (%lint-checker-store-connection): New parameter. (check-derivation): Arrange the code so that it's possible to either run with the store from the new parameter, or open a new connection via the with-store syntax. --- guix/lint.scm | 42 ++++++++++++++++++++++++++---------------- 1 file changed, 26 insertions(+), 16 deletions(-) diff --git a/guix/lint.scm b/guix/lint.scm index cd2ea571ed..19498db857 100644 --- a/guix/lint.scm +++ b/guix/lint.scm @@ -100,7 +100,9 @@ lint-checker? lint-checker-name lint-checker-description - lint-checker-check)) + lint-checker-check + + %lint-checker-store-connection)) ;;; @@ -142,6 +144,9 @@ ((_ package (G_ message) rest ...) (%make-warning package message rest ...)))) +(define %lint-checker-store-connection + (make-parameter #f)) + ;;; ;;; Checkers @@ -887,7 +892,7 @@ descriptions maintained upstream." (define (check-derivation package) "Emit a warning if we fail to compile PACKAGE to a derivation." - (define (try system) + (define (try store system) (catch #t (lambda () (guard (c ((store-protocol-error? c) @@ -900,25 +905,30 @@ descriptions maintained upstream." (G_ "failed to create ~a derivation: ~a") (list system (condition-message c))))) - (with-store store - ;; Disable grafts since it can entail rebuilds. - (parameterize ((%graft? #f)) - (package-derivation store package system #:graft? #f) - - ;; If there's a replacement, make sure we can compute its - ;; derivation. - (match (package-replacement package) - (#f #t) - (replacement - (package-derivation store replacement system - #:graft? #f))))))) + ;; Disable grafts since it can entail rebuilds. + (parameterize ((%graft? #f)) + (package-derivation store package system #:graft? #f) + + ;; If there's a replacement, make sure we can compute its + ;; derivation. + (match (package-replacement package) + (#f #t) + (replacement + (package-derivation store replacement system + #:graft? #f)))))) (lambda args (make-warning package (G_ "failed to create ~a derivation: ~s") (list system args))))) - (filter lint-warning? - (map try (package-supported-systems package)))) + (define (check-with-store store) + (filter lint-warning? + (map (cut try store <>) (package-supported-systems package)))) + + (or (and=> (%lint-checker-store-connection) + check-with-store) + (with-store store + (check-with-store store)))) (define (check-license package) "Warn about type errors of the 'license' field of PACKAGE." From patchwork Thu Dec 26 18:01:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Baines X-Patchwork-Id: 19453 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id A46F8179A0; Thu, 26 Dec 2019 18:02:13 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 6DDA517928 for ; Thu, 26 Dec 2019 18:02:13 +0000 (GMT) Received: from localhost ([::1]:55640 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ikXSe-0008RI-S0 for patchwork@mira.cbaines.net; Thu, 26 Dec 2019 13:02:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:60646) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ikXSV-0008Nn-KP for guix-patches@gnu.org; Thu, 26 Dec 2019 13:02:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ikXSU-0004dH-IU for guix-patches@gnu.org; Thu, 26 Dec 2019 13:02:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:50243) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ikXSU-0004dB-Fd for guix-patches@gnu.org; Thu, 26 Dec 2019 13:02:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ikXSU-0007hP-E3 for guix-patches@gnu.org; Thu, 26 Dec 2019 13:02:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#38754] [PATCH 2/2] scripts: lint: Set the %link-checker-store-connection parameter. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 26 Dec 2019 18:02:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 38754 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 38754@debbugs.gnu.org Received: via spool by 38754-submit@debbugs.gnu.org id=B38754.157738327129484 (code B ref 38754); Thu, 26 Dec 2019 18:02:02 +0000 Received: (at 38754) by debbugs.gnu.org; 26 Dec 2019 18:01:11 +0000 Received: from localhost ([127.0.0.1]:56215 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ikXRf-0007fU-Cx for submit@debbugs.gnu.org; Thu, 26 Dec 2019 13:01:11 -0500 Received: from mira.cbaines.net ([212.71.252.8]:45646) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ikXRZ-0007f1-Tx for 38754@debbugs.gnu.org; Thu, 26 Dec 2019 13:01:09 -0500 Received: from localhost (140.126.9.51.dyn.plus.net [51.9.126.140]) by mira.cbaines.net (Postfix) with ESMTPSA id 021F717992 for <38754@debbugs.gnu.org>; Thu, 26 Dec 2019 18:01:04 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 5f0a2294 for <38754@debbugs.gnu.org>; Thu, 26 Dec 2019 18:01:04 +0000 (UTC) From: Christopher Baines Date: Thu, 26 Dec 2019 18:01:04 +0000 Message-Id: <20191226180104.10888-2-mail@cbaines.net> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20191226180104.10888-1-mail@cbaines.net> References: <8736d7rnzg.fsf@cbaines.net> <20191226180104.10888-1-mail@cbaines.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches If set, this parameter provides a store connection used by the derivation linter. Without this being set, the derivation linter establishes a new connection for each package. With this change, I saw the time taken to lint all packages with the derivation linter drop from over 4 minutes to around 3 minutes. * guix/scripts/lint.scm (guix-lint): Set the %lint-checker-store-connection) parameter. --- guix/scripts/lint.scm | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/guix/scripts/lint.scm b/guix/scripts/lint.scm index 8d08c484f5..47c104217d 100644 --- a/guix/scripts/lint.scm +++ b/guix/scripts/lint.scm @@ -28,6 +28,7 @@ (define-module (guix scripts lint) #:use-module (guix packages) + #:use-module (guix store) #:use-module (guix lint) #:use-module (guix ui) #:use-module (guix scripts) @@ -167,12 +168,15 @@ run the checkers on all packages.\n")) (_ #f)) (reverse opts))) (checkers (or (assoc-ref opts 'checkers) %all-checkers))) - (cond - ((assoc-ref opts 'list?) - (list-checkers-and-exit checkers)) - ((null? args) - (fold-packages (lambda (p r) (run-checkers p checkers)) '())) - (else - (for-each (lambda (spec) - (run-checkers (specification->package spec) checkers)) - args))))) + (with-store store + (parameterize + ((%lint-checker-store-connection store)) + (cond + ((assoc-ref opts 'list?) + (list-checkers-and-exit checkers)) + ((null? args) + (fold-packages (lambda (p r) (run-checkers p checkers)) '())) + (else + (for-each (lambda (spec) + (run-checkers (specification->package spec) checkers)) + args)))))))