From patchwork Wed Dec 19 20:13:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Efraim Flashner X-Patchwork-Id: 491 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id BFA5B169A3; Wed, 19 Dec 2018 20:14:15 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) by mira.cbaines.net (Postfix) with ESMTP id DC006169A1 for ; Wed, 19 Dec 2018 20:14:14 +0000 (GMT) Received: from localhost ([::1]:33976 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gZiEP-00048F-IA for patchwork@mira.cbaines.net; Wed, 19 Dec 2018 15:14:13 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58296) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gZiEJ-00046S-0Q for guix-patches@gnu.org; Wed, 19 Dec 2018 15:14:11 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gZiEE-0004DR-K2 for guix-patches@gnu.org; Wed, 19 Dec 2018 15:14:06 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:50858) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gZiEE-0004DI-F1 for guix-patches@gnu.org; Wed, 19 Dec 2018 15:14:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gZiEE-0002Hu-8m for guix-patches@gnu.org; Wed, 19 Dec 2018 15:14:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#33805] [PATCH]: refresh: Allow searching recursively Resent-From: Efraim Flashner Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 19 Dec 2018 20:14:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 33805 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 33805@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.15452504248749 (code B ref -1); Wed, 19 Dec 2018 20:14:02 +0000 Received: (at submit) by debbugs.gnu.org; 19 Dec 2018 20:13:44 +0000 Received: from localhost ([127.0.0.1]:55116 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gZiDv-0002H2-QR for submit@debbugs.gnu.org; Wed, 19 Dec 2018 15:13:44 -0500 Received: from eggs.gnu.org ([208.118.235.92]:48550) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gZiDt-0002Gq-TP for submit@debbugs.gnu.org; Wed, 19 Dec 2018 15:13:42 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gZiDj-0003pL-0k for submit@debbugs.gnu.org; Wed, 19 Dec 2018 15:13:36 -0500 Received: from lists.gnu.org ([2001:4830:134:3::11]:48720) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gZiDg-0003oD-Nc for submit@debbugs.gnu.org; Wed, 19 Dec 2018 15:13:30 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58126) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gZiDe-0003f2-1X for guix-patches@gnu.org; Wed, 19 Dec 2018 15:13:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gZiDY-0003lC-BW for guix-patches@gnu.org; Wed, 19 Dec 2018 15:13:25 -0500 Received: from flashner.co.il ([178.62.234.194]:47138) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gZiDW-0003jb-Gr for guix-patches@gnu.org; Wed, 19 Dec 2018 15:13:20 -0500 Received: from localhost (unknown [141.226.9.73]) by flashner.co.il (Postfix) with ESMTPSA id DF56440041 for ; Wed, 19 Dec 2018 20:13:14 +0000 (UTC) Date: Wed, 19 Dec 2018 22:13:13 +0200 From: Efraim Flashner Message-ID: <20181219201313.GC2581@macbook41> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.11.0 (2018-11-25) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Currently no test cases since `guix refresh` needs network access. I currently have two implementations for refresh-recursive, the commented one I started with and was adding cases which caused errors for guix refresh. The second, with par-for-each¹, has an undefined return value, so it doesn't care about errors (libuv-julia, autoconf-wrapper, etc). I'm pretty sure the par-for-each implementation is faster too. ¹ https://www.gnu.org/software/guile/manual/guile.html#Parallel-Forms diff --git a/doc/guix.texi b/doc/guix.texi index 20b5013fd..2888d1efb 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -7370,6 +7370,22 @@ are many packages, though, for which it lacks a method to determine whether a new upstream release is available. However, the mechanism is extensible, so feel free to get in touch with us to add a new method! +@table @code + +@item --recursive +Consider the packages specified, and all the packages upon which they depend. + +@example +$ guix refresh --recursive coreutils +gnu/packages/acl.scm:35:2: warning: no updater for acl +gnu/packages/m4.scm:30:12: info: 1.4.18 is already the latest version of m4 +gnu/packages/xml.scm:68:2: warning: no updater for expat +gnu/packages/multiprecision.scm:40:12: info: 6.1.2 is already the latest version of gmp +@dots{} +@end example + +@end table + Sometimes the upstream name differs from the package name used in Guix, and @command{guix refresh} needs a little help. Most updaters honor the @code{upstream-name} property in package definitions, which can be used @@ -7543,6 +7559,22 @@ hop@@2.4.0 geiser@@0.4 notmuch@@0.18 mu@@0.9.9.5 cflow@@1.4 idutils@@4.6 @dots{} The command above lists a set of packages that could be built to check for compatibility with an upgraded @code{flex} package. +@table @code + +@item --list-recursive +List all the packages which one or more packages depend upon. + +@example +$ guix refresh --list-recursive flex +flex@2.6.4 depends on the following 25 packages: perl@5.28.0 help2man@1.47.6 +bison@3.0.5 indent@2.2.10 tar@1.30 gzip@1.9 bzip2@1.0.6 xz@5.2.4 file@5.33 @dote{} +@end example + +@end table + +The command above lists a set of packages which, when changed, would cause +@code{flex} to be rebuilt. + The following options can be used to customize GnuPG operation: @table @code diff --git a/guix/scripts/refresh.scm b/guix/scripts/refresh.scm index 1d86f949c..08fd25118 100644 --- a/guix/scripts/refresh.scm +++ b/guix/scripts/refresh.scm @@ -5,6 +5,7 @@ ;;; Copyright © 2015 Alex Kost ;;; Copyright © 2016 Ben Woodcroft ;;; Copyright © 2017 Mathieu Othacehe +;;; Copyright © 2018 Efraim Flashner ;;; ;;; This file is part of GNU Guix. ;;; @@ -40,6 +41,7 @@ #:use-module (ice-9 regex) #:use-module (ice-9 vlist) #:use-module (ice-9 format) + #:use-module (ice-9 threads) ; par-for-each #:use-module (srfi srfi-1) #:use-module (srfi srfi-11) #:use-module (srfi srfi-26) @@ -88,6 +90,12 @@ (option '(#\l "list-dependent") #f #f (lambda (opt name arg result) (alist-cons 'list-dependent? #t result))) + (option '(#\r "recursive") #f #f + (lambda (opt name arg result) + (alist-cons 'recursive? #t result))) + (option '("list-recursive") #f #f + (lambda (opt name arg result) + (alist-cons 'list-recursive? #t result))) (option '("keyring") #t #f (lambda (opt name arg result) @@ -140,6 +148,10 @@ specified with `--select'.\n")) (display (G_ " -l, --list-dependent list top-level dependent packages that would need to be rebuilt as a result of upgrading PACKAGE...")) + (display (G_ " + -r, --recursive check the PACKAGE and its inputs for upgrades")) + (display (G_ " + --list-recursive list all the packages that PACKAGE depends on")) (newline) (display (G_ " --keyring=FILE use FILE as the keyring of upstream OpenPGP keys")) @@ -323,6 +335,50 @@ dependent packages are rebuilt: ~{~a~^ ~}~%" (map full-name covering)))) (return #t)))) +(define (refresh-recursive packages) + "Check all of the package inputs of PACKAGES for newer upstream versions." + (mlet %store-monad ((edges (node-edges %bag-node-type + ;; Here we don't want the -boot0 packages. + (fold-packages cons '())))) + (let ((dependent (node-transitive-edges packages edges))) + ;; par-for-each has an undefined return value, so packages which cause + ;; errors can be ignored. + (par-for-each (lambda (package) + ;(for-each (lambda (package) + ; (unless (any (cute string-suffix? <> package) + ; ;; A partial list of package suffixes that + ; ;; cause guix-refresh to error. + ; '("bootstrap" "for-build" "intermediate" + ; "minimal" "union" "without-mesa" "wrapper")) + (guix-refresh package)) + ; ) + (map package-name dependent))) + (return #t))) + +(define (list-recursive packages) + "List all the things that would cause PACKAGES to be rebuilt if they are changed." + ;; Using %BAG-NODE-TYPE is more accurate than using %PACKAGE-NODE-TYPE + ;; because it includes implicit dependencies. + (define (full-name package) + (string-append (package-name package) "@" + (package-version package))) + + (mlet %store-monad ((edges (node-edges %bag-node-type + ;; Here we don't want the -boot0 packages. + (fold-packages cons '())))) + (let ((dependent (node-transitive-edges packages edges))) + (match packages + ((x) + (format (current-output-port) + (G_ "~a depends on the following ~d packages: ~{~a~^ ~}~%.") + (full-name x) (length dependent) (map full-name dependent))) + (lst + (format (current-output-port) + (G_ "The following ~d packages \ +all are dependent packages: ~{~a~^ ~}~%") + (length dependent) (map full-name dependent)))) + (return #t)))) + ;;; ;;; Manifest. @@ -402,7 +458,9 @@ update would trigger a complete rebuild." (let* ((opts (parse-options)) (update? (assoc-ref opts 'update?)) (updaters (options->updaters opts)) + (recursive? (assoc-ref opts 'recursive?)) (list-dependent? (assoc-ref opts 'list-dependent?)) + (list-recursive? (assoc-ref opts 'list-recursive?)) (key-download (assoc-ref opts 'key-download)) ;; Warn about missing updaters when a package is explicitly given on @@ -441,6 +499,10 @@ update would trigger a complete rebuild." (cond (list-dependent? (list-dependents packages)) + (list-recursive? + (list-recursive packages)) + (recursive? + (refresh-recursive packages)) (update? (parameterize ((%openpgp-key-server (or (assoc-ref opts 'key-server)