From patchwork Wed Apr 21 12:21:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Othacehe X-Patchwork-Id: 28721 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 4ADE627BC79; Wed, 21 Apr 2021 13:22:16 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id EE0B627BC7B for ; Wed, 21 Apr 2021 13:22:10 +0100 (BST) Received: from localhost ([::1]:41408 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lZBru-0002Xq-5Z for patchwork@mira.cbaines.net; Wed, 21 Apr 2021 08:22:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48628) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lZBrm-0002X3-Ng for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:45576) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lZBrm-00079q-GI for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lZBrm-0005a5-A5 for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#47929] [PATCH 1/5] ci: Add manifest support to channel-with-substitutes-available. References: <20210421121610.2045-1-othacehe@gnu.org> In-Reply-To: <20210421121610.2045-1-othacehe@gnu.org> Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 21 Apr 2021 12:22:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47929 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 47929@debbugs.gnu.org Cc: Mathieu Othacehe Received: via spool by 47929-submit@debbugs.gnu.org id=B47929.161900768921373 (code B ref 47929); Wed, 21 Apr 2021 12:22:02 +0000 Received: (at 47929) by debbugs.gnu.org; 21 Apr 2021 12:21:29 +0000 Received: from localhost ([127.0.0.1]:57111 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lZBrE-0005YX-VN for submit@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:29 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35874) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lZBrD-0005YB-0x for 47929@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:27 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54763) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lZBr7-0006lD-SI for 47929@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:21 -0400 Received: from [2a01:e0a:19b:d9a0:9576:8bbf:4795:82ee] (port=38154 helo=localhost.localdomain) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lZBr6-0003pa-P3; Wed, 21 Apr 2021 08:21:21 -0400 From: Mathieu Othacehe Date: Wed, 21 Apr 2021 14:21:04 +0200 Message-Id: <20210421122108.2344-1-othacehe@gnu.org> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/ci.scm (%default-guix-specification, %default-package-specification): New variables. (, ): New records. (job, job-history, sort-history-by-coverage, channel-commit, package->job-name, manifest->jobs): New procedures. (find-latest-commit-with-substitutes): Rename it into ... (latest-checkouts-with-substitutes): ... this new procedure. (channel-with-substitutes-available): Add an optional manifest argument and honor it. * doc/guix.texi (Channels with Substitutes): Update it. --- doc/guix.texi | 31 ++++++-- guix/ci.scm | 205 ++++++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 207 insertions(+), 29 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index b9019d5550..c39bbdb3d5 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -5201,11 +5201,32 @@ server at @url{https://ci.guix.gnu.org}. "https://ci.guix.gnu.org")) @end lisp -Note that this does not mean that all the packages that you will -install after running @command{guix pull} will have available -substitutes. It only ensures that @command{guix pull} will not try to -compile package definitions. This is particularly useful when using -machines with limited resources. +It is also possible to ask @command{guix pull} to use the latest commit +with the maximal number of available substitutes for a given manifest +this way: + +@lisp +(use-modules (guix ci)) + +(list (channel-with-substitutes-available + %default-guix-channel + "https://ci.guix.gnu.org" + "/path/to/manifest)) +@end lisp + +or this way: + +@lisp +(use-modules (guix ci)) + +(list (channel-with-substitutes-available + %default-guix-channel + "https://ci.guix.gnu.org" + (specifications->manifest + '("git" "emacs-minimal")))) +@end lisp + +This is particularly useful when using machines with limited resources. @node Creating a Channel @section Creating a Channel diff --git a/guix/ci.scm b/guix/ci.scm index c70e5bb9e6..780e90ef32 100644 --- a/guix/ci.scm +++ b/guix/ci.scm @@ -18,10 +18,16 @@ ;;; along with GNU Guix. If not, see . (define-module (guix ci) + #:use-module (gnu packages) + #:use-module (guix channels) #:use-module (guix http-client) + #:use-module (guix packages) + #:use-module (guix profiles) + #:use-module (guix ui) #:use-module (guix utils) #:use-module (json) #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) #:use-module (ice-9 match) #:use-module (guix i18n) #:use-module (guix diagnostics) @@ -58,6 +64,7 @@ latest-evaluations evaluations-for-commit + manifest->jobs channel-with-substitutes-available)) ;;; Commentary: @@ -67,6 +74,14 @@ ;;; ;;; Code: +;; The name of the CI specification building the 'guix-modular' package. +(define %default-guix-specification + (make-parameter "guix")) + +;; The default name of the CI specification building all the packages. +(define %default-package-specification + (make-parameter "master")) + (define-json-mapping make-build-product build-product? json->build-product @@ -109,6 +124,24 @@ (map json->checkout (vector->list checkouts))))) +(define-json-mapping make-job job? + json->job + (name job-name) ;string + (build job-build) ;integer + (status job-status)) ;integer + +(define-json-mapping make-history history? + json->history + (evaluation history-evaluation) ;integer + (checkouts history-checkouts "checkouts" ;* + (lambda (checkouts) + (map json->checkout + (vector->list checkouts)))) + (jobs history-jobs "jobs" + (lambda (jobs) + (map json->job + (vector->list jobs))))) + (define %query-limit ;; Max number of builds requested in queries. 1000) @@ -172,34 +205,158 @@ as one of their inputs." (evaluation-checkouts evaluation))) (latest-evaluations url limit))) -(define (find-latest-commit-with-substitutes url) - "Return the latest commit with available substitutes for the Guix package -definitions at URL. Return false if no commit were found." - (let* ((job-name (string-append "guix." (%current-system))) - (build (match (latest-builds url 1 - #:job job-name - #:status 0) ;success - ((build) build) - (_ #f))) - (evaluation (and build - (evaluation url (build-evaluation build)))) - (commit (and evaluation - (match (evaluation-checkouts evaluation) - ((checkout) - (checkout-commit checkout)))))) - commit)) - -(define (channel-with-substitutes-available chan url) +(define* (job url name #:key evaluation) + "Return the job which name is NAME for the given EVALUATION, from the CI +server at URL." + (map json->job + (vector->list + (json->scm + (http-fetch + (format #f "~a/api/jobs?evaluation=~a&names=~a" + url evaluation name)))))) + +(define* (jobs-history url jobs + #:key + (specification "master") + (limit 20)) + "Return the job history for the SPECIFICATION jobs which names are part of +the JOBS list, from the CI server at URL. Limit the history to the latest +LIMIT evaluations. " + (let ((names (string-join jobs ","))) + (map json->history + (vector->list + (json->scm + (http-fetch + (format #f "~a/api/jobs/history?spec=~a&names=~a&nr=~a" + url specification names (number->string limit)))))))) + +(define (sort-history-by-coverage history) + "Sort and return the given evaluation HISTORY list by descending successful +jobs count. This means that the first element of the list will be the +evaluation with the higher successful jobs count." + (let ((coverage + (map (cut fold + (lambda (status prev) + (if (eq? status 0) ;successful + (1+ prev) + prev)) + 0 <>) + (map (compose + (cut map job-status <>) history-jobs) + history)))) + (map (match-lambda + ((cov . hist) hist)) + (sort (map cons coverage history) + (match-lambda* + (((c1 . h1) (c2 . h2)) + (> c1 c2))))))) + +(define (channel-commit checkouts channel) + "Return the CHANNEL commit from CHECKOUTS." + (any (lambda (checkout) + (and (string=? (checkout-channel checkout) channel) + (checkout-commit checkout))) + checkouts)) + +(define (package->job-name package) + "Return the CI job name for the given PACKAGE name." + (string-append package "." (%current-system))) + +(define (manifest->jobs manifest) + "Return the list of job names that are part of the given MANIFEST." + (define (load-manifest file) + (let ((user-module (make-user-module '((guix profiles) (gnu))))) + (load* file user-module))) + + (let* ((manifest (cond + ((string? manifest) + (load-manifest manifest)) + ((manifest? manifest) + manifest) + (else #f))) + (packages (delete-duplicates + (map manifest-entry-item + (manifest-transitive-entries manifest)) + eq?))) + (map (lambda (package) + (package->job-name (package-name package))) + packages))) + +(define* (latest-checkouts-with-substitutes url jobs) + "Return a list of latest checkouts, sorted by descending substitutes +coverage of the given JOBS list on the CI server at URL. Only evaluations for +which the Guix package is built are considered. + +If JOBS is false, return a list of latest checkouts for which the Guix package +is built. Return false if no checkouts were found." + (define guix-history + (filter (lambda (hist) + (let ((jobs (history-jobs hist))) + (match jobs + ((job) + (eq? (job-status job) 0)) + (else #f)))) + (jobs-history url (list (package->job-name "guix")) + #:specification + (%default-guix-specification)))) + + (define (guix-commit checkouts) + (let ((name (symbol->string + (channel-name %default-guix-channel)))) + (channel-commit checkouts name))) + + (define (guix-package-available? hist) + (any (lambda (guix-hist) + (string=? (guix-commit + (history-checkouts hist)) + (guix-commit + (history-checkouts guix-hist))) + hist) + guix-history)) + + (define (first-checkout checkouts) + (match checkouts + ((checkouts _ ...) + checkouts) + (() #f))) + + (if jobs + (let* ((jobs-history + (sort-history-by-coverage + (jobs-history url jobs + #:specification + (%default-package-specification)))) + (checkouts + (map history-checkouts + (filter-map guix-package-available? + jobs-history)))) + (first-checkout checkouts)) + (first-checkout + (map history-checkouts guix-history)))) + +(define* (channel-with-substitutes-available chan url + #:optional manifest) "Return a channel inheriting from CHAN but which commit field is set to the latest commit with available substitutes for the Guix package definitions at -URL. The current system is taken into account. +URL. If the MANIFEST argument is passed, return the latest commit with the +maximal substitutes coverage of MANIFEST. MANIFEST can be an absolute path as +a string, or a record. The current system is taken into account. If no commit with available substitutes were found, the commit field is set to false and a warning message is printed." - (let ((commit (find-latest-commit-with-substitutes url))) - (unless commit + (let* ((jobs (and manifest + (manifest->jobs manifest))) + (checkouts + (latest-checkouts-with-substitutes url jobs))) + (unless checkouts (warning (G_ "could not find available substitutes at ~a~%") url)) - (channel - (inherit chan) - (commit commit)))) + (let* ((name (channel-name chan)) + (name-str (if (symbol? name) + (symbol->string name) + name)) + (commit (and checkouts + (channel-commit checkouts name-str)))) + (channel + (inherit chan) + (commit commit))))) From patchwork Wed Apr 21 12:21:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Othacehe X-Patchwork-Id: 28725 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 4C42E27BC79; Wed, 21 Apr 2021 13:22:44 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id A1A1C27BC7A for ; Wed, 21 Apr 2021 13:22:43 +0100 (BST) Received: from localhost ([::1]:42024 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lZBsQ-0002oE-Rq for patchwork@mira.cbaines.net; Wed, 21 Apr 2021 08:22:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48632) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lZBrn-0002XA-0m for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:45577) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lZBrm-0007AQ-Ps for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lZBrm-0005aC-MU for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#47929] [PATCH 2/5] scripts: pull: Load (gnu packages) module. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 21 Apr 2021 12:22:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47929 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 47929@debbugs.gnu.org Cc: Mathieu Othacehe Received: via spool by 47929-submit@debbugs.gnu.org id=B47929.161900769021380 (code B ref 47929); Wed, 21 Apr 2021 12:22:02 +0000 Received: (at 47929) by debbugs.gnu.org; 21 Apr 2021 12:21:30 +0000 Received: from localhost ([127.0.0.1]:57114 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lZBrF-0005Yc-Mb for submit@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:30 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35904) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lZBrE-0005YD-8p for 47929@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:28 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54764) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lZBr9-0006ll-3S for 47929@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:23 -0400 Received: from [2a01:e0a:19b:d9a0:9576:8bbf:4795:82ee] (port=38154 helo=localhost.localdomain) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lZBr7-0003pa-Ky; Wed, 21 Apr 2021 08:21:22 -0400 From: Mathieu Othacehe Date: Wed, 21 Apr 2021 14:21:05 +0200 Message-Id: <20210421122108.2344-2-othacehe@gnu.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421122108.2344-1-othacehe@gnu.org> References: <20210421122108.2344-1-othacehe@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches This allows to pass a manifest to channel-with-substitutes-available this way: (channel-with-substitutes-available %default-guix-channel "https://ci.guix.gnu.org" (specifications->manifest '("git" "emacs-minimal"))) * guix/scripts/pull.scm (channel-list): Load the (gnu packages) module when evaluating the user channels list. --- guix/scripts/pull.scm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm index 07613240a8..662239b492 100644 --- a/guix/scripts/pull.scm +++ b/guix/scripts/pull.scm @@ -707,7 +707,8 @@ transformations specified in OPTS (resulting from '--url', '--commit', or (string-append %sysconfdir "/guix/channels.scm")) (define (load-channels file) - (let ((result (load* file (make-user-module '((guix channels)))))) + (let ((result (load* file (make-user-module '((guix channels) + (gnu packages)))))) (if (and (list? result) (every channel? result)) result (leave (G_ "'~a' did not return a list of channels~%") file)))) From patchwork Wed Apr 21 12:21:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Othacehe X-Patchwork-Id: 28722 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id D4E6927BC79; Wed, 21 Apr 2021 13:22:16 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 6C2E527BC7C for ; Wed, 21 Apr 2021 13:22:11 +0100 (BST) Received: from localhost ([::1]:41434 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lZBru-0002Yb-Gu for patchwork@mira.cbaines.net; Wed, 21 Apr 2021 08:22:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48642) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lZBrn-0002XH-Du for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:45578) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lZBrn-0007Aw-6H for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lZBrn-0005aK-2l for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#47929] [PATCH 3/5] ci: Add dashboard procedures. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 21 Apr 2021 12:22:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47929 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 47929@debbugs.gnu.org Cc: Mathieu Othacehe Received: via spool by 47929-submit@debbugs.gnu.org id=B47929.161900769521404 (code B ref 47929); Wed, 21 Apr 2021 12:22:03 +0000 Received: (at 47929) by debbugs.gnu.org; 21 Apr 2021 12:21:35 +0000 Received: from localhost ([127.0.0.1]:57118 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lZBrL-0005Z9-6u for submit@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:35 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35906) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lZBrF-0005YF-4C for 47929@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:29 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54765) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lZBr9-0006mc-Vk for 47929@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:23 -0400 Received: from [2a01:e0a:19b:d9a0:9576:8bbf:4795:82ee] (port=38154 helo=localhost.localdomain) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lZBr8-0003pa-Tj; Wed, 21 Apr 2021 08:21:23 -0400 From: Mathieu Othacehe Date: Wed, 21 Apr 2021 14:21:06 +0200 Message-Id: <20210421122108.2344-3-othacehe@gnu.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421122108.2344-1-othacehe@gnu.org> References: <20210421122108.2344-1-othacehe@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/ci.scm (dashboard-url, dashboard-register): New procedures. --- guix/ci.scm | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/guix/ci.scm b/guix/ci.scm index 780e90ef32..78ab739340 100644 --- a/guix/ci.scm +++ b/guix/ci.scm @@ -65,6 +65,8 @@ evaluations-for-commit manifest->jobs + dashboard-url + dashboard-register channel-with-substitutes-available)) ;;; Commentary: @@ -282,6 +284,26 @@ evaluation with the higher successful jobs count." (package->job-name (package-name package))) packages))) +(define (dashboard-url url id) + "Return the url of the dashboard with the given ID on the CI server at URL." + (format #f "~a/dashboard/~a" url id)) + +(define* (dashboard-register url packages + #:key + (specification "master")) + "Register a dashboard for the packages jobs of the given SPECIFICATION using +the CI server at URL. Returns the newly created dashboard id or false if it +could not be created." + (let* ((jobs (manifest->jobs + (packages->manifest packages))) + (names (string-join jobs ",")) + (id (json->scm + (http-fetch + (format #f "~a/api/dashboard/register?spec=~a&names=~a" + url specification names))))) + (and id + (assoc-ref id "id")))) + (define* (latest-checkouts-with-substitutes url jobs) "Return a list of latest checkouts, sorted by descending substitutes coverage of the given JOBS list on the CI server at URL. Only evaluations for From patchwork Wed Apr 21 12:21:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Othacehe X-Patchwork-Id: 28723 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id E38ED27BC79; Wed, 21 Apr 2021 13:22:17 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 9F10227BC7A for ; Wed, 21 Apr 2021 13:22:13 +0100 (BST) Received: from localhost ([::1]:41536 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lZBrw-0002bE-NX for patchwork@mira.cbaines.net; Wed, 21 Apr 2021 08:22:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48648) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lZBrn-0002XP-Qn for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:45579) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lZBrn-0007BN-JD for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lZBrn-0005aR-Fr for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#47929] [PATCH 4/5] scripts: weather: Add packages dashboard support. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 21 Apr 2021 12:22:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47929 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 47929@debbugs.gnu.org Cc: Mathieu Othacehe Received: via spool by 47929-submit@debbugs.gnu.org id=B47929.161900769621411 (code B ref 47929); Wed, 21 Apr 2021 12:22:03 +0000 Received: (at 47929) by debbugs.gnu.org; 21 Apr 2021 12:21:36 +0000 Received: from localhost ([127.0.0.1]:57120 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lZBrL-0005ZB-He for submit@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:35 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35908) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lZBrH-0005YI-Ow for 47929@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:31 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54766) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lZBrC-0006nq-Jr for 47929@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:26 -0400 Received: from [2a01:e0a:19b:d9a0:9576:8bbf:4795:82ee] (port=38154 helo=localhost.localdomain) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lZBr9-0003pa-Vd; Wed, 21 Apr 2021 08:21:24 -0400 From: Mathieu Othacehe Date: Wed, 21 Apr 2021 14:21:07 +0200 Message-Id: <20210421122108.2344-4-othacehe@gnu.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421122108.2344-1-othacehe@gnu.org> References: <20210421122108.2344-1-othacehe@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/scripts/weather.scm (display-dashboard-url): New procedure. (guix-weather): Call it. --- guix/scripts/weather.scm | 32 +++++++++++++++++++++++--------- 1 file changed, 23 insertions(+), 9 deletions(-) diff --git a/guix/scripts/weather.scm b/guix/scripts/weather.scm index 5164fe0494..be0b2e3509 100644 --- a/guix/scripts/weather.scm +++ b/guix/scripts/weather.scm @@ -499,6 +499,17 @@ SERVER. Display information for packages with at least THRESHOLD dependents." #f systems)))) +(define (display-dashboard-url server packages) + "Display a link to the dashboard for PACKAGES on the given CI SERVER." + (let* ((id (dashboard-register server packages)) + (url (and id (dashboard-url server id)))) + (when url + (format #t "~%") + (format #t (G_ "The packages dashboard is available ~a.~%") + (if (supports-hyperlinks?) + (hyperlink url (G_ "here")) + (format #f "here: ~a" url)))))) + ;;; ;;; Entry point. @@ -554,15 +565,18 @@ SERVER. Display information for packages with at least THRESHOLD dependents." (report-server-coverage server items #:display-missing? (assoc-ref opts 'display-missing?))) - (match (assoc-ref opts 'coverage) - (#f #f) - (threshold - ;; PACKAGES may include non-package objects coming from a - ;; manifest. Filter them out. - (report-package-coverage server - (filter package? packages) - systems - #:threshold threshold))) + + ;; PACKAGES may include non-package objects coming from a + ;; manifest. Filter them out. + (let ((packages (filter package? packages))) + (match (assoc-ref opts 'coverage) + (#f #f) + (threshold + (report-package-coverage server + packages + systems + #:threshold threshold))) + (display-dashboard-url server packages)) (= 1 coverage)) urls)))))) From patchwork Wed Apr 21 12:21:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Othacehe X-Patchwork-Id: 28724 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id E6DF727BC7B; Wed, 21 Apr 2021 13:22:43 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 7CED927BC79 for ; Wed, 21 Apr 2021 13:22:43 +0100 (BST) Received: from localhost ([::1]:42004 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lZBsQ-0002nj-Lh for patchwork@mira.cbaines.net; Wed, 21 Apr 2021 08:22:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48650) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lZBro-0002Xn-7t for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:45580) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lZBrn-0007Bn-VA for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lZBrn-0005aY-S5 for guix-patches@gnu.org; Wed, 21 Apr 2021 08:22:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#47929] [PATCH 5/5] ui: Disable hyperlink support inside screen. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 21 Apr 2021 12:22:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47929 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 47929@debbugs.gnu.org Cc: Mathieu Othacehe Received: via spool by 47929-submit@debbugs.gnu.org id=B47929.161900769621417 (code B ref 47929); Wed, 21 Apr 2021 12:22:03 +0000 Received: (at 47929) by debbugs.gnu.org; 21 Apr 2021 12:21:36 +0000 Received: from localhost ([127.0.0.1]:57122 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lZBrL-0005ZI-RG for submit@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:36 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35910) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lZBrI-0005YK-JL for 47929@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:32 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54767) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lZBrD-0006p0-Ec for 47929@debbugs.gnu.org; Wed, 21 Apr 2021 08:21:27 -0400 Received: from [2a01:e0a:19b:d9a0:9576:8bbf:4795:82ee] (port=38154 helo=localhost.localdomain) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lZBrC-0003pa-GI; Wed, 21 Apr 2021 08:21:26 -0400 From: Mathieu Othacehe Date: Wed, 21 Apr 2021 14:21:08 +0200 Message-Id: <20210421122108.2344-5-othacehe@gnu.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421122108.2344-1-othacehe@gnu.org> References: <20210421122108.2344-1-othacehe@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Inside screen, the OSC escape sequence is interpreted but the link is not clickable. * guix/ui.scm (supports-hyperlinks?): Return false if STY is set. --- guix/ui.scm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/guix/ui.scm b/guix/ui.scm index 7fbd4c63a2..56fbbb3db3 100644 --- a/guix/ui.scm +++ b/guix/ui.scm @@ -1486,7 +1486,8 @@ documented at ;; However, Emacs comint as of 26.3 does not ignore it and instead lets it ;; through, hence the 'INSIDE_EMACS' special case below. (and (isatty?* port) - (not (getenv "INSIDE_EMACS")))) + (not (or (getenv "INSIDE_EMACS") + (getenv "STY"))))) ;screen doesn't support hyperlinks. (define* (file-hyperlink file #:optional (text file)) "Return TEXT with escapes for a hyperlink to FILE."