From patchwork Mon Dec 7 15:39:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mathieu Othacehe X-Patchwork-Id: 25639 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 2A6C527BC00; Mon, 7 Dec 2020 15:41:11 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 099C027BC01 for ; Mon, 7 Dec 2020 15:41:10 +0000 (GMT) Received: from localhost ([::1]:38874 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmIdR-0003LV-8N for patchwork@mira.cbaines.net; Mon, 07 Dec 2020 10:41:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45230) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kmIdK-0003L4-Kh for guix-patches@gnu.org; Mon, 07 Dec 2020 10:41:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:43459) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kmIdK-0001R9-DP for guix-patches@gnu.org; Mon, 07 Dec 2020 10:41:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kmIdK-0004Gn-Bi for guix-patches@gnu.org; Mon, 07 Dec 2020 10:41:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45104] pull: Add a "with-substitutes" option. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 07 Dec 2020 15:41:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 45104 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: 45104@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.160735560516320 (code B ref -1); Mon, 07 Dec 2020 15:41:02 +0000 Received: (at submit) by debbugs.gnu.org; 7 Dec 2020 15:40:05 +0000 Received: from localhost ([127.0.0.1]:55002 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmIcP-0004F9-23 for submit@debbugs.gnu.org; Mon, 07 Dec 2020 10:40:05 -0500 Received: from lists.gnu.org ([209.51.188.17]:59678) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmIcM-0004Ez-Md for submit@debbugs.gnu.org; Mon, 07 Dec 2020 10:40:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:44856) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kmIcM-0002Zp-Dc for guix-patches@gnu.org; Mon, 07 Dec 2020 10:40:02 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:57408) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmIcM-0000wv-4V for guix-patches@gnu.org; Mon, 07 Dec 2020 10:40:02 -0500 Received: from [2a01:e0a:19b:d9a0:34f0:bcad:cad1:16d6] (port=34604 helo=cervin) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kmIcL-0006bB-7Q for guix-patches@gnu.org; Mon, 07 Dec 2020 10:40:01 -0500 From: Mathieu Othacehe Date: Mon, 07 Dec 2020 16:39:59 +0100 Message-ID: <87eek1vd4g.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Hello, When "guix pull" is run before the CI server is done baking substitutes, the user may need to build some derivations, which can be long on low-end hardware. This patch adds an option to "guix pull" so that it updates to the latest commit with available substitutes. This work is still in progress, but I'd like to gather some impressions on that before going further. Thanks, Mathieu From d399f8dbb9e38a82241b9048b8b04758fae10005 Mon Sep 17 00:00:00 2001 From: Mathieu Othacehe Date: Mon, 7 Dec 2020 16:12:22 +0100 Subject: [PATCH] scripts: pull: Add "with-substitutes" option. --- guix/scripts/pull.scm | 38 +++++++++++++++++++++++++++++++++++--- 1 file changed, 35 insertions(+), 3 deletions(-) diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm index 83cdc1d1eb..4609f8614e 100644 --- a/guix/scripts/pull.scm +++ b/guix/scripts/pull.scm @@ -2,6 +2,7 @@ ;;; Copyright © 2013, 2014, 2015, 2017, 2018, 2019, 2020 Ludovic Courtès ;;; Copyright © 2017 Marius Bakke ;;; Copyright © 2020 Tobias Geerinckx-Rice +;;; Copyright © 2020 Mathieu Othacehe ;;; ;;; This file is part of GNU Guix. ;;; @@ -20,6 +21,7 @@ (define-module (guix scripts pull) #:use-module ((guix ui) #:hide (display-profile-content)) + #:use-module (guix ci) #:use-module (guix colors) #:use-module (guix utils) #:use-module ((guix status) #:select (with-status-verbosity)) @@ -64,6 +66,7 @@ #:re-export (display-profile-content channel-commit-hyperlink) #:export (channel-list + find-lastest-commit-with-substitutes guix-pull)) @@ -169,6 +172,9 @@ Download and deploy the latest version of Guix.\n")) (lambda (opt name arg result) (alist-cons 'validate-pull warn-about-backward-updates result))) + (option '("with-substitutes") #f #f + (lambda (opt name arg result) + (alist-cons 'with-substitutes? #t result))) (option '("disable-authentication") #f #f (lambda (opt name arg result) (alist-cons 'authenticate-channels? #f result))) @@ -526,6 +532,26 @@ true, display what would be built without actually building it." (leave (G_ "while creating symlink '~a': ~a~%") link (strerror (system-error-errno args)))))))) + +;;; +;;; Substitutes. +;;; + +(define (find-lastest-commit-with-substitutes) + (let ((urls %default-substitute-urls)) + (any (lambda (url) + (let* ((build (match (latest-builds url 1 + #:job "guix.x86_64-linux" + #:status 0) ;success + ((build) build))) + (evaluation (evaluation url + (build-evaluation build))) + (commit (match (evaluation-checkouts evaluation) + ((checkout) + (checkout-commit checkout))))) + commit)) + urls))) + ;;; ;;; Queries. @@ -731,8 +757,9 @@ Use '~/.config/guix/channels.scm' instead.")) (let ((ref (assoc-ref opts 'ref)) (url (or (assoc-ref opts 'repository-url) - (environment-variable)))) - (if (or ref url) + (environment-variable))) + (with-substitutes? (assoc-ref opts 'with-substitutes?))) + (if (or ref url with-substitutes?) (match (find guix-channel? channels) ((? channel? guix) ;; Apply '--url', '--commit', and '--branch' to the 'guix' channel. @@ -745,7 +772,12 @@ Use '~/.config/guix/channels.scm' instead.")) (channel (inherit guix) (url url) (commit #f) (branch branch))) (#f - (channel (inherit guix) (url url)))) + (let ((commit + (and with-substitutes? + (find-lastest-commit-with-substitutes)))) + (channel (inherit guix) + (url url) + (commit commit))))) (remove guix-channel? channels)))) (#f ;no 'guix' channel, failure will ensue channels)) -- 2.29.2