From patchwork Thu Oct 19 20:33:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 55062 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 5859627BBEA; Thu, 19 Oct 2023 21:34:47 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 82A2127BBE2 for ; Thu, 19 Oct 2023 21:34:42 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtZj2-0006JI-Cp; Thu, 19 Oct 2023 16:34:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtZj1-0006Im-JM for guix-patches@gnu.org; Thu, 19 Oct 2023 16:34:35 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qtZj1-0006zs-Bj for guix-patches@gnu.org; Thu, 19 Oct 2023 16:34:35 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qtZjR-0000wC-M2; Thu, 19 Oct 2023 16:35:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42146] [PATCH 1/3] build: Relocate record and associated procedures here. References: <20200630220913.14665-1-kuba@kadziolka.net> In-Reply-To: <20200630220913.14665-1-kuba@kadziolka.net> Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: kuba@kadziolka.net, maxim.cournoyer@gmail.com, guix-patches@gnu.org Resent-Date: Thu, 19 Oct 2023 20:35:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42146 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 42146@debbugs.gnu.org Cc: Maxim Cournoyer , Jakub =?utf-8?b?S8SFZHpp?= =?utf-8?b?b8WCa2E=?= , Maxim Cournoyer X-Debbugs-Original-Xcc: Jakub =?utf-8?b?S8SFZHppb8WCa2E=?= , Maxim Cournoyer Received: via spool by 42146-submit@debbugs.gnu.org id=B42146.16977476873567 (code B ref 42146); Thu, 19 Oct 2023 20:35:01 +0000 Received: (at 42146) by debbugs.gnu.org; 19 Oct 2023 20:34:47 +0000 Received: from localhost ([127.0.0.1]:37942 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtZj9-0000vI-AK for submit@debbugs.gnu.org; Thu, 19 Oct 2023 16:34:47 -0400 Received: from mail-qk1-x72c.google.com ([2607:f8b0:4864:20::72c]:53679) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtZj3-0000uv-P6 for 42146@debbugs.gnu.org; Thu, 19 Oct 2023 16:34:41 -0400 Received: by mail-qk1-x72c.google.com with SMTP id af79cd13be357-7789cb322deso3477585a.3 for <42146@debbugs.gnu.org>; Thu, 19 Oct 2023 13:34:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697747645; x=1698352445; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=b8visCM1R1nOmFaZmfOgo/UFjx68QHhn/qnzsKsQeAg=; b=ZN3ZytrOSkoPV3PTQaE1mAar2RlNnWmqgMs1eUJnLtPW5QYVB8f3z+ETfhqDnFw1DS XyavybpqF03vu1wUn37gRq17HNTkbrOjgAw4oyHai8tgBky7hY6NGNRQaA/QxtH0hIAW Pb0u8CKyr2zoBfMzP4MnLVK05A79UctLns1v85Ta/8/wkalN6l36ICOFAi7ESSKQon4+ 1Ae8JJTz3BU8D0tV1gLHkHWCJ1HetFbnWCCw56TCEjuCuy7/Hbdtjw62LL5lqX1PJ2Em T5U0x3yELnvQ1o/Z7GAqvIOeIP4v7BWHxMdrbd2lt6sY9z6JIlAoh3YHe0WyIL4bC4oO eWBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697747645; x=1698352445; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=b8visCM1R1nOmFaZmfOgo/UFjx68QHhn/qnzsKsQeAg=; b=ZxcAquP9GiFSEo55HE/e7UF9MDsvAS9ZPWpca4BgE2OTuZn++bxCM8Vz4pWkIgiu8n SCyNxBgNdbpxrkHJyZivfM2vWBjhk42Cg6BOvpQ5VcbK7Lv6DbTDMTCXwJu7JWjhcWPd Lr/9mdtMdzqjth6e+o4LJtDKp1QVlwkrZs8MPGYzSd1XrF/uqqcfE4iVMcbhD1y3Lrjb 6Y9/YmGl0na0+4auI/al8y10TAzAOSvDKRcwLda6EB77r3PZ7ACOvRn0WcU6borfd1XS JwfcqC9PBFfIF5slrnZEnMm5GbbtnYtoS9tYFfWLUFgG1dtJM2rc5Kdk1RSRPuV7zzIh I87A== X-Gm-Message-State: AOJu0Yy4UcYO5QrjiSDMvsyioUbWrb8JzRF4JdfVrpGJwbDOvFyxRhSO zG+bBTapLnbVAlTYyTfp3ccFG1eoEnfN8Q== X-Google-Smtp-Source: AGHT+IFA9mzRE/o856F/91XgvZa7sQElpq+DUlYXjrLLxGzsRTInleEEv8Plk+fACFYJFEpXs4lzng== X-Received: by 2002:a05:620a:4548:b0:76f:2899:3a96 with SMTP id u8-20020a05620a454800b0076f28993a96mr3923889qkp.0.1697747644695; Thu, 19 Oct 2023 13:34:04 -0700 (PDT) Received: from localhost.localdomain (dsl-156-111.b2b2c.ca. [66.158.156.111]) by smtp.gmail.com with ESMTPSA id 2-20020a05620a070200b0077413b342e9sm76591qkc.128.2023.10.19.13.34.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 13:34:03 -0700 (PDT) From: Maxim Cournoyer Date: Thu, 19 Oct 2023 16:33:32 -0400 Message-ID: X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * etc/teams.scm.in (, make-regexp*, regexp*-exec): Move to... * guix/build/utils.scm: ... here. (list-matches*): New procedure. Change-Id: I566ac372f7d8ba08de94e19b54dcc68da2106a23 --- etc/teams.scm.in | 19 +------------------ guix/build/utils.scm | 39 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+), 18 deletions(-) base-commit: d59653b7c9e43ebdbba20e2ca071429507f94c67 diff --git a/etc/teams.scm.in b/etc/teams.scm.in index 55242caad1..8af25b9802 100644 --- a/etc/teams.scm.in +++ b/etc/teams.scm.in @@ -34,29 +34,12 @@ (srfi srfi-9) (srfi srfi-26) (ice-9 format) - (ice-9 regex) (ice-9 match) (ice-9 rdelim) + (guix build utils) (guix ui) (git)) -(define-record-type - (%make-regexp* pat flag rx) - regexp*? - (pat regexp*-pattern) - (flag regexp*-flag) - (rx regexp*-rx)) - -;;; Work around regexp implementation. -;;; This record allows to track the regexp pattern and then display it. -(define* (make-regexp* pat #:optional (flag regexp/extended)) - "Alternative to `make-regexp' producing annotated objects." - (%make-regexp* pat flag (make-regexp pat flag))) - -(define (regexp*-exec rx* str) - "Execute the RX* regexp, a object." - (regexp-exec (regexp*-rx rx*) str)) - (define-record-type (make-team id name description members scope) team? diff --git a/guix/build/utils.scm b/guix/build/utils.scm index 8e630ad586..2b3a8e278b 100644 --- a/guix/build/utils.scm +++ b/guix/build/utils.scm @@ -9,6 +9,7 @@ ;;; Copyright © 2020, 2021 Maxim Cournoyer ;;; Copyright © 2021, 2022 Maxime Devos ;;; Copyright © 2021 Brendan Tildesley +;;; Copyright © 2022 Simon Tournier ;;; Copyright © 2023 Carlo Zancanaro ;;; ;;; This file is part of GNU Guix. @@ -28,6 +29,7 @@ (define-module (guix build utils) #:use-module (srfi srfi-1) + #:use-module (srfi srfi-9) #:use-module (srfi srfi-11) #:use-module (srfi srfi-26) #:use-module (srfi srfi-34) @@ -55,6 +57,14 @@ (define-module (guix build utils) package-name->name+version parallel-job-count + make-regexp* + regexp*-exec + regexp*? + regexp*-pattern + regexp*-flag + regexp*-rx + list-matches* + compressor tarball? %xz-parallel-args @@ -163,6 +173,35 @@ (define-syntax-rule (define-constant name val) (module-replace! (current-module) '(setvbuf))) (else #f)) + +;;; +;;; Extend regexp objects with a pattern field. +;;; +(define-record-type + (%make-regexp* pat flag rx) + regexp*? + (pat regexp*-pattern) ;the regexp pattern, a string + (flag regexp*-flag) ;regexp flags + (rx regexp*-rx)) ;the compiled regexp object + +;;; Work around regexp implementation. +;;; This record allows to track the regexp pattern and then display it. +(define* (make-regexp* pat #:optional (flag regexp/extended)) + "Alternative to `make-regexp' producing annotated objects." + (%make-regexp* pat flag (make-regexp pat flag))) + +(define (regexp*-exec rx* str) + "Execute the RX* regexp, a object." + (regexp-exec (regexp*-rx rx*) str)) + +(define* (list-matches* regexp str #:optional (flags regexp/extended)) + "Like 'list-matches', but also accepting a regexp* as REGEXP." + (match regexp + ((or (? string?) (? regexp?)) + (list-matches regexp str flags)) + ((? regexp*?) + (list-matches (regexp*-rx regexp) str flags)))) + ;;; ;;; Compression helpers. From patchwork Thu Oct 19 20:33:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 55063 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 7E09B27BBE9; Thu, 19 Oct 2023 21:35:01 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 66C3527BBE2 for ; Thu, 19 Oct 2023 21:34:59 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtZj3-0006Ke-RZ; Thu, 19 Oct 2023 16:34:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtZj1-0006Iu-VH for guix-patches@gnu.org; Thu, 19 Oct 2023 16:34:35 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qtZj1-0006zz-MV for guix-patches@gnu.org; Thu, 19 Oct 2023 16:34:35 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qtZjS-0000wI-4d; Thu, 19 Oct 2023 16:35:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42146] [PATCH 2/3] build: substitute: Error when no substitutions were done. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: kuba@kadziolka.net, maxim.cournoyer@gmail.com, guix-patches@gnu.org Resent-Date: Thu, 19 Oct 2023 20:35:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42146 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 42146@debbugs.gnu.org Cc: Jakub =?utf-8?b?S8SFZHppb8WCa2E=?= , Maxim Cournoyer , Jakub =?utf-8?b?S8SFZHppb8WC?= =?utf-8?b?a2E=?= , Maxim Cournoyer X-Debbugs-Original-Xcc: Jakub =?utf-8?b?S8SFZHppb8WCa2E=?= , Maxim Cournoyer Received: via spool by 42146-submit@debbugs.gnu.org id=B42146.16977476883573 (code B ref 42146); Thu, 19 Oct 2023 20:35:02 +0000 Received: (at 42146) by debbugs.gnu.org; 19 Oct 2023 20:34:48 +0000 Received: from localhost ([127.0.0.1]:37944 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtZjD-0000vU-8E for submit@debbugs.gnu.org; Thu, 19 Oct 2023 16:34:47 -0400 Received: from mail-qk1-x72d.google.com ([2607:f8b0:4864:20::72d]:47290) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtZj6-0000uz-DZ for 42146@debbugs.gnu.org; Thu, 19 Oct 2023 16:34:41 -0400 Received: by mail-qk1-x72d.google.com with SMTP id af79cd13be357-7788db95652so3878185a.2 for <42146@debbugs.gnu.org>; Thu, 19 Oct 2023 13:34:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697747648; x=1698352448; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oTAbCUEBY4/PaAlLb3JRnahAdJdeX/zaSLEJfY7+90g=; b=aigGrMq5FpRYclutciNokFrhbtz5FbX8/+vifefVTLPWmE6/OA5/tVoojMfaJcXZ2Z 0hzO5oSgshZ5p4o2hS9/rksQRywXE7d4I3d8SODSiarDrLKn/3rEICH0Pjc9ndIg3QWF BN3UypVZSFu3Kfx1rPtHJY1DHNhG8rew6u1u68UeNp1R/yENLp0zyPu/jlePukzWmJHB Ag3rinz+5fn7QHn9mn5ZVpSLK//X9aVf1hx0vgrskD8itsVICYINn0Ok36PqoNHW7rkK XGrOhqXonWe/GrwpsT/B2KK+EiOzkPF/1hVJNGvURb9eggwx2NTerRgfxQWniKvLXrWT G98A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697747648; x=1698352448; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oTAbCUEBY4/PaAlLb3JRnahAdJdeX/zaSLEJfY7+90g=; b=IADZxhjW81S5sZXWI4KRaCu9Y7pSues3bHaW0ZtJrjeJRZszBHwDHNIFzLI5YF4h8W EFy6qCNR8+ElPAMt6yA/AI9bR+qUiTdrMNRMM3UNu5kLbOpPyuo4CaS851w9cnZ2+yxA qOWzTtoT5dcqdiS4+eCXLPg+z9ChpLRezZpwkNgb7nlNTJywQNNsf4cIpMPUCdJrIoDA wT79PgUYSww/sKDdt50u7lPDdO2hiy0HOawzhEPxigDIx5YqiaCbYOGS/5Ee+Al5DfE/ 9wDjpRJ5e2GFPQw8ADK2+B6KrFDyvvEkit99K+QjIAPKfIAOcplRAG+S6jzJ6/kOYKwx jWsw== X-Gm-Message-State: AOJu0Yzz7GcXz3qcHVlpl1ACsfT/UMSRLZwWXGhIwWgF0AbFDgLoP9XK LIhVf8UWPJ+YYfHtRSMp+ELYu5LRg7J3ZA== X-Google-Smtp-Source: AGHT+IH3r47NdrEpahDPXiMGSZCz+rO/hZa9dCzcOtNVbE0KEcgI/qFhbXrZPzm4LX87wc0K+K5MrA== X-Received: by 2002:a05:620a:29c2:b0:775:903e:388c with SMTP id s2-20020a05620a29c200b00775903e388cmr3062568qkp.2.1697747647630; Thu, 19 Oct 2023 13:34:07 -0700 (PDT) Received: from localhost.localdomain (dsl-156-111.b2b2c.ca. [66.158.156.111]) by smtp.gmail.com with ESMTPSA id 2-20020a05620a070200b0077413b342e9sm76591qkc.128.2023.10.19.13.34.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 13:34:07 -0700 (PDT) From: Maxim Cournoyer Date: Thu, 19 Oct 2023 16:33:33 -0400 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches From: Jakub Kądziołka * guix/build/utils.scm (substitute, substitute*) [require-matches?]: New argument. * tests/build-utils.scm ("substitute*"): New test group. ("substitute*, no match error") ("substitute*, partial no match error"): New tests. Co-authored-by: Maxim Cournoyer Change-Id: I66ed33d72aa73cd35e5642521efec70bf756f86e --- guix/build/utils.scm | 94 +++++++++++++++++++++++++++++++++---------- tests/build-utils.scm | 63 +++++++++++++++++++---------- 2 files changed, 114 insertions(+), 43 deletions(-) diff --git a/guix/build/utils.scm b/guix/build/utils.scm index 2b3a8e278b..7bfb6560e1 100644 --- a/guix/build/utils.scm +++ b/guix/build/utils.scm @@ -6,7 +6,8 @@ ;;; Copyright © 2018, 2022 Arun Isaac ;;; Copyright © 2018, 2019 Ricardo Wurmus ;;; Copyright © 2020 Efraim Flashner -;;; Copyright © 2020, 2021 Maxim Cournoyer +;;; Copyright © 2020 Jakub Kądziołka +;;; Copyright © 2020, 2021, 2023 Maxim Cournoyer ;;; Copyright © 2021, 2022 Maxime Devos ;;; Copyright © 2021 Brendan Tildesley ;;; Copyright © 2022 Simon Tournier @@ -971,24 +972,53 @@ (define (replace-char c1 c2 s) c)) s))) -(define (substitute file pattern+procs) +(define* (substitute file pattern+procs #:key (require-matches? #t)) "PATTERN+PROCS is a list of regexp/two-argument-procedure pairs. For each line of FILE, and for each PATTERN that it matches, call the corresponding PROC as (PROC LINE MATCHES); PROC must return the line that will be written as a substitution of the original line. Be careful about using '$' to match the -end of a line; by itself it won't match the terminating newline of a line." - (let ((rx+proc (map (match-lambda - (((? regexp? pattern) . proc) +end of a line; by itself it won't match the terminating newline of a line. + +By default, SUBSTITUTE will raise a &message condition if one of the patterns +fails to match. REQUIRE-MATCHES? can be set to false when lack of matches is +acceptable (e.g. if you have multiple potential patterns not guaranteed to be +found in FILE)." + (define (rx->pattern m) + (match m + ((? regexp? pattern) + "") + ((? regexp*? pattern) + (regexp*-pattern pattern)) + ((? string? pattern) + pattern))) + + (define (make-condition failed-matches) + (condition + (&message + (message (format #f "substitute: `~a': no match for patterns `~a'" + file failed-matches))))) + + (let ((rx+proc (map (match-lambda + (((or (? regexp? pattern) (? regexp*? pattern)) . proc) (cons pattern proc)) ((pattern . proc) - (cons (make-regexp pattern regexp/extended) - proc))) - pattern+procs))) + (cons (make-regexp* pattern regexp/extended) proc))) + pattern+procs))) (with-atomic-file-replacement file (lambda (in out) - (let loop ((line (read-line in 'concat))) + (let loop ((line (read-line in 'concat)) + (ok-matches '()) + (failed-matches '())) (if (eof-object? line) - #t + (when require-matches? + (let ((failed-patterns (lset-difference + string=? + (delete-duplicates + (map rx->pattern failed-matches)) + (delete-duplicates + (map rx->pattern ok-matches))))) + (when (not (null? failed-patterns)) + (raise (make-condition failed-patterns))))) ;; Work around the fact that Guile's regexp-exec does not handle ;; NUL characters (a limitation of the underlying GNU libc's ;; regexec) by temporarily replacing them by an unused private @@ -998,19 +1028,30 @@ (define (substitute file pattern+procs) (unused-private-use-code-point line)) #\nul)) (line* (replace-char #\nul nul* line)) - (line1* (fold (lambda (r+p line) - (match r+p - ((regexp . proc) - (match (list-matches regexp line) - ((and m+ (_ _ ...)) - (proc line m+)) - (_ line))))) - line* - rx+proc)) + (results ;line, ok-matches and failed-matches + (fold (lambda (r+p results) + (let ((line (first results)) + (ok-matches (second results)) + (failed-matches (third results))) + (match r+p + ((regexp . proc) + (match (list-matches* regexp line) + ((and m+ (_ _ ...)) + (list (proc line m+) + (cons regexp ok-matches) + failed-matches)) + (_ + (list line + ok-matches + (cons regexp failed-matches)))))))) + (list line* '() '()) + rx+proc)) + (line1* (first results)) + (ok-matches (second results)) + (failed-matches (third results)) (line1 (replace-char nul* #\nul line1*))) (display line1 out) - (loop (read-line in 'concat))))))))) - + (loop (read-line in 'concat) ok-matches failed-matches)))))))) (define-syntax let-matches ;; Helper macro for `substitute*'. @@ -1048,9 +1089,17 @@ (define-syntax substitute* Alternatively, FILE may be a list of file names, in which case they are all subject to the substitutions. +By default, SUBSTITUTE* will raise a &message condition if one of the patterns +fails to match on one of the files; REQUIRE-MATCHES? may be set to false to +avoid an error being raised in such condition. + Be careful about using '$' to match the end of a line; by itself it won't match the terminating newline of a line." ((substitute* file ((regexp match-var ...) body ...) ...) + (substitute* file #:require-matches? #t + ((regexp match-var ...) body ...) ...)) + ((substitute* file #:require-matches? require-matches? + ((regexp match-var ...) body ...) ...) (let () (define (substitute-one-file file-name) (substitute @@ -1074,7 +1123,8 @@ (define-syntax substitute* (begin body ...) (substring l o (match:start m)) r)))))))) - ...))) + ...) + #:require-matches? require-matches?)) (match file ((files (... ...)) diff --git a/tests/build-utils.scm b/tests/build-utils.scm index 3babf5d544..890fbca16f 100644 --- a/tests/build-utils.scm +++ b/tests/build-utils.scm @@ -1,7 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2012, 2015, 2016, 2019, 2020 Ludovic Courtès ;;; Copyright © 2019 Ricardo Wurmus -;;; Copyright © 2021 Maxim Cournoyer +;;; Copyright © 2021, 2023 Maxim Cournoyer ;;; Copyright © 2021 Maxime Devos ;;; Copyright © 2021 Brendan Tildesley ;;; @@ -289,26 +289,47 @@ (define (arg-test bash-args) (test-assert "wrap-script, argument handling, bash --norc" (arg-test " --norc")) -(test-equal "substitute*, text contains a NUL byte, UTF-8" - "c\0d" - (with-fluids ((%default-port-encoding "UTF-8") - (%default-port-conversion-strategy 'error)) - ;; The GNU libc is locale sensitive. Depending on the value of LANG, the - ;; test could fail with "string contains #\\nul character: ~S" or "cannot - ;; convert wide string to output locale". - (setlocale LC_ALL "en_US.UTF-8") - (call-with-temporary-output-file - (lambda (file port) - (format port "a\0b") - (flush-output-port port) - - (substitute* file - (("a") "c") - (("b") "d")) - - (with-input-from-file file - (lambda _ - (get-string-all (current-input-port)))))))) +(test-group "substitute*" + (define-syntax-rule (define-substitute*-test test-type name expected + content clauses ...) + (test-type + name + expected + (with-fluids ((%default-port-encoding "UTF-8") + (%default-port-conversion-strategy 'error)) + ;; The GNU libc is locale sensitive. Depending on the value of LANG, + ;; the test could fail with "string contains #\\nul character: ~S" or + ;; "cannot convert wide string to output locale". + (setlocale LC_ALL "en_US.UTF-8") + (call-with-temporary-output-file + (lambda (file port) + (format port content) + (flush-output-port port) + + (substitute* file + clauses ...) + + (with-input-from-file file + (lambda _ + (get-string-all (current-input-port))))))))) + + (define-substitute*-test test-equal + "substitute*, text contains a NUL byte, UTF-8" + "c\0d" ;expected + "a\0b" ;content + (("a") "c") + (("b") "d")) + + (define-substitute*-test test-error "substitute*, no match error" + #t ;expected + "a\0b" ;content + (("Oops!") "c")) + + (define-substitute*-test test-error "substitute*, partial no match error" + #t ;expected + "a\0b" ;content + (("a") "c" + ("Oops!") "c"))) (test-equal "search-input-file: exception if not found" `((path) From patchwork Thu Oct 19 20:33:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 55064 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 8F26527BBEA; Thu, 19 Oct 2023 21:35:17 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id C84DC27BBE2 for ; Thu, 19 Oct 2023 21:35:16 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtZj4-0006L4-6E; Thu, 19 Oct 2023 16:34:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtZj2-0006Iz-22 for guix-patches@gnu.org; Thu, 19 Oct 2023 16:34:36 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qtZj1-000701-Oy for guix-patches@gnu.org; Thu, 19 Oct 2023 16:34:35 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qtZjS-0000wR-G5; Thu, 19 Oct 2023 16:35:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42146] [PATCH 3/3] build: bootstrap-configure: Allow lack of matches in substitute. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: kuba@kadziolka.net, maxim.cournoyer@gmail.com, guix-patches@gnu.org Resent-Date: Thu, 19 Oct 2023 20:35:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42146 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 42146@debbugs.gnu.org Cc: Jakub =?utf-8?b?S8SFZHppb8WCa2E=?= , Maxim Cournoyer , Jakub =?utf-8?b?S8SFZHppb8WC?= =?utf-8?b?a2E=?= , Maxim Cournoyer X-Debbugs-Original-Xcc: Jakub =?utf-8?b?S8SFZHppb8WCa2E=?= , Maxim Cournoyer Received: via spool by 42146-submit@debbugs.gnu.org id=B42146.16977476893582 (code B ref 42146); Thu, 19 Oct 2023 20:35:02 +0000 Received: (at 42146) by debbugs.gnu.org; 19 Oct 2023 20:34:49 +0000 Received: from localhost ([127.0.0.1]:37946 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtZjF-0000vh-44 for submit@debbugs.gnu.org; Thu, 19 Oct 2023 16:34:49 -0400 Received: from mail-qk1-x736.google.com ([2607:f8b0:4864:20::736]:46194) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtZj9-0000v2-52 for 42146@debbugs.gnu.org; Thu, 19 Oct 2023 16:34:44 -0400 Received: by mail-qk1-x736.google.com with SMTP id af79cd13be357-7788fb06997so4984485a.0 for <42146@debbugs.gnu.org>; Thu, 19 Oct 2023 13:34:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697747650; x=1698352450; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r4Q9z/d9a8INLP5xbQe9cf87MBMZAubiof6Yyn2YFqk=; b=XU2qcXCic7CtC3U6rZZsrzgNdL6xmjp+0/8Fr3h05S4DHQSifeWWUySNaRSc5+Z3EZ o5e6+VJFvsFMBpSJhvkH+CsSamYQZZRu6XT0M6G2H/aV0oHuS0eYvXoNKnAAz5lhH09J O0tXpfx+2+hms9i4mafcEgOKdQ1tSiZwHoXK1K6w6diaQFXcbRW9V7IWrSBLfzmMLcNJ viRFwRclMi1BnA5Ocw1eLBU3hFxtWyH1ko2lGcYCvm32RgOQUzfZZ41/xlLq/gbX3oEO hfotn7Hz3TBmyEQr2qZ5KECiEt42cZMV/oHLvG6BgwLxgWpqis7J0z40Lz7J2lmd/fW4 EFgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697747650; x=1698352450; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=r4Q9z/d9a8INLP5xbQe9cf87MBMZAubiof6Yyn2YFqk=; b=S5D4/S/Ffe6ipqqSGYNiuJVp//GteoixmBgasgHLckwYFHR7lFR0AUDNeLVBOZ+7vm RLWcSkQPm5JD26QD9CCbjm/bVqM+/VY8IlTzB17JcuzSmIPLpH27Q4Ga4k6JqHPlVtm4 10d8V0Be6KIMOgt/MbvER1uqraXFmAjheHuIYTQicshU2/9oRR94MY8wMAeyF/35mPsz 8sdoikNT68TeSARVa6BerxvTkv0qUj1ot2DNNDNtFw7ziwsfy65DfoifcC6iV83kg7Oz mtVSPoAzwY2agXUeIXI/zlAijiBo328RewWGGFmWoNWyME2kGhW0Rk/+MufWgZ9kVjn4 pyyw== X-Gm-Message-State: AOJu0Yzx7/+rmIv2yHDqV8ylv6Zpvw8DK8iWHgEGG43YlxYeas4S4WSz gV79w+5AxBIi/i1HCLIRPQJiepiOfI2waw== X-Google-Smtp-Source: AGHT+IFAl3wHepBCuXv7hTlSDku/g7bxD+o60LKTzOyTWfkHQlZU7JJLfblDAc02FszRhsTDFCnjdA== X-Received: by 2002:a05:620a:430f:b0:76e:f638:bcd9 with SMTP id u15-20020a05620a430f00b0076ef638bcd9mr3523055qko.38.1697747650613; Thu, 19 Oct 2023 13:34:10 -0700 (PDT) Received: from localhost.localdomain (dsl-156-111.b2b2c.ca. [66.158.156.111]) by smtp.gmail.com with ESMTPSA id 2-20020a05620a070200b0077413b342e9sm76591qkc.128.2023.10.19.13.34.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 13:34:10 -0700 (PDT) From: Maxim Cournoyer Date: Thu, 19 Oct 2023 16:33:34 -0400 Message-ID: <28fd2bd2039b4a4c0fe8fdaf1adbfb5956f683d5.1697747385.git.maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches From: Jakub Kądziołka Matches are not required here, as not every file will use every variable. * guix/build/gnu-bootstrap.scm (bootstrap-configure): Pass #:require-matches? #f to substitute*. Signed-off-by: Maxim Cournoyer --- guix/build/gnu-bootstrap.scm | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/guix/build/gnu-bootstrap.scm b/guix/build/gnu-bootstrap.scm index b4257a3717..d044c8acd9 100644 --- a/guix/build/gnu-bootstrap.scm +++ b/guix/build/gnu-bootstrap.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2020, 2022 Timothy Sample +;;; Copyright © 2020 Jakub Kądziołka ;;; ;;; This file is part of GNU Guix. ;;; @@ -51,6 +52,7 @@ (define (bootstrap-configure name version modules scripts) (let ((target (string-drop-right template 3))) (copy-file template target) (substitute* target + #:require-matches? #f (("@PACKAGE_NAME@") name) (("@VERSION@") version)))) (append-map (lambda (dir) (find-files dir "\\.in$")) @@ -60,14 +62,14 @@ (define (bootstrap-configure name version modules scripts) (let ((target (string-drop-right template 3))) (copy-file template target) (substitute* target + #:require-matches? #f (("@GUILE@") guile) (("@MODDIR@") moddir) (("@GODIR@") godir)) (chmod target #o755))) (find-files scripts (lambda (fn st) - (string-suffix? ".in" fn)))) - #t))) + (string-suffix? ".in" fn))))))) (define (bootstrap-build modules) "Create a procedure that builds an early bootstrap package. The