From patchwork Fri Oct 20 00:57:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 55068 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 8DD7127BBE9; Fri, 20 Oct 2023 01:59:30 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id A5E1727BBE2 for ; Fri, 20 Oct 2023 01:59:25 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtdqZ-0003lt-F9; Thu, 19 Oct 2023 20:58:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtdqW-0003lP-Kn for guix-patches@gnu.org; Thu, 19 Oct 2023 20:58:37 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qtdqV-0007Mf-Pf; Thu, 19 Oct 2023 20:58:35 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qtdqw-0003kr-JB; Thu, 19 Oct 2023 20:59:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42146] [PATCH v3 1/3] build: Relocate record and associated procedures here. References: <20200630220913.14665-1-kuba@kadziolka.net> In-Reply-To: <20200630220913.14665-1-kuba@kadziolka.net> Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: kuba@kadziolka.net, maxim.cournoyer@gmail.com, ludo@gnu.org, guix-patches@gnu.org Resent-Date: Fri, 20 Oct 2023 00:59:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42146 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 42146@debbugs.gnu.org Cc: Maxim Cournoyer , Jakub =?utf-8?b?S8SFZHpp?= =?utf-8?b?b8WCa2E=?= , Maxim Cournoyer , Ludovic =?utf-8?q?Court=C3=A8s?= X-Debbugs-Original-Xcc: Jakub =?utf-8?b?S8SFZHppb8WCa2E=?= , Maxim Cournoyer , Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 42146-submit@debbugs.gnu.org id=B42146.169776351814380 (code B ref 42146); Fri, 20 Oct 2023 00:59:02 +0000 Received: (at 42146) by debbugs.gnu.org; 20 Oct 2023 00:58:38 +0000 Received: from localhost ([127.0.0.1]:38246 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtdqU-0003jb-Ix for submit@debbugs.gnu.org; Thu, 19 Oct 2023 20:58:38 -0400 Received: from mail-qv1-xf2f.google.com ([2607:f8b0:4864:20::f2f]:47167) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtdqO-0003j9-BQ for 42146@debbugs.gnu.org; Thu, 19 Oct 2023 20:58:32 -0400 Received: by mail-qv1-xf2f.google.com with SMTP id 6a1803df08f44-66d13ac2796so1888046d6.2 for <42146@debbugs.gnu.org>; Thu, 19 Oct 2023 17:58:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697763475; x=1698368275; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=b8visCM1R1nOmFaZmfOgo/UFjx68QHhn/qnzsKsQeAg=; b=fYv1je81APKTsJHbV9G021GqdvuqdJEOrNmp2WQ0WUiazmrxHuYtz55PnspYBzl4Jv +SJvcGjrxjruNQubqTyG7mBtoZsH/ahkc2MiJMfhEnoibnrTnGmG5VGE4d62N0PliE7w dRDRHSSoVQKm/kFSg5cYVNB0dOQkxSdKTDh0bIp/MaqY6WAdYy3yvx5pewi4cgMKgtAt ER8vRy2C7hcyzSIMWjQ1ey+ZZzIPG2eWu53aKXlEu+TtMUE3ZJmEeu1RcuAGRvgSCdbe RATJ1pr8FtANKROMccMw+9qnB3LJN6ZwIIlnRScAtDCDEn0+5RPoHwG6S8DDNxQ9m83y 2GCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697763475; x=1698368275; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=b8visCM1R1nOmFaZmfOgo/UFjx68QHhn/qnzsKsQeAg=; b=jitFJH448g91fKKm8O63DROdN0WXC6kDIFLu4/y8v5qBNghHnNqJoZh36eC6wHGfN5 guzbG/HSupt4R6wi8fzzYTMwR90i+5mRKQvzRSK2lrxj3zO8S4OB365NL/aSTcnFWWmc pP2RFI1dmoGCxOn2z2nhMIbe08Da4h+tzjW/Zp5sghLiRWV14uiq2Kc6w9ptWVEz3h4q KKOVPP4lkio5vKwUmK8lx9ZxAXPcuj90idZ1ZUYwdUd/1H6T3w81unkBJzp7kAJTDvCi oLFyNpMaPXieuzsBSLmYf29PlX6bstVMTnEVfmpSr6EQ6mT4fHHajGxsZnhakHqMtsRy qMVw== X-Gm-Message-State: AOJu0YwJ8ifcE6Qv/WP766lzudr0cvdBf66kYDUlFIbyaQdN0FfXmzir +z4UOR48fF/UXxh/W3FTkhFspNuNvwAVxQ== X-Google-Smtp-Source: AGHT+IEJVakeO2yux+B7vvrMP2473/ogqwLqgym3fKCN/Q3Do0ZvCANcovJqXZBFZSkb7PR+sQaKaQ== X-Received: by 2002:a05:6214:230f:b0:66d:62b7:53f4 with SMTP id gc15-20020a056214230f00b0066d62b753f4mr517492qvb.45.1697763475332; Thu, 19 Oct 2023 17:57:55 -0700 (PDT) Received: from localhost.localdomain (dsl-156-111.b2b2c.ca. [66.158.156.111]) by smtp.gmail.com with ESMTPSA id h9-20020a0cf8c9000000b0065d0a4262e0sm279097qvo.70.2023.10.19.17.57.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 17:57:54 -0700 (PDT) From: Maxim Cournoyer Date: Thu, 19 Oct 2023 20:57:38 -0400 Message-ID: X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * etc/teams.scm.in (, make-regexp*, regexp*-exec): Move to... * guix/build/utils.scm: ... here. (list-matches*): New procedure. Change-Id: I566ac372f7d8ba08de94e19b54dcc68da2106a23 --- etc/teams.scm.in | 19 +------------------ guix/build/utils.scm | 39 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+), 18 deletions(-) base-commit: d59653b7c9e43ebdbba20e2ca071429507f94c67 diff --git a/etc/teams.scm.in b/etc/teams.scm.in index 55242caad1..8af25b9802 100644 --- a/etc/teams.scm.in +++ b/etc/teams.scm.in @@ -34,29 +34,12 @@ (srfi srfi-9) (srfi srfi-26) (ice-9 format) - (ice-9 regex) (ice-9 match) (ice-9 rdelim) + (guix build utils) (guix ui) (git)) -(define-record-type - (%make-regexp* pat flag rx) - regexp*? - (pat regexp*-pattern) - (flag regexp*-flag) - (rx regexp*-rx)) - -;;; Work around regexp implementation. -;;; This record allows to track the regexp pattern and then display it. -(define* (make-regexp* pat #:optional (flag regexp/extended)) - "Alternative to `make-regexp' producing annotated objects." - (%make-regexp* pat flag (make-regexp pat flag))) - -(define (regexp*-exec rx* str) - "Execute the RX* regexp, a object." - (regexp-exec (regexp*-rx rx*) str)) - (define-record-type (make-team id name description members scope) team? diff --git a/guix/build/utils.scm b/guix/build/utils.scm index 8e630ad586..2b3a8e278b 100644 --- a/guix/build/utils.scm +++ b/guix/build/utils.scm @@ -9,6 +9,7 @@ ;;; Copyright © 2020, 2021 Maxim Cournoyer ;;; Copyright © 2021, 2022 Maxime Devos ;;; Copyright © 2021 Brendan Tildesley +;;; Copyright © 2022 Simon Tournier ;;; Copyright © 2023 Carlo Zancanaro ;;; ;;; This file is part of GNU Guix. @@ -28,6 +29,7 @@ (define-module (guix build utils) #:use-module (srfi srfi-1) + #:use-module (srfi srfi-9) #:use-module (srfi srfi-11) #:use-module (srfi srfi-26) #:use-module (srfi srfi-34) @@ -55,6 +57,14 @@ (define-module (guix build utils) package-name->name+version parallel-job-count + make-regexp* + regexp*-exec + regexp*? + regexp*-pattern + regexp*-flag + regexp*-rx + list-matches* + compressor tarball? %xz-parallel-args @@ -163,6 +173,35 @@ (define-syntax-rule (define-constant name val) (module-replace! (current-module) '(setvbuf))) (else #f)) + +;;; +;;; Extend regexp objects with a pattern field. +;;; +(define-record-type + (%make-regexp* pat flag rx) + regexp*? + (pat regexp*-pattern) ;the regexp pattern, a string + (flag regexp*-flag) ;regexp flags + (rx regexp*-rx)) ;the compiled regexp object + +;;; Work around regexp implementation. +;;; This record allows to track the regexp pattern and then display it. +(define* (make-regexp* pat #:optional (flag regexp/extended)) + "Alternative to `make-regexp' producing annotated objects." + (%make-regexp* pat flag (make-regexp pat flag))) + +(define (regexp*-exec rx* str) + "Execute the RX* regexp, a object." + (regexp-exec (regexp*-rx rx*) str)) + +(define* (list-matches* regexp str #:optional (flags regexp/extended)) + "Like 'list-matches', but also accepting a regexp* as REGEXP." + (match regexp + ((or (? string?) (? regexp?)) + (list-matches regexp str flags)) + ((? regexp*?) + (list-matches (regexp*-rx regexp) str flags)))) + ;;; ;;; Compression helpers. From patchwork Fri Oct 20 00:57:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 55067 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id CA81327BBEC; Fri, 20 Oct 2023 01:59:26 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id AA0C427BBE9 for ; Fri, 20 Oct 2023 01:59:25 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtdqa-0003mB-NG; Thu, 19 Oct 2023 20:58:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtdqW-0003lQ-Kn for guix-patches@gnu.org; Thu, 19 Oct 2023 20:58:37 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qtdqV-0007Mg-Pr; Thu, 19 Oct 2023 20:58:35 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qtdqw-0003kl-7C; Thu, 19 Oct 2023 20:59:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42146] [PATCH v3 2/3] build: substitute: Error when no substitutions were done. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: kuba@kadziolka.net, maxim.cournoyer@gmail.com, ludo@gnu.org, guix-patches@gnu.org Resent-Date: Fri, 20 Oct 2023 00:59:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42146 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 42146@debbugs.gnu.org Cc: Jakub =?utf-8?b?S8SFZHppb8WCa2E=?= , Maxim Cournoyer , Jakub =?utf-8?b?S8SFZHppb8WC?= =?utf-8?b?a2E=?= , Maxim Cournoyer , Ludovic =?utf-8?q?Court=C3=A8s?= X-Debbugs-Original-Xcc: Jakub =?utf-8?b?S8SFZHppb8WCa2E=?= , Maxim Cournoyer , Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 42146-submit@debbugs.gnu.org id=B42146.169776351414362 (code B ref 42146); Fri, 20 Oct 2023 00:59:02 +0000 Received: (at 42146) by debbugs.gnu.org; 20 Oct 2023 00:58:34 +0000 Received: from localhost ([127.0.0.1]:38243 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtdqT-0003jY-OW for submit@debbugs.gnu.org; Thu, 19 Oct 2023 20:58:34 -0400 Received: from mail-vs1-xe33.google.com ([2607:f8b0:4864:20::e33]:54714) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtdqR-0003jD-CA for 42146@debbugs.gnu.org; Thu, 19 Oct 2023 20:58:32 -0400 Received: by mail-vs1-xe33.google.com with SMTP id ada2fe7eead31-4580a2ec248so87551137.1 for <42146@debbugs.gnu.org>; Thu, 19 Oct 2023 17:58:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697763478; x=1698368278; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2eiEVVW1YIPDVs3/30xqVWj4WCnrhALxoLlMnwbevBc=; b=IoKCbPRX35cahet1V/Dzbj+HX18OnMOHFotZRlCTd0/B2kTHs+RJ+oWcO9lZYOmEtI IioWxCnUsx+iYskfZ/Ia0gFGf2Q4DBgTtfhTNflV+aZPRUk1HAt1WBGoGN5Yq22feKdL DpObCGX4Fe9zc5iB/Y+bFifLdMgNL7fzVrI3rfHdjH8c/hyq/JSzlywM1ZRXq9yLXd6p Uk3AYW2OWCneLucuGejW6+T4ZLJ0Eayg9Oqwhb81Tbun6IvD71Xc2Ahh8U7FInv9ZSRR 8eIu2PyHsF2U6Afqz/5YE0TA54GtoUe9LxXQdL7m5uXHub5U1QGX0SsVCnGvxIUS5z88 Bupg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697763478; x=1698368278; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2eiEVVW1YIPDVs3/30xqVWj4WCnrhALxoLlMnwbevBc=; b=rPZ7FPqphoYnjKWiQXsX5Jzd3eWsiX2jVdmppg82AQZIU9VccaN0uz59d/GqcA3/bV Vf2674tTO7TvV7/XY6lB7Pu1HYyVsGB5EOc+qhYYz+IxfpJW5n8QC0xS8ctSBE0d1+5t ZDvyOUgMzWt/NwqoXJRS2byes2CRnscOsjBcVS9IOcJAiVkSt5KCPB1T66zJroxxVEB1 BN1yO1QHkaiRXzfUHE6wYPCWGsibAOKX6NN0ZOTlhJT/aurOXisP/VBSskO5EKySsxRl +cTEfcbZ6H6HAZnMp/GKCh7RYLxIgvUshSLoCzVXV2eUQ3+jXQHYy0Ngv7Ia6qnMSWKZ Dk6g== X-Gm-Message-State: AOJu0Ywt1LJcsT05HvPH15a66wqfiunS6j/VTDmRqA+lZo4t762dKpcA yFeSz0uAXSxeq7qdmebYmXcgh5rgVsUp3g== X-Google-Smtp-Source: AGHT+IFfRU5dHNsQoMCxDMmAySoleppZPRIpE66fblrMLPEF5zzKO8dSii5CS3s278SeXBnLIP/A1A== X-Received: by 2002:a05:6102:20c4:b0:457:7138:fa74 with SMTP id i4-20020a05610220c400b004577138fa74mr552529vsr.35.1697763478521; Thu, 19 Oct 2023 17:57:58 -0700 (PDT) Received: from localhost.localdomain (dsl-156-111.b2b2c.ca. [66.158.156.111]) by smtp.gmail.com with ESMTPSA id h9-20020a0cf8c9000000b0065d0a4262e0sm279097qvo.70.2023.10.19.17.57.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 17:57:58 -0700 (PDT) From: Maxim Cournoyer Date: Thu, 19 Oct 2023 20:57:39 -0400 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches From: Jakub Kądziołka * guix/build/utils.scm (substitute, substitute*) [require-matches?]: New argument. * tests/build-utils.scm ("substitute*"): New test group. ("substitute*, no match error") ("substitute*, partial no match error"): New tests. Co-authored-by: Maxim Cournoyer Change-Id: I66ed33d72aa73cd35e5642521efec70bf756f86e --- guix/build/utils.scm | 93 +++++++++++++++++++++++++++++++++---------- tests/build-utils.scm | 68 +++++++++++++++++++++---------- 2 files changed, 118 insertions(+), 43 deletions(-) diff --git a/guix/build/utils.scm b/guix/build/utils.scm index 2b3a8e278b..8e4b8321dd 100644 --- a/guix/build/utils.scm +++ b/guix/build/utils.scm @@ -6,7 +6,8 @@ ;;; Copyright © 2018, 2022 Arun Isaac ;;; Copyright © 2018, 2019 Ricardo Wurmus ;;; Copyright © 2020 Efraim Flashner -;;; Copyright © 2020, 2021 Maxim Cournoyer +;;; Copyright © 2020 Jakub Kądziołka +;;; Copyright © 2020, 2021, 2023 Maxim Cournoyer ;;; Copyright © 2021, 2022 Maxime Devos ;;; Copyright © 2021 Brendan Tildesley ;;; Copyright © 2022 Simon Tournier @@ -111,8 +112,14 @@ (define-module (guix build utils) modify-phases with-atomic-file-replacement + %substitute-requires-matches? substitute substitute* + &substitute-error + substitute-error? + substitute-error-file + substitute-error-patterns + dump-port set-file-time patch-shebang @@ -971,24 +978,51 @@ (define (replace-char c1 c2 s) c)) s))) -(define (substitute file pattern+procs) +(define-condition-type &substitute-error &error + substitute-error? + (file substitute-error-file) + (patterns substitute-error-patterns)) + +(define %substitute-requires-matches? + (make-parameter #t)) + +(define* (substitute file pattern+procs + #:key (require-matches? (%substitute-requires-matches?))) "PATTERN+PROCS is a list of regexp/two-argument-procedure pairs. For each line of FILE, and for each PATTERN that it matches, call the corresponding PROC as (PROC LINE MATCHES); PROC must return the line that will be written as a substitution of the original line. Be careful about using '$' to match the -end of a line; by itself it won't match the terminating newline of a line." - (let ((rx+proc (map (match-lambda - (((? regexp? pattern) . proc) +end of a line; by itself it won't match the terminating newline of a line. + +By default, SUBSTITUTE will raise a &substitute-error condition if one of the +patterns fails to match. REQUIRE-MATCHES? can be set to false when lack of +matches is acceptable (e.g. if you have multiple potential patterns not +guaranteed to be found in FILE)." + (define (rx->pattern m) + (match m + ((? regexp? pattern) + "") + ((? regexp*? pattern) + (regexp*-pattern pattern)) + ((? string? pattern) + pattern))) + + (let ((rx+proc (map (match-lambda + (((or (? regexp? pattern) (? regexp*? pattern)) . proc) (cons pattern proc)) ((pattern . proc) - (cons (make-regexp pattern regexp/extended) - proc))) - pattern+procs))) + (cons (make-regexp* pattern regexp/extended) proc))) + pattern+procs))) (with-atomic-file-replacement file (lambda (in out) - (let loop ((line (read-line in 'concat))) + (let loop ((line (read-line in 'concat)) + (unmatched-regexps (map first rx+proc))) (if (eof-object? line) - #t + (when (and require-matches? (not (null? unmatched-regexps))) + (raise (condition + (&substitute-error + (file file) + (patterns (map rx->pattern unmatched-regexps)))))) ;; Work around the fact that Guile's regexp-exec does not handle ;; NUL characters (a limitation of the underlying GNU libc's ;; regexec) by temporarily replacing them by an unused private @@ -998,19 +1032,23 @@ (define (substitute file pattern+procs) (unused-private-use-code-point line)) #\nul)) (line* (replace-char #\nul nul* line)) - (line1* (fold (lambda (r+p line) - (match r+p - ((regexp . proc) - (match (list-matches regexp line) - ((and m+ (_ _ ...)) - (proc line m+)) - (_ line))))) - line* - rx+proc)) + (results ;line, unmatched-regexps + (fold (lambda (r+p results) + (let ((line (first results)) + (unmatched (second results))) + (match r+p + ((regexp . proc) + (match (list-matches* regexp line) + ((and m+ (_ _ ...)) + (list (proc line m+) + (delq regexp unmatched))) + (_ (list line unmatched))))))) + (list line* unmatched-regexps) + rx+proc)) + (line1* (first results)) (line1 (replace-char nul* #\nul line1*))) (display line1 out) - (loop (read-line in 'concat))))))))) - + (loop (read-line in 'concat) (second results))))))))) (define-syntax let-matches ;; Helper macro for `substitute*'. @@ -1048,9 +1086,19 @@ (define-syntax substitute* Alternatively, FILE may be a list of file names, in which case they are all subject to the substitutions. +By default, SUBSTITUTE* will raise a &message condition if one of the patterns +fails to match on one of the files; REQUIRE-MATCHES? may be set to false to +avoid an error being raised in such condition. + Be careful about using '$' to match the end of a line; by itself it won't match the terminating newline of a line." ((substitute* file ((regexp match-var ...) body ...) ...) + (substitute* file + ((regexp match-var ...) body ...) ... + #:require-matches? #t)) + ((substitute* file + ((regexp match-var ...) body ...) ... + #:require-matches? require-matches?) (let () (define (substitute-one-file file-name) (substitute @@ -1074,7 +1122,8 @@ (define-syntax substitute* (begin body ...) (substring l o (match:start m)) r)))))))) - ...))) + ...) + #:require-matches? require-matches?)) (match file ((files (... ...)) diff --git a/tests/build-utils.scm b/tests/build-utils.scm index 3babf5d544..35c66faa3c 100644 --- a/tests/build-utils.scm +++ b/tests/build-utils.scm @@ -1,7 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2012, 2015, 2016, 2019, 2020 Ludovic Courtès ;;; Copyright © 2019 Ricardo Wurmus -;;; Copyright © 2021 Maxim Cournoyer +;;; Copyright © 2021, 2023 Maxim Cournoyer ;;; Copyright © 2021 Maxime Devos ;;; Copyright © 2021 Brendan Tildesley ;;; @@ -289,26 +289,52 @@ (define (arg-test bash-args) (test-assert "wrap-script, argument handling, bash --norc" (arg-test " --norc")) -(test-equal "substitute*, text contains a NUL byte, UTF-8" - "c\0d" - (with-fluids ((%default-port-encoding "UTF-8") - (%default-port-conversion-strategy 'error)) - ;; The GNU libc is locale sensitive. Depending on the value of LANG, the - ;; test could fail with "string contains #\\nul character: ~S" or "cannot - ;; convert wide string to output locale". - (setlocale LC_ALL "en_US.UTF-8") - (call-with-temporary-output-file - (lambda (file port) - (format port "a\0b") - (flush-output-port port) - - (substitute* file - (("a") "c") - (("b") "d")) - - (with-input-from-file file - (lambda _ - (get-string-all (current-input-port)))))))) +(define-syntax-rule (define-substitute*-test test-type name expected + content clauses ...) + (test-type + name + expected + (with-fluids ((%default-port-encoding "UTF-8") + (%default-port-conversion-strategy 'error)) + ;; The GNU libc is locale sensitive. Depending on the value of LANG, + ;; the test could fail with "string contains #\\nul character: ~S" or + ;; "cannot convert wide string to output locale". + (setlocale LC_ALL "en_US.UTF-8") + (call-with-temporary-output-file + (lambda (file port) + (format port content) + (flush-output-port port) + + (substitute* file + clauses ...) + + (with-input-from-file file + (lambda _ + (get-string-all (current-input-port))))))))) + +(define-substitute*-test test-equal + "substitute*, text contains a NUL byte, UTF-8" + "c\0d" ;expected + "a\0b" ;content + (("a") "c") + (("b") "d")) + +(define-substitute*-test test-error "substitute*, no match error" + #t ;expected + "a\0b" ;content + (("Oops!") "c")) + +(define-substitute*-test test-equal "substitute*, no match, ignored" + "abc" ;expected + "abc" ;content + (("Oops!") "c") + #:require-matches? #f) + +(define-substitute*-test test-error "substitute*, partial no match error" + #t ;expected + "a\0b" ;content + (("a") "c" + ("Oops!") "c")) (test-equal "search-input-file: exception if not found" `((path) From patchwork Fri Oct 20 00:57:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 55066 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 0B77727BBE2; Fri, 20 Oct 2023 01:59:00 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 4670D27BBE9 for ; Fri, 20 Oct 2023 01:58:58 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qtdqb-0003mH-3g; Thu, 19 Oct 2023 20:58:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qtdqX-0003lj-Vw for guix-patches@gnu.org; Thu, 19 Oct 2023 20:58:39 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qtdqW-0007Nn-NQ; Thu, 19 Oct 2023 20:58:36 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qtdqx-0003l3-3M; Thu, 19 Oct 2023 20:59:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#42146] [PATCH v3 3/3] build: bootstrap-configure: Allow lack of matches in substitute. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: kuba@kadziolka.net, maxim.cournoyer@gmail.com, ludo@gnu.org, guix-patches@gnu.org Resent-Date: Fri, 20 Oct 2023 00:59:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42146 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 42146@debbugs.gnu.org Cc: Jakub =?utf-8?b?S8SFZHppb8WCa2E=?= , Maxim Cournoyer , Jakub =?utf-8?b?S8SFZHppb8WC?= =?utf-8?b?a2E=?= , Maxim Cournoyer , Ludovic =?utf-8?q?Court=C3=A8s?= X-Debbugs-Original-Xcc: Jakub =?utf-8?b?S8SFZHppb8WCa2E=?= , Maxim Cournoyer , Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 42146-submit@debbugs.gnu.org id=B42146.169776352014389 (code B ref 42146); Fri, 20 Oct 2023 00:59:03 +0000 Received: (at 42146) by debbugs.gnu.org; 20 Oct 2023 00:58:40 +0000 Received: from localhost ([127.0.0.1]:38248 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtdqZ-0003k0-Ou for submit@debbugs.gnu.org; Thu, 19 Oct 2023 20:58:40 -0400 Received: from mail-qv1-xf2b.google.com ([2607:f8b0:4864:20::f2b]:57766) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qtdqU-0003jI-8T for 42146@debbugs.gnu.org; Thu, 19 Oct 2023 20:58:34 -0400 Received: by mail-qv1-xf2b.google.com with SMTP id 6a1803df08f44-66d1ef93284so1846436d6.3 for <42146@debbugs.gnu.org>; Thu, 19 Oct 2023 17:58:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697763481; x=1698368281; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Okco+31FqmL/2OCY1wTsGhnDcafE/pDvBRDkBKQDPMQ=; b=aD+J7qj0wzdhyo/s9o/3AXEL9x8CD1em8R2Uv0XT/GngQFOEA0Ix+I8kK5HrLpBBEF KNvgbAI+cRKYyI2sXjBoGvZkihQTh3T6AePJ1kMnW1BEOXgrSBIBp8W4aGg/ZXxwYwTJ Vvf4yaUgfC3odrFRSCiEuPoq2UKX+TDCnRNHFT8aP8dyjqjzA/8Y2XcEZctx1U/NSraH eAj2BL4ry5bh92M50x5NLMBoUM2G2gvYcTD5DMZuRRrAsGZZO+0p6GoBNyNwqNKCAKE2 AnWOZTn6QGBciMF4WpZw+B7+YZO+VoN2RME7VhcWVsyeGQg1TMgKcDk75qmMGAAoAvn3 PpMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697763481; x=1698368281; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Okco+31FqmL/2OCY1wTsGhnDcafE/pDvBRDkBKQDPMQ=; b=PDtSoIX3DO67IsKMPhqlLqR4Wu7bhKvbvBA3rHLoDcxCmYq4ZmJwexsdbDbwCPLS6H KHqPsV2rWqt3SJvBot/o1VxLIsZfa90ENWsjFu2LMSxoXILQ3SUVJGReegyJk/nWJYsI 5C2sk5S1DPCm6xoVVZVAkvRyFa2dgffO3xxIByMvHi37wgnhDxyRny0kjk8TkbJ1vNSO AHOBPRcdBY266CXuRMNO+s3K+Nndy15lynHkP7Ua88a6k2vxL0P/NhhkSQbrvE9NUx1i U4EYzWUfwx8AH4Cf3LWPxJq36Mz5Xa8CsvPo2r+SkQAzbprS7Nc2/pAqIB09XLRT7jgi sC7A== X-Gm-Message-State: AOJu0YwrplRQ8NAbe0WGf8ELyQ7u+L8CSX4B45oHPzWfWhq+BhcB2TMr /yzV4LnMeYL+QQwDqOYYet3Za0OK0L23sA== X-Google-Smtp-Source: AGHT+IHOeyg2TlEhzsDG7QYDtqv0C6wCL/uCNhGfkHtGddGZCAeNQkqnKQPedJhYS+Ay1qDn0g0RDg== X-Received: by 2002:a05:6214:519e:b0:66d:140a:18f3 with SMTP id kl30-20020a056214519e00b0066d140a18f3mr541647qvb.45.1697763481643; Thu, 19 Oct 2023 17:58:01 -0700 (PDT) Received: from localhost.localdomain (dsl-156-111.b2b2c.ca. [66.158.156.111]) by smtp.gmail.com with ESMTPSA id h9-20020a0cf8c9000000b0065d0a4262e0sm279097qvo.70.2023.10.19.17.58.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 17:58:01 -0700 (PDT) From: Maxim Cournoyer Date: Thu, 19 Oct 2023 20:57:40 -0400 Message-ID: <0d8764ba1b635d49c07cc1d47f3dcc620bbf482d.1697763444.git.maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches From: Jakub Kądziołka Matches are not required here, as not every file will use every variable. * guix/build/gnu-bootstrap.scm (bootstrap-configure): Pass #:require-matches? #f to substitute*. Signed-off-by: Maxim Cournoyer Change-Id: I7eee433a3af5fa310b3f2e8b8b58ac9befb4b56a --- guix/build/gnu-bootstrap.scm | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/guix/build/gnu-bootstrap.scm b/guix/build/gnu-bootstrap.scm index b4257a3717..b097f410ad 100644 --- a/guix/build/gnu-bootstrap.scm +++ b/guix/build/gnu-bootstrap.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2020, 2022 Timothy Sample +;;; Copyright © 2020 Jakub Kądziołka ;;; ;;; This file is part of GNU Guix. ;;; @@ -52,7 +53,8 @@ (define (bootstrap-configure name version modules scripts) (copy-file template target) (substitute* target (("@PACKAGE_NAME@") name) - (("@VERSION@") version)))) + (("@VERSION@") version) + #:require-matches? #f))) (append-map (lambda (dir) (find-files dir "\\.in$")) modules)) (for-each (lambda (template) @@ -62,12 +64,12 @@ (define (bootstrap-configure name version modules scripts) (substitute* target (("@GUILE@") guile) (("@MODDIR@") moddir) - (("@GODIR@") godir)) + (("@GODIR@") godir) + #:require-matches? #f) (chmod target #o755))) (find-files scripts (lambda (fn st) - (string-suffix? ".in" fn)))) - #t))) + (string-suffix? ".in" fn))))))) (define (bootstrap-build modules) "Create a procedure that builds an early bootstrap package. The