From patchwork Mon Aug 21 18:06:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 53075 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id BDF5D27BBE9; Mon, 21 Aug 2023 19:10:16 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id E667627BBE2 for ; Mon, 21 Aug 2023 19:10:14 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY9LV-0005DK-CP; Mon, 21 Aug 2023 14:09:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY9Kr-00054D-5o for guix-patches@gnu.org; Mon, 21 Aug 2023 14:09:06 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY9Kp-0004a3-73; Mon, 21 Aug 2023 14:09:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qY9Ko-0007RH-HA; Mon, 21 Aug 2023 14:09:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65230] [PATCH v3 01/10] gnu-maintenance: Make base-url argument of import-html-release required. References: In-Reply-To: Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 21 Aug 2023 18:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65230 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65230@debbugs.gnu.org Cc: Maxim Cournoyer , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65230-submit@debbugs.gnu.org id=B65230.169264133928577 (code B ref 65230); Mon, 21 Aug 2023 18:09:02 +0000 Received: (at 65230) by debbugs.gnu.org; 21 Aug 2023 18:08:59 +0000 Received: from localhost ([127.0.0.1]:57663 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9Kk-0007Qq-V5 for submit@debbugs.gnu.org; Mon, 21 Aug 2023 14:08:59 -0400 Received: from mail-qt1-x82d.google.com ([2607:f8b0:4864:20::82d]:47187) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9Kk-0007QX-8X for 65230@debbugs.gnu.org; Mon, 21 Aug 2023 14:08:58 -0400 Received: by mail-qt1-x82d.google.com with SMTP id d75a77b69052e-41095b84d8eso10020551cf.2 for <65230@debbugs.gnu.org>; Mon, 21 Aug 2023 11:08:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692641330; x=1693246130; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=nPe+PQu+Le99kNcteUXZMEhcKyG+iLaCRslNHkWva4o=; b=qINudUHfxQDw+kaXnVBiLiVfLdJUkPW63b9W8OaKiAMBE8tcvZqeUNW0GFcOD1PMex 22ZCYNyWPgjEVS1IcpXJ3AKj+YGFcaI0y+Ae5myK4Z/Rs2wXt27WTLGYEm53Wt3uBv3O O6dfH/7JzkSiCIfRPNApBqUB8JEJEx1twtOD+SN2AnzIfMCfxWwjFgbCNBKGkdImq5Dx 2MZJoN2Bmh+5zyU8zPYYE6a0jivhK12UnvsjAHtE/92Ot7E50hXR1wpGHBFjqg0KGKx0 bJU3G7dLqX+36Td4Q98kXJ+wUi2erzFMKp2uMnHk1BYyp/EGLWb99wyc8RVIPUda9Jhx qh1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692641330; x=1693246130; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=nPe+PQu+Le99kNcteUXZMEhcKyG+iLaCRslNHkWva4o=; b=dTP1NuaCgNu5bj0087OD4WZ3YaZKupYnO2dloV5XuYK9ktNmG8v1NqsvMvZ9WnvNdf eOF3dzkcDGU4GOWnUQwJ3xL/+CJ+aJFdCWMyLq2oURQuEqUl4xNzwKeOO4kizrvHByXe JPCAQSPLi4OzYkpS4lwRcCMV6wBjndrqWtfE0Y+Hp7jKKkRu9YKnLnbnHTWZ3SYUDQ0H /RTshbTitBsvhxw/CCi4kxn5X3jWKe0r6IOI/fHwVFbBIhQjug4KdDko7LhfF0u33wEC bw3rQ+L/yzn+/YTcR60cH5MQMBU+P6aT7Y6CPxJN6o3p1dX5HQP8bdKEaT17ejdyEV6Q 6KFA== X-Gm-Message-State: AOJu0YwrDc6oxi0M6wOdxDUxUvudDZY4i+5+sJMZkTbeznm7MC++qF3N +TPcq5PZsCTlP7O5rMyxQP2ou2Q3yOI= X-Google-Smtp-Source: AGHT+IGug6flSl3RimMvfIUH+z/+gSob1RZ9AbzR+xLk1RKKTacOMAkXHZ+GFf7yuU6t7AQyB1gYow== X-Received: by 2002:ac8:5b56:0:b0:40f:da10:2426 with SMTP id n22-20020ac85b56000000b0040fda102426mr8808863qtw.56.1692641330011; Mon, 21 Aug 2023 11:08:50 -0700 (PDT) Received: from localhost.localdomain (dsl-10-148-105.b2b2c.ca. [72.10.148.105]) by smtp.gmail.com with ESMTPSA id t16-20020a05620a035000b00767303dc070sm2635523qkm.8.2023.08.21.11.08.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 11:08:49 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Aug 2023 14:06:05 -0400 Message-ID: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches It doesn't make sense to have it default to something like "https://kernel.org/pub"; it should always be provided explicitly. * guix/gnu-maintenance.scm (import-html-release) <#:base-url>: Turn keyword argument into a positional argument. Update doc. * guix/gnu-maintenance.scm (import-savannah-release): Adjust call accordingly. (import-kernel.org-release): Likewise. (import-html-updatable-release): Likewise. --- (no changes since v1) guix/gnu-maintenance.scm | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) base-commit: b03f4ff76e7502d4636ff805727df437c17de21a diff --git a/guix/gnu-maintenance.scm b/guix/gnu-maintenance.scm index 32712f7218..b95a45824e 100644 --- a/guix/gnu-maintenance.scm +++ b/guix/gnu-maintenance.scm @@ -483,15 +483,14 @@ (define (html-links sxml) (_ links)))) -(define* (import-html-release package +(define* (import-html-release base-url package #:key (version #f) - (base-url "https://kernel.org/pub") (directory (string-append "/" package)) file->signature) - "Return an for the latest release of PACKAGE (a string) on -SERVER under DIRECTORY, or #f. Optionally include a VERSION string to fetch a -specific version. + "Return an for the latest release of PACKAGE (a string) +under DIRECTORY at BASE-URL, or #f. Optionally include a VERSION string to +fetch a specific version. BASE-URL should be the URL of an HTML page, typically a directory listing as found on 'https://kernel.org/pub'. @@ -730,9 +729,8 @@ (define* (import-savannah-release package #:key (version #f)) (directory (dirname (uri-path uri)))) ;; Note: We use the default 'file->signature', which adds ".sig", ".asc", ;; or whichever detached signature naming scheme PACKAGE uses. - (import-html-release package + (import-html-release %savannah-base package #:version version - #:base-url %savannah-base #:directory directory))) (define* (latest-sourceforge-release package #:key (version #f)) @@ -824,9 +822,8 @@ (define* (import-kernel.org-release package #:key (version #f)) ((uri mirrors ...) uri)))) (package (package-upstream-name package)) (directory (dirname (uri-path uri)))) - (import-html-release package + (import-html-release %kernel.org-base package #:version version - #:base-url %kernel.org-base #:directory directory #:file->signature file->signature))) @@ -870,9 +867,8 @@ (define* (import-html-updatable-release package #:key (version #f)) (dirname (uri-path uri)))) (package (package-upstream-name package))) (false-if-networking-error - (import-html-release package + (import-html-release base package #:version version - #:base-url base #:directory directory)))) (define %gnu-updater From patchwork Mon Aug 21 18:06:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 53076 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 6EF7227BBEA; Mon, 21 Aug 2023 19:10:58 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id CF20C27BBE2 for ; Mon, 21 Aug 2023 19:10:57 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY9MZ-0005eC-It; Mon, 21 Aug 2023 14:10:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY9Lk-0005MS-Or for guix-patches@gnu.org; Mon, 21 Aug 2023 14:10:06 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY9Lk-0004nE-Ge; Mon, 21 Aug 2023 14:10:00 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qY9Ll-0007VJ-US; Mon, 21 Aug 2023 14:10:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65230] [PATCH v3 02/10] gnu-maintenance: Fix docstring. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 21 Aug 2023 18:10:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65230 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65230@debbugs.gnu.org Cc: Maxim Cournoyer , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65230-submit@debbugs.gnu.org id=B65230.169264134428632 (code B ref 65230); Mon, 21 Aug 2023 18:10:01 +0000 Received: (at 65230) by debbugs.gnu.org; 21 Aug 2023 18:09:04 +0000 Received: from localhost ([127.0.0.1]:57669 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9Kq-0007Ri-D2 for submit@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:04 -0400 Received: from mail-qk1-x736.google.com ([2607:f8b0:4864:20::736]:57774) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9Km-0007Qc-If for 65230@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:01 -0400 Received: by mail-qk1-x736.google.com with SMTP id af79cd13be357-76db0439387so16250585a.3 for <65230@debbugs.gnu.org>; Mon, 21 Aug 2023 11:08:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692641332; x=1693246132; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=q3JO7BEWz5sr9o2eOQRUBQ3oLEReWz5mfl3yoNMGD5g=; b=fXucFT35a846+01XGKwR4CisAi4qFavMgmb0ugwsZcRRY4Zc2vNNWcFdlmlFWY9YfU 1z/8O6m48O05/pI01hMEkiAxdnI/QFaAH5MY77EnH44xVoVImuyzBHivc71WrMpEWDU3 qzNTELr+4uheKHqpBmexmIcekiHgOyt8tnCxA1nSTe1sNFYXRd8PnuNcuihhS0vOSeRW fGWPEwl5yfDah11ToP3FxFt1FM8a6hQpj7odLfwYTIYTNLxC/ZQ+cTfqvBNHfACL2JaI WM5Eu51yFpHUzrdW3i8J22rfaVQqx4gF+Ny7osM2wzeS+tGQXEQ2+gDTfRZ2c83LyZsi jDqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692641332; x=1693246132; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=q3JO7BEWz5sr9o2eOQRUBQ3oLEReWz5mfl3yoNMGD5g=; b=H5tWLuiNY9veVmZOZ86eNa06vSZYonAgAaBhHwmfD90PFlBH/lCl4sYOKbg9yqXUHY 4jCqZH/kBH4iyqSi099iYGpySjkrrn1IFmX+kHi7dZvncRJfxSRCnPRnKd6l2mPEgNac /sxWmKUqnHa/yA1p43/0oqVLe/O9Td55wEhrDm+lXVSIHq1NkFP3DigNpFXwN5pT3cq+ hSrz8u96hf5rP7Vj+4F7zEgpuVyk/w/l0xgSk7dFx4x3hF1MgRscF7RCi5ZAaQ3ifyoI uXtcHA4fi6tb6gF1+O+5btOr84zrnaDFpW5pIw9CdJJo470bRUWmCnD7t1NMSFw8Pnhy /erQ== X-Gm-Message-State: AOJu0Yzlmi4W6lt1cM4JGFWL606b8lgww4M05Bhl1AJakij+THTUieCf y3DN67QB4HQO/J9XtGB4LdWiDJDskyg= X-Google-Smtp-Source: AGHT+IFDgUUQSZYN9E5F6XaxTS89ZXsoW65oQZ2V0P9NXIrnVC2M3zFCxFM8dCp31F912/RZItbxdQ== X-Received: by 2002:a05:620a:d93:b0:76d:83ae:fdcd with SMTP id q19-20020a05620a0d9300b0076d83aefdcdmr9152948qkl.57.1692641332384; Mon, 21 Aug 2023 11:08:52 -0700 (PDT) Received: from localhost.localdomain (dsl-10-148-105.b2b2c.ca. [72.10.148.105]) by smtp.gmail.com with ESMTPSA id t16-20020a05620a035000b00767303dc070sm2635523qkm.8.2023.08.21.11.08.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 11:08:51 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Aug 2023 14:06:06 -0400 Message-ID: <9767dadfe8c826b6060c9ba46c5aae66083b0c5a.1692641173.git.maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> References: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/gnu-maintenance.scm (import-kernel.org-release): Fix docstring. --- (no changes since v1) guix/gnu-maintenance.scm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/guix/gnu-maintenance.scm b/guix/gnu-maintenance.scm index b95a45824e..a314923d3b 100644 --- a/guix/gnu-maintenance.scm +++ b/guix/gnu-maintenance.scm @@ -489,7 +489,7 @@ (define* (import-html-release base-url package (directory (string-append "/" package)) file->signature) "Return an for the latest release of PACKAGE (a string) -under DIRECTORY at BASE-URL, or #f. Optionally include a VERSION string to +under DIRECTORY at BASE-URL, or #f. Optionally include a VERSION string to fetch a specific version. BASE-URL should be the URL of an HTML page, typically a directory listing as @@ -806,7 +806,7 @@ (define* (import-xorg-release package #:key (version #f)) (string-append "/pub/xorg/" (dirname (uri-path uri))))))) (define* (import-kernel.org-release package #:key (version #f)) - "Return the latest release of PACKAGE, the name of a kernel.org package. + "Return the latest release of PACKAGE, a Linux kernel package. Optionally include a VERSION string to fetch a specific version." (define %kernel.org-base ;; This URL and sub-directories thereof are nginx-generated directory From patchwork Mon Aug 21 18:06:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 53084 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 0E61327BBEC; Mon, 21 Aug 2023 19:12:21 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 230BB27BBE2 for ; Mon, 21 Aug 2023 19:12:17 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY9N3-0005nm-GE; Mon, 21 Aug 2023 14:11:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY9Ll-0005Mc-Mo for guix-patches@gnu.org; Mon, 21 Aug 2023 14:10:06 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY9Ll-0004nY-DB; Mon, 21 Aug 2023 14:10:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qY9Lm-0007VR-AK; Mon, 21 Aug 2023 14:10:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65230] [PATCH v3 03/10] gnu-maintenance: Extract url->links procedure. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 21 Aug 2023 18:10:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65230 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65230@debbugs.gnu.org Cc: Maxim Cournoyer , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65230-submit@debbugs.gnu.org id=B65230.169264134528643 (code B ref 65230); Mon, 21 Aug 2023 18:10:02 +0000 Received: (at 65230) by debbugs.gnu.org; 21 Aug 2023 18:09:05 +0000 Received: from localhost ([127.0.0.1]:57671 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9Kq-0007Rl-MI for submit@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:05 -0400 Received: from mail-qt1-x836.google.com ([2607:f8b0:4864:20::836]:47196) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9Ko-0007Qg-Bf for 65230@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:02 -0400 Received: by mail-qt1-x836.google.com with SMTP id d75a77b69052e-41095b84d8eso10020981cf.2 for <65230@debbugs.gnu.org>; Mon, 21 Aug 2023 11:08:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692641334; x=1693246134; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lVilgTs+co6otIBGSPy1QW/2lzqmpgq2oXWxpSWuqAA=; b=Ye64T8EWuk1U9w/9vcy84mzThlo9pF2ORdN6Z50hVssIMO5rSVIJcas1LHxaQIC6V0 NCeF6rIVP/6H5iTbczR5xpWbUNjg58szlx4UxjDuVOK3PdoUPGsrARPOxuYjh8jQCRcy Fm6+oMHf/ZM0WGqsr27PFy6NTLT/VtNGpvXZ26YVtBZ0UP2TH3ceKcHP5e0Jymbb1fYv RRgUTdmgU+P5v4+5LgQl768v4BQ99DIkkIcE8zhn4lddIsqx1a/5WBFEf95j5KRokl8c qpjrQzlogMRGia2onJ0LQTkoTe/0kM3GzDRkzqzagf2gTsoltB5+ydMPnwBlaEdOg5dw 2uTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692641334; x=1693246134; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lVilgTs+co6otIBGSPy1QW/2lzqmpgq2oXWxpSWuqAA=; b=e96cmA9cl2viygZpW4mMrKzQatkZnV5WJb7N5J/e7W3sqbRGsMOq0b+Hi2VT/+Fqza Vl8GXNobdj2NuPiCx0MAEuHmcjonjVsKMaLLrD59n/ig/E1ZxBU+ui4Vzj0JTv8ZQL5h REcRo6KiqvZsm9GFFDz5t3ee6f6+3jaDdI5rrZjclF1+/fms05TT9MOfMOqrNr9u7bJo AxEC3C4mWzFe+ceF0O9JKSk6g+vB5MhFfO/8cXb0FREHCsDxLH1l+ntT1UIl6CrqYk7T 5/lyDChcgq3EYAGoNkEvmXyoNxkmR7bNUwHJ1kUYJaJasb85+jUtoP1QvaIZSp3sTcSA QN0g== X-Gm-Message-State: AOJu0YzSeAJ3/9UlaFrxPDOoD0FqhjqmDHOaKNekbOU4in/KrLPN3drI X6EfuvkVTTGa+0NWCm6V4bESaeSPqgA= X-Google-Smtp-Source: AGHT+IFXlCRtohHJXzz29xxq9NiNjGYRveKG8vlnhzSFU74ngaQzTG2lYKaivD45wy7tGRqp7C/qFQ== X-Received: by 2002:ac8:7fc9:0:b0:410:60a4:ffbc with SMTP id b9-20020ac87fc9000000b0041060a4ffbcmr11036989qtk.66.1692641334321; Mon, 21 Aug 2023 11:08:54 -0700 (PDT) Received: from localhost.localdomain (dsl-10-148-105.b2b2c.ca. [72.10.148.105]) by smtp.gmail.com with ESMTPSA id t16-20020a05620a035000b00767303dc070sm2635523qkm.8.2023.08.21.11.08.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 11:08:54 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Aug 2023 14:06:07 -0400 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> References: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/gnu-maintenance.scm (url->links): New procedure. (import-html-release): Use it. --- (no changes since v1) guix/gnu-maintenance.scm | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/guix/gnu-maintenance.scm b/guix/gnu-maintenance.scm index a314923d3b..2e0fc3e8ab 100644 --- a/guix/gnu-maintenance.scm +++ b/guix/gnu-maintenance.scm @@ -483,6 +483,14 @@ (define (html-links sxml) (_ links)))) +(define (url->links url) + "Return the unique links on the HTML page accessible at URL." + (let* ((uri (string->uri url)) + (port (http-fetch/cached uri #:ttl 3600)) + (sxml (html->sxml port))) + (close-port port) + (delete-duplicates (html-links sxml)))) + (define* (import-html-release base-url package #:key (version #f) @@ -499,12 +507,10 @@ (define* (import-html-release base-url package if any. Otherwise, FILE->SIGNATURE must be a procedure; it is passed a source file URL and must return the corresponding signature URL, or #f it signatures are unavailable." - (let* ((uri (string->uri (if (string-null? directory) - base-url - (string-append base-url directory "/")))) - (port (http-fetch/cached uri #:ttl 3600)) - (sxml (html->sxml port)) - (links (delete-duplicates (html-links sxml)))) + (let* ((url (if (string-null? directory) + base-url + (string-append base-url directory "/"))) + (links (url->links url))) (define (file->signature/guess url) (let ((base (basename url))) (any (lambda (link) @@ -562,7 +568,6 @@ (define* (import-html-release base-url package (define candidates (filter-map url->release links)) - (close-port port) (match candidates (() #f) ((first . _) From patchwork Mon Aug 21 18:06:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 53083 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 4F3D727BBEA; Mon, 21 Aug 2023 19:12:10 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 9043127BBE2 for ; Mon, 21 Aug 2023 19:12:09 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY9MV-0005e5-Rk; Mon, 21 Aug 2023 14:10:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY9Ll-0005Md-NL for guix-patches@gnu.org; Mon, 21 Aug 2023 14:10:06 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY9Ll-0004na-EH; Mon, 21 Aug 2023 14:10:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qY9Ln-0007Vm-38; Mon, 21 Aug 2023 14:10:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65230] [PATCH v3 04/10] gnu-maintenance: Fix indentation. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 21 Aug 2023 18:10:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65230 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65230@debbugs.gnu.org Cc: Maxim Cournoyer , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65230-submit@debbugs.gnu.org id=B65230.169264135128692 (code B ref 65230); Mon, 21 Aug 2023 18:10:03 +0000 Received: (at 65230) by debbugs.gnu.org; 21 Aug 2023 18:09:11 +0000 Received: from localhost ([127.0.0.1]:57683 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9Kw-0007SW-IG for submit@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:11 -0400 Received: from mail-qk1-x731.google.com ([2607:f8b0:4864:20::731]:61505) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9Kq-0007Qy-Lw for 65230@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:08 -0400 Received: by mail-qk1-x731.google.com with SMTP id af79cd13be357-76d93cf8e13so214566885a.2 for <65230@debbugs.gnu.org>; Mon, 21 Aug 2023 11:09:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692641336; x=1693246136; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NChkL5OrIwj9vZ+wWHvnTwIVDB6osNDrfs6xUqJhXf0=; b=dZcHCTna5sSBqHfK2JdqGf2DwNuq/LsbAobrm7EvZiH2tMIm7Q76ugKj4HQJTo37cC ahJ0e/lZssotPi7pcVdIrkCgctIuKzwRV6ryKhgAsf4ofW6X5jGOEgaUrxPlS/Old0zZ 4tgZp0gC9/BKxY1EGV3gZpdd2l7QZvjsZ7zDOnPqGdfUMpsObRxiWDo5cegoCWNTCRa/ Bf26kDp3l0D/bGkODnfeG5iRVTFoMdwus59LRR2voibK8DPVEVZLCzvSTdZy2TiCCBOC 5I8gDBsm9UytO/1v5t2/eHSnYQVJrn3LuXUaE4vzRfvN/sOJT1FEQSY7i3+NO4DZ6SFm PKGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692641336; x=1693246136; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NChkL5OrIwj9vZ+wWHvnTwIVDB6osNDrfs6xUqJhXf0=; b=d5j6HWkvdJiFUL1GKuzrKyVSDkC8bHcV+3YacT/W1KnOAj7pnEgYGHX+GiY5euQM+/ xDoIOcb6xXTyNEoN6olO+M/GgUxp4cbqXBpfWtctuBOC9vMYOdQpAu+9lbcKrwse5Bw4 xzItu+7E40lCXHh11MBAtQ+Z+PPc6cNLLxXlVazM728sjjEaludLDAttJs4UF3NaCptL 3QWZK80TzO6fZpQ9CdgusgEIlpYPWe2Hpf1ipKCD9stdN5xNXDERO5VIXN3rvtl95imq W0HLzuDIvp1VE081lmFPvVY3Wvitn1fJAC7m/fRU/Xa+PWMWtRm2dT0EkvEkaL04WvjE PZaw== X-Gm-Message-State: AOJu0YxQGFYy3UOXeuIr7IfPM/A+56fjtqDo0T4+4xxGCynv1vxHdmId YI3nyB8oIah+VmZQbL+C2ALptgkgGXM= X-Google-Smtp-Source: AGHT+IEk5qTix8tXeKuq3LI7kmBHRASOZoXfKJdIsRazkozUI/imoplNiQ/Axc+KIBHFEqAMtUMqCw== X-Received: by 2002:a05:620a:40c6:b0:76c:ecef:28c with SMTP id g6-20020a05620a40c600b0076cecef028cmr10446562qko.57.1692641336642; Mon, 21 Aug 2023 11:08:56 -0700 (PDT) Received: from localhost.localdomain (dsl-10-148-105.b2b2c.ca. [72.10.148.105]) by smtp.gmail.com with ESMTPSA id t16-20020a05620a035000b00767303dc070sm2635523qkm.8.2023.08.21.11.08.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 11:08:56 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Aug 2023 14:06:08 -0400 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> References: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/gnu-maintenance.scm: Re-indent file. --- (no changes since v1) guix/gnu-maintenance.scm | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/guix/gnu-maintenance.scm b/guix/gnu-maintenance.scm index 2e0fc3e8ab..67abbc1c5a 100644 --- a/guix/gnu-maintenance.scm +++ b/guix/gnu-maintenance.scm @@ -578,11 +578,11 @@ (define* (import-html-release base-url package (coalesce-sources candidates)) ;; Select the most recent release and return it. (reduce (lambda (r1 r2) - (if (version>? (upstream-source-version r1) - (upstream-source-version r2)) - r1 r2)) - first - (coalesce-sources candidates))))))) + (if (version>? (upstream-source-version r1) + (upstream-source-version r2)) + r1 r2)) + first + (coalesce-sources candidates))))))) ;;; @@ -656,20 +656,20 @@ (define* (import-gnu-release package #:key (version #f)) (tarballs (filter (lambda (file) (string=? version (tarball->version file))) relevant))) - (match tarballs - (() #f) - (_ - (upstream-source - (package name) - (version version) - (urls (map (lambda (file) - (string-append "mirror://gnu/" - (string-drop file - (string-length "/gnu/")))) - ;; Sort so that the tarball with the same compression - ;; format as currently used in PACKAGE comes first. - (sort tarballs better-tarball?))) - (signature-urls (map (cut string-append <> ".sig") urls)))))))) + (match tarballs + (() #f) + (_ + (upstream-source + (package name) + (version version) + (urls (map (lambda (file) + (string-append "mirror://gnu/" + (string-drop file + (string-length "/gnu/")))) + ;; Sort so that the tarball with the same compression + ;; format as currently used in PACKAGE comes first. + (sort tarballs better-tarball?))) + (signature-urls (map (cut string-append <> ".sig") urls)))))))) (define %package-name-rx ;; Regexp for a package name, e.g., "foo-X.Y". Since TeXmacs uses From patchwork Mon Aug 21 18:06:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 53080 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 7AAB327BBEA; Mon, 21 Aug 2023 19:11:44 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 8002427BBE2 for ; Mon, 21 Aug 2023 19:11:43 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY9Mc-0005fW-1h; Mon, 21 Aug 2023 14:10:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY9Ln-0005Ms-UJ for guix-patches@gnu.org; Mon, 21 Aug 2023 14:10:08 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY9Lm-0004su-Oc; Mon, 21 Aug 2023 14:10:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qY9Lm-0007Va-Mp; Mon, 21 Aug 2023 14:10:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65230] [PATCH v3 05/10] gnu-maintenance: Accept package object in 'import-html-release' procedure. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 21 Aug 2023 18:10:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65230 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65230@debbugs.gnu.org Cc: Maxim Cournoyer , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65230-submit@debbugs.gnu.org id=B65230.169264135028679 (code B ref 65230); Mon, 21 Aug 2023 18:10:02 +0000 Received: (at 65230) by debbugs.gnu.org; 21 Aug 2023 18:09:10 +0000 Received: from localhost ([127.0.0.1]:57680 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9Kw-0007SS-3N for submit@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:10 -0400 Received: from mail-qk1-x734.google.com ([2607:f8b0:4864:20::734]:59897) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9Ks-0007R9-Vy for 65230@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:07 -0400 Received: by mail-qk1-x734.google.com with SMTP id af79cd13be357-76ce59842c1so201793185a.3 for <65230@debbugs.gnu.org>; Mon, 21 Aug 2023 11:09:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692641339; x=1693246139; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fSvqP932FLPbM8fa3EzPSgM/uOJTV+oVvRl1fFsvuP4=; b=WJYmcwJUggTGNjz5xuCOP8bvHQzVmZMREqALUkiu/Dus0EdJEDtE4pI3b2TXC6JxvO LgACb/JDAanEb2q4SAQzBNuLqK28fK936E8RfhAckTP7qK88kZUL42AyWw/0sdPCV509 RgcSu52dQy5awR9nsqpubWq0yKjW+XflovLmWtbTETFOZjsLHGKdBTe93mOaan1QNJ2U gZaRa2sfOFBwSgvaMFS84wbm26xG1FjRlwcqiZstp6Rl8TwMHChIapokoy6ZRlGeDDZe Js5idX8zsVze0v6X8b2t4O5IsdBuT87ZqcZaFBH32qqMIVWjYr3asL9+dHO05GJuGyet Z54w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692641339; x=1693246139; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fSvqP932FLPbM8fa3EzPSgM/uOJTV+oVvRl1fFsvuP4=; b=c4SyO25LglcQ06wdQokNoBJTBdJVJTE2kP/2lbpENgEO7fbfpH+WGGpWCq/3bc+/TP vp0JsQufXXRLQE1ZIryjK7UzoBq4opvO68OfAGMzrJOOfEJEiB4CAmedydk5Bw/AURid cKDCodd2qr7npK3xVffUeowpL6xdJ5Qn2urcvPkV3Ug5dxbukhZXkffCyl4esVggHIM1 iIchKL/JdMV95bZloV6n84ZxB5ZpIV2NFRog7WfWlVW6ftintV7OeP3TQbLoEtE8BDPf Fsp+x9duNvwBBiwKUZVglvOny75cfdXCViMwCFvLMtFy+OiWB2TJMQqtRqLXuFsFz2JJ Ly2w== X-Gm-Message-State: AOJu0YzuEx0xVq0lTI7n60ufGH8Bx5y+0MyJagnklLaPiQOaxy7nyhM6 tKcfbq4yXzhWwIS9dHkUgXLNYlCUJoU= X-Google-Smtp-Source: AGHT+IHU9IyPP/PUIe4vGBV0P432JuvlWFJNrroIF+GcTrOylsNtEBkPU3Bw+n3mYmu5t11HHv/QXQ== X-Received: by 2002:a05:620a:3913:b0:76c:d4be:7c93 with SMTP id qr19-20020a05620a391300b0076cd4be7c93mr8185977qkn.20.1692641338801; Mon, 21 Aug 2023 11:08:58 -0700 (PDT) Received: from localhost.localdomain (dsl-10-148-105.b2b2c.ca. [72.10.148.105]) by smtp.gmail.com with ESMTPSA id t16-20020a05620a035000b00767303dc070sm2635523qkm.8.2023.08.21.11.08.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 11:08:58 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Aug 2023 14:06:09 -0400 Message-ID: <7d7a0eb99954ffcf862634983fa7298df071fa5a.1692641173.git.maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> References: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches This is in preparation for a new URL rewriting feature, which will need to have the current version information available. * guix/gnu-maintenance.scm (import-html-release): Update doc. Adjust default value of the DIRECTORY argument. Bind PACKAGE in lexical scope so that its value there is unchanged. (import-savannah-release, import-kernel.org-release) (import-html-updatable-release): Adjust accordingly. --- (no changes since v1) guix/gnu-maintenance.scm | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/guix/gnu-maintenance.scm b/guix/gnu-maintenance.scm index 67abbc1c5a..13d6c1c7f2 100644 --- a/guix/gnu-maintenance.scm +++ b/guix/gnu-maintenance.scm @@ -494,11 +494,12 @@ (define (url->links url) (define* (import-html-release base-url package #:key (version #f) - (directory (string-append "/" package)) + (directory (string-append + "/" (package-upstream-name package))) file->signature) - "Return an for the latest release of PACKAGE (a string) -under DIRECTORY at BASE-URL, or #f. Optionally include a VERSION string to -fetch a specific version. + "Return an for the latest release of PACKAGE under +DIRECTORY at BASE-URL, or #f. Optionally include a VERSION string to fetch a +specific version. BASE-URL should be the URL of an HTML page, typically a directory listing as found on 'https://kernel.org/pub'. @@ -507,7 +508,8 @@ (define* (import-html-release base-url package if any. Otherwise, FILE->SIGNATURE must be a procedure; it is passed a source file URL and must return the corresponding signature URL, or #f it signatures are unavailable." - (let* ((url (if (string-null? directory) + (let* ((package (package-upstream-name package)) + (url (if (string-null? directory) base-url (string-append base-url directory "/"))) (links (url->links url))) @@ -730,7 +732,6 @@ (define* (import-savannah-release package #:key (version #f)) (match (origin-uri (package-source package)) ((? string? uri) uri) ((uri mirrors ...) uri)))) - (package (package-upstream-name package)) (directory (dirname (uri-path uri)))) ;; Note: We use the default 'file->signature', which adds ".sig", ".asc", ;; or whichever detached signature naming scheme PACKAGE uses. @@ -825,7 +826,6 @@ (define* (import-kernel.org-release package #:key (version #f)) (match (origin-uri (package-source package)) ((? string? uri) uri) ((uri mirrors ...) uri)))) - (package (package-upstream-name package)) (directory (dirname (uri-path uri)))) (import-html-release %kernel.org-base package #:version version @@ -869,8 +869,7 @@ (define* (import-html-updatable-release package #:key (version #f)) "://" (uri-host uri)))) (directory (if custom "" - (dirname (uri-path uri)))) - (package (package-upstream-name package))) + (dirname (uri-path uri))))) (false-if-networking-error (import-html-release base package #:version version From patchwork Mon Aug 21 18:06:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 53079 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 211E627BBE2; Mon, 21 Aug 2023 19:11:26 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 729A327BBE9 for ; Mon, 21 Aug 2023 19:11:25 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY9Md-0005gC-BQ; Mon, 21 Aug 2023 14:10:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY9Ln-0005Mt-Up for guix-patches@gnu.org; Mon, 21 Aug 2023 14:10:08 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY9Ln-0004xj-7w; Mon, 21 Aug 2023 14:10:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qY9Ln-0007Vv-GG; Mon, 21 Aug 2023 14:10:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65230] [PATCH v3 06/10] gnu-maintenance: Document nested procedures in 'import-html-release'. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 21 Aug 2023 18:10:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65230 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65230@debbugs.gnu.org Cc: Maxim Cournoyer , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65230-submit@debbugs.gnu.org id=B65230.169264135128701 (code B ref 65230); Mon, 21 Aug 2023 18:10:03 +0000 Received: (at 65230) by debbugs.gnu.org; 21 Aug 2023 18:09:11 +0000 Received: from localhost ([127.0.0.1]:57685 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9Kx-0007Sf-5v for submit@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:11 -0400 Received: from mail-qk1-x735.google.com ([2607:f8b0:4864:20::735]:59898) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9Ku-0007Re-Lh for 65230@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:09 -0400 Received: by mail-qk1-x735.google.com with SMTP id af79cd13be357-76ce59842c1so201794385a.3 for <65230@debbugs.gnu.org>; Mon, 21 Aug 2023 11:09:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692641340; x=1693246140; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AMByyaIJSuomag/N+KKSRiluVBRVpJ6bNmbJjcSE/M8=; b=JB55eIN3+YDjHsl/pr+1X4HOaf3IB9XXpZBMCai5sLmf/vYABkaAZ5GkRVPb9xfxyD Wptx4s7WCPpYhyAVOgnDcR4zVswwJqbhhpDzgODbhqVL7G1r+1oGdcUI8Xf4b9c8rLFx 2ZGuTU4zctRZeVzV8kAQ6uDIL1itRf4KBbgd21ned8+M47IAKEDETS/B+NjFgEeSf0V4 OXS+/+YZauA+RgIvdOFylntILMspiLLmgO99uLDicICTTfH/NwUNUx4jx5bY30DJ2nw7 wlzxbiEOYxLVSh89c4LT2dkRvzs5U+JJssFFJPb40OCrjHgq+skMJqnt9ZgoktIMc1rs M1fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692641340; x=1693246140; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AMByyaIJSuomag/N+KKSRiluVBRVpJ6bNmbJjcSE/M8=; b=L8sVKBGbMQc284zt0/M8pTcTfkfzwjKgGa5iVuxAyZ39wdeiMLs93BOZA9oj1owKOl xwmPS95C5/qixZ0pJp9NzgqU0qZbK51FbEIXyr9e6AGZKCfwHeJQ63INGBDswfhh+HSf UBQfDF8rlLQKNVJXltJBeVTobYtuY3BtglPEUWZxlQWy0dkCkQoWuM3LB6aOn8Pf+hh0 YAqS/UqHD/9WYVWBx6kOGLp1+Jef8kCBTRxSrBbD6A5S0PkC+tpDLf9zIZvzDfIWdzBG BRq0trm/8iDkocZWOjI097+PulX1lcOf/aLKOfuuUrOnITggLFopdS9DvE65RzwLB5gM 1jrA== X-Gm-Message-State: AOJu0YzgTVJ+V//+XKX3bLxz++uIWVAoAIAc+Z1Pc15GhgjEf7SZbf9Y tkEyLB6Rg8w75r5KTzgrwU2DawzrAkQ= X-Google-Smtp-Source: AGHT+IGX/S6c9nYyrF/boXzI7rhcyt7f0+ecsD7PMvxxqs9SbPUA38jIsCZgcUoHSs3/4XcAldHI5A== X-Received: by 2002:a05:620a:2ea:b0:76c:9ac2:4e63 with SMTP id a10-20020a05620a02ea00b0076c9ac24e63mr6059025qko.77.1692641340583; Mon, 21 Aug 2023 11:09:00 -0700 (PDT) Received: from localhost.localdomain (dsl-10-148-105.b2b2c.ca. [72.10.148.105]) by smtp.gmail.com with ESMTPSA id t16-20020a05620a035000b00767303dc070sm2635523qkm.8.2023.08.21.11.09.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 11:09:00 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Aug 2023 14:06:10 -0400 Message-ID: <77681888f701b5381cae110887308101655c987a.1692641173.git.maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> References: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/gnu-maintenance.scm (import-html-release): Add docstring to the 'file->signature/guess' and 'url->release' nested procedures. --- (no changes since v1) guix/gnu-maintenance.scm | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/guix/gnu-maintenance.scm b/guix/gnu-maintenance.scm index 13d6c1c7f2..9bab8e9e5f 100644 --- a/guix/gnu-maintenance.scm +++ b/guix/gnu-maintenance.scm @@ -514,6 +514,7 @@ (define* (import-html-release base-url package (string-append base-url directory "/"))) (links (url->links url))) (define (file->signature/guess url) + "Return the first link that matches a signature extension, else #f." (let ((base (basename url))) (any (lambda (link) (any (lambda (extension) @@ -524,6 +525,8 @@ (define* (import-html-release base-url package links))) (define (url->release url) + "Return an object if a release file was found at URL, +else #f." (let* ((base (basename url)) (base-url (string-append base-url directory)) (url (cond ((and=> (string->uri url) uri-scheme) ;full URL? @@ -574,7 +577,7 @@ (define* (import-html-release base-url package (() #f) ((first . _) (if version - ;; find matching release version and return it + ;; Find matching release version and return it. (find (lambda (upstream) (string=? (upstream-source-version upstream) version)) (coalesce-sources candidates)) From patchwork Mon Aug 21 18:06:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 53081 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 61F0A27BBEA; Mon, 21 Aug 2023 19:11:56 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 4526D27BBE2 for ; Mon, 21 Aug 2023 19:11:55 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY9Mx-0005kB-V6; Mon, 21 Aug 2023 14:11:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY9Ln-0005Mr-Ui for guix-patches@gnu.org; Mon, 21 Aug 2023 14:10:08 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY9Ll-0004nq-Oc; Mon, 21 Aug 2023 14:10:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qY9Lo-0007WB-0v; Mon, 21 Aug 2023 14:10:04 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65230] [PATCH v3 07/10] gnu-maintenance: Extract 'canonicalize-url' from 'import-html-release'. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 21 Aug 2023 18:10:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65230 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65230@debbugs.gnu.org Cc: Maxim Cournoyer , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65230-submit@debbugs.gnu.org id=B65230.169264135928738 (code B ref 65230); Mon, 21 Aug 2023 18:10:03 +0000 Received: (at 65230) by debbugs.gnu.org; 21 Aug 2023 18:09:19 +0000 Received: from localhost ([127.0.0.1]:57690 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9L4-0007TQ-Q9 for submit@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:19 -0400 Received: from mail-qk1-x736.google.com ([2607:f8b0:4864:20::736]:55303) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9Kw-0007S2-BT for 65230@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:11 -0400 Received: by mail-qk1-x736.google.com with SMTP id af79cd13be357-76da22c090dso119952685a.2 for <65230@debbugs.gnu.org>; Mon, 21 Aug 2023 11:09:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692641342; x=1693246142; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yvDbRzPxClF/CYFUkNp+pm1g6bxCyyal/mrJmNA9dow=; b=CnIQk1qVzU6/LV8mhboMLp76DpTvPWEDnThRU+zsTuC9YsiXW1XlgAtWraEa30sBcP lvFuJgjpne7dABklAbqTM2JUng+44vfYuUZKRsHkShvVeTyOn/oDK8Nsz9yCRUZ21Plr XOkvtwr10m8XGsr3X0sgfkvqyRwhVYqhrjs+wTzOvn5ok4vabOzkMbdZmXsJ9cyckAt3 A4EQqGEduz/zHYcxaTFtbcmnmRy8Xt5rHpWWnBWieDDTyOyy+Op2iC68IykTaphx3Zh5 qYnp+LWbQlSF/6niANayGpMJ3rddU1Qym/kflwj2y+TtyAMUbcv2JAFz8HuGc2Nhbiuf XYDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692641342; x=1693246142; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yvDbRzPxClF/CYFUkNp+pm1g6bxCyyal/mrJmNA9dow=; b=cw1zQbGOTEjfqjG8L+m0ijTUo/B+COKkKr3gn68/Ge9HI5LuzYD3WXyLp+ITue7pXE hMlBOpDJeHH/TtTTBARAriPh2UyFUC31OjnJY6NrWQ969DTbFjt/xvr1Tl2qAguWjaya 3ylxJjS4AuuZbNDn9bAXfp79uOFvkT08DMS+aIFye2XB6kiF/PSdeGxVZWmdldLxmajh HQDUbcRY4/ST8HQdhTRwQc2P3wnrcwa8cioNnQiGRJ6eLFtTV6M/NS4DMsAeNddaNA6y WzcAi/cgsM4AcFO+zgJO8ave6Ae5I9h9k8OV0mzoVVUJmoZg0aU48GimPLzfhaZ03F8t 9zcw== X-Gm-Message-State: AOJu0YywZX/bddPwaJVOOL8tuts0ip7jKsNjsYRb1CWofBqagHE1Kh05 gY6kOo6AWrfnRvkD6IV+vB+0oMNAMlE= X-Google-Smtp-Source: AGHT+IEfbSF87ZIyLUv1gFFveLEXNtRROmVMNC8/CG5jIWE7R0C0WxBFqjuz6CZpcUDz1Cg651wpxw== X-Received: by 2002:a05:620a:40c8:b0:76c:b682:f503 with SMTP id g8-20020a05620a40c800b0076cb682f503mr8811757qko.60.1692641342336; Mon, 21 Aug 2023 11:09:02 -0700 (PDT) Received: from localhost.localdomain (dsl-10-148-105.b2b2c.ca. [72.10.148.105]) by smtp.gmail.com with ESMTPSA id t16-20020a05620a035000b00767303dc070sm2635523qkm.8.2023.08.21.11.09.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 11:09:02 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Aug 2023 14:06:11 -0400 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> References: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/gnu-maintenance.scm (canonicalize-url): New procedure, extracted from... (import-html-release): ... here. Use it. Rename inner PACKAGE variable to NAME, to explicit it is a string and not a package object. --- (no changes since v1) guix/gnu-maintenance.scm | 70 +++++++++++++++++++--------------------- 1 file changed, 34 insertions(+), 36 deletions(-) diff --git a/guix/gnu-maintenance.scm b/guix/gnu-maintenance.scm index 9bab8e9e5f..abba891d4b 100644 --- a/guix/gnu-maintenance.scm +++ b/guix/gnu-maintenance.scm @@ -491,6 +491,33 @@ (define (url->links url) (close-port port) (delete-duplicates (html-links sxml)))) +(define (canonicalize-url url base-url) + "Make relative URL absolute, by appending URL to BASE-URL as required. If +URL is a directory instead of a file, it should be suffixed with a slash (/)." + (cond ((and=> (string->uri url) uri-scheme) + ;; Fully specified URL. + url) + ((string-prefix? "//" url) + ;; Full URL lacking a URI scheme. Reuse the URI scheme of the + ;; document that contains the URL. + (string-append (symbol->string (uri-scheme (string->uri base-url))) + ":" url)) + ((string-prefix? "/" url) + ;; Absolute URL. + (let ((uri (string->uri base-url))) + (uri->string + (build-uri (uri-scheme uri) + #:host (uri-host uri) + #:port (uri-port uri) + #:path url)))) + ;; URL is relative to BASE-URL, which is assumed to be a directory. + ((string-suffix? "/" base-url) + (string-append base-url url)) + (else + ;; URL is relative to BASE-URL, which is assumed to denote a file + ;; within a directory. + (string-append (dirname base-url) "/" url)))) + (define* (import-html-release base-url package #:key (version #f) @@ -508,11 +535,12 @@ (define* (import-html-release base-url package if any. Otherwise, FILE->SIGNATURE must be a procedure; it is passed a source file URL and must return the corresponding signature URL, or #f it signatures are unavailable." - (let* ((package (package-upstream-name package)) + (let* ((name (package-upstream-name package)) (url (if (string-null? directory) base-url (string-append base-url directory "/"))) - (links (url->links url))) + (links (map (cut canonicalize-url <> url) (url->links url)))) + (define (file->signature/guess url) "Return the first link that matches a signature extension, else #f." (let ((base (basename url))) @@ -526,42 +554,12 @@ (define* (import-html-release base-url package (define (url->release url) "Return an object if a release file was found at URL, -else #f." - (let* ((base (basename url)) - (base-url (string-append base-url directory)) - (url (cond ((and=> (string->uri url) uri-scheme) ;full URL? - url) - ;; full URL, except for URI scheme. Reuse the URI - ;; scheme of the document that contains the link. - ((string-prefix? "//" url) - (string-append - (symbol->string (uri-scheme (string->uri base-url))) - ":" url)) - ((string-prefix? "/" url) ;absolute path? - (let ((uri (string->uri base-url))) - (uri->string - (build-uri (uri-scheme uri) - #:host (uri-host uri) - #:port (uri-port uri) - #:path url)))) - - ;; URL is a relative path and BASE-URL may or may not - ;; end in slash. - ((string-suffix? "/" base-url) - (string-append base-url url)) - (else - ;; If DIRECTORY is non-empty, assume BASE-URL - ;; denotes a directory; otherwise, assume BASE-URL - ;; denotes a file within a directory, and that URL - ;; is relative to that directory. - (string-append (if (string-null? directory) - (dirname base-url) - base-url) - "/" url))))) - (and (release-file? package base) +else #f. URL is assumed to fully specified." + (let ((base (basename url))) + (and (release-file? name base) (let ((version (tarball->version base))) (upstream-source - (package package) + (package name) (version version) ;; uri-mirror-rewrite: Don't turn nice mirror:// URIs into ftp:// ;; URLs during "guix refresh -u". From patchwork Mon Aug 21 18:06:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 53077 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 6BCC727BBEA; Mon, 21 Aug 2023 19:11:04 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 9EF8A27BBE2 for ; Mon, 21 Aug 2023 19:11:02 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY9Mb-0005em-Hf; Mon, 21 Aug 2023 14:10:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY9Ln-0005Mq-U1 for guix-patches@gnu.org; Mon, 21 Aug 2023 14:10:08 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY9Lm-0004ui-Tr; Mon, 21 Aug 2023 14:10:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qY9Lo-0007WP-HF; Mon, 21 Aug 2023 14:10:04 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65230] [PATCH v3 08/10] gnu-maintenance: Add support to rewrite version in URL path. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 21 Aug 2023 18:10:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65230 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65230@debbugs.gnu.org Cc: Maxim Cournoyer , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65230-submit@debbugs.gnu.org id=B65230.169264136028747 (code B ref 65230); Mon, 21 Aug 2023 18:10:04 +0000 Received: (at 65230) by debbugs.gnu.org; 21 Aug 2023 18:09:20 +0000 Received: from localhost ([127.0.0.1]:57692 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9L5-0007TT-8l for submit@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:20 -0400 Received: from mail-qk1-x72c.google.com ([2607:f8b0:4864:20::72c]:42019) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9Ky-0007SD-PF for 65230@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:14 -0400 Received: by mail-qk1-x72c.google.com with SMTP id af79cd13be357-76da88804c1so51464285a.0 for <65230@debbugs.gnu.org>; Mon, 21 Aug 2023 11:09:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692641345; x=1693246145; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pSgDAuzYQ4pY5Tno0h9v46bbcunDuIvk9rvy84FGWMI=; b=mM5uY9tP6Z9JSTGkDU2wjC1XNWgBtlUpvv4qOJem5gI/Me5Yz73eyRLS7T7lUDDt8J VZudBIV7lIX2QY6U9vLnhMbWUhIQ07XHcPkRf8pL5118p8MFOsj3Y4gxnR22cUEzE9w7 W9IHQsW19FeHRBb9npg+CU0uRDolhXBkJbXjO0zWxRCkXh4QS4VdXm5o4j0o792ug57l MD1oevoNkXUCBqjQtUyglAxFh0bVE1r/ZaZdTn5DnhAEUIahtf8FwNIV6ZpdR4+FGmz+ /G6rGvnfKBW4x84UfimWg0XptT+jfRFPZ1l6seeiv66M2qd+l//VkWXg6YN/FySvukmY EVTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692641345; x=1693246145; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pSgDAuzYQ4pY5Tno0h9v46bbcunDuIvk9rvy84FGWMI=; b=IOw4frF6WpyDK1HD15NHBXWVw4euEYE0/6RUjX4wreNgne2+2j5WUmtLqlH2n1Dt0t 84+Hi1SrqL5jO2gMF7bMV8GeEMtUVw8dD6xkMys3RCcEqc5z1G9XaJwFMNiCopssFUn1 AZnD33jRE1Cy2UXpYu6PRBR2w2cD20aQYcnaXp1oUT6aXW5tDWmtPUCqRMsIYPf8Tm2L PBLES1C7d8+nYYZ3uYcGGmBwjX/8d4DTVj1vxMMtVUj6pb1vUXB3UmMA12A4pYpjshuf A/37XhW9q/8Qpqa/jUWnSZOVI9LZga74T8U/VDiTKNqwNPJhSdJcl80GVWD8rFSXahSF m9cw== X-Gm-Message-State: AOJu0YwOONxhKTz5sNNlZPvWWAwu/bhgkWxBbX3BFyX1RbowJJO8SL5j 5Vp4UPNq0lnubEMz6VdFSiUDADGTRWw= X-Google-Smtp-Source: AGHT+IHyo7xtbxJ2byIfrK9etrKrkBy8eRCDsZEWvAQdDtvM63dsv6s7tQJOIbO31fMI7xShpfhaIA== X-Received: by 2002:a05:620a:3915:b0:76c:a911:f74f with SMTP id qr21-20020a05620a391500b0076ca911f74fmr10318512qkn.27.1692641344581; Mon, 21 Aug 2023 11:09:04 -0700 (PDT) Received: from localhost.localdomain (dsl-10-148-105.b2b2c.ca. [72.10.148.105]) by smtp.gmail.com with ESMTPSA id t16-20020a05620a035000b00767303dc070sm2635523qkm.8.2023.08.21.11.09.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 11:09:03 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Aug 2023 14:06:12 -0400 Message-ID: <7566bf9f3d57c65f4a7d2a09924aaf7252016f09.1692641173.git.maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> References: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches Fixes . Previously, the generic HTML updater would only look for the list of files found at the parent of its current source URL, ignoring that the URL may embed the version elsewhere in its path. This could cause 'guix refresh' to report no updates available, while in fact there were, such as for 'libuv'. * guix/gnu-maintenance.scm (strip-trailing-slash): New procedure. (%version-rx): New variable. (rewrite-url): New procedure. (import-html-release): New rewrite-url? argument. When true, use the above procedure. (import-html-updatable-release): Call import-html-release with #:rewrite-url set to #t. * tests/gnu-maintenance.scm ("rewrite-url, to-version specified") ("rewrite-url, without to-version"): New tests. --- (no changes since v1) guix/gnu-maintenance.scm | 102 ++++++++++++++++++++++++++++++++++++-- tests/gnu-maintenance.scm | 43 ++++++++++++++++ 2 files changed, 142 insertions(+), 3 deletions(-) diff --git a/guix/gnu-maintenance.scm b/guix/gnu-maintenance.scm index abba891d4b..3cd84ee3d7 100644 --- a/guix/gnu-maintenance.scm +++ b/guix/gnu-maintenance.scm @@ -3,6 +3,7 @@ ;;; Copyright © 2012, 2013 Nikita Karetnikov ;;; Copyright © 2021 Simon Tournier ;;; Copyright © 2022 Maxime Devos +;;; Copyright © 2023 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -26,6 +27,7 @@ (define-module (guix gnu-maintenance) #:use-module (ice-9 regex) #:use-module (ice-9 match) #:use-module (srfi srfi-1) + #:use-module (srfi srfi-2) #:use-module (srfi srfi-11) #:use-module (srfi srfi-26) #:use-module (rnrs io ports) @@ -61,6 +63,7 @@ (define-module (guix gnu-maintenance) gnu-package? uri-mirror-rewrite + rewrite-url release-file? releases @@ -518,9 +521,93 @@ (define (canonicalize-url url base-url) ;; within a directory. (string-append (dirname base-url) "/" url)))) +(define (strip-trailing-slash s) + "Strip any trailing slash from S, a string." + (if (string-suffix? "/" s) + (string-drop-right s 1) + s)) + +;;; TODO: Extend to support the RPM and GNOME version schemes? +(define %version-rx "[0-9.]+") + +(define* (rewrite-url url version #:key to-version) + "Rewrite URL so that the URL path components matching the current VERSION or +VERSION-MAJOR.VERSION-MINOR are updated with that of the latest version found +by crawling the corresponding URL directories. Alternatively, when TO-VERSION +is specified, rewrite version matches directly to it without crawling URL. + +For example, the URL +\"https://dist.libuv.org/dist/v1.45.0/libuv-v1.45.0.tar.gz\" could be +rewritten to something like +\"https://dist.libuv.org/dist/v1.46.0/libuv-v1.46.0.tar.gz\"." + ;; XXX: major-minor may be #f if version is not a triplet but a single + ;; number such as "2". + (let* ((major-minor (false-if-exception (version-major+minor version))) + (to-major-minor (false-if-exception + (and=> to-version version-major+minor))) + (uri (string->uri url)) + (url-prefix (string-drop-right url (string-length (uri-path uri)))) + (url-prefix-components (string-split url-prefix #\/)) + (path (uri-path uri)) + ;; Strip a forward slash on the path to avoid a double slash when + ;; string-joining later. + (path (if (string-prefix? "/" path) + (string-drop path 1) + path)) + (path-components (string-split path #\/))) + (string-join + (reverse + (fold + (lambda (s parents) + (if to-version + ;; Direct rewrite case; the archive is assumed to exist. + (let ((u (string-replace-substring s version to-version))) + (cons (if (and major-minor to-major-minor) + (string-replace-substring u major-minor to-major-minor) + u) + parents)) + ;; More involved HTML crawl case. + (let* ((pattern (if major-minor + (format #f "(~a|~a)" version major-minor) + (format #f "(~a)" version))) + (m (string-match pattern s))) + (if m + ;; Crawl parent and rewrite current component. + (let* ((parent-url (string-join (reverse parents) "/")) + (links (url->links parent-url)) + ;; The pattern matching the version. + (pattern (string-append "^" (match:prefix m) + "(" %version-rx ")" + (match:suffix m) "$")) + (candidates (filter-map + (lambda (l) + ;; Links may be followed by a + ;; trailing '/' in the case of + ;; directories. + (and-let* + ((l (strip-trailing-slash l)) + (m (string-match pattern l)) + (v (match:substring m 1))) + (cons v l))) + links))) + ;; Retrieve the item having the largest version. + (if (null? candidates) + (error "no candidates found in rewrite-url") + (cons (cdr (first (sort candidates + (lambda (x y) + (version>? (car x) + (car y)))))) + parents))) + ;; No version found in path component; continue. + (cons s parents))))) + (reverse url-prefix-components) + path-components)) + "/"))) + (define* (import-html-release base-url package #:key - (version #f) + rewrite-url? + version (directory (string-append "/" (package-upstream-name package))) file->signature) @@ -534,11 +621,19 @@ (define* (import-html-release base-url package When FILE->SIGNATURE is omitted or #f, guess the detached signature file name, if any. Otherwise, FILE->SIGNATURE must be a procedure; it is passed a source file URL and must return the corresponding signature URL, or #f it signatures -are unavailable." - (let* ((name (package-upstream-name package)) +are unavailable. + +When REWRITE-URL? is #t, versioned components in BASE-URL and/or DIRECTORY are +also updated to the latest version, as explained in the doc of the +\"rewrite-url\" procedure used." + (let* ((current-version (package-version package)) + (name (package-upstream-name package)) (url (if (string-null? directory) base-url (string-append base-url directory "/"))) + (url (if rewrite-url? + (rewrite-url url current-version #:to-version version) + url)) (links (map (cut canonicalize-url <> url) (url->links url)))) (define (file->signature/guess url) @@ -873,6 +968,7 @@ (define* (import-html-updatable-release package #:key (version #f)) (dirname (uri-path uri))))) (false-if-networking-error (import-html-release base package + #:rewrite-url? #t #:version version #:directory directory)))) diff --git a/tests/gnu-maintenance.scm b/tests/gnu-maintenance.scm index 516e02ec6a..196a6f9092 100644 --- a/tests/gnu-maintenance.scm +++ b/tests/gnu-maintenance.scm @@ -147,4 +147,47 @@ (define-module (test-gnu-maintenance) (equal? (list expected-signature-url) (upstream-source-signature-urls update)))))) +(test-equal "rewrite-url, to-version specified" + "https://download.qt.io/official_releases/qt/6.5/6.5.2/\ +submodules/qtbase-everywhere-src-6.5.2.tar.xz" + (rewrite-url "https://download.qt.io/official_releases/qt/6.3/6.3.2/\ +submodules/qtbase-everywhere-src-6.3.2.tar.xz" "6.3.2" #:to-version "6.5.2")) + +(test-equal "rewrite-url, without to-version" + "https://dist.libuv.org/dist/v1.46.0/libuv-v1.46.0.tar.gz" + (with-http-server + ;; First reply, crawling https://dist.libuv.org/dist/. + `((200 "\ + + +Index of dist + +../ +v1.44.0/ +v1.44.1/ +v1.44.2/ +v1.45.0/ +v1.46.0/ + +") + ;; Second reply, crawling https://dist.libuv.org/dist/v1.46.0/. + (200 "\ + + +Index of dist/v1.46.0 + +../ + + libuv-v1.46.0-dist.tar.gz +libuv-v1.46.0-dist.tar.gz.sign + + libuv-v1.46.0.tar.gz + + libuv-v1.46.0.tar.gz.sign + +")) + (rewrite-url "https://dist.libuv.org/dist/v1.45.0/libuv-v1.45.0.tar.gz" + "1.45.0"))) + (test-end) From patchwork Mon Aug 21 18:06:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 53078 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 8F17327BBE9; Mon, 21 Aug 2023 19:11:24 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 0B57427BBE2 for ; Mon, 21 Aug 2023 19:11:22 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY9Md-0005gD-B4; Mon, 21 Aug 2023 14:10:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY9Ln-0005Mp-U4 for guix-patches@gnu.org; Mon, 21 Aug 2023 14:10:08 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY9Lm-0004sv-OW; Mon, 21 Aug 2023 14:10:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qY9Lp-0007WY-0P; Mon, 21 Aug 2023 14:10:05 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65230] [PATCH v3 09/10] gnu-maintenance: Allow mirror URLs to fallback to the generic HTML updater. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 21 Aug 2023 18:10:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65230 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65230@debbugs.gnu.org Cc: Maxim Cournoyer , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65230-submit@debbugs.gnu.org id=B65230.169264136128756 (code B ref 65230); Mon, 21 Aug 2023 18:10:04 +0000 Received: (at 65230) by debbugs.gnu.org; 21 Aug 2023 18:09:21 +0000 Received: from localhost ([127.0.0.1]:57694 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9L6-0007Tc-97 for submit@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:20 -0400 Received: from mail-qt1-x82c.google.com ([2607:f8b0:4864:20::82c]:57396) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9L0-0007SR-Nc for 65230@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:16 -0400 Received: by mail-qt1-x82c.google.com with SMTP id d75a77b69052e-40fd276621aso23901731cf.2 for <65230@debbugs.gnu.org>; Mon, 21 Aug 2023 11:09:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692641346; x=1693246146; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mrcr7qd3HJBLWUbh5Y66xNjCZ58YmhSyEVgbiO1yMd8=; b=k4qXqDxXk36L0l1tjoNIOuvsuGKX2RzJOK2Z50rJNUibvfkk1AoGqVqdS42kmLfYSV +aHU1NNjUEhwSTLFItBM3wNGsKbUf/bdBpaM3wAjKND/saCbYlasi8dzwiYIEUpxxc8K BfizPCODkx+5727hzdmZQg/NDCRyC+3c63DG1YZrH1DqAadksglF7+AuMTICIgYVtimf K+XxKgXgjDkER2dgJPj2+R7tHAaiUkP4OEkz/0KedHp4WyYPKswbNNIRClUqldvloBr3 w6u8sWPj6xdYdjzY5NygJ+TIXxwXhFt88eg/SsCqPEba9W5bHW4PuOvO7d2hSA76xXX2 4W9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692641346; x=1693246146; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mrcr7qd3HJBLWUbh5Y66xNjCZ58YmhSyEVgbiO1yMd8=; b=L/IGywyHMaMM1fhKJuBMGl/GkA5wu24O2PJUF6SvQTpwQbiaEEP7JEdcmeIEfnjQlG lBASWY86IE1q2dM1kN1xk2fEeBT8nzVAiJORF/oLmUAmxNJSIaeQ1YeLk0uyuTYLJ2B3 bB14syJ3DvK75Lf0txW/IEHMPxLSQ9vYBix+TRtYK07YyDYIYerRIVXjgRu/Ypf1Hd1n bNfWFiMkKQNDhNgGs0brjm4dHbgzrFueepSIuPLy9oC5f1LK9z4bmozVN1vwScMbcy0q L7H+srLzXPVLcKmN21TV7cYJ404vKbmhQBWhbMWZuyJkx3qHHd60ZNdZ4r+vh2nAZCHZ h+Tg== X-Gm-Message-State: AOJu0Yy/OxScMGBafI4PyJUrBo1ptSnB71ys6J4vioILfxvxFGllwAfV DrZSwvD17b8gLJrZmfG0EHSHnoKXlow= X-Google-Smtp-Source: AGHT+IEQsbqhIg1K02RS+C5Xt1l2DRsuS4MbM7H6fCeW2ZhpPuNlD/nh2XR4+sVYPDqaeeOylH3f1w== X-Received: by 2002:ac8:570a:0:b0:409:f273:e28d with SMTP id 10-20020ac8570a000000b00409f273e28dmr7536750qtw.62.1692641346535; Mon, 21 Aug 2023 11:09:06 -0700 (PDT) Received: from localhost.localdomain (dsl-10-148-105.b2b2c.ca. [72.10.148.105]) by smtp.gmail.com with ESMTPSA id t16-20020a05620a035000b00767303dc070sm2635523qkm.8.2023.08.21.11.09.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 11:09:06 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Aug 2023 14:06:13 -0400 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> References: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/gnu-maintenance.scm (http-url?): Extract from html-updatable-package?, modify to return the HTTP URL, and support the mirror:// scheme. (%disallowed-hosting-sites): New variable, extracted from html-updatable-package. (html-updatable-package?): Rewrite a mirror:// URL to an HTTP or HTTPS one. * guix/download.scm (%mirrors): Update comment. --- (no changes since v2) Changes in v2: - Update %mirrors comment to mention speed-related exceptions guix/download.scm | 5 +++- guix/gnu-maintenance.scm | 58 +++++++++++++++++++++++++--------------- 2 files changed, 41 insertions(+), 22 deletions(-) diff --git a/guix/download.scm b/guix/download.scm index 30d7c5a86e..334290c7fb 100644 --- a/guix/download.scm +++ b/guix/download.scm @@ -51,7 +51,10 @@ (define-module (guix download) ;;; Code: (define %mirrors - ;; Mirror lists used when `mirror://' URLs are passed. + ;; Mirror lists used when `mirror://' URLs are passed. The first mirror + ;; entry of each set should ideally be the most authoritative one, as that's + ;; what the generic HTML updater will pick to look for updates, with + ;; possible exceptions when the authoritative mirror is too slow. (let* ((gnu-mirrors '(;; This one redirects to a (supposedly) nearby and (supposedly) ;; up-to-date mirror. diff --git a/guix/gnu-maintenance.scm b/guix/gnu-maintenance.scm index 3cd84ee3d7..2574e0f827 100644 --- a/guix/gnu-maintenance.scm +++ b/guix/gnu-maintenance.scm @@ -928,27 +928,40 @@ (define* (import-kernel.org-release package #:key (version #f)) #:directory directory #:file->signature file->signature))) -(define html-updatable-package? - ;; Return true if the given package may be handled by the generic HTML - ;; updater. - (let ((hosting-sites '("github.com" "github.io" "gitlab.com" - "notabug.org" "sr.ht" "gitlab.inria.fr" - "ftp.gnu.org" "download.savannah.gnu.org" - "pypi.org" "crates.io" "rubygems.org" - "bioconductor.org"))) - (define http-url? - (url-predicate (lambda (url) - (match (string->uri url) - (#f #f) - (uri - (let ((scheme (uri-scheme uri)) - (host (uri-host uri))) - (and (memq scheme '(http https)) - (not (member host hosting-sites))))))))) - - (lambda (package) - (or (assoc-ref (package-properties package) 'release-monitoring-url) - (http-url? package))))) +;;; These sites are disallowed for the generic HTML updater as there are +;;; better means to query them. +(define %disallowed-hosting-sites + '("github.com" "github.io" "gitlab.com" + "notabug.org" "sr.ht" "gitlab.inria.fr" + "ftp.gnu.org" "download.savannah.gnu.org" + "pypi.org" "crates.io" "rubygems.org" + "bioconductor.org")) + +(define (http-url? url) + "Return URL if URL has HTTP or HTTPS as its protocol. If URL uses the +special mirror:// protocol, substitute it with the first HTTP or HTTPS URL +prefix from its set." + (match (string->uri url) + (#f #f) + (uri + (let ((scheme (uri-scheme uri)) + (host (uri-host uri))) + (or (and (memq scheme '(http https)) + (not (member host %disallowed-hosting-sites)) + url) + (and (eq? scheme 'mirror) + (and=> (find http-url? + (assoc-ref %mirrors + (string->symbol host))) + (lambda (url) + (string-append (strip-trailing-slash url) + (uri-path uri)))))))))) + +(define (html-updatable-package? package) + "Return true if the given package may be handled by the generic HTML +updater." + (or (assoc-ref (package-properties package) 'release-monitoring-url) + ((url-predicate http-url?) package))) (define* (import-html-updatable-release package #:key (version #f)) "Return the latest release of PACKAGE. Do that by crawling the HTML page of @@ -956,6 +969,9 @@ (define* (import-html-updatable-release package #:key (version #f)) string to fetch a specific version." (let* ((uri (string->uri (match (origin-uri (package-source package)) + ((? (cut string-prefix? "mirror://" <>) url) + ;; Retrieve the authoritative HTTP URL from a mirror. + (http-url? url)) ((? string? url) url) ((url _ ...) url)))) (custom (assoc-ref (package-properties package) From patchwork Mon Aug 21 18:06:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 53082 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id B7B4C27BBE2; Mon, 21 Aug 2023 19:12:05 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 8796627BBE9 for ; Mon, 21 Aug 2023 19:12:04 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY9N6-0005qg-JD; Mon, 21 Aug 2023 14:11:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY9Ls-0005NP-QI for guix-patches@gnu.org; Mon, 21 Aug 2023 14:10:13 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY9Lo-000500-W8; Mon, 21 Aug 2023 14:10:07 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qY9Lp-0007Wn-EK; Mon, 21 Aug 2023 14:10:05 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65230] [PATCH v3 10/10] gnu-maintenance: Consider Qt source tarballs as "release files". Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 21 Aug 2023 18:10:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65230 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65230@debbugs.gnu.org Cc: Maxim Cournoyer , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65230-submit@debbugs.gnu.org id=B65230.169264136128762 (code B ref 65230); Mon, 21 Aug 2023 18:10:05 +0000 Received: (at 65230) by debbugs.gnu.org; 21 Aug 2023 18:09:21 +0000 Received: from localhost ([127.0.0.1]:57696 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9L6-0007Tl-Ub for submit@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:21 -0400 Received: from mail-qt1-x82a.google.com ([2607:f8b0:4864:20::82a]:60577) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qY9L2-0007Sq-Ic for 65230@debbugs.gnu.org; Mon, 21 Aug 2023 14:09:16 -0400 Received: by mail-qt1-x82a.google.com with SMTP id d75a77b69052e-4039f7e1d3aso24862641cf.0 for <65230@debbugs.gnu.org>; Mon, 21 Aug 2023 11:09:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692641348; x=1693246148; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KWXYeNMQQXuJ4AlJoauOKszS8fSMwi4+qwJkA4epQvM=; b=aRZXM+fQIiOpxbUHfCNcXTknGzjwMDWpZCsRjhQiIslVYElrFwHWPsHrNgZt+Un8is mnnvuF34fpoRO3dF5MqZ9Tzz3MgTEZ6TMzXjLGlulaCjyHwNrO06j32m1R03M4zXFHZ8 5O92izWkY3zA5zCQvy3asUPXut0LFjjFcniP4rp/7ABVEpmctsZgBrweZU27DuczjPa/ n/RK/AVnJc2I85cldyf+GfCJyaTHdrSX3mtxfAzf2G5pn0AtFnhwBbhcG5oFxb2ojf60 cIdmcvGrlNLeF/tKZBmk7lTpwL+6Eg5mD48lpJQ4aLqUxXNT5cTwMdC1mNB7QiGfzqLX 9bug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692641348; x=1693246148; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KWXYeNMQQXuJ4AlJoauOKszS8fSMwi4+qwJkA4epQvM=; b=I11JYYrxA/EQQqzxLEgUyN53Fu5OzWxYRKMs2Y1jZC4XgAOweDWjDb0kk80/236Ygw vXZSJEkr2ORHp7ZwBz3eAJvOsGgBV/SVmV/VW9vECcnOHlVaHyTmDvis4JydSNhBV7b4 gQ2bvXtfhbE2kS2SzzVOPo04yUKFbKCwb7Ln3DfXQKr8VhsNfJebP/hPaMBAIAHW2Sj/ sFdK6P7GJUIxiUELx76WJeFKPJxnUOlRIbHhb3Z/F/18FJPBpzldtGbXahb0vXXWPkg/ huJje4eeVKzmCVaipyOeOGhDLYlm/apgTGu/2UWuM9R7n2a9JkGquGQ/UQik+wXEgoDW DZ/g== X-Gm-Message-State: AOJu0YzqNxbO0uxumnOM0jRzaPuYjVVTcd1Ha3zfRjmT7itKF0dBTAjE 6tDu93c78MMe+GPe8Fo13B248jQsF50= X-Google-Smtp-Source: AGHT+IH0E9v85g9SjsXZ0Z/7H2++Q9nTFMMb9ds0aM0XpKH7SfSGYSxSErOTh7XIwVqEh5lelPfhtw== X-Received: by 2002:ac8:5a8f:0:b0:406:cadc:ff35 with SMTP id c15-20020ac85a8f000000b00406cadcff35mr8016812qtc.54.1692641348477; Mon, 21 Aug 2023 11:09:08 -0700 (PDT) Received: from localhost.localdomain (dsl-10-148-105.b2b2c.ca. [72.10.148.105]) by smtp.gmail.com with ESMTPSA id t16-20020a05620a035000b00767303dc070sm2635523qkm.8.2023.08.21.11.09.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 11:09:08 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Aug 2023 14:06:14 -0400 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> References: <5465490ad96f27fecbb69f9bd6f1607d29af6c23.1692641173.git.maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/gnu-maintenance.scm (release-file?): Use positive logic in doc. Add a special case for Qt source archives. * tests/gnu-maintenance.scm ("release-file?"): Update test. --- Changes in v3: - Move a couple Qt-specific commits to the qt-updates branch Changes in v2: - Also special case release file of Qt Creator guix/gnu-maintenance.scm | 18 +++++++++++++----- tests/gnu-maintenance.scm | 5 ++++- 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/guix/gnu-maintenance.scm b/guix/gnu-maintenance.scm index 2574e0f827..04827a9f27 100644 --- a/guix/gnu-maintenance.scm +++ b/guix/gnu-maintenance.scm @@ -258,8 +258,7 @@ (define %alpha-tarball-rx (make-regexp "^.*-.*[0-9](-|~|\\.)?(alpha|beta|rc|RC|cvs|svn|git)-?[0-9\\.]*\\.tar\\.")) (define (release-file? project file) - "Return #f if FILE is not a release tarball of PROJECT, otherwise return -true." + "Return true if FILE is a release tarball of PROJECT." (and (not (member (file-extension file) '("sig" "sign" "asc" "md5sum" "sha1sum" "sha256sum"))) @@ -268,12 +267,21 @@ (define (release-file? project file) ;; Filter out unrelated files, like `guile-www-1.1.1'. ;; Case-insensitive for things like "TeXmacs" vs. "texmacs". ;; The "-src" suffix is for "freefont-src-20120503.tar.gz". + ;; The '-everywhere-src' suffix is for Qt modular components. (and=> (match:substring match 1) (lambda (name) (or (string-ci=? name project) - (string-ci=? name - (string-append project - "-src"))))))) + (string-ci=? name (string-append project "-src")) + (string-ci=? + name (string-append project "-everywhere-src")) + ;; For older Qt releases such as version 5. + (string-ci=? + name (string-append + project "-everywhere-opensource-src")) + ;; For Qt Creator. + (string-ci=? + name (string-append + project "-opensource-src"))))))) (not (regexp-exec %alpha-tarball-rx file)) (let ((s (tarball-sans-extension file))) (regexp-exec %package-name-rx s)))) diff --git a/tests/gnu-maintenance.scm b/tests/gnu-maintenance.scm index 196a6f9092..61ae295b96 100644 --- a/tests/gnu-maintenance.scm +++ b/tests/gnu-maintenance.scm @@ -40,7 +40,10 @@ (define-module (test-gnu-maintenance) ("exiv2" "exiv2-0.27.3-Source.tar.gz") ("mpg321" "mpg321_0.3.2.orig.tar.gz") ("bvi" "bvi-1.4.1.src.tar.gz") - ("hostscope" "hostscope-V2.1.tgz"))) + ("hostscope" "hostscope-V2.1.tgz") + ("qtbase" "qtbase-everywhere-src-6.5.2.tar.xz") + ("qtbase" "qtbase-everywhere-opensource-src-5.15.10.tar.xz") + ("qt-creator" "qt-creator-opensource-src-11.0.1.tar.xz"))) (every (lambda (project+file) (not (apply release-file? project+file))) '(("guile" "guile-www-1.1.1.tar.gz")