From patchwork Mon Apr 28 16:23:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hilton Chain X-Patchwork-Id: 42126 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 4D42A27BC4B; Mon, 28 Apr 2025 17:35:26 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,FROM_SUSPICIOUS_NTLD,MAILING_LIST_MULTI,PDS_OTHER_BAD_TLD, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_VALIDITY_CERTIFIED,RCVD_IN_VALIDITY_RPBL, RCVD_IN_VALIDITY_SAFE,SPF_HELO_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 67BEE27BC49 for ; Mon, 28 Apr 2025 17:35:25 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u9RRw-0000iq-Kg; Mon, 28 Apr 2025 12:35:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u9RPG-0007f4-Fx for guix-patches@gnu.org; Mon, 28 Apr 2025 12:32:40 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1u9RP7-0004Rk-JM for guix-patches@gnu.org; Mon, 28 Apr 2025 12:32:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:References:In-Reply-To:Date:From:To:Subject; bh=4dOltDErW8w9b5cCO6Tsfvk+Q+HwUkFuJaMDDKPG4NU=; b=IIZpCSfHlHrRsMzs7q58QZ6ICCfzrhf9OLV54ry1YcXLvGZwS1Z9oHolStJmLs3oLp0i7QtV2GwQCEjN2repyEaoXh1Rdk4kiWxpX3F4vtMplVfKLCWS93WvgA2jZ2WaEZbBMAVRyaYAvjBnaLPhPr/pFNB3SNQhqwUcVY8B9HHcTyDL4RUo/9ph1/k16K99CzP2ll2d0VZnfRT1vWSdqet2/WcgVRn5x8ombIYUw8j49Se7of8mJNWVtacQ8cOqadQfufQfd6efUhlq9kGqSD2qoGyNS9hGy41fxrHEqjeFsNhP0GD5Bn48BxpENSqq13Q601Xe6o27llPSnvmAZA==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1u9RP7-0003Mg-3A; Mon, 28 Apr 2025 12:32:25 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#77093] [PATCH v4 rust-team 18/22] build-system: cargo: Add =?utf-8?b?4oCYY2FyZ28taW5wdXRz4oCZLg==?= Resent-From: Hilton Chain Original-Sender: "Debbugs-submit" Resent-CC: divya@subvertising.org, efraim@flashner.co.il, hako@ultrarare.space, steve@futurile.net, guix-patches@gnu.org Resent-Date: Mon, 28 Apr 2025 16:32:24 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77093 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 77093@debbugs.gnu.org Cc: Hilton Chain , Divya Ranjan Pattanaik , Efraim Flashner , Hilton Chain , Steve George X-Debbugs-Original-Xcc: Divya Ranjan Pattanaik , Efraim Flashner , Hilton Chain , Steve George Received: via spool by 77093-submit@debbugs.gnu.org id=B77093.174585793712775 (code B ref 77093); Mon, 28 Apr 2025 16:32:24 +0000 Received: (at 77093) by debbugs.gnu.org; 28 Apr 2025 16:32:17 +0000 Received: from localhost ([127.0.0.1]:32901 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u9ROu-0003Ic-N9 for submit@debbugs.gnu.org; Mon, 28 Apr 2025 12:32:16 -0400 Received: from mail.boiledscript.com ([49.12.134.184]:51860) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u9ROY-0003DU-TG for 77093@debbugs.gnu.org; Mon, 28 Apr 2025 12:31:55 -0400 From: Hilton Chain DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ultrarare.space; s=mail; t=1745857909; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JQ3Omu14dEkErKXoH+/KoXcdAVtBDHePzGKsA3ZtN1A=; b=EKB4QzRHi7EuACK1Z5RMyxYt6g5bq6t7HfSnVIVTNimOR3wlHqM28Xt9VvA7C071J+zQhQ NdWvEVSBHa7f2IyFQS7QuK6nyKr/hMEYSQLFgReG4SYtIK/ZDxyMNvRRQDzN0lBgLYW2xz MCkFT1imbr8dOa4zrOiWuzJ4V5NELjVthksSmo6ccwbQTl6ovX8hlng9a2vaFfStPjPdzK 9xghudWxWuXoHdWipzVNAboThAt6Jrolt/d9Drcqt7t3OsSo3FjXAVqMAVV7ixnCKeF1wt w7Ii39NgPAKTYe5exKhbtn2+DdgKkUGw1BPq4Vc4xyQ3ZmrqL8qrPIchw2Ma/Q== Date: Tue, 29 Apr 2025 00:23:15 +0800 Message-ID: <6c08a9385bdc1fb8fcdfa980a5aa3af08066c215.1745855744.git.hako@ultrarare.space> In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * gnu/packages/rust-crates.scm: New file. * gnu/packages/rust-sources.scm: New file. * gnu/local.mk (GNU_SYSTEM_MODULES): Regisiter them. * guix/build-system/cargo.scm (define-cargo-inputs): New macro. (crate-source, cargo-inputs): New procedures. * guix/import/crate.scm: Hide ‘crate-source’ from (guix build-system cargo). * etc/teams/rust/audit-rust-crates: New file. * etc/teams/rust/cleanup-crates.sh: New file. * etc/teams/rust/rust-crates.tmpl: New file. * etc/teams/rust/unpack-new-crates.sh: New file. Change-Id: I2f2d705a3e376ed3c646f31b824052a2278d4fb3 --- etc/teams/rust/audit-rust-crates | 70 +++++++++++++++++++++++++++++ etc/teams/rust/cleanup-crates.sh | 37 +++++++++++++++ etc/teams/rust/rust-crates.tmpl | 44 ++++++++++++++++++ etc/teams/rust/unpack-new-crates.sh | 45 +++++++++++++++++++ gnu/local.mk | 2 + gnu/packages/rust-crates.scm | 44 ++++++++++++++++++ gnu/packages/rust-sources.scm | 29 ++++++++++++ guix/build-system/cargo.scm | 46 ++++++++++++++++++- guix/import/crate.scm | 2 +- 9 files changed, 317 insertions(+), 2 deletions(-) create mode 100755 etc/teams/rust/audit-rust-crates create mode 100755 etc/teams/rust/cleanup-crates.sh create mode 100644 etc/teams/rust/rust-crates.tmpl create mode 100755 etc/teams/rust/unpack-new-crates.sh create mode 100644 gnu/packages/rust-crates.scm create mode 100644 gnu/packages/rust-sources.scm diff --git a/etc/teams/rust/audit-rust-crates b/etc/teams/rust/audit-rust-crates new file mode 100755 index 0000000000..d5546fd1e1 --- /dev/null +++ b/etc/teams/rust/audit-rust-crates @@ -0,0 +1,70 @@ +#!/usr/bin/env -S gawk -f +# GNU Guix --- Functional package management for GNU +# Copyright © 2025 Efraim Flashner +# +# This file is part of GNU Guix. +# +# GNU Guix is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or (at +# your option) any later version. +# +# GNU Guix is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Guix. If not, see . + +# To run: +# ./etc/teams/rust/audit-rust-crates ./path/to/file.scm +# Prints the output of cargo-audit to the shell. + +# Make sure we have cargo-audit in our PATH +BEGIN { + if (system("which cargo-audit 1> /dev/null")) + exit 1; + # Parse a record at a time. + RS = "\n\n" + cargoAudit = "cargo-audit audit --file -" +} + +# Check the crate-source origin-only inputs +/crate-source/ { + for(i=3; i <= NF-2; i++) { + if($i == "(crate-source") { + cargoLock = cargoLock "[[package]]\nname = " $(i+1) "\nversion = " $(i+2) "\n" + next + } + } +} + +# Check the crates packaged from crates.io tarballs +/crate-uri/ { + for(i=3; i <= NF; i++) { + if($i == "(version") + crateVersion = $(i+1) + if($i == "(crate-uri") + crateName = $(i+1) + } + gsub(/)/, "", crateVersion) + cargoLock = cargoLock "[[package]]\nname = " crateName "\nversion = " crateVersion "\n" +} + +# The xxxx-cargo-input variables have a set style +# TODO: Replace the last dash between the name and the version with a space! +# This doesn't take into account swapping between "-" and "_" so we skip it. +#( $2 ~ /-cargo-inputs/ ) { +# sub(/-cargo-inputs/, "", $2) +# gsub(/)/, "", $0) +# gsub(/rust-/, "", $0) +# #gensub(/([[:alpha:]])-([[:digit:]]+)/, "\\1 \\2", "g", $i) +# print "[[package]]\nname = \"" $2 "\"\nversion = \"1.0.0\"\ndependencies = [" +# for (i = 4; i <= NF; i++) { +# print "\"" $i "\"," +# } +# print "]" +#} + +END { print cargoLock | cargoAudit } diff --git a/etc/teams/rust/cleanup-crates.sh b/etc/teams/rust/cleanup-crates.sh new file mode 100755 index 0000000000..eca37ca00c --- /dev/null +++ b/etc/teams/rust/cleanup-crates.sh @@ -0,0 +1,37 @@ +#!/bin/sh +# GNU Guix --- Functional package management for GNU +# Copyright © 2025 Hilton Chain +# +# This file is part of GNU Guix. +# +# GNU Guix is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or (at +# your option) any later version. +# +# GNU Guix is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Guix. If not, see . + +FILE=gnu/packages/rust-crates.scm +PATTERN='^(define rust-' + +grep "$PATTERN" $FILE | cut -d' ' -f2 | while IFS= read -r crate +do + if [ "$(grep -wc "$crate" $FILE)" -eq 1 ]; then + echo "\ +(begin + (use-modules (guix utils)) + (let ((source-properties + (find-definition-location \"$FILE\" '$crate #:define-prefix 'define))) + (and=> source-properties delete-expression)))" | + guix repl -t machine + fi +done + +# Delete extra newlines. +sed --in-place ':a;N;$!ba;s/\n\n\+/\n\n/g' $FILE diff --git a/etc/teams/rust/rust-crates.tmpl b/etc/teams/rust/rust-crates.tmpl new file mode 100644 index 0000000000..a2c7fcd27d --- /dev/null +++ b/etc/teams/rust/rust-crates.tmpl @@ -0,0 +1,44 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2025 Hilton Chain +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu packages rust-crates) + #:use-module (guix gexp) + #:use-module (guix packages) + #:use-module (guix download) + #:use-module (guix git-download) + #:use-module (guix build-system cargo) + #:use-module (gnu packages rust-sources) + #:export (lookup-cargo-inputs)) + +;;; +;;; This file is managed by ‘guix import’. Do NOT add definitions manually. +;;; + +;;; +;;; Rust libraries fetched from crates.io and non-workspace development +;;; snapshots. +;;; + +(define qqqq-separator 'begin-of-crates) + +(define ssss-separator 'end-of-crates) + + +;;; +;;; Cargo inputs. +;;; diff --git a/etc/teams/rust/unpack-new-crates.sh b/etc/teams/rust/unpack-new-crates.sh new file mode 100755 index 0000000000..7ff1080655 --- /dev/null +++ b/etc/teams/rust/unpack-new-crates.sh @@ -0,0 +1,45 @@ +#!/bin/sh +# GNU Guix --- Functional package management for GNU +# Copyright © 2025 Hilton Chain +# +# This file is part of GNU Guix. +# +# GNU Guix is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or (at +# your option) any later version. +# +# GNU Guix is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Guix. If not, see . + +# Usage: ./etc/teams/rust/unpack-new-crates.sh +# Then inspect DIR with your tool of choice. +# +# For example: +# ./etc/teams/rust/unpack-new-crates.sh tmp-rust-crates && +# emacs --eval '(find-dired "tmp-rust-crates" "")' +# +# Note this uses ‘git diff’, only unstaged one are unpacked. + +UNPACK_DIR="$1" +CHANGES="$(git diff gnu/packages/rust-crates.scm | grep -E '^\+\(define rust-' | cut -f2 -d' ')" + +rm -rf "$UNPACK_DIR" +mkdir -p "$UNPACK_DIR" + +for crate in $CHANGES +do + built="$(./pre-inst-env guix build -e "(@@ (gnu packages rust-crates) "$crate")" -v0 2>/dev/null)" + if [[ -n $built ]]; then + if [[ -d $built ]]; then + cp -r "$built" "$UNPACK_DIR" + else + tar xf "$built" -C "$UNPACK_DIR" + fi + fi +done diff --git a/gnu/local.mk b/gnu/local.mk index 4854e2fe92..c60c7f1fc7 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -616,6 +616,8 @@ GNU_SYSTEM_MODULES = \ %D%/packages/rush.scm \ %D%/packages/rust.scm \ %D%/packages/rust-apps.scm \ + %D%/packages/rust-crates.scm \ + %D%/packages/rust-sources.scm \ %D%/packages/samba.scm \ %D%/packages/sagemath.scm \ %D%/packages/sawfish.scm \ diff --git a/gnu/packages/rust-crates.scm b/gnu/packages/rust-crates.scm new file mode 100644 index 0000000000..a2c7fcd27d --- /dev/null +++ b/gnu/packages/rust-crates.scm @@ -0,0 +1,44 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2025 Hilton Chain +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu packages rust-crates) + #:use-module (guix gexp) + #:use-module (guix packages) + #:use-module (guix download) + #:use-module (guix git-download) + #:use-module (guix build-system cargo) + #:use-module (gnu packages rust-sources) + #:export (lookup-cargo-inputs)) + +;;; +;;; This file is managed by ‘guix import’. Do NOT add definitions manually. +;;; + +;;; +;;; Rust libraries fetched from crates.io and non-workspace development +;;; snapshots. +;;; + +(define qqqq-separator 'begin-of-crates) + +(define ssss-separator 'end-of-crates) + + +;;; +;;; Cargo inputs. +;;; diff --git a/gnu/packages/rust-sources.scm b/gnu/packages/rust-sources.scm new file mode 100644 index 0000000000..f8d25bbc2c --- /dev/null +++ b/gnu/packages/rust-sources.scm @@ -0,0 +1,29 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2025 Hilton Chain +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu packages rust-sources) + #:use-module (guix gexp) + #:use-module (guix packages) + #:use-module (guix download) + #:use-module (guix git-download) + #:use-module (guix build-system cargo)) + +;;; +;;; Cargo workspaces and Rust libraries requiring external inputs to unbundle. +;;; These packages are hidden, as they are not interesting to users. +;;; diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm index 7a07003262..659ad95a35 100644 --- a/guix/build-system/cargo.scm +++ b/guix/build-system/cargo.scm @@ -25,12 +25,15 @@ ;;; along with GNU Guix. If not, see . (define-module (guix build-system cargo) + #:use-module (guix diagnostics) + #:use-module (guix i18n) #:use-module (guix search-paths) #:use-module (guix store) #:use-module (guix utils) #:use-module (guix gexp) #:use-module (guix monads) #:use-module (guix packages) + #:use-module (guix download) #:use-module (guix platform) #:use-module (guix build-system) #:use-module (guix build-system gnu) @@ -45,7 +48,10 @@ (define-module (guix build-system cargo) crate-url crate-url? crate-uri - crate-name->package-name)) + crate-name->package-name + crate-source + define-cargo-inputs + cargo-inputs)) (define %crate-base-url (make-parameter "https://crates.io")) @@ -62,6 +68,44 @@ (define (crate-uri name version) (define (crate-name->package-name name) (downstream-package-name "rust-" name)) +;; NOTE: Only use this procedure in (gnu packages rust-crates). +(define* (crate-source name version hash #:key (patches '()) (snippet #f)) + (origin + (method url-fetch) + (uri (crate-uri name version)) + (file-name + (string-append (crate-name->package-name name) "-" version ".tar.gz")) + (sha256 (base32 hash)) + (modules '((guix build utils))) + (patches patches) + (snippet snippet))) + +(define-syntax define-cargo-inputs + (syntax-rules (=>) + ((_ lookup inputs ...) + (define lookup + (let ((table (make-hash-table))) + (letrec-syntax ((record + (syntax-rules (=>) + ((_) #t) + ((_ (name => lst) rest (... ...)) + (begin + (hashq-set! table 'name (filter identity lst)) + (record rest (... ...))))))) + (record inputs ...) + (lambda (name) + "Return the inputs for NAME." + (hashq-ref table name)))))))) + +(define* (cargo-inputs name #:key (module '(gnu packages rust-crates))) + "Lookup Cargo inputs for NAME defined in MODULE, return an empty list if +unavailable." + (let ((lookup (module-ref (resolve-interface module) 'lookup-cargo-inputs))) + (or (lookup name) + (begin + (warning (G_ "no Cargo inputs available for '~a'~%") name) + '())))) + (define (default-rust target) "Return the default Rust package." ;; Lazily resolve the binding to avoid a circular dependency. diff --git a/guix/import/crate.scm b/guix/import/crate.scm index 14e6e28c5b..a6f247bbae 100644 --- a/guix/import/crate.scm +++ b/guix/import/crate.scm @@ -27,7 +27,7 @@ (define-module (guix import crate) #:use-module (guix base32) - #:use-module (guix build-system cargo) + #:use-module ((guix build-system cargo) #:hide (crate-source)) #:use-module (guix diagnostics) #:use-module (gcrypt hash) #:use-module (guix http-client)