From patchwork Mon Jun 21 06:11:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 30563 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 137CC27BC81; Mon, 21 Jun 2021 07:13:30 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 529B827BC78 for ; Mon, 21 Jun 2021 07:13:29 +0100 (BST) Received: from localhost ([::1]:38956 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lvDBY-0008Hg-D3 for patchwork@mira.cbaines.net; Mon, 21 Jun 2021 02:13:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47926) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lvDB9-0007uD-5H for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:51893) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lvDB8-0001re-Tw for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lvDB8-0006cQ-Qs for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#49149] [PATCH] tentatively reuse rlib for cargo-build-system Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 21 Jun 2021 06:13:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49149 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 49149@debbugs.gnu.org Cc: Maxim Cournoyer Received: via spool by 49149-submit@debbugs.gnu.org id=B49149.162425596525337 (code B ref 49149); Mon, 21 Jun 2021 06:13:02 +0000 Received: (at 49149) by debbugs.gnu.org; 21 Jun 2021 06:12:45 +0000 Received: from localhost ([127.0.0.1]:35187 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lvDAr-0006aa-9V for submit@debbugs.gnu.org; Mon, 21 Jun 2021 02:12:45 -0400 Received: from mail-qt1-f172.google.com ([209.85.160.172]:42567) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lvDAp-0006aH-Lk for 49149@debbugs.gnu.org; Mon, 21 Jun 2021 02:12:44 -0400 Received: by mail-qt1-f172.google.com with SMTP id x21so3454466qtq.9 for <49149@debbugs.gnu.org>; Sun, 20 Jun 2021 23:12:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Z1X73fhCPAWV4PwnGSeUz2Pz/GvBtru7kZ+p33Emoek=; b=GuVtRoLhhdpOGPTdMwy4gt9GgE4WvLvEzCfMaMuukpbekRumujLdSylI/6F3d8n9pM 79/TgxkIeUhFvSu6QC9lfDd7ahTnA56X9gDbrKlrFeHlvPTKHpZVyIxUxLkEh6eieqy4 fSy9f5BW1PiAuzBl/OV7NrBsLGQ0d1xkqgJ2cq2bshhwbDjohk6MjpF+GyGoW/EsVUbG 3XmYwnEjvIcx/pICaJfNFrdHLbTMkvwqwbtELRfNiUMZypT9xwmodWjzWJOiXBMvuVvk PRP+1bEztvVZ8mvwNGjILazJkfkwxD6YEVruG8NBmbUrVFY4ce0TmMgNYY7anTa6byCY hcXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Z1X73fhCPAWV4PwnGSeUz2Pz/GvBtru7kZ+p33Emoek=; b=LkhCCkQYItm/AfKyqsK9twi70loI7xqOkTIYV8AcAGQDmRRpO08zaQlax4GhZ/Rblx OQyWh3G+P+vtMuMFU2FHi6kaGbg30rnjGRnnFlOfbeukZ9lhMzsU89YYEfUWlJLlHht8 3qngbNv3BpYx0DtQJpH62rWchp+Fc8y6C+5BFHcnNgLw9hauRLdCvOEc4MuSmQRUzHNY IEcQ2x/+i31pChYlcdTrhKpEcZ0hpacQSipwYtPQBjjtfvm+C2EAe/WisVFJuTwA6gwN GowYbOKWLP/tyuvKLo4Tk1OctbHIw0FuY5ILyAOwQwHvvCCJPX6y3K0E9RteVZN+gX/L FAyg== X-Gm-Message-State: AOAM5301yz6SJbMJa3Vz1Qai6kkbAg6DpUleDIkrO8GX+rqwBcolfChD fGpnZOJZP+9jVObWYwQZlDwROtTp0jNi7A== X-Google-Smtp-Source: ABdhPJy7aedqmMEGhYnzMyYOWC2VuLOhQuuTQ1+xIYxITN/jYtZ4S57u6YSJf1Pzm65/QAwssLXwvA== X-Received: by 2002:ac8:7516:: with SMTP id u22mr22216679qtq.160.1624255958050; Sun, 20 Jun 2021 23:12:38 -0700 (PDT) Received: from localhost.localdomain (dsl-148-219.b2b2c.ca. [66.158.148.219]) by smtp.gmail.com with ESMTPSA id i11sm8478663qke.74.2021.06.20.23.12.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Jun 2021 23:12:37 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Jun 2021 02:11:59 -0400 Message-Id: <20210621061205.31878-2-maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210621061205.31878-1-maxim.cournoyer@gmail.com> References: <20210621061205.31878-1-maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches --- guix/build-system/cargo.scm | 3 +- guix/build/cargo-build-system.scm | 78 ++++++++++++++++++++++++++----- 2 files changed, 68 insertions(+), 13 deletions(-) diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm index e53d2a7523..9ef9f6b149 100644 --- a/guix/build-system/cargo.scm +++ b/guix/build-system/cargo.scm @@ -271,7 +271,8 @@ any dependent crates. This can be a benefits: (build-inputs `(("cargo" ,rust "cargo") ("rustc" ,rust) ,@(expand-crate-sources cargo-inputs cargo-development-inputs) - ,@native-inputs)) + ,@native-inputs + ,@(if target '() inputs))) (outputs outputs) (build cargo-build) (arguments (strip-keyword-arguments private-keywords arguments))))) diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm index 0a95672b00..e68f20e463 100644 --- a/guix/build/cargo-build-system.scm +++ b/guix/build/cargo-build-system.scm @@ -5,6 +5,7 @@ ;;; Copyright © 2019, 2020, 2021 Efraim Flashner ;;; Copyright © 2020 Jakub Kądziołka ;;; Copyright © 2020 Marius Bakke +;;; Copyright © 2021 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -24,7 +25,7 @@ (define-module (guix build cargo-build-system) #:use-module ((guix build gnu-build-system) #:prefix gnu:) #:use-module (guix build json) - #:use-module (guix build utils) + #:use-module ((guix build utils) #:hide (delete)) #:use-module (guix build cargo-utils) #:use-module (ice-9 popen) #:use-module (ice-9 rdelim) @@ -34,7 +35,10 @@ #:use-module (srfi srfi-1) #:use-module (srfi srfi-26) #:export (%standard-phases - cargo-build)) + cargo-build + + rust-version + rust-library-prefix)) ;; Commentary: ;; @@ -42,6 +46,25 @@ ;; ;; Code: +(define (rust-version rust) + "Return the version triplet (major.minor.patch) as a string, given RUST, a +store file name." + (let* ((version (last (string-split rust #\-))) + (components (string-split version #\.)) + (major+minor+patch (take components 3))) + (string-join major+minor+patch "."))) + +(define (rust-library-prefix/relative inputs) + "Return the relative versioned Rust library prefix where Rust libraries are +to be installed." + (string-append "lib/rust/" (rust-version (assoc-ref inputs "rustc")))) + +(define (rust-library-prefix inputs outputs) + "Return the absolute versioned Rust library prefix where Rust libraries are +to be installed." + (string-append (assoc-ref outputs "out") "/" + (rust-library-prefix/relative inputs))) + (define (manifest-targets) "Extract all targets from the Cargo.toml manifest" (let* ((port (open-input-pipe "cargo read-manifest")) @@ -73,6 +96,16 @@ Cargo.toml file present at its root." " | cut -d/ -f2" " | grep -q '^Cargo.toml$'"))))) +(define (rlib? file) + "Check if FILE has the .rlib extension." + (string-suffix? ".rlib" file)) + +(define (inputs->directories inputs) + "Extract the directory part from INPUTS." + (match inputs + (((names . directories) ...) + directories))) + (define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys) (define (inputs->rust-inputs inputs) "Filter using the label part from INPUTS." @@ -80,11 +113,6 @@ Cargo.toml file present at its root." (match input ((name . _) (rust-package? name)))) inputs)) - (define (inputs->directories inputs) - "Extract the directory part from INPUTS." - (match inputs - (((names . directories) ...) - directories))) (let ((rust-inputs (inputs->directories (inputs->rust-inputs inputs)))) (unless (null? rust-inputs) @@ -185,6 +213,22 @@ directory = '" port) (generate-all-checksums vendor-dir) #t) +(define* (populate-cargo-cache #:key inputs outputs #:allow-other-keys) + "Populate the 'target/release' directory with any pre-built Rust libraries, +to avoid rebuilding them from sources when possible." + (let* ((rust-lib-prefix (rust-library-prefix/relative inputs)) + (input-dirs (inputs->directories inputs)) + (rust-lib-dirs (filter (lambda (f) + (file-exists? (string-append + f "/" rust-lib-prefix))) + input-dirs)) + (rlibs (delete-duplicates (append-map (cut find-files <> "\\.rlib$") + rust-lib-dirs)))) + (pk 'rust-lib-dirs rust-lib-dirs) + (pk 'rlibs rlibs) + (for-each (cut install-file <> "target/release") rlibs) + (invoke "find" "target"))) + (define* (build #:key skip-build? (features '()) @@ -228,7 +272,9 @@ directory = '" port) "Install a given Cargo package." (let* ((out (assoc-ref outputs "out")) (registry (string-append out "/share/cargo/registry")) - (sources (string-append out "/share/cargo/src"))) + (sources (string-append out "/share/cargo/src")) + (libdir (rust-library-prefix inputs outputs)) + (release-dir "target/release")) (mkdir-p out) ;; Make cargo reuse all the artifacts we just built instead @@ -237,10 +283,17 @@ directory = '" port) ;; Only install crates which include binary targets, ;; otherwise cargo will raise an error. - (or skip-build? - (not (has-executable-target?)) - (invoke "cargo" "install" "--no-track" "--path" "." "--root" out - "--features" (string-join features))) + (unless skip-build? + ;; Install binaries. + (when (has-executable-target?) + (apply invoke "cargo" "install" "--no-track" "--path" "." "--root" out + (if (not (null? features)) + (list "--features" (string-join features)) + '()))) + ;; Install static libraries. + (for-each (lambda (file) + (install-file (string-append release-dir "/" file) libdir)) + (scandir release-dir (cut string-suffix? ".rlib" <>)))) (when install-source? ;; Install crate tarballs and unpacked sources for later use. @@ -260,6 +313,7 @@ directory = '" port) (modify-phases gnu:%standard-phases (delete 'bootstrap) (replace 'configure configure) + (add-before 'build 'populate-cargo-cache populate-cargo-cache) (replace 'build build) (replace 'check check) (replace 'install install)