Message ID | 8734r143fv.fsf_-_@gnu.org |
---|---|
State | New |
Headers |
Return-Path: <guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org> X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 6F66B27BBEA; Wed, 1 May 2024 22:41:25 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 56A0527BBE2 for <patchwork@mira.cbaines.net>; Wed, 1 May 2024 22:41:20 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from <guix-patches-bounces@gnu.org>) id 1s2Hgx-0008H8-EU; Wed, 01 May 2024 17:40:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1s2Hgv-0008GZ-Fn for guix-patches@gnu.org; Wed, 01 May 2024 17:40:41 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1s2Hgv-0001gv-7z for guix-patches@gnu.org; Wed, 01 May 2024 17:40:41 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1s2HhG-0001Ej-Jy for guix-patches@gnu.org; Wed, 01 May 2024 17:41:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#70492] bug#66866: Grafting breaks cross-compilation Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= <ludo@gnu.org> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org> Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 01 May 2024 21:41:02 +0000 Resent-Message-ID: <handler.70492.B70492.17145996084690@debbugs.gnu.org> Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 70492 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: dan <i@dan.games> Cc: David Elsing <david.elsing@posteo.net>, dev@jpoiret.xyz, Philip McGrath <philip@philipmcgrath.com>, Mathieu Othacehe <othacehe@gnu.org>, efraim@flashner.co.il, 66866@debbugs.gnu.org, 70492@debbugs.gnu.org, janneke@gnu.org Received: via spool by 70492-submit@debbugs.gnu.org id=B70492.17145996084690 (code B ref 70492); Wed, 01 May 2024 21:41:02 +0000 Received: (at 70492) by debbugs.gnu.org; 1 May 2024 21:40:08 +0000 Received: from localhost ([127.0.0.1]:39597 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces@debbugs.gnu.org>) id 1s2HgN-0001DZ-Hp for submit@debbugs.gnu.org; Wed, 01 May 2024 17:40:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53606) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@gnu.org>) id 1s2HgK-0001D2-UZ; Wed, 01 May 2024 17:40:06 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@gnu.org>) id 1s2Hfs-0001IY-Ky; Wed, 01 May 2024 17:39:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=PKTJuTUWxkH1yaMMlZ+oMbsulqQ7RHSARVY7zE5+BHQ=; b=dTixssOHGGxuezOc38/1 Dz8N/Jgb5ZzwZ9DpXtVZmQcAFnV9pAXMEelzxCV4ZrJjCtcKJAhD8zYG9uL7sKGJ+/DepQ9/FfrD1 ++8A9YwZkdIXMxHGf0f8mQ8e7iYDueSP8hpt3olrUOQeRNC52VDibw293AkktLsdhurwWVVPx7ENe 757FT6oLkq6jWMV+7SmboAg7LbXZ2gNeXs44GKHnE5vehOKJAaaPAitKcKI3DkvxOvjhveWNixyNU yzUyD2MXMUixhd4zErDy+wqCRm0dgWuibiFCuv0Gokp1LkvGcpmqmB/CF+LTmfnwClgJRKZ/CcV2t l+VEUa0kpPLBmw==; From: Ludovic =?utf-8?q?Court=C3=A8s?= <ludo@gnu.org> In-Reply-To: <87frwn308k.fsf@dan.games> (dan's message of "Tue, 19 Mar 2024 01:30:31 +0800") References: <87ttnh5b4g.fsf@gnu.org> <86h6ien6bg.fsf@posteo.net> <87frwn308k.fsf@dan.games> Date: Wed, 01 May 2024 23:39:32 +0200 Message-ID: <8734r143fv.fsf_-_@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: <guix-patches.gnu.org> List-Unsubscribe: <https://lists.gnu.org/mailman/options/guix-patches>, <mailto:guix-patches-request@gnu.org?subject=unsubscribe> List-Archive: <https://lists.gnu.org/archive/html/guix-patches> List-Post: <mailto:guix-patches@gnu.org> List-Help: <mailto:guix-patches-request@gnu.org?subject=help> List-Subscribe: <https://lists.gnu.org/mailman/listinfo/guix-patches>, <mailto:guix-patches-request@gnu.org?subject=subscribe> Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches |
Series |
[bug#70492] bug#66866: Grafting breaks cross-compilation
|
|
Commit Message
Ludovic Courtès
May 1, 2024, 9:39 p.m. UTC
Hi, dan <i@dan.games> skribis: > I spent some time digging into the rabbit hole. After changing > the lower function of the copy-build-system to look more like the > lower function of the gnu-build-system, I'm able to cross compile > alsa-lib without the --no-grafts flag. The changes I made are > like: > > diff --git a/guix/build-system/copy.scm > b/guix/build-system/copy.scm > index d58931b33c..74304b4bfb 100644 > --- a/guix/build-system/copy.scm > +++ b/guix/build-system/copy.scm > @@ -66,13 +66,13 @@ (define* (lower name > (bag > (name name) > (system system) > - (host-inputs `(,@(if source > + (build-inputs `(,@(if source > `(("source" ,source)) > '()) > - ,@inputs > + ,@native-inputs > ;; Keep the standard inputs of > 'gnu-build-system'. > ,@(standard-packages))) > - (build-inputs native-inputs) > + (host-inputs inputs) > (outputs outputs) > (build copy-build) > (arguments (strip-keyword-arguments private-keywords > arguments)))) > > Can we put everything inside build-inputs? From my understanding, > copy-build-system shouldn't care about cross-compilation at all. Intuitively, if ‘copy-build-system’ is about copying architecture-independent files, then it should do the same thing whether or not we are cross-compiling. However, users can and do add phases whose result is architecture-dependent. Small sample: • ‘desec-certbot-hook’ captures a reference to curl, so it would get the wrong one when cross-compiling if we assumed build-inputs = host-inputs. • ‘chez-scheme-for-racket-bootstrap-bootfiles’ builds stuff when cross-compiling. Philip, could you explain the intent and what you expect here? So it would seem we can’t just assume everything is a native input like https://issues.guix.gnu.org/70492 does. Now, as David and you found out, the use of inputs in build-system/copy.scm:lower is bogus. It seems that it can be fixed by following the intended definition of build/host inputs, as David suggested: But wait! That’s all theoretical because the bag always has (target #f) and ‘copy-build’ bundles build and host inputs together, as if doing a native build. So it seems like https://issues.guix.gnu.org/70492 (putting everything in ‘build-inputs’) is OK, after all. But still, there seem to be some expectation that ‘copy-build-system’ can support cross-compilation for real, so maybe we should add a ‘copy-cross-build’ procedure in addition to the patch above. Thoughts? Ludo’.
Comments
Hello, I think I finally understand why the problem only occurs with grafts if a dependency uses copy-build-system. It is actually somewhat complicated. When a package is lowered into a derivation by 'package->derivation' in guix/packages.scm, a list of potentially applicable grafts is created by the 'bag-grafts' procedure. This works by first traversing the bag for native packages (where bag-build-inputs and bag-target-inputs are followed recursively; and also bag-host-inputs if not cross building) and building the native derivations of the grafted versions of the packages which have a replacement (such as glibc at the time of writing). Building the grafted package (in the 'input-graft' procedure) is done again with 'package->derivation', which eventually calls 'graft-derivation' in guix/grafts.scm, which in turn calls 'non-self-references', where the ungrafted package is actually built (ignoring the store monad indirection), not just its derivation. In the case of cross builds, the bag is also traversed for the target dependencies, where the bag-host-inputs are followed recursively. Analogously, this causes the ungrafted packages and their ungrafted replacement to be cross built to compute the grafted derivations. As dan found out, the 'lower' procedure in guix/build-system/copy.scm incorrectly puts standard-packages into the host inputs. They contain the glibc-final package in gnu/packages/commencement.scm (which inherits the replacement of glibc). Because copy-build-system puts them into bag-host-inputs, the package replacement of the glibc-final package is then cross built without grafts when calculating its grafted derivation, which it does not support. Note that the glibc-final package and its package replacement are distinct from the glibc package and its replacement in gnu/packages/base.scm. I confirmed this by running --8<---------------cut here---------------start------------->8--- ,use (guix) (with-store store (run-with-store store (package->cross-derivation (package-replacement (@@ (gnu packages commencement) glibc-final)) "i686-linux-gnu" #:graft? #f))) --8<---------------cut here---------------end--------------->8--- in the REPL, which returns the same incorrect glibc derivation that is attempted to be built as a dependency when running `guix build alsa-lib --target=i686-linux-gnu`. It is actually possible to compute the graft derivations only when needed by a store reference (which does not change the derivation of the grafted package). Then, this problem does not occur even without the fix for the copy-build-system, as the invalid derivations are never actually built. With substitutes or after running GC, this may even prevent building a package with replacement when the requested package output which (transitively) depends on it has no (transitive) store reference to it. I made a patch for that here: https://issues.guix.gnu.org/70895 Regarding the changes to the copy-build-system: Ludovic Courtès <ludo@gnu.org> writes: > But still, there seem to be some expectation that ‘copy-build-system’ > can support cross-compilation for real, so maybe we should add a > ‘copy-cross-build’ procedure in addition to the patch above. In guix/build-system/copy.scm, it is described as an extension of gnu-build-system and the manual says "It adds much of the gnu-build-system packages to the set of inputs. Because of this, the copy-build-system does not require all the boilerplate code often needed for the trivial-build-system." Therefore, I think it makes sense to add the copy-cross-build procedure so that copy-build-system actually behaves like gnu-build-system. Cheers, David
diff --git a/guix/build-system/copy.scm b/guix/build-system/copy.scm index d58931b33c2..cf0214320bf 100644 --- a/guix/build-system/copy.scm +++ b/guix/build-system/copy.scm @@ -66,13 +66,13 @@ (define* (lower name (bag (name name) (system system) - (host-inputs `(,@(if source + (build-inputs `(,@(if source `(("source" ,source)) '()) - ,@inputs - ;; Keep the standard inputs of 'gnu-build-system'. - ,@(standard-packages))) - (build-inputs native-inputs) + ,@native-inputs + ;; Keep the standard inputs of 'gnu-build-system'. + ,@(standard-packages))) + (host-inputs inputs) (outputs outputs) (build copy-build) (arguments (strip-keyword-arguments private-keywords arguments))))