From patchwork Fri Jan 18 09:53:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 762 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id B624916A88; Fri, 18 Jan 2019 09:54:10 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 5CDAF16A7D for ; Fri, 18 Jan 2019 09:54:10 +0000 (GMT) Received: from localhost ([127.0.0.1]:35176 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gkQqo-0001OP-0V for patchwork@mira.cbaines.net; Fri, 18 Jan 2019 04:54:10 -0500 Received: from eggs.gnu.org ([209.51.188.92]:41492) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gkQqi-0001MF-5E for guix-patches@gnu.org; Fri, 18 Jan 2019 04:54:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gkQqg-0001F7-DN for guix-patches@gnu.org; Fri, 18 Jan 2019 04:54:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:36418) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gkQqg-0001Ep-9k for guix-patches@gnu.org; Fri, 18 Jan 2019 04:54:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gkQqg-0001eb-7v for guix-patches@gnu.org; Fri, 18 Jan 2019 04:54:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#34122] [PATCH 2/3] inferior: 'gexp->derivation-in-inferior' honors EXP's load path. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 18 Jan 2019 09:54:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 34122 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 34122@debbugs.gnu.org Received: via spool by 34122-submit@debbugs.gnu.org id=B34122.15478052346330 (code B ref 34122); Fri, 18 Jan 2019 09:54:02 +0000 Received: (at 34122) by debbugs.gnu.org; 18 Jan 2019 09:53:54 +0000 Received: from localhost ([127.0.0.1]:35696 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gkQqX-0001dw-LO for submit@debbugs.gnu.org; Fri, 18 Jan 2019 04:53:53 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:37962) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gkQqV-0001da-KP for 34122@debbugs.gnu.org; Fri, 18 Jan 2019 04:53:52 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id EF12FC39; Fri, 18 Jan 2019 10:53:50 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Iv0ppbL0vFcW; Fri, 18 Jan 2019 10:53:50 +0100 (CET) Received: from gnu.org (unknown [IPv6:2001:660:6102:320:e120:2c8f:8909:cdfe]) by hera.aquilenet.fr (Postfix) with ESMTPSA id BD40C2341; Fri, 18 Jan 2019 10:53:49 +0100 (CET) From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Fri, 18 Jan 2019 10:53:43 +0100 Message-Id: <20190118095344.12927-2-ludo@gnu.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190118095344.12927-1-ludo@gnu.org> References: <20190118095344.12927-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: rekado@elephly.net Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Previously the imported modules and extensions of EXP would be missing from the load path of 'guix repl'. * guix/inferior.scm (gexp->derivation-in-inferior)[script]: New variable. [trampoline]: Write (primitive-load #$script) to PIPE. Add #$output. * tests/channels.scm ("channel-instances->manifest")[depends?]: Check for requisites rather than direct references. Adjust callers accordingly. --- guix/inferior.scm | 13 ++++++++++--- tests/channels.scm | 16 ++++++++++------ 2 files changed, 20 insertions(+), 9 deletions(-) diff --git a/guix/inferior.scm b/guix/inferior.scm index 4dfb242e44..9f19e7d316 100644 --- a/guix/inferior.scm +++ b/guix/inferior.scm @@ -491,6 +491,10 @@ PACKAGE must be live." "Return a derivation that evaluates EXP with GUIX, an instance of Guix as returned for example by 'channel-instances->derivation'. Other arguments are passed as-is to 'gexp->derivation'." + (define script + ;; EXP wrapped with a proper (set! %load-path …) prologue. + (scheme-file "inferior-script.scm" exp)) + (define trampoline ;; This is a crude way to run EXP on GUIX. TODO: use 'raw-derivation' and ;; make 'guix repl' the "builder"; this will require "opening up" the @@ -501,9 +505,12 @@ passed as-is to 'gexp->derivation'." (let ((pipe (open-pipe* OPEN_WRITE #+(file-append guix "/bin/guix") "repl" "-t" "machine"))) - ;; Unquote EXP right here so that its references to #$output - ;; propagate to the surrounding gexp. - (write '#$exp pipe) ;XXX: load path for EXP? + + ;; XXX: EXP presumably refers to #$output but that reference is lost + ;; so explicitly reference it here. + #$output + + (write `(primitive-load #$script) pipe) (unless (zero? (close-pipe pipe)) (error "inferior failed" #+guix))))) diff --git a/tests/channels.scm b/tests/channels.scm index 7df1b8c5fe..8540aef435 100644 --- a/tests/channels.scm +++ b/tests/channels.scm @@ -24,6 +24,7 @@ #:use-module (guix store) #:use-module ((guix grafts) #:select (%graft?)) #:use-module (guix derivations) + #:use-module (guix sets) #:use-module (guix gexp) #:use-module (srfi srfi-1) #:use-module (srfi srfi-26) @@ -187,12 +188,15 @@ (manifest-entries manifest)) (define (depends? drv in out) - ;; Return true if DRV depends on all of IN and none of OUT. - (let ((lst (map derivation-input-path (derivation-inputs drv))) + ;; Return true if DRV depends (directly or indirectly) on all of IN + ;; and none of OUT. + (let ((set (list->set + (requisites store + (list (derivation-file-name drv))))) (in (map derivation-file-name in)) (out (map derivation-file-name out))) - (and (every (cut member <> lst) in) - (not (any (cut member <> lst) out))))) + (and (every (cut set-contains? set <>) in) + (not (any (cut set-contains? set <>) out))))) (define (lookup name) (run-with-store store @@ -212,8 +216,8 @@ (depends? drv1 (list drv0) (list drv2 drv3)) (depends? drv2 - (list drv1) (list drv0 drv3)) + (list drv1) (list drv3)) (depends? drv3 - (list drv2 drv0) (list drv1)))))))) + (list drv2 drv0) (list)))))))) (test-end "channels") From patchwork Fri Jan 18 09:53:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 763 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 8018816A88; Fri, 18 Jan 2019 09:54:16 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id F2F8C16A7D for ; Fri, 18 Jan 2019 09:54:15 +0000 (GMT) Received: from localhost ([127.0.0.1]:35180 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gkQqt-0001Tg-JM for patchwork@mira.cbaines.net; Fri, 18 Jan 2019 04:54:15 -0500 Received: from eggs.gnu.org ([209.51.188.92]:41519) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gkQqm-0001RE-An for guix-patches@gnu.org; Fri, 18 Jan 2019 04:54:09 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gkQqi-0001HS-5P for guix-patches@gnu.org; Fri, 18 Jan 2019 04:54:07 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:36419) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gkQqg-0001FX-R5 for guix-patches@gnu.org; Fri, 18 Jan 2019 04:54:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gkQqg-0001ej-O0 for guix-patches@gnu.org; Fri, 18 Jan 2019 04:54:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#34122] [PATCH 3/3] channels: Build channel modules in an inferior. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 18 Jan 2019 09:54:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 34122 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 34122@debbugs.gnu.org Received: via spool by 34122-submit@debbugs.gnu.org id=B34122.15478052376340 (code B ref 34122); Fri, 18 Jan 2019 09:54:02 +0000 Received: (at 34122) by debbugs.gnu.org; 18 Jan 2019 09:53:57 +0000 Received: from localhost ([127.0.0.1]:35698 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gkQqb-0001eC-0u for submit@debbugs.gnu.org; Fri, 18 Jan 2019 04:53:57 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:37972) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gkQqW-0001di-ED for 34122@debbugs.gnu.org; Fri, 18 Jan 2019 04:53:52 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id BC61F2346; Fri, 18 Jan 2019 10:53:51 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KWlnYoyMc5Jd; Fri, 18 Jan 2019 10:53:50 +0100 (CET) Received: from gnu.org (unknown [IPv6:2001:660:6102:320:e120:2c8f:8909:cdfe]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 1A96B2347; Fri, 18 Jan 2019 10:53:50 +0100 (CET) From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Fri, 18 Jan 2019 10:53:44 +0100 Message-Id: <20190118095344.12927-3-ludo@gnu.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190118095344.12927-1-ludo@gnu.org> References: <20190118095344.12927-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: rekado@elephly.net Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches This ensures that channel modules are compiled with the right Guile, that they get to see the right modules, and so on. IOW, it avoids bugs such as those addressed by commits 3c0e16391ed9a3e3e4611b940fb393c5f2ecea63 and cb341c121919877ae6267a6460c0c17536d06eff. * guix/channels.scm (standard-module-derivation): Add 'core' parameter. Rewrite in terms of 'gexp->derivation-in-inferior'. (build-from-source): Add #:core parameter and pass it to 'standard-module-derivation'. (build-channel-instance): Add 'core' parameter and pass it on. (channel-instance-derivations)[dependencies]: Remove. Adjust 'build-channel-instance' call. --- guix/channels.scm | 83 +++++++++++++++++++++-------------------------- 1 file changed, 37 insertions(+), 46 deletions(-) diff --git a/guix/channels.scm b/guix/channels.scm index b9ce2aa024..eb56c821e5 100644 --- a/guix/channels.scm +++ b/guix/channels.scm @@ -218,45 +218,48 @@ of COMMIT at URL. Use NAME as the channel name." ;; place a set of compiled Guile modules in ~/.config/guix/latest. 1) -(define (standard-module-derivation name source dependencies) - "Return a derivation that builds the Scheme modules in SOURCE and that -depend on DEPENDENCIES, a list of lowerable objects. The assumption is that -SOURCE contains package modules to be added to '%package-module-path'." - (define modules - (scheme-modules* source)) - +(define (standard-module-derivation name source core dependencies) + "Return a derivation that builds with CORE, a Guix instance, the Scheme +modules in SOURCE and that depend on DEPENDENCIES, a list of lowerable +objects. The assumption is that SOURCE contains package modules to be added +to '%package-module-path'." ;; FIXME: We should load, say SOURCE/.guix-channel.scm, which would allow ;; channel publishers to specify things such as the sub-directory where .scm ;; files live, files to exclude from the channel, preferred substitute URLs, ;; etc. - (mlet* %store-monad ((compiled - (compiled-modules modules - #:name name - #:module-path (list source) - #:extensions dependencies))) - (gexp->derivation name - (with-extensions dependencies - (with-imported-modules '((guix build utils)) - #~(begin - (use-modules (guix build utils)) + (define build + ;; This is code that we'll run in CORE, a Guix instance, with its own + ;; modules and so on. That way, we make sure these modules are built for + ;; the right Guile version, with the right dependencies, and that they get + ;; to see the right (gnu packages …) modules. + (with-extensions dependencies + #~(begin + (use-modules (guix build compile) + (guix build utils) + (srfi srfi-26)) - (let ((go (string-append #$output "/lib/guile/" - (effective-version) - "/site-ccache")) - (scm (string-append #$output - "/share/guile/site/" - (effective-version)))) - (mkdir-p (dirname go)) - (symlink #$compiled go) - (mkdir-p (dirname scm)) - (symlink #$source scm)))))))) + (define go + (string-append #$output "/lib/guile/" (effective-version) + "/site-ccache")) + (define scm + (string-append #$output "/share/guile/site/" + (effective-version))) + + (compile-files #$source go + (find-files #$source "\\.scm$")) + (mkdir-p (dirname scm)) + (symlink #$source scm) + scm))) + + (gexp->derivation-in-inferior name build core)) (define* (build-from-source name source - #:key verbose? commit + #:key core verbose? commit (dependencies '())) "Return a derivation to build Guix from SOURCE, using the self-build script -contained therein. Use COMMIT as the version string." +contained therein; use COMMIT as the version string. When CORE is true, build +package modules under SOURCE using CORE, an instance of Guix." ;; Running the self-build script makes it easier to update the build ;; procedure: the self-build script of the Guix-to-be-installed contains the ;; right dependencies, build procedure, etc., which the Guix-in-use may not @@ -278,9 +281,10 @@ contained therein. Use COMMIT as the version string." #:pull-version %pull-version)) ;; Build a set of modules that extend Guix using the standard method. - (standard-module-derivation name source dependencies))) + (standard-module-derivation name source core dependencies))) -(define* (build-channel-instance instance #:optional (dependencies '())) +(define* (build-channel-instance instance + #:optional core (dependencies '())) "Return, as a monadic value, the derivation for INSTANCE, a channel instance. DEPENDENCIES is a list of extensions providing Guile modules that INSTANCE depends on." @@ -288,6 +292,7 @@ INSTANCE depends on." (channel-name (channel-instance-channel instance))) (channel-instance-checkout instance) #:commit (channel-instance-commit instance) + #:core core #:dependencies dependencies)) (define (resolve-dependencies instances) @@ -328,17 +333,6 @@ INSTANCES." (guix-channel? (channel-instance-channel instance))) instances)) - (define dependencies - ;; Dependencies of CORE-INSTANCE. - ;; FIXME: It would be best not to hard-wire this information here and - ;; instead query it to CORE-INSTANCE. - (list (module-ref (resolve-interface '(gnu packages gnupg)) - 'guile-gcrypt) - (module-ref (resolve-interface '(gnu packages guile)) - 'guile-git) - (module-ref (resolve-interface '(gnu packages guile)) - 'guile-bytestructures))) - (define edges (resolve-dependencies instances)) @@ -348,10 +342,7 @@ INSTANCES." (mlet %store-monad ((core (instance->derivation core-instance)) (deps (mapm %store-monad instance->derivation (edges instance)))) - (build-channel-instance instance - (cons core - (append deps - dependencies))))) + (build-channel-instance instance core deps))) instance)) (mapm %store-monad instance->derivation instances))