From patchwork Mon Jun 10 21:41:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 14281 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id CED7017064; Mon, 10 Jun 2019 22:42:14 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id B251717063 for ; Mon, 10 Jun 2019 22:42:13 +0100 (BST) Received: from localhost ([::1]:49908 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1haS3R-0003Zt-C6 for patchwork@mira.cbaines.net; Mon, 10 Jun 2019 17:42:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35145) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1haS3J-0003Sk-05 for guix-patches@gnu.org; Mon, 10 Jun 2019 17:42:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1haS3H-0001Kp-BD for guix-patches@gnu.org; Mon, 10 Jun 2019 17:42:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:44042) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1haS3H-0001Kh-7N for guix-patches@gnu.org; Mon, 10 Jun 2019 17:42:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1haS3H-0006K1-3h for guix-patches@gnu.org; Mon, 10 Jun 2019 17:42:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#36162] [PATCH 4/4] Add (guix remote). Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 10 Jun 2019 21:42:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36162 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 36162@debbugs.gnu.org Received: via spool by 36162-submit@debbugs.gnu.org id=B36162.156020291424266 (code B ref 36162); Mon, 10 Jun 2019 21:42:03 +0000 Received: (at 36162) by debbugs.gnu.org; 10 Jun 2019 21:41:54 +0000 Received: from localhost ([127.0.0.1]:57583 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1haS38-0006JJ-7d for submit@debbugs.gnu.org; Mon, 10 Jun 2019 17:41:54 -0400 Received: from eggs.gnu.org ([209.51.188.92]:54483) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1haS35-0006IV-EU for 36162@debbugs.gnu.org; Mon, 10 Jun 2019 17:41:51 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:60155) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1haS30-000191-8c; Mon, 10 Jun 2019 17:41:46 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=33892 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1haS2z-0001fJ-P7; Mon, 10 Jun 2019 17:41:46 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 10 Jun 2019 23:41:30 +0200 Message-Id: <20190610214130.19378-4-ludo@gnu.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190610214130.19378-1-ludo@gnu.org> References: <20190610214130.19378-1-ludo@gnu.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/remote.scm: New file. * Makefile.am (MODULES): Add it. --- Makefile.am | 1 + guix/remote.scm | 130 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 131 insertions(+) create mode 100644 guix/remote.scm diff --git a/Makefile.am b/Makefile.am index 0aa92ecfb9..42307abaed 100644 --- a/Makefile.am +++ b/Makefile.am @@ -274,6 +274,7 @@ if HAVE_GUILE_SSH MODULES += \ guix/ssh.scm \ + guix/remote.scm \ guix/scripts/copy.scm \ guix/store/ssh.scm diff --git a/guix/remote.scm b/guix/remote.scm new file mode 100644 index 0000000000..cc051dee8a --- /dev/null +++ b/guix/remote.scm @@ -0,0 +1,130 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2019 Ludovic Courtès +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix remote) + #:use-module (guix ssh) + #:use-module (guix gexp) + #:use-module (guix inferior) + #:use-module (guix store) + #:use-module (guix monads) + #:use-module (guix modules) + #:use-module (guix derivations) + #:use-module (ssh popen) + #:use-module (srfi srfi-1) + #:use-module (ice-9 match) + #:export (remote-eval)) + +;;; Commentary: +;;; +;;; Evaluate a gexp on a remote machine, over SSH, ensuring that all the +;;; elements the gexp refers to are deployed beforehand. This is useful for +;;; expressions that have side effects; for pure expressions, you would rather +;;; build a derivation remotely or offload it. +;;; +;;; Code: + +(define (remote-pipe-for-gexp lowered session) + "Return a remote pipe for the given SESSION to evaluate LOWERED." + (define shell-quote + (compose object->string object->string)) + + (apply open-remote-pipe* session OPEN_READ + (string-append (derivation->output-path + (lowered-gexp-guile lowered)) + "/bin/guile") + "--no-auto-compile" + (append (append-map (lambda (directory) + `("-L" ,directory)) + (lowered-gexp-load-path lowered)) + (append-map (lambda (directory) + `("-C" ,directory)) + (lowered-gexp-load-path lowered)) + `("-c" + ,(shell-quote (lowered-gexp-sexp lowered)))))) + +(define (%remote-eval lowered session) + "Evaluate LOWERED, a lowered gexp, in SESSION. This assumes that all the +prerequisites of EXP are already available on the host at SESSION." + (let* ((pipe (remote-pipe-for-gexp lowered session)) + (result (read-repl-response pipe))) + (close-port pipe) + result)) + +(define (trampoline exp) + "Return a \"trampoline\" gexp that evaluates EXP and writes the evaluation +result to the current output port using the (guix repl) protocol." + (define program + (scheme-file "remote-exp.scm" exp)) + + (with-imported-modules (source-module-closure '((guix repl))) + #~(begin + (use-modules (guix repl)) + (send-repl-response '(primitive-load #$program) + (current-output-port)) + (force-output)))) + +(define* (remote-eval exp session + #:key + (build-locally? #t) + (module-path %load-path) + (socket-name "/var/guix/daemon-socket/socket")) + "Evaluate EXP, a gexp, on the host at SESSION, an SSH session. Ensure that +all the elements EXP refers to are built and deployed to SESSION beforehand. +When BUILD-LOCALLY? is true, said dependencies are built locally and sent to +the remote store afterwards; otherwise, dependencies are built directly on the +remote store." + (mlet %store-monad ((lowered (lower-gexp (trampoline exp) + #:module-path %load-path)) + (remote -> (connect-to-remote-daemon session + socket-name))) + (define inputs + (cons (gexp-input (lowered-gexp-guile lowered)) + (lowered-gexp-inputs lowered))) + + (define to-build + (map (lambda (input) + (if (derivation? (gexp-input-thing input)) + (cons (gexp-input-thing input) + (gexp-input-output input)) + (gexp-input-thing input))) + inputs)) + + (if build-locally? + (let ((to-send (map (lambda (input) + (match (gexp-input-thing input) + ((? derivation? drv) + (derivation->output-path + drv (gexp-input-output input))) + ((? store-path? item) + item))) + inputs))) + (mbegin %store-monad + (built-derivations to-build) + ((store-lift send-files) to-send remote #:recursive? #t) + (return (%remote-eval lowered session)))) + (let ((to-send (map (lambda (input) + (match (gexp-input-thing input) + ((? derivation? drv) + (derivation-file-name drv)) + ((? store-path? item) + item))) + inputs))) + (mbegin %store-monad + ((store-lift send-files) to-send remote #:recursive? #t) + (return (build-derivations remote to-build)) + (return (%remote-eval lowered session)))))))