From patchwork Thu Sep 23 03:11:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jesse Gibbons X-Patchwork-Id: 33251 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id C981D27BBE3; Thu, 23 Sep 2021 04:13:10 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 082F827BBE1 for ; Thu, 23 Sep 2021 04:13:10 +0100 (BST) Received: from localhost ([::1]:42598 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mTFAb-0003qx-0S for patchwork@mira.cbaines.net; Wed, 22 Sep 2021 23:13:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44406) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mTFAV-0003qn-1Y for guix-patches@gnu.org; Wed, 22 Sep 2021 23:13:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:40778) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mTFAU-0007Vr-Qm for guix-patches@gnu.org; Wed, 22 Sep 2021 23:13:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mTFAU-0001s4-G2 for guix-patches@gnu.org; Wed, 22 Sep 2021 23:13:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50708] [PATCH] gnu: web: Add jupyter-service Resent-From: Jesse Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 23 Sep 2021 03:13:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50708 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Maxim Cournoyer Cc: 50708@debbugs.gnu.org Received: via spool by 50708-submit@debbugs.gnu.org id=B50708.16323667277111 (code B ref 50708); Thu, 23 Sep 2021 03:13:02 +0000 Received: (at 50708) by debbugs.gnu.org; 23 Sep 2021 03:12:07 +0000 Received: from localhost ([127.0.0.1]:52324 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mTF9a-0001qc-Fk for submit@debbugs.gnu.org; Wed, 22 Sep 2021 23:12:07 -0400 Received: from mail-pj1-f42.google.com ([209.85.216.42]:51188) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mTF9W-0001q0-DY for 50708@debbugs.gnu.org; Wed, 22 Sep 2021 23:12:05 -0400 Received: by mail-pj1-f42.google.com with SMTP id k23so3506984pji.0 for <50708@debbugs.gnu.org>; Wed, 22 Sep 2021 20:12:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=laUvx3uNfs8iUflqLLmh0MDTx9yClTFc+GZLEWYkxpA=; b=o6CKlRoIX4U6cb3nPu1cQcd0HVy5KqSJvncM5KPie9fNeUKkV1wOq+A45RJxLO49B5 /Nz6+gLvbYwLBQ5V6EmmLXcqEtw/Q66bTbIJxclF9YAwjHQJH8luoCyeAYuHFIHp5rBo LXXII+oTBoR0zbIoYSIJ0/EZVxBk5ygf83l/Z/YxKOeZrQ/GkdAtQTzpiM8Rzt0yG+Ex 8ojKRB1kT3NvYOu7ZWfTMTlVNmJvo+x3PJfA49eLEDMH9BV/Pr2tfC0X8sKB0mSqlYgj MR/msyAdh0Pnrh0wuApY64WZqByjoSrdYmj5uw5cu3GA5Uc0iB1b/PGqAYi+U2S9v+j9 pclw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=laUvx3uNfs8iUflqLLmh0MDTx9yClTFc+GZLEWYkxpA=; b=cNvaumxJz1K5cQmyMhEJXN+5IydGzHjX3HepctaKBsacrpjp958zJYSfidg5MeEN7w Db9njlAfg5rd1LWQxwQH3a/ZESc7YQf/2KZqmvaYfg2+I+lxX9hof6vnbA/wiiPohKrf ez8SoVBpn9ZCrbBkqC1sB9tVBDSL2u092n8NHqxasrxXFdzO56B5tp4KlzkXiMyBNCjH qi7yXzoIj+dI3VHz4dJBtz6H/34r7fo6ytEKqTRtt3sDmfdsVxqj1myydla/O677iEyP vLKVcPbxpRt5Zz3MaAMcToTSLzt/VIgJ+wtUpF23alQits53XzxsySSkgtfdQlV8sWYl r+og== X-Gm-Message-State: AOAM533chwhNZ54JsXmxpXM+f/AQVm6ZpRIh9uVaQEcwaqYDK9LmYtdQ D3WAKfF6BNtPuhBBpPrLSH9X/mODaSk1/w== X-Google-Smtp-Source: ABdhPJwsBVWw1ojTOhz2lYWg7t5iBHByXnpz+Yqb9UiKgjnK/wM/sSM5SOW4RWHLRDHvqjMRH9oiYw== X-Received: by 2002:a17:902:7e84:b0:13a:3396:1c04 with SMTP id z4-20020a1709027e8400b0013a33961c04mr1926615pla.84.1632366716150; Wed, 22 Sep 2021 20:11:56 -0700 (PDT) Received: from [192.168.1.220] ([38.141.58.134]) by smtp.gmail.com with ESMTPSA id g3sm3551784pjm.22.2021.09.22.20.11.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 22 Sep 2021 20:11:55 -0700 (PDT) References: <565a1b21-aefa-4129-a024-52517fdff9bd@gmail.com> <74f78a5c-3f79-7485-0cf2-ae1fbb1969dd@gmail.com> <87pmt0kp83.fsf_-_@gmail.com> From: Jesse Message-ID: <23294845-1ac9-b2ef-c056-89f6ee51435f@gmail.com> Date: Wed, 22 Sep 2021 21:11:52 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Icedove/78.13.0 MIME-Version: 1.0 In-Reply-To: <87pmt0kp83.fsf_-_@gmail.com> Content-Language: en-US X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Attached is a patch that takes into account your feedback. On 9/22/21 8:19 PM, Maxim Cournoyer wrote: > Hello Jesse, > > Jesse writes: ... > + #:environment-variables ; TODO use search-paths magic instead of hard-coding these things. > + (list "GI_TYPELIB_PATH=/run/current-system/profile/lib/girepository-1.0" > + "GUILE_LOAD_COMPILED_PATH=/run/current-system/profile/lib/guile/3.0/site-ccache:/run/current-system/profile/share/guile/site/3.0" > + "GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/3.0" > + "HOME=/var/lib/jupyter" > + "JUPYTER_PATH=/run/current-system/profile/share/jupyter" > + "PATH=/run/current-system/profile/bin:/run/current-system/profile/sbin" > + "PYTHONPATH=/run/current-system/profile/lib/python3.8/site-packages" > + "R_LIBS_SITE=/run/current-system/profile/site-library/" > + "TEXMF=/run/current-system/profile/share/texmf-dist" > + "TEXMFCNF=/run/current-system/profile//share/texmf-dist/web2c" > >> + "XDG_DATA_DIRS=/run/current-system/profile/share") >> + > Hmm. Would it work if search paths were set on the 'jupyter' package > instead? The above is too fragile, with hard coded versions baked in. > > ... > I haven't tried it, but it looks promising. If we can find a better > solution than the hard coded version values in the environment variables > list, I think it'd be in good shape for inclusion, although a system > test would be a nice to have. > > Thank you! > > Maxim The weird thing about this particular service is it is more useful with libraries Jupyter and the kernels are not dependent on. This part is leftover from when I wrote it several months ago for a home server. (I actually used this service to work on notebooks using the Guile kernel working with sqlite and to try the other kernels.) I found that it couldn't find the packages I need and the web shell was broken, so I settled for hard-coding these variables until I could find time to fix them. I included a TODO with an idea I have not yet implemented: use the search-paths to give Jupyter the environment variables it needs the same way a profile sets up its search paths. I thought it might be a good idea to associate the service with its own profile for security reasons. This would also fix a bug where the Jupyter Notebook icon appears on the desktop but fails to start a new Jupyter server. Another idea I had is the service could be isolated in its own container with the network exposed and the home directory shared, but I have no idea how that would work or if it would even add more security. However, this could potentially allow multiple Jupyter services to run simultaneously on different ports. However, one problem is the resulting server might be unusable because the login information would be in a log file hidden in a container. I'll work on the search-paths idea and send an update when I'm done. I'm also open to other suggestions on how to fix the hard-coded values. I don't think this will be ready to merge until that problem is fixed, but I've attached the patch anyway for curious users who want to try it out. I would appreciate help implementing the container idea if it's feasible. -Jesse From f96e41a7222b51cb4ff6e16657de2e87fea78472 Mon Sep 17 00:00:00 2001 From: Jesse Date: Mon, 20 Sep 2021 16:01:22 -0600 Subject: [PATCH] gnu: web: Add jupyter-service * gnu/services/web.scm (gitile-service-type): New variable. * doc/guix.texi (Version Control Services): Document it. --- doc/guix.texi | 52 +++++++++++++++++++++ gnu/services/web.scm | 105 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 157 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index 6436e83a7c..b92246cc8a 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -26197,6 +26197,58 @@ The file which should store the logging output of Agate. @end table @end deftp +@subsubheading Jupyter +@cindex jupyter +The Jupyter Notebook is a web application that allows you to create +and share documents that contain live code, equations, visualizations +and explanatory text. Uses include: data cleaning and transformation, +numerical simulation, statistical modeling, machine learning and much +more. + +@deffn (Scheme Variable) jupyter-service-type +This is the type of the Jupyter service, whose value should be an +@code{jupyter-service-type} object, as in this example: + +@lisp +(service jupyter-service-type + (jupyter-configuration + (log-file "/var/log/jupyter.log") + (server-config + (kernels (list jupyter-guile-kernel))))) + +@end lisp + +The example above tells the Jupyter service to provide the +@code{jupyter-guile-kernel} as an optional kernel. +@end deffn + +@deftp {Data Type} jupyter-configuration +Data type representing the configuration of Agate. + +@table @asis +@item @code{jupyter-package} (default: @code{jupyter}) +The Jupyter package to use. + + +@item @code{kernels} (default: @code{'()} +A list of kernel packages to use with the Jupyter service. + +@item @code{root-dir} (default: @file{"/var/lib/jupyter/notebooks"}) +The directory where the Jupyter server looks for Jupyter notebooks. + +@item @code{log-file} (default: @code{#f}) +The location of the log file. If #f is given, no log is produced. + +@item @code{shell} (default: @code{(file-append bash "/bin/bash")} +The location of the shell used when starting a web terminal (if it is configured to allow terminals). + +@item @code{server-config} (default: @code{#f}) +A file containing the Jupyter server's config file. If @code{#f} is given, an empty file is produced, and the default settings are used. + +@end table +@end deftp + + @node Certificate Services @subsection Certificate Services diff --git a/gnu/services/web.scm b/gnu/services/web.scm index 6a093cf4e4..76700da412 100644 --- a/gnu/services/web.scm +++ b/gnu/services/web.scm @@ -15,6 +15,7 @@ ;;; Copyright © 2020 Arun Isaac ;;; Copyright © 2020 Oleg Pykhalov ;;; Copyright © 2020, 2021 Alexandru-Sergiu Marton +;;; Copyright © 2020, 2021 Jesse Gibbons ;;; ;;; This file is part of GNU Guix. ;;; @@ -41,11 +42,13 @@ #:use-module (gnu system shadow) #:use-module (gnu packages admin) #:use-module (gnu packages base) + #:use-module (gnu packages bash) #:use-module (gnu packages databases) #:use-module (gnu packages web) #:use-module (gnu packages patchutils) #:use-module (gnu packages php) #:use-module (gnu packages python) + #:use-module (gnu packages python-xyz) #:use-module (gnu packages gnupg) #:use-module (gnu packages guile) #:use-module (gnu packages logging) @@ -91,6 +94,8 @@ httpd-service-type + jupyter-configuration + nginx-configuration nginx-configuration? nginx-configuration-nginx @@ -1994,3 +1999,103 @@ root=/srv/gemini (service-extension shepherd-root-service-type agate-shepherd-service))) (default-value (agate-configuration)))) + + +;;; +;;; Jupyter +;;; + +(define-public default-jupyter-config "#") + +(define-record-type* jupyter-configuration + this-jupyter-configuration jupyter-configuration? + (jupyter-package jupyter-configuration-jupyter-package + (default jupyter)) + (kernels jupyter-configuration-kernels + (default '())) + (root-dir jupyter-configuration-root-dir + (default "/var/lib/jupyter/notebooks")) + (log-file jupyter-configuration-log-file + (default #f)) + (shell jupyter-configuration-shell + (default (file-append bash "/bin/bash"))) + (server-config jupyter-configuration-server-config + (default #f))); TODO: Make configuration DSL. + + +(define (search-path-string search-path-pair) + (string-append (search-path-specification-variable (car search-path-pair)) + "=" + (cdr search-path-pair))) + +;;;TODO: Add actions to list jupyter servers, change passwords, etc. +(define (jupyter-shepherd-service config) + (list (shepherd-service + (provision '(jupyter)) ;TODO: Add magic to allow multiple Jupyter servers + (requirement '(loopback)) + (start #~(make-forkexec-constructor + (list "/run/current-system/profile/bin/jupyter" + "notebook" + (string-append "--config=" + #$(or (jupyter-configuration-server-config config) + (plain-file "jupyter_notebook_config.py" + default-jupyter-config)) + #$(jupyter-configuration-root-dir config))) + #:user "jupyter" + #:group "jupyter" + #:environment-variables ; TODO use search-paths magic instead of hard-coding these things. + (list "GI_TYPELIB_PATH=/run/current-system/profile/lib/girepository-1.0" + "GUILE_LOAD_COMPILED_PATH=/run/current-system/profile/lib/guile/3.0/site-ccache:/run/current-system/profile/share/guile/site/3.0" + "GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/3.0" + "HOME=/var/lib/jupyter" + "JUPYTER_PATH=/run/current-system/profile/share/jupyter" + "PATH=/run/current-system/profile/bin:/run/current-system/profile/sbin" + "PYTHONPATH=/run/current-system/profile/lib/python3.8/site-packages" + "R_LIBS_SITE=/run/current-system/profile/site-library/" + "TEXMF=/run/current-system/profile/share/texmf-dist" + "TEXMFCNF=/run/current-system/profile/share/texmf-dist/web2c" + "XDG_DATA_DIRS=/run/current-system/profile/share") + + #:directory #$(jupyter-configuration-root-dir config) + #:log-file #$(jupyter-configuration-log-file config))) + (stop #~(make-kill-destructor)) + (documentation "Runs a Jupyter Notebook server. A Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations, and explanatory text.")))) + +(define (jupyter-account config) + (list + (user-account + (name "jupyter") + (group "jupyter") + (comment "Jupyter Notebook Server") + (home-directory "/var/lib/jupyter") + (shell (jupyter-configuration-shell config)) + (system? #t)) + (user-group + (name "jupyter") + (system? #t)))) + +(define (jupyter-profile config) + (cons* + (jupyter-configuration-jupyter-package config) + (jupyter-configuration-kernels config))) + +(define (jupyter-activation config) + #~(begin + (let ((root-dir #$(jupyter-configuration-root-dir config)) + (pw (getpwnam "jupyter"))) + (unless (file-exists? root-dir) + (mkdir root-dir) + (chown root-dir (passwd:uid pw) + (passwd:gid pw)) + (chmod root-dir #o700))))) + +(define-public jupyter-service-type + (service-type + (name "jupyter") + (extensions (list + (service-extension shepherd-root-service-type jupyter-shepherd-service) + (service-extension account-service-type jupyter-account) + (service-extension activation-service-type jupyter-activation) + (service-extension profile-service-type jupyter-profile))) + (description "Runs a Jupyter Notebook server. A Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations, and explanatory text.") + (default-value (jupyter-configuration))))