From patchwork Sat Sep 16 20:10:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Saku Laesvuori X-Patchwork-Id: 53966 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 89B8627BBEA; Sat, 16 Sep 2023 21:11:09 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 8055627BBE2 for ; Sat, 16 Sep 2023 21:11:05 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhbd3-00055v-23; Sat, 16 Sep 2023 16:10:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhbd0-00055Z-Ub for guix-patches@gnu.org; Sat, 16 Sep 2023 16:10:55 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhbd0-000090-ML for guix-patches@gnu.org; Sat, 16 Sep 2023 16:10:54 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qhbd7-0000t8-R4 for guix-patches@gnu.org; Sat, 16 Sep 2023 16:11:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#61740] [PATCH] services: Add rspamd-service-type. Resent-From: Saku Laesvuori Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 16 Sep 2023 20:11:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 61740 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: moreinfo patch To: Felix Lechner Cc: Thomas Ieong , Bruno Victal , Ludovic =?utf-8?q?Court=C3=A8s?= , 61740@debbugs.gnu.org Received: via spool by 61740-submit@debbugs.gnu.org id=B61740.16948950373379 (code B ref 61740); Sat, 16 Sep 2023 20:11:01 +0000 Received: (at 61740) by debbugs.gnu.org; 16 Sep 2023 20:10:37 +0000 Received: from localhost ([127.0.0.1]:48588 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhbci-0000sQ-4s for submit@debbugs.gnu.org; Sat, 16 Sep 2023 16:10:37 -0400 Received: from vmi571514.contaboserver.net ([75.119.130.101]:40888 helo=mail.laesvuori.fi) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhbce-0000sD-GI for 61740@debbugs.gnu.org; Sat, 16 Sep 2023 16:10:35 -0400 Received: from X-kone (88-113-24-127.elisa-laajakaista.fi [88.113.24.127]) by mail.laesvuori.fi (Postfix) with ESMTPSA id 03B023400C6; Sat, 16 Sep 2023 22:10:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=laesvuori.fi; s=mail; t=1694895055; bh=ai7qN6wZqCQhLnHdwhbFjXUqGUciXO3VUaawG0cZzdI=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=ej/kyQMEjt+ScxkvoYBQAr7UxFEBjCn/wLj9V+XYkV57rcQ7iizF1VztD/5rLazBe rjxoMJczpVN9aYB2ff2wsp86DuiOL6wv8D4SlYtb9RzYqqOetJkqfJ8BGoZLA2Yu4R xni21ffcdgPVi4JDbLHiDD3syo5kBtG66NPyTVns= Date: Sat, 16 Sep 2023 23:10:20 +0300 Message-ID: References: <87sf7fqi3x.fsf@lease-up.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <87sf7fqi3x.fsf@lease-up.com> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Saku Laesvuori X-ACL-Warn: , Saku Laesvuori via Guix-patches X-Patchwork-Original-From: Saku Laesvuori via Guix-patches via From: Saku Laesvuori Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches On Fri, Sep 15, 2023 at 01:47:14PM -0700, Felix Lechner wrote: > Hi Saku, > > > I happened to need rspamd myself > > So do I but it does not seem to start locally. It created some folders > and, per the log, the configuration file passed the syntax check, but > then the boot stalls. > > I used (service rspamd-service-type) and nothing else in my system > configuration. Should it be sufficient? Thanks! Oops, the version I tested (and am running right now) used activation-service-type and I forgot to test that it was still working after moving the code from there to the shepherd service. A fixed v3 is attached. From 886c32091bdc032c4180d490125a131df79b705c Mon Sep 17 00:00:00 2001 Message-ID: <886c32091bdc032c4180d490125a131df79b705c.1694894798.git.saku@laesvuori.fi> From: Thomas Ieong Date: Thu, 23 Feb 2023 21:16:14 +0100 Subject: [PATCH v3] services: Add rspamd-service-type. * gnu/services/mail.scm (rspamd-service-type): New variable. * gnu/tests/mail.scm (%test-rspamd): New variable. * doc/guix.texi: Document it. Co-authored-by: Saku Laesvuori --- This version imports (gnu build activation) in the shepherd start action and removes the syntax check, because I expect the service to fail anyway if the configuration file syntax is invalid and the check seemed to prevent booting (no idea why). doc/guix.texi | 61 +++++++++++++ gnu/services/mail.scm | 197 +++++++++++++++++++++++++++++++++++++++++- gnu/tests/mail.scm | 87 ++++++++++++++++++- 3 files changed, 343 insertions(+), 2 deletions(-) base-commit: 2d4d147839b81ba8761c9e50cabe9b60025dc670 diff --git a/doc/guix.texi b/doc/guix.texi index f82bb99069..04e4a60f97 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -119,6 +119,8 @@ Copyright @copyright{} 2023 Zheng Junjie@* Copyright @copyright{} 2023 Brian Cully@* Copyright @copyright{} 2023 Felix Lechner@* +Copyright @copyright{} 2023 Thomas Ieong@* +Copyright @copyright{} 2023 Saku Laesvuori@* Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or @@ -27393,6 +27395,65 @@ Mail Services @end table @end deftp +@subsubheading Rspamd Service +@cindex email +@cindex spam + +@defvar rspamd-service-type +This is the type of the @uref{https://rspamd.com/, Rspamd} filtering +system whose value should be a @code{rspamd-configuration}. +@end defvar + +@c %start of fragment + +@deftp {Data Type} rspamd-configuration +Available @code{rspamd-configuration} fields are: + +@table @asis +@item @code{package} (default: @code{rspamd}) (type: file-like) +The package that provides rspamd. + +@item @code{config-file} (type: file-like) +File-like object of the configuration file to use. By default all +workers are enabled except fuzzy and they are binded to their usual +ports, e.g localhost:11334, localhost:11333 and so on + +@item @code{local.d-files} (default: @code{()}) (type: directory-tree) +Configuration files in local.d, provided as a list of two element lists +where the first element is the filename and the second one is a +file-like object. Settings in these files will be merged with the +defaults. + +@item @code{override.d-files} (default: @code{()}) (type: directory-tree) +Configuration files in override.d, provided as a list of two element +lists where the first element is the filename and the second one is a +file-like object. Settings in these files will override the defaults. + +@item @code{user} (default: @code{"rspamd"}) (type: string) +The user to run rspamd as. + +@item @code{group} (default: @code{"rspamd"}) (type: string) +The group to run rspamd as. + +@item @code{pid-file} (default: @code{"/var/run/rspamd/rspamd.pid"}) (type: string) +Where to store the PID file. + +@item @code{debug?} (default: @code{#f}) (type: boolean) +Force debug output. + +@item @code{insecure?} (default: @code{#f}) (type: boolean) +Ignore running workers as privileged users (insecure). + +@item @code{skip-template?} (default: @code{#f}) (type: boolean) +Do not apply Jinja templates. + +@end table + +@end deftp + + +@c %end of fragment + @node Messaging Services @subsection Messaging Services diff --git a/gnu/services/mail.scm b/gnu/services/mail.scm index 12dcc8e71d..d8720907c8 100644 --- a/gnu/services/mail.scm +++ b/gnu/services/mail.scm @@ -5,6 +5,8 @@ ;;; Copyright © 2017, 2020 Tobias Geerinckx-Rice ;;; Copyright © 2019 Kristofer Buffington ;;; Copyright © 2020 Jonathan Brielmaier +;;; Copyright © 2023 Thomas Ieong +;;; Copyright © 2023 Saku Laesvuori ;;; ;;; This file is part of GNU Guix. ;;; @@ -80,7 +82,11 @@ (define-module (gnu services mail) radicale-configuration radicale-configuration? radicale-service-type - %default-radicale-config-file)) + %default-radicale-config-file + + rspamd-configuration + rspamd-service-type + %default-rspamd-config-file)) ;;; Commentary: ;;; @@ -1987,3 +1993,192 @@ (define radicale-service-type (service-extension account-service-type (const %radicale-accounts)) (service-extension activation-service-type radicale-activation))) (default-value (radicale-configuration)))) + +;;; +;;; Rspamd. +;;; + +(define (directory-tree? xs) + (match xs + (((file-name file-like) ...) + (and (every string? file-name) + (every file-like? file-like))) + (_ #f))) + +(define-configuration/no-serialization rspamd-configuration + (package + (file-like rspamd) + "The package that provides rspamd.") + (config-file + (file-like %default-rspamd-config-file) + "File-like object of the configuration file to use. By default +all workers are enabled except fuzzy and they are binded +to their usual ports, e.g localhost:11334, localhost:11333 and so on") + (local.d-files + (directory-tree '()) + "Configuration files in local.d, provided as a list of two element lists where +the first element is the filename and the second one is a file-like object. Settings +in these files will be merged with the defaults.") + (override.d-files + (directory-tree '()) + "Configuration files in override.d, provided as a list of two element lists where +the first element is the filename and the second one is a file-like object. Settings +in these files will override the defaults.") + (user + (string "rspamd") + "The user to run rspamd as.") + (group + (string "rspamd") + "The group to run rspamd as.") + (pid-file + (string "/var/run/rspamd/rspamd.pid") + "Where to store the PID file.") + (debug? + (boolean #f) + "Force debug output.") + (insecure? + (boolean #f) + "Ignore running workers as privileged users (insecure).") + (skip-template? + (boolean #f) + "Do not apply Jinja templates.")) + +(define %default-rspamd-config-file + (plain-file "rspamd.conf" " +.include \"$CONFDIR/common.conf\" + +options { + pidfile = \"$RUNDIR/rspamd.pid\"; + .include \"$CONFDIR/options.inc\" + .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/options.inc\" + .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/options.inc\" +} + +logging { + type = \"file\"; + filename = \"$LOGDIR/rspamd.log\"; + .include \"$CONFDIR/logging.inc\" + .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/logging.inc\" + .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/logging.inc\" +} + +worker \"normal\" { + bind_socket = \"localhost:11333\"; + .include \"$CONFDIR/worker-normal.inc\" + .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-normal.inc\" + .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-normal.inc\" +} + +worker \"controller\" { + bind_socket = \"localhost:11334\"; + .include \"$CONFDIR/worker-controller.inc\" + .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-controller.inc\" + .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-controller.inc\" +} + +worker \"rspamd_proxy\" { + bind_socket = \"localhost:11332\"; + .include \"$CONFDIR/worker-proxy.inc\" + .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-proxy.inc\" + .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-proxy.inc\" +} + +# Local fuzzy storage is disabled by default + +worker \"fuzzy\" { + bind_socket = \"localhost:11335\"; + count = -1; # Disable by default + .include \"$CONFDIR/worker-fuzzy.inc\" + .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-fuzzy.inc\" + .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-fuzzy.inc\" +} +")) + +(define (rspamd-accounts config) + (match-record config + (user group) + (list + (user-group + (name group) + (system? #t)) + (user-account + (name user) + (group group) + (system? #t) + (comment "Rspamd daemon") + (home-directory "/var/empty") + (shell (file-append shadow "/sbin/nologin")))))) + +(define (rspamd-shepherd-service config) + (match-record config + (package config-file user group pid-file debug? insecure? skip-template? + local.d-files override.d-files) + (list + (shepherd-service + (provision '(rspamd)) + (documentation "Run the rspamd daemon.") + (requirement '(networking)) + (start (let ((rspamd (file-append package "/bin/rspamd")) + (local-confdir + (file-union + "rspamd-local-confdir" + `(("local.d" ,(file-union "local.d" local.d-files)) + ("override.d" ,(file-union "override.d" override.d-files)))))) + (with-imported-modules (source-module-closure '((gnu build activation))) + #~(begin + (use-modules (gnu build activation)) ; for mkdir-p/perms + (let ((user (getpwnam #$user))) + (mkdir-p/perms "/var/run/rspamd" user #o755) + (mkdir-p/perms "/var/log/rspamd" user #o755) + (mkdir-p/perms "/var/lib/rspamd" user #o755)) + (make-forkexec-constructor + (list #$rspamd "-c" #$config-file + "--var" (string-append "LOCAL_CONFDIR=" #$local-confdir) + "--no-fork" + #$@(if debug? + '("--debug") + '()) + #$@(if insecure? + '("--insecure") + '()) + #$@(if skip-template? + '("--skip-template") + '())) + #:user #$user + #:group #$group))))) + (stop #~(make-kill-destructor)) + (actions + (list + (shepherd-configuration-action config-file) + (shepherd-action + (name 'reload) + (documentation "Reload rspamd.") + (procedure + #~(lambda (pid) + (if pid + (begin + (kill pid SIGHUP) + (display "Service rspamd has been reloaded")) + (format #t "Service rspamd is not running."))))) + (shepherd-action + (name 'reopenlog) + (documentation "Reopen log files.") + (procedure + #~(lambda (pid) + (if pid + (begin + (kill pid SIGUSR1) + (display "Reopening the logs for rspamd")) + (format #t "Service rspamd is not running."))))))))))) + +(define rspamd-service-type + (service-type + (name 'rspamd) + (description "Run the rapid spam filtering system.") + (extensions + (list + (service-extension shepherd-root-service-type rspamd-shepherd-service) + (service-extension account-service-type rspamd-accounts) + (service-extension profile-service-type + (compose list rspamd-configuration-package)))) + (default-value (rspamd-configuration)))) diff --git a/gnu/tests/mail.scm b/gnu/tests/mail.scm index dcb8f08ea8..4dae6886b2 100644 --- a/gnu/tests/mail.scm +++ b/gnu/tests/mail.scm @@ -6,6 +6,7 @@ ;;; Copyright © 2018 Clément Lassieur ;;; Copyright © 2019 Christopher Baines ;;; Copyright © 2019, 2020 Tobias Geerinckx-Rice +;;; Copyright © 2023 Thomas Ieong ;;; ;;; This file is part of GNU Guix. ;;; @@ -40,7 +41,8 @@ (define-module (gnu tests mail) #:export (%test-opensmtpd %test-exim %test-dovecot - %test-getmail)) + %test-getmail + %test-rspamd)) (define %opensmtpd-os (simple-operating-system @@ -579,3 +581,86 @@ (define %test-getmail (name "getmail") (description "Connect to a running Getmail server.") (value (run-getmail-test)))) + +(define %rspamd-os + (simple-operating-system + (service dhcp-client-service-type) + (service rspamd-service-type))) + +(define (run-rspamd-test) + "Return a test of an OS running Rspamd service." + + (define rspamd-ports + '((22664 . 11332) ;; proxy worker + (22666 . 11333) ;; normal worker + (22668 . 11334) ;; web controller + (22670 . 11335))) ;; fuzzy worker + + (define vm + (virtual-machine + (operating-system (marionette-operating-system + %rspamd-os + #:imported-modules '((gnu services herd)))) + (port-forwardings rspamd-ports))) + + (define test + (with-imported-modules '((gnu build marionette)) + #~(begin + (use-modules (srfi srfi-64) + (srfi srfi-11) + (gnu build marionette) + (web uri) + (web client) + (web response)) + + (define marionette + (make-marionette '(#$vm))) + + (test-runner-current (system-test-runner #$output)) + (test-begin "rspamd") + + (test-assert "service is running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (start-service 'rspamd)) + marionette)) + + + ;; Check mympd-service-type commit for reference + ;; TODO: For this test we need to authorize the controller to + ;; listen on other interfaces, e.g *:11334 instead of localhost:11334 + + ;; Check that we can access the web ui + (test-equal "http-get" + 200 + (begin + (let-values (((response text) + (http-get "http://localhost:22668/" + #:decode-body? #t))) + (response-code response)))) + + (test-assert "rspamd socket ready" + (wait-for-unix-socket + "/var/lib/rspamd/rspamd.sock" + marionette)) + + (test-assert "rspamd pid ready" + (marionette-eval + '(file-exists? "/var/run/rspamd/rspamd.pid") + marionette)) + + (test-assert "rspamd log file" + (marionette-eval + '(file-exists? "/var/log/rspamd/rspamd.log") + marionette)) + + (test-end)))) + + (gexp->derivation "rspamd-test" test)) + +(define %test-rspamd + (system-test + (name "rspamd") + (description "Send an email to a running rspamd server.") + (value (run-rspamd-test))))