From patchwork Tue Sep 5 19:06:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Saku Laesvuori X-Patchwork-Id: 53522 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 4341427BBEA; Tue, 5 Sep 2023 20:07:24 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id BFA4C27BBE9 for ; Tue, 5 Sep 2023 20:07:21 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdbOB-00084S-Dk; Tue, 05 Sep 2023 15:07:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdbO9-00083z-LW for guix-patches@gnu.org; Tue, 05 Sep 2023 15:07:01 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qdbO9-00076W-Dw for guix-patches@gnu.org; Tue, 05 Sep 2023 15:07:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qdbOA-00040p-1A for guix-patches@gnu.org; Tue, 05 Sep 2023 15:07:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#61740] [PATCH] services: Add rspamd-service-type. Resent-From: Saku Laesvuori Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 05 Sep 2023 19:07:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 61740 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: moreinfo patch To: Ludovic =?utf-8?q?Court=C3=A8s?= Cc: Thomas Ieong , Bruno Victal , 61740@debbugs.gnu.org Received: via spool by 61740-submit@debbugs.gnu.org id=B61740.169394077415367 (code B ref 61740); Tue, 05 Sep 2023 19:07:01 +0000 Received: (at 61740) by debbugs.gnu.org; 5 Sep 2023 19:06:14 +0000 Received: from localhost ([127.0.0.1]:59298 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdbNN-0003zm-Kk for submit@debbugs.gnu.org; Tue, 05 Sep 2023 15:06:14 -0400 Received: from vmi571514.contaboserver.net ([75.119.130.101]:60246 helo=mail.laesvuori.fi) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdbNJ-0003zZ-4K for 61740@debbugs.gnu.org; Tue, 05 Sep 2023 15:06:12 -0400 Received: from X-kone (88-113-24-127.elisa-laajakaista.fi [88.113.24.127]) by mail.laesvuori.fi (Postfix) with ESMTPSA id 7F87F342FBE; Tue, 5 Sep 2023 21:06:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=laesvuori.fi; s=mail; t=1693940792; bh=xNL1ukIUb7BzTB2WUCm7CKx2GmLgBANjCpaEgNROKSw=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=WkVIgODCD1fin1RdkFkDDo8fwGb4WVmPxUOK4Wz6lKCt98gv+pAMm33O7+b/7ICLY zDGkMLEBh0yzeprobqbyZ9AtGwpAtuOi0GLztSaOaKiAAbB0d1+UHrahsybqUCEu2p A1SQckdC7Aiq/U1/BC8p+TF7x7vQbQM/MamHwric= Date: Tue, 5 Sep 2023 22:06:01 +0300 Message-ID: References: <68b32de839c2abda283be3539eef38aebd53d82e.1677183321.git.th.ieong@free.fr> <87v8dppmy2.fsf_-_@gnu.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <87v8dppmy2.fsf_-_@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Saku Laesvuori X-ACL-Warn: , Saku Laesvuori via Guix-patches X-Patchwork-Original-From: Saku Laesvuori via Guix-patches via From: Saku Laesvuori Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches > Hi Thomas, > > It’s been a while. :-) Did you have time to consider Bruno’s > suggestions to send an updated patch? > > https://issues.guix.gnu.org/61740 > > Thanks, > Ludo’. I happened to need rspamd myself so I cleaned this patch a little and thought it would be useful to submit a v2 of it. I don't really know how co-authored patches should be sent (because I expect the From: in the patch to interfere with email) so I added it as an attachment. From 0de51c84aaccfa389276188cc617ddb6c05772f1 Mon Sep 17 00:00:00 2001 Message-ID: <0de51c84aaccfa389276188cc617ddb6c05772f1.1693939190.git.saku@laesvuori.fi> From: Thomas Ieong Date: Thu, 23 Feb 2023 21:16:14 +0100 Subject: [PATCH v2] services: Add rspamd-service-type. * gnu/services/mail.scm (rspamd-service-type): New variable. * gnu/tests/mail.scm (%test-rspamd): New variable. * doc/guix.texi: Document it. Co-authored-by: Saku Laesvuori --- doc/guix.texi | 61 +++++++++++++ gnu/services/mail.scm | 201 +++++++++++++++++++++++++++++++++++++++++- gnu/tests/mail.scm | 87 +++++++++++++++++- 3 files changed, 347 insertions(+), 2 deletions(-) base-commit: 2d4d147839b81ba8761c9e50cabe9b60025dc670 diff --git a/doc/guix.texi b/doc/guix.texi index f82bb99069..04e4a60f97 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -119,6 +119,8 @@ Copyright @copyright{} 2023 Zheng Junjie@* Copyright @copyright{} 2023 Brian Cully@* Copyright @copyright{} 2023 Felix Lechner@* +Copyright @copyright{} 2023 Thomas Ieong@* +Copyright @copyright{} 2023 Saku Laesvuori@* Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or @@ -27393,6 +27395,65 @@ Mail Services @end table @end deftp +@subsubheading Rspamd Service +@cindex email +@cindex spam + +@defvar rspamd-service-type +This is the type of the @uref{https://rspamd.com/, Rspamd} filtering +system whose value should be a @code{rspamd-configuration}. +@end defvar + +@c %start of fragment + +@deftp {Data Type} rspamd-configuration +Available @code{rspamd-configuration} fields are: + +@table @asis +@item @code{package} (default: @code{rspamd}) (type: file-like) +The package that provides rspamd. + +@item @code{config-file} (type: file-like) +File-like object of the configuration file to use. By default all +workers are enabled except fuzzy and they are binded to their usual +ports, e.g localhost:11334, localhost:11333 and so on + +@item @code{local.d-files} (default: @code{()}) (type: directory-tree) +Configuration files in local.d, provided as a list of two element lists +where the first element is the filename and the second one is a +file-like object. Settings in these files will be merged with the +defaults. + +@item @code{override.d-files} (default: @code{()}) (type: directory-tree) +Configuration files in override.d, provided as a list of two element +lists where the first element is the filename and the second one is a +file-like object. Settings in these files will override the defaults. + +@item @code{user} (default: @code{"rspamd"}) (type: string) +The user to run rspamd as. + +@item @code{group} (default: @code{"rspamd"}) (type: string) +The group to run rspamd as. + +@item @code{pid-file} (default: @code{"/var/run/rspamd/rspamd.pid"}) (type: string) +Where to store the PID file. + +@item @code{debug?} (default: @code{#f}) (type: boolean) +Force debug output. + +@item @code{insecure?} (default: @code{#f}) (type: boolean) +Ignore running workers as privileged users (insecure). + +@item @code{skip-template?} (default: @code{#f}) (type: boolean) +Do not apply Jinja templates. + +@end table + +@end deftp + + +@c %end of fragment + @node Messaging Services @subsection Messaging Services diff --git a/gnu/services/mail.scm b/gnu/services/mail.scm index 12dcc8e71d..43d39ecfe6 100644 --- a/gnu/services/mail.scm +++ b/gnu/services/mail.scm @@ -5,6 +5,8 @@ ;;; Copyright © 2017, 2020 Tobias Geerinckx-Rice ;;; Copyright © 2019 Kristofer Buffington ;;; Copyright © 2020 Jonathan Brielmaier +;;; Copyright © 2023 Thomas Ieong +;;; Copyright © 2023 Saku Laesvuori ;;; ;;; This file is part of GNU Guix. ;;; @@ -80,7 +82,11 @@ (define-module (gnu services mail) radicale-configuration radicale-configuration? radicale-service-type - %default-radicale-config-file)) + %default-radicale-config-file + + rspamd-configuration + rspamd-service-type + %default-rspamd-config-file)) ;;; Commentary: ;;; @@ -1987,3 +1993,196 @@ (define radicale-service-type (service-extension account-service-type (const %radicale-accounts)) (service-extension activation-service-type radicale-activation))) (default-value (radicale-configuration)))) + +;;; +;;; Rspamd. +;;; + +(define (directory-tree? xs) + (match xs + (((file-name file-like) ...) + (and (every string? file-name) + (every file-like? file-like))) + (_ #f))) + +(define-configuration/no-serialization rspamd-configuration + (package + (file-like rspamd) + "The package that provides rspamd.") + (config-file + (file-like %default-rspamd-config-file) + "File-like object of the configuration file to use. By default +all workers are enabled except fuzzy and they are binded +to their usual ports, e.g localhost:11334, localhost:11333 and so on") + (local.d-files + (directory-tree '()) + "Configuration files in local.d, provided as a list of two element lists where +the first element is the filename and the second one is a file-like object. Settings +in these files will be merged with the defaults.") + (override.d-files + (directory-tree '()) + "Configuration files in override.d, provided as a list of two element lists where +the first element is the filename and the second one is a file-like object. Settings +in these files will override the defaults.") + (user + (string "rspamd") + "The user to run rspamd as.") + (group + (string "rspamd") + "The group to run rspamd as.") + (pid-file + (string "/var/run/rspamd/rspamd.pid") + "Where to store the PID file.") + (debug? + (boolean #f) + "Force debug output.") + (insecure? + (boolean #f) + "Ignore running workers as privileged users (insecure).") + (skip-template? + (boolean #f) + "Do not apply Jinja templates.")) + +(define %default-rspamd-config-file + (plain-file "rspamd.conf" " +.include \"$CONFDIR/common.conf\" + +options { + pidfile = \"$RUNDIR/rspamd.pid\"; + .include \"$CONFDIR/options.inc\" + .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/options.inc\" + .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/options.inc\" +} + +logging { + type = \"file\"; + filename = \"$LOGDIR/rspamd.log\"; + .include \"$CONFDIR/logging.inc\" + .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/logging.inc\" + .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/logging.inc\" +} + +worker \"normal\" { + bind_socket = \"localhost:11333\"; + .include \"$CONFDIR/worker-normal.inc\" + .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-normal.inc\" + .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-normal.inc\" +} + +worker \"controller\" { + bind_socket = \"localhost:11334\"; + .include \"$CONFDIR/worker-controller.inc\" + .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-controller.inc\" + .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-controller.inc\" +} + +worker \"rspamd_proxy\" { + bind_socket = \"localhost:11332\"; + .include \"$CONFDIR/worker-proxy.inc\" + .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-proxy.inc\" + .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-proxy.inc\" +} + +# Local fuzzy storage is disabled by default + +worker \"fuzzy\" { + bind_socket = \"localhost:11335\"; + count = -1; # Disable by default + .include \"$CONFDIR/worker-fuzzy.inc\" + .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-fuzzy.inc\" + .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-fuzzy.inc\" +} +")) + +(define (rspamd-accounts config) + (match-record config + (user group) + (list + (user-group + (name group) + (system? #t)) + (user-account + (name user) + (group group) + (system? #t) + (comment "Rspamd daemon") + (home-directory "/var/empty") + (shell (file-append shadow "/sbin/nologin")))))) + +(define (rspamd-shepherd-service config) + (match-record config + (package config-file user group pid-file debug? insecure? skip-template? + local.d-files override.d-files) + (list + (shepherd-service + (provision '(rspamd)) + (documentation "Run the rspamd daemon.") + (requirement '(networking)) + (start (let ((rspamd (file-append package "/bin/rspamd")) + (local-confdir + (file-union + "rspamd-local-confdir" + `(("local.d" ,(file-union "local.d" local.d-files)) + ("override.d" ,(file-union "override.d" override.d-files)))))) + #~(begin + (use-modules (guix build utils) + (ice-9 match)) + (let ((user (getpwnam #$user))) + (mkdir-p/perms "/var/run/rspamd" user #o755) + (mkdir-p/perms "/var/log/rspamd" user #o755) + (mkdir-p/perms "/var/lib/rspamd" user #o755)) + ;; Check configuration file syntax. + (system* (string-append #$package "/bin/rspamadm") + "configtest" + "-c" #$config-file) + (make-forkexec-constructor + (list #$rspamd "-c" #$config-file + "--var" (string-append "LOCAL_CONFDIR=" #$local-confdir) + "--no-fork" + #$@(if debug? + '("--debug") + '()) + #$@(if insecure? + '("--insecure") + '()) + #$@(if skip-template? + '("--skip-template") + '())) + #:user #$user + #:group #$group)))) + (stop #~(make-kill-destructor)) + (actions + (list + (shepherd-configuration-action config-file) + (shepherd-action + (name 'reload) + (documentation "Reload rspamd.") + (procedure + #~(lambda (pid) + (if pid + (begin + (kill pid SIGHUP) + (display "Service rspamd has been reloaded")) + (format #t "Service rspamd is not running."))))) + (shepherd-action + (name 'reopenlog) + (documentation "Reopen log files.") + (procedure + #~(lambda (pid) + (if pid + (begin + (kill pid SIGUSR1) + (display "Reopening the logs for rspamd")) + (format #t "Service rspamd is not running."))))))))))) + +(define rspamd-service-type + (service-type + (name 'rspamd) + (description "Run the rapid spam filtering system.") + (extensions + (list + (service-extension shepherd-root-service-type rspamd-shepherd-service) + (service-extension account-service-type rspamd-accounts) + (service-extension profile-service-type + (compose list rspamd-configuration-package)))) + (default-value (rspamd-configuration)))) diff --git a/gnu/tests/mail.scm b/gnu/tests/mail.scm index dcb8f08ea8..4dae6886b2 100644 --- a/gnu/tests/mail.scm +++ b/gnu/tests/mail.scm @@ -6,6 +6,7 @@ ;;; Copyright © 2018 Clément Lassieur ;;; Copyright © 2019 Christopher Baines ;;; Copyright © 2019, 2020 Tobias Geerinckx-Rice +;;; Copyright © 2023 Thomas Ieong ;;; ;;; This file is part of GNU Guix. ;;; @@ -40,7 +41,8 @@ (define-module (gnu tests mail) #:export (%test-opensmtpd %test-exim %test-dovecot - %test-getmail)) + %test-getmail + %test-rspamd)) (define %opensmtpd-os (simple-operating-system @@ -579,3 +581,86 @@ (define %test-getmail (name "getmail") (description "Connect to a running Getmail server.") (value (run-getmail-test)))) + +(define %rspamd-os + (simple-operating-system + (service dhcp-client-service-type) + (service rspamd-service-type))) + +(define (run-rspamd-test) + "Return a test of an OS running Rspamd service." + + (define rspamd-ports + '((22664 . 11332) ;; proxy worker + (22666 . 11333) ;; normal worker + (22668 . 11334) ;; web controller + (22670 . 11335))) ;; fuzzy worker + + (define vm + (virtual-machine + (operating-system (marionette-operating-system + %rspamd-os + #:imported-modules '((gnu services herd)))) + (port-forwardings rspamd-ports))) + + (define test + (with-imported-modules '((gnu build marionette)) + #~(begin + (use-modules (srfi srfi-64) + (srfi srfi-11) + (gnu build marionette) + (web uri) + (web client) + (web response)) + + (define marionette + (make-marionette '(#$vm))) + + (test-runner-current (system-test-runner #$output)) + (test-begin "rspamd") + + (test-assert "service is running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (start-service 'rspamd)) + marionette)) + + + ;; Check mympd-service-type commit for reference + ;; TODO: For this test we need to authorize the controller to + ;; listen on other interfaces, e.g *:11334 instead of localhost:11334 + + ;; Check that we can access the web ui + (test-equal "http-get" + 200 + (begin + (let-values (((response text) + (http-get "http://localhost:22668/" + #:decode-body? #t))) + (response-code response)))) + + (test-assert "rspamd socket ready" + (wait-for-unix-socket + "/var/lib/rspamd/rspamd.sock" + marionette)) + + (test-assert "rspamd pid ready" + (marionette-eval + '(file-exists? "/var/run/rspamd/rspamd.pid") + marionette)) + + (test-assert "rspamd log file" + (marionette-eval + '(file-exists? "/var/log/rspamd/rspamd.log") + marionette)) + + (test-end)))) + + (gexp->derivation "rspamd-test" test)) + +(define %test-rspamd + (system-test + (name "rspamd") + (description "Send an email to a running rspamd server.") + (value (run-rspamd-test))))