diff mbox series

[bug#61740] services: Add rspamd-service-type.

Message ID e2kz4e54vf2gbop4n6b5xnbwiqdgelbiug4in76z5ozm4rfdhy@hvrgaiostytj
State New
Headers show
Series [bug#61740] services: Add rspamd-service-type. | expand

Commit Message

Saku Laesvuori Sept. 16, 2023, 8:10 p.m. UTC
On Fri, Sep 15, 2023 at 01:47:14PM -0700, Felix Lechner wrote:
> Hi Saku,
> 
> > I happened to need rspamd myself
> 
> So do I but it does not seem to start locally. It created some folders
> and, per the log, the configuration file passed the syntax check, but
> then the boot stalls.
> 
> I used (service rspamd-service-type) and nothing else in my system
> configuration. Should it be sufficient? Thanks!

Oops, the version I tested (and am running right now) used
activation-service-type and I forgot to test that it was still working
after moving the code from there to the shepherd service. A fixed v3 is
attached.
From 886c32091bdc032c4180d490125a131df79b705c Mon Sep 17 00:00:00 2001
Message-ID: <886c32091bdc032c4180d490125a131df79b705c.1694894798.git.saku@laesvuori.fi>
From: Thomas Ieong <th.ieong@free.fr>
Date: Thu, 23 Feb 2023 21:16:14 +0100
Subject: [PATCH v3] services: Add rspamd-service-type.

* gnu/services/mail.scm (rspamd-service-type): New variable.
* gnu/tests/mail.scm (%test-rspamd): New variable.
* doc/guix.texi: Document it.

Co-authored-by: Saku Laesvuori <saku@laesvuori.fi>
---
This version imports (gnu build activation) in the shepherd start action
and removes the syntax check, because I expect the service to fail
anyway if the configuration file syntax is invalid and the check seemed
to prevent booting (no idea why).

 doc/guix.texi         |  61 +++++++++++++
 gnu/services/mail.scm | 197 +++++++++++++++++++++++++++++++++++++++++-
 gnu/tests/mail.scm    |  87 ++++++++++++++++++-
 3 files changed, 343 insertions(+), 2 deletions(-)


base-commit: 2d4d147839b81ba8761c9e50cabe9b60025dc670

Comments

Bruno Victal Dec. 6, 2023, 2:58 p.m. UTC | #1
Hi Saku,

Some comments:

> +(define (directory-tree? xs)
> +  (match xs
> +    (((file-name file-like) ...)
> +     (and (every string? file-name)
> +          (every file-like? file-like)))
> +    (_ #f)))

You can express this more compactly as:

--8<---------------cut here---------------start------------->8---
(define directory-tree?
  (match-lambda
    ((((? string?) (? file-like?)) ...) #t)
    (_ #f)))
--8<---------------cut here---------------end--------------->8---

> +  (user
> +   (string "rspamd")
> +   "The user to run rspamd as.")
> +  (group
> +   (string "rspamd")
> +   "The group to run rspamd as.")

How about using user-account and user-group records instead? (see
vnstat-service-type for an example)

> +  (pid-file
> +   (string "/var/run/rspamd/rspamd.pid")
> +   "Where to store the PID file.")

Is it useful to expose this?


> +  (insecure?
> +   (boolean #f)
> +   "Ignore running workers as privileged users (insecure).")

To me it seems redundant to restate “(insecure)” in the description.

> +                     (make-forkexec-constructor
> +                      (list #$rspamd "-c" #$config-file

I'd prefer the long-name --config over the shorter ones here.

> +                            "--var" (string-append "LOCAL_CONFDIR=" #$local-confdir)

Curiously I don't see this listed in the 'rspamd' manpage although
it is on the 'rspamadm' one. Can you confirm whether this works
and if so, report to upstream that their docs are missing this?

> +     (service-extension profile-service-type
> +                        (compose list rspamd-configuration-package))

What's the motivation for adding the rspamd package to the profile?

> +(define %rspamd-os
> +  (simple-operating-system
> +   (service dhcp-client-service-type)
> +   (service rspamd-service-type)))

Is 'dhcp-client-service-type' needed for this system test?
I haven't tested it but it looks unnecessary to me.

> +          ;; Check that we can access the web ui
> +          (test-equal "http-get"
> +            200
> +            (begin
> +              (let-values (((response text)
> +                            (http-get "http://localhost:22668/"
> +                                      #:decode-body? #t)))
> +                (response-code response))))

IMO if you're only interested in the HTTP response code a http-head
is the better option, unless the program handles those requests
differently. Also, since 'text' isn't used you can simplify this to:

--8<---------------cut here---------------start------------->8---
;; Don't forget to remove the unused (srfi srfi-11) import.

(test-equal "Web UI is accessible"
  200
  (response-code (http-head "http://localhost:22668/")))
--8<---------------cut here---------------end--------------->8---

> +          (test-assert "rspamd pid ready"
> +            (marionette-eval
> +             '(file-exists? "/var/run/rspamd/rspamd.pid")
> +             marionette))

There's a procedure dedicated for this:

--8<---------------cut here---------------start------------->8---
(test-assert "rspamd pid ready"
  (wait-for-file #$(rspamd-configuration-pid-file (rspamd-configuration)) marionette)))
--8<---------------cut here---------------end--------------->8---

Note: I used (rspamd-configuration) since it you're using the default
rspamd-configuration value here.

> +(define %test-rspamd
> +  (system-test
> +   (name "rspamd")
> +   (description "Send an email to a running rspamd server.")
> +   (value (run-rspamd-test))))

I'd change the description to something like "Basic rspamd service test."
as the current one is misleading.
diff mbox series

Patch

diff --git a/doc/guix.texi b/doc/guix.texi
index f82bb99069..04e4a60f97 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -119,6 +119,8 @@ 
 Copyright @copyright{} 2023 Zheng Junjie@*
 Copyright @copyright{} 2023 Brian Cully@*
 Copyright @copyright{} 2023 Felix Lechner@*
+Copyright @copyright{} 2023 Thomas Ieong@*
+Copyright @copyright{} 2023 Saku Laesvuori@*
 
 Permission is granted to copy, distribute and/or modify this document
 under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -27393,6 +27395,65 @@  Mail Services
 @end table
 @end deftp
 
+@subsubheading Rspamd Service
+@cindex email
+@cindex spam
+
+@defvar rspamd-service-type
+This is the type of the @uref{https://rspamd.com/, Rspamd} filtering
+system whose value should be a @code{rspamd-configuration}.
+@end defvar
+
+@c %start of fragment
+
+@deftp {Data Type} rspamd-configuration
+Available @code{rspamd-configuration} fields are:
+
+@table @asis
+@item @code{package} (default: @code{rspamd}) (type: file-like)
+The package that provides rspamd.
+
+@item @code{config-file} (type: file-like)
+File-like object of the configuration file to use.  By default all
+workers are enabled except fuzzy and they are binded to their usual
+ports, e.g localhost:11334, localhost:11333 and so on
+
+@item @code{local.d-files} (default: @code{()}) (type: directory-tree)
+Configuration files in local.d, provided as a list of two element lists
+where the first element is the filename and the second one is a
+file-like object.  Settings in these files will be merged with the
+defaults.
+
+@item @code{override.d-files} (default: @code{()}) (type: directory-tree)
+Configuration files in override.d, provided as a list of two element
+lists where the first element is the filename and the second one is a
+file-like object.  Settings in these files will override the defaults.
+
+@item @code{user} (default: @code{"rspamd"}) (type: string)
+The user to run rspamd as.
+
+@item @code{group} (default: @code{"rspamd"}) (type: string)
+The group to run rspamd as.
+
+@item @code{pid-file} (default: @code{"/var/run/rspamd/rspamd.pid"}) (type: string)
+Where to store the PID file.
+
+@item @code{debug?} (default: @code{#f}) (type: boolean)
+Force debug output.
+
+@item @code{insecure?} (default: @code{#f}) (type: boolean)
+Ignore running workers as privileged users (insecure).
+
+@item @code{skip-template?} (default: @code{#f}) (type: boolean)
+Do not apply Jinja templates.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
 @node Messaging Services
 @subsection Messaging Services
 
diff --git a/gnu/services/mail.scm b/gnu/services/mail.scm
index 12dcc8e71d..d8720907c8 100644
--- a/gnu/services/mail.scm
+++ b/gnu/services/mail.scm
@@ -5,6 +5,8 @@ 
 ;;; Copyright © 2017, 2020 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2019 Kristofer Buffington <kristoferbuffington@gmail.com>
 ;;; Copyright © 2020 Jonathan Brielmaier <jonathan.brielmaier@web.de>
+;;; Copyright © 2023 Thomas Ieong <th.ieong@free.fr>
+;;; Copyright © 2023 Saku Laesvuori <saku@laesvuori.fi>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -80,7 +82,11 @@  (define-module (gnu services mail)
             radicale-configuration
             radicale-configuration?
             radicale-service-type
-            %default-radicale-config-file))
+            %default-radicale-config-file
+
+            rspamd-configuration
+            rspamd-service-type
+            %default-rspamd-config-file))
 
 ;;; Commentary:
 ;;;
@@ -1987,3 +1993,192 @@  (define radicale-service-type
           (service-extension account-service-type (const %radicale-accounts))
           (service-extension activation-service-type radicale-activation)))
    (default-value (radicale-configuration))))
+
+;;;
+;;; Rspamd.
+;;;
+
+(define (directory-tree? xs)
+  (match xs
+    (((file-name file-like) ...)
+     (and (every string? file-name)
+          (every file-like? file-like)))
+    (_ #f)))
+
+(define-configuration/no-serialization rspamd-configuration
+  (package
+   (file-like rspamd)
+   "The package that provides rspamd.")
+  (config-file
+   (file-like %default-rspamd-config-file)
+   "File-like object of the configuration file to use.  By default
+all workers are enabled except fuzzy and they are binded
+to their usual ports, e.g localhost:11334, localhost:11333 and so on")
+  (local.d-files
+   (directory-tree '())
+   "Configuration files in local.d, provided as a list of two element lists where
+the first element is the filename and the second one is a file-like object.  Settings
+in these files will be merged with the defaults.")
+  (override.d-files
+   (directory-tree '())
+   "Configuration files in override.d, provided as a list of two element lists where
+the first element is the filename and the second one is a file-like object.  Settings
+in these files will override the defaults.")
+  (user
+   (string "rspamd")
+   "The user to run rspamd as.")
+  (group
+   (string "rspamd")
+   "The group to run rspamd as.")
+  (pid-file
+   (string "/var/run/rspamd/rspamd.pid")
+   "Where to store the PID file.")
+  (debug?
+   (boolean #f)
+   "Force debug output.")
+  (insecure?
+   (boolean #f)
+   "Ignore running workers as privileged users (insecure).")
+  (skip-template?
+   (boolean #f)
+   "Do not apply Jinja templates."))
+
+(define %default-rspamd-config-file
+  (plain-file "rspamd.conf" "
+.include \"$CONFDIR/common.conf\"
+
+options {
+    pidfile = \"$RUNDIR/rspamd.pid\";
+    .include \"$CONFDIR/options.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/options.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/options.inc\"
+}
+
+logging {
+    type = \"file\";
+    filename = \"$LOGDIR/rspamd.log\";
+    .include \"$CONFDIR/logging.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/logging.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/logging.inc\"
+}
+
+worker \"normal\" {
+    bind_socket = \"localhost:11333\";
+    .include \"$CONFDIR/worker-normal.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-normal.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-normal.inc\"
+}
+
+worker \"controller\" {
+    bind_socket = \"localhost:11334\";
+    .include \"$CONFDIR/worker-controller.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-controller.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-controller.inc\"
+}
+
+worker \"rspamd_proxy\" {
+    bind_socket = \"localhost:11332\";
+    .include \"$CONFDIR/worker-proxy.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-proxy.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-proxy.inc\"
+}
+
+# Local fuzzy storage is disabled by default
+
+worker \"fuzzy\" {
+    bind_socket = \"localhost:11335\";
+    count = -1; # Disable by default
+    .include \"$CONFDIR/worker-fuzzy.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-fuzzy.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-fuzzy.inc\"
+}
+"))
+
+(define (rspamd-accounts config)
+  (match-record config <rspamd-configuration>
+    (user group)
+    (list
+     (user-group
+      (name group)
+      (system? #t))
+     (user-account
+      (name user)
+      (group group)
+      (system? #t)
+      (comment "Rspamd daemon")
+      (home-directory "/var/empty")
+      (shell (file-append shadow "/sbin/nologin"))))))
+
+(define (rspamd-shepherd-service config)
+  (match-record config <rspamd-configuration>
+    (package config-file user group pid-file debug? insecure? skip-template?
+     local.d-files override.d-files)
+    (list
+     (shepherd-service
+      (provision '(rspamd))
+      (documentation "Run the rspamd daemon.")
+      (requirement '(networking))
+      (start (let ((rspamd (file-append package "/bin/rspamd"))
+                   (local-confdir
+                     (file-union
+                      "rspamd-local-confdir"
+                      `(("local.d" ,(file-union "local.d" local.d-files))
+                        ("override.d" ,(file-union "override.d" override.d-files))))))
+               (with-imported-modules (source-module-closure '((gnu build activation)))
+                 #~(begin
+                     (use-modules (gnu build activation)) ; for mkdir-p/perms
+                     (let ((user (getpwnam #$user)))
+                       (mkdir-p/perms "/var/run/rspamd" user #o755)
+                       (mkdir-p/perms "/var/log/rspamd" user #o755)
+                       (mkdir-p/perms "/var/lib/rspamd" user #o755))
+                     (make-forkexec-constructor
+                      (list #$rspamd "-c" #$config-file
+                            "--var" (string-append "LOCAL_CONFDIR=" #$local-confdir)
+                            "--no-fork"
+                            #$@(if debug?
+                                 '("--debug")
+                                 '())
+                            #$@(if insecure?
+                                 '("--insecure")
+                                 '())
+                            #$@(if skip-template?
+                                 '("--skip-template")
+                                 '()))
+                      #:user #$user
+                      #:group #$group)))))
+      (stop #~(make-kill-destructor))
+      (actions
+       (list
+        (shepherd-configuration-action config-file)
+        (shepherd-action
+         (name 'reload)
+         (documentation "Reload rspamd.")
+         (procedure
+          #~(lambda (pid)
+              (if pid
+                (begin
+                  (kill pid SIGHUP)
+                  (display "Service rspamd has been reloaded"))
+                (format #t "Service rspamd is not running.")))))
+        (shepherd-action
+         (name 'reopenlog)
+         (documentation "Reopen log files.")
+         (procedure
+          #~(lambda (pid)
+              (if pid
+                (begin
+                  (kill pid SIGUSR1)
+                  (display "Reopening the logs for rspamd"))
+                (format #t "Service rspamd is not running.")))))))))))
+
+(define rspamd-service-type
+  (service-type
+   (name 'rspamd)
+   (description "Run the rapid spam filtering system.")
+   (extensions
+    (list
+     (service-extension shepherd-root-service-type rspamd-shepherd-service)
+     (service-extension account-service-type rspamd-accounts)
+     (service-extension profile-service-type
+                        (compose list rspamd-configuration-package))))
+   (default-value (rspamd-configuration))))
diff --git a/gnu/tests/mail.scm b/gnu/tests/mail.scm
index dcb8f08ea8..4dae6886b2 100644
--- a/gnu/tests/mail.scm
+++ b/gnu/tests/mail.scm
@@ -6,6 +6,7 @@ 
 ;;; Copyright © 2018 Clément Lassieur <clement@lassieur.org>
 ;;; Copyright © 2019 Christopher Baines <mail@cbaines.net>
 ;;; Copyright © 2019, 2020 Tobias Geerinckx-Rice <me@tobias.gr>
+;;; Copyright © 2023 Thomas Ieong <th.ieong@free.fr>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -40,7 +41,8 @@  (define-module (gnu tests mail)
   #:export (%test-opensmtpd
             %test-exim
             %test-dovecot
-            %test-getmail))
+            %test-getmail
+            %test-rspamd))
 
 (define %opensmtpd-os
   (simple-operating-system
@@ -579,3 +581,86 @@  (define %test-getmail
    (name "getmail")
    (description "Connect to a running Getmail server.")
    (value (run-getmail-test))))
+
+(define %rspamd-os
+  (simple-operating-system
+   (service dhcp-client-service-type)
+   (service rspamd-service-type)))
+
+(define (run-rspamd-test)
+  "Return a test of an OS running Rspamd service."
+
+  (define rspamd-ports
+    '((22664 . 11332)    ;; proxy worker
+      (22666 . 11333)    ;; normal worker
+      (22668 . 11334)    ;; web controller
+      (22670 . 11335)))  ;; fuzzy worker
+
+  (define vm
+    (virtual-machine
+     (operating-system (marionette-operating-system
+                        %rspamd-os
+                        #:imported-modules '((gnu services herd))))
+     (port-forwardings rspamd-ports)))
+
+  (define test
+    (with-imported-modules '((gnu build marionette))
+      #~(begin
+          (use-modules (srfi srfi-64)
+                       (srfi srfi-11)
+                       (gnu build marionette)
+                       (web uri)
+                       (web client)
+                       (web response))
+
+          (define marionette
+            (make-marionette '(#$vm)))
+
+          (test-runner-current (system-test-runner #$output))
+          (test-begin "rspamd")
+
+          (test-assert "service is running"
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (start-service 'rspamd))
+             marionette))
+
+
+          ;; Check mympd-service-type commit for reference
+          ;; TODO: For this test we need to authorize the controller to
+          ;; listen on other interfaces, e.g *:11334 instead of localhost:11334
+
+          ;; Check that we can access the web ui
+          (test-equal "http-get"
+            200
+            (begin
+              (let-values (((response text)
+                            (http-get "http://localhost:22668/"
+                                      #:decode-body? #t)))
+                (response-code response))))
+
+          (test-assert "rspamd socket ready"
+            (wait-for-unix-socket
+             "/var/lib/rspamd/rspamd.sock"
+             marionette))
+
+          (test-assert "rspamd pid ready"
+            (marionette-eval
+             '(file-exists? "/var/run/rspamd/rspamd.pid")
+             marionette))
+
+          (test-assert "rspamd log file"
+            (marionette-eval
+             '(file-exists? "/var/log/rspamd/rspamd.log")
+             marionette))
+
+          (test-end))))
+
+  (gexp->derivation "rspamd-test" test))
+
+(define %test-rspamd
+  (system-test
+   (name "rspamd")
+   (description "Send an email to a running rspamd server.")
+   (value (run-rspamd-test))))