diff mbox series

[bug#69513] services: Add restic-backup service.

Message ID 3afc07b0f3e6663a9fb64203544bce1659f97364.1709412684.git.goodoldpaul@autistici.org
State New
Headers show
Series [bug#69513] services: Add restic-backup service. | expand

Commit Message

Giacomo Leidi March 2, 2024, 8:51 p.m. UTC
* gnu/services/backup.scm: New file.
* gnu/local.mk: Add this.
* doc/guix.texi: Document this.

Change-Id: I9efd5559bb445b484107a7c27c2d0a65ccad1e66
---
 doc/guix.texi           |  95 +++++++++++++++++++++++-
 gnu/local.mk            |   1 +
 gnu/services/backup.scm | 160 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 255 insertions(+), 1 deletion(-)
 create mode 100644 gnu/services/backup.scm


base-commit: 6f5ea7ac1acb3d1c53baf7620cca66cc87fe5a73

Comments

Ludovic Courtès March 29, 2024, 10:36 p.m. UTC | #1
Hi,

Giacomo Leidi <goodoldpaul@autistici.org> skribis:

> * gnu/services/backup.scm: New file.
> * gnu/local.mk: Add this.
> * doc/guix.texi: Document this.
>
> Change-Id: I9efd5559bb445b484107a7c27c2d0a65ccad1e66

[...]

> +@subsubheading Backup services

Please capitalize headings: “Backup Services”.

We should probably move documentation of ‘syncthing-service-type’ here,
even if they live in different modules for now.

> +The @code{(gnu services backup)} module offers services for backing up
> +file system trees. For now, it provides the @code{restic-backup-service-type}.
                    ^
Nitpick: Please leave two spaces after an end-of-sentence period (for
easier Emacs navigation, readability, and consistency).

> +To backup a list of file system trees to a pre-initialized, end-to-end
> +encrypted, deduplicated data repository, you could so with the
> +@code{restic-backup-service-type}. For example with the following
> +configuration:

How about:

  With @code{restic-backup-service-type}, you can periodically back up
  directories and files with @uref{https://restic.net/, Restic}, which
  supports end-to-end encryption and deduplication.  Consider the
  following configuration:

?

> +Each @code{restic-backup-job} translates to an mcron job which sets the
> +@code{RESTIC_PASSWORD} environment variable by reading the first line of

@env{RESTIC_PASSWORD}

> +@item @code{specification} (type: gexp-or-string)
> +A string or a gexp that will be passed as time specification in the
> +mcron job specification (@pxref{Syntax, mcron job specifications,,
> +mcron,GNU@tie{}mcron}).

Maybe ‘schedule’ rather than ‘specification’, to clarify what’s being
specified?  (That’s the name I chose in <unattended-upgrade-configuration>.)

> +@item @code{included} (default: @code{'()}) (type: list-of-lowerables)
> +A list of values that are lowered to strings representing filesystem
> +paths.  These are the paths that will be recursively included in the
> +current job.

In GNU and Guix, “path” is used to denote “search paths”; in other
cases, we write “file name” or “file”.  So I’d suggest something like:

  The list of files or directories to be backed up.

The ‘files-to-backup’ (or ‘files’?) may be more descriptive that
‘included’.

> +  (password-file
> +   (string)
> +   "The path of a password file, readable by the configured @code{user}, that

“Name of the password file”

> +will be used to set the @code{RESTIC_PASSWORD} environment variable for the

s/@code/@env/

> +    (program-file
> +     "restic-backup-job.scm"
> +     (with-imported-modules (source-module-closure
> +                             '((guix build utils)))
> +       #~(begin
> +           (use-modules (guix build utils)
> +                        (ice-9 popen)
> +                        (ice-9 rdelim))
> +           (setenv "RESTIC_PASSWORD"
> +                   (with-input-from-file #$password-file read-line))
> +
> +           (execlp #$restic #$@verbose
> +                   "-r" #$repository
> +                   #$@extra-flags
> +                   "backup" #$@included))))))

I believe (guix build utils) is unused, in which case you can remove it.

The ‘execlp’ call lacks argv[0]; it should look like this:

  (execlp #$restic #$restic #$@verbose "-r" …)

(Notice that #$restic appears twice.)

Could you send an updated patch?

Thanks,
Ludo’.
Giacomo Leidi April 2, 2024, 8:33 p.m. UTC | #2
Hello Ludo',

thank you for your insight, I should have addressed all of your 
comments. I'm sending an updated patch.


cheers

giacomo
Giacomo Leidi May 1, 2024, 9:14 p.m. UTC | #3
Hello Ludo' ,


I'm sending a v3 rebased on current master. In this revision I added the 
possibility to manually trigger a backup without waiting for the 
scheduled job to run.

Thank you for your work,


giacomo
Richard Sent May 30, 2024, 7:23 p.m. UTC | #4
Hi all,

> Please consider adding a system test for this: as previously
> discussed, we try hard to have tests for every system service.

I have a few changes I plan on submitting soon regarding the restic
service. FYSA Giacomo, I'm writing system tests as well, although those
tests will obviously be limited to local repositories.

For the curious I'm working on the following changes:

1. Add an init? field to attempt to bootstrap local repository
initialization, ala https://github.com/NixOS/nixpkgs/pull/307962

2. Add support for Restic's password command feature in addition to the
existing password file

3. Either add rclone to the profile or add an extra-packages field to
restic-backup-job so restic-required packages can be located in one
spot.

4. The aforementioned tests

I haven't used restic before so this might be harder than I think it'll
be. So far things seem to be going well.
diff mbox series

Patch

diff --git a/doc/guix.texi b/doc/guix.texi
index 87fe9f803c..4e53d22c5a 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -111,7 +111,7 @@ 
 Copyright @copyright{} 2022 John Kehayias@*
 Copyright @copyright{} 2022⁠–⁠2023 Bruno Victal@*
 Copyright @copyright{} 2022 Ivan Vilata-i-Balaguer@*
-Copyright @copyright{} 2023 Giacomo Leidi@*
+Copyright @copyright{} 2023, 2024 Giacomo Leidi@*
 Copyright @copyright{} 2022 Antero Mejr@*
 Copyright @copyright{} 2023 Karl Hallsby@*
 Copyright @copyright{} 2023 Nathaniel Nicandro@*
@@ -41045,6 +41045,99 @@  Miscellaneous Services
 
 @c End of auto-generated fail2ban documentation.
 
+@cindex Backup
+@subsubheading Backup services
+
+The @code{(gnu services backup)} module offers services for backing up
+file system trees. For now, it provides the @code{restic-backup-service-type}.
+
+To backup a list of file system trees to a pre-initialized, end-to-end
+encrypted, deduplicated data repository, you could so with the
+@code{restic-backup-service-type}. For example with the following
+configuration:
+
+@lisp
+(service restic-backup-service-type
+         (restic-backup-configuration
+           (jobs
+             (list (restic-backup-job
+                     (repository "rclone:remote-ftp:backup/restic")
+                     (password-file "/root/.restic")
+                     ;; Every day at 23.
+                     (specification "0 23 * * *")
+                     (included '("/root/.restic"
+                                 "/root/.config/rclone"
+                                 "/etc/ssh/ssh_host_rsa_key"
+                                 "/etc/ssh/ssh_host_rsa_key.pub"
+                                 "/etc/guix/signing-key.pub"
+                                 "/etc/guix/signing-key.sec")))))))
+@end lisp
+
+Each @code{restic-backup-job} translates to an mcron job which sets the
+@code{RESTIC_PASSWORD} environment variable by reading the first line of
+@code{password-file} and runs @command{restic backup}.
+
+@c %start of fragment
+
+@deftp {Data Type} restic-backup-configuration
+Available @code{restic-backup-configuration} fields are:
+
+@table @asis
+@item @code{jobs} (default: @code{'()}) (type: list-of-restic-backup-jobs)
+The list of backup jobs for the current system.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} restic-backup-job
+Available @code{restic-backup-job} fields are:
+
+@table @asis
+@item @code{restic} (default: @code{restic}) (type: package)
+The restic package to be used for the current job.
+
+@item @code{user} (default: @code{"root"}) (type: string)
+The user used for running the current job.
+
+@item @code{repository} (type: string)
+The restic repository target of this job.
+
+@item @code{password-file} (type: string)
+The path of a password file, readable by the configured @code{user},
+that will be used to set the @code{RESTIC_PASSWORD} environment variable
+for the current job.
+
+@item @code{specification} (type: gexp-or-string)
+A string or a gexp that will be passed as time specification in the
+mcron job specification (@pxref{Syntax, mcron job specifications,,
+mcron,GNU@tie{}mcron}).
+
+@item @code{included} (default: @code{'()}) (type: list-of-lowerables)
+A list of values that are lowered to strings representing filesystem
+paths.  These are the paths that will be recursively included in the
+current job.
+
+@item @code{verbose?} (default: @code{#f}) (type: boolean)
+Whether to enable verbose output for the current backup job.
+
+@item @code{extra-flags} (default: @code{'()}) (type: list-of-lowerables)
+A list of values that are lowered to strings.  These will be passed as
+command-line arguments to the current job @command{restic backup}
+invokation.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
 @node Setuid Programs
 @section Setuid Programs
 
diff --git a/gnu/local.mk b/gnu/local.mk
index cabd82f532..bf911327f4 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -693,6 +693,7 @@  GNU_SYSTEM_MODULES =				\
   %D%/services/auditd.scm			\
   %D%/services/avahi.scm			\
   %D%/services/base.scm				\
+  %D%/services/backup.scm			\
   %D%/services/certbot.scm			\
   %D%/services/cgit.scm			\
   %D%/services/ci.scm				\
diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm
new file mode 100644
index 0000000000..e9172af8c4
--- /dev/null
+++ b/gnu/services/backup.scm
@@ -0,0 +1,160 @@ 
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services backup)
+  #:use-module (gnu packages backup)
+  #:use-module (gnu services)
+  #:use-module (gnu services configuration)
+  #:use-module (gnu services mcron)
+  #:use-module (guix gexp)
+  #:use-module (guix modules)
+  #:use-module (guix packages)
+  #:use-module (srfi srfi-1)
+  #:export (restic-backup-job
+            restic-backup-job?
+            restic-backup-job-fields
+            restic-backup-job-restic
+            restic-backup-job-user
+            restic-backup-job-repository
+            restic-backup-job-password-file
+            restic-backup-job-included
+            restic-backup-job-verbose?
+            restic-backup-job-extra-flags
+
+            restic-backup-configuration
+            restic-backup-configuration?
+            restic-backup-configuration-fields
+            restic-backup-configuration-jobs
+
+            restic-backup-job-program
+            restic-backup-job->mcron-job
+            restic-backup-service-type))
+
+(define (gexp-or-string? value)
+  (or (gexp? value)
+      (string? value)))
+
+(define (lowerable? value)
+  (or (file-like? value)
+      (gexp-or-string? value)))
+
+(define list-of-lowerables?
+  (list-of lowerable?))
+
+(define-configuration/no-serialization restic-backup-job
+  (restic
+   (package restic)
+   "The restic package to be used for the current job.")
+  (user
+   (string "root")
+   "The user used for running the current job.")
+  (repository
+   (string)
+   "The restic repository target of this job.")
+  (password-file
+   (string)
+   "The path of a password file, readable by the configured @code{user}, that
+will be used to set the @code{RESTIC_PASSWORD} environment variable for the
+current job.")
+  (specification
+   (gexp-or-string)
+   "A string or a gexp that will be passed as time specification in the mcron
+job specification (@pxref{Syntax, mcron job specifications,, mcron,
+GNU@tie{}mcron}).")
+  (included
+   (list-of-lowerables '())
+   "A list of values that are lowered to strings representing filesystem paths.
+These are the paths that will be recursively included in the current job.")
+  (verbose?
+   (boolean #f)
+   "Whether to enable verbose output for the current backup job.")
+  (extra-flags
+   (list-of-lowerables '())
+   "A list of values that are lowered to strings.  These will be passed as
+command-line arguments to the current job @command{restic backup} invokation."))
+
+(define list-of-restic-backup-jobs?
+  (list-of restic-backup-job?))
+
+(define-configuration/no-serialization restic-backup-configuration
+  (jobs
+   (list-of-restic-backup-jobs '())
+   "The list of backup jobs for the current system."))
+
+(define (restic-backup-job-program config)
+  (let ((restic
+         (file-append (restic-backup-job-restic config) "/bin/restic"))
+        (repository
+         (restic-backup-job-repository config))
+        (password-file
+         (restic-backup-job-password-file config))
+        (included
+         (restic-backup-job-included config))
+        (extra-flags
+         (restic-backup-job-extra-flags config))
+        (verbose
+         (if (restic-backup-job-verbose? config)
+             '("--verbose")
+             '())))
+    (program-file
+     "restic-backup-job.scm"
+     (with-imported-modules (source-module-closure
+                             '((guix build utils)))
+       #~(begin
+           (use-modules (guix build utils)
+                        (ice-9 popen)
+                        (ice-9 rdelim))
+           (setenv "RESTIC_PASSWORD"
+                   (with-input-from-file #$password-file read-line))
+
+           (execlp #$restic #$@verbose
+                   "-r" #$repository
+                   #$@extra-flags
+                   "backup" #$@included))))))
+
+(define (restic-backup-job->mcron-job config)
+  (let ((user
+         (restic-backup-job-user config))
+        (specification
+         (restic-backup-job-specification config))
+        (program
+         (restic-backup-job-program config)))
+    #~(job #$specification
+           #$program
+           #:user #$user)))
+
+(define restic-backup-service-type
+  (service-type (name 'restic-backup)
+                (extensions
+                 (list
+                  (service-extension mcron-service-type
+                                     (lambda (config)
+                                       (map restic-backup-job->mcron-job
+                                            (restic-backup-configuration-jobs
+                                             config))))))
+                (compose concatenate)
+                (extend
+                 (lambda (config jobs)
+                   (restic-backup-configuration
+                    (inherit config)
+                    (jobs (append (restic-backup-configuration-jobs config)
+                                  jobs)))))
+                (default-value (restic-backup-configuration))
+                (description
+                 "This service configures @code{mcron} jobs for running backups
+with @code{restic}.")))