From patchwork Sun Jan 19 22:04:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: paul X-Patchwork-Id: 37169 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 018E727BBE9; Sun, 19 Jan 2025 22:05:20 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-6.4 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED, RCVD_IN_VALIDITY_CERTIFIED,RCVD_IN_VALIDITY_RPBL,RCVD_IN_VALIDITY_SAFE, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id E04C627BBE2 for ; Sun, 19 Jan 2025 22:05:17 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tZdPo-00040K-1y; Sun, 19 Jan 2025 17:05:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tZdPl-0003zw-CS for guix-patches@gnu.org; Sun, 19 Jan 2025 17:05:06 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tZdPi-00008a-Dn; Sun, 19 Jan 2025 17:05:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:From:To:In-Reply-To:References:Subject; bh=Z/4vWsWRY35iW7yItPwCX4x+WW7Mg4MHy4+T5WA0Rrs=; b=hzS/x9zarbbBmyd374Ld5g4KYso2bi+Gv8tMa7seeF2zVATvhw8AsrFasobZI4xYr3l2oNWj9Mb2ynAC4Kj3aXf2qRuuL7Pe2IiaA2PyErtda3quW4UOqNzGj2WeTirJ5XKF89ZrxgLUgISXXH8+Kl7g9YdWftPrw3vqAHGCcSeNqIgurj52lQuMPj43Y7hcRSOR36EU2Fl5pwEnf+wCnuaWSIZAHl3vC2h8orrUAhTNgOmRW2iG6yWscUIlRiezChMEPb2tggesn//yjE3qD4uXozSDfgMLn0iA/DeoYzHZhy/2eg4w2Idiw95eHUdj/wM7ZxPktA+GuJYjFK3MGg==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1tZdPi-0006kC-84; Sun, 19 Jan 2025 17:05:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#75045] [PATCH v2] services: restic-backup: Implement as a Shepherd timer. References: In-Reply-To: Resent-From: Giacomo Leidi Original-Sender: "Debbugs-submit" Resent-CC: ludo@gnu.org, maxim.cournoyer@gmail.com, guix-patches@gnu.org Resent-Date: Sun, 19 Jan 2025 22:05:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75045 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 75045@debbugs.gnu.org Cc: Giacomo Leidi , Ludovic =?utf-8?q?Court=C3=A8?= =?utf-8?q?s?= , Maxim Cournoyer X-Debbugs-Original-Xcc: Ludovic =?utf-8?q?Court=C3=A8s?= , Maxim Cournoyer Received: via spool by 75045-submit@debbugs.gnu.org id=B75045.173732428625895 (code B ref 75045); Sun, 19 Jan 2025 22:05:02 +0000 Received: (at 75045) by debbugs.gnu.org; 19 Jan 2025 22:04:46 +0000 Received: from localhost ([127.0.0.1]:47677 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tZdPR-0006jb-8O for submit@debbugs.gnu.org; Sun, 19 Jan 2025 17:04:45 -0500 Received: from confino.investici.org ([93.190.126.19]:47119) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tZdPN-0006jQ-MF for 75045@debbugs.gnu.org; Sun, 19 Jan 2025 17:04:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=autistici.org; s=stigmate; t=1737324280; bh=Z/4vWsWRY35iW7yItPwCX4x+WW7Mg4MHy4+T5WA0Rrs=; h=From:To:Cc:Subject:Date:From; b=VQAQNoM9lnLPsGPIA9/DjgHzma9dlxb3yxkQ9499GKJdXvRoz9HoOIZSPuYryQulg RSPCwkul/2kEbDISE7gsLWk4JpFTzshf0Jtmj6qiyIZiytyvdCfJXMPJtAiTjBCnUo W6kIz7FQJ35f7Av7vo1Ok3k2k3EkLXYIMv7agejc= Received: from mx1.investici.org (unknown [127.0.0.1]) by confino.investici.org (Postfix) with ESMTP id 4YbnZJ5S0sz10yH; Sun, 19 Jan 2025 22:04:40 +0000 (UTC) Received: from [93.190.126.19] (mx1.investici.org [93.190.126.19]) (Authenticated sender: goodoldpaul@autistici.org) by localhost (Postfix) with ESMTPSA id 4YbnZJ4MJ3z10xb; Sun, 19 Jan 2025 22:04:40 +0000 (UTC) Date: Sun, 19 Jan 2025 23:04:00 +0100 Message-ID: <9b62d9a0cd9f00b074a5871729286925788a0eab.1737324240.git.goodoldpaul@autistici.org> X-Mailer: git-send-email 2.47.1 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Giacomo Leidi X-ACL-Warn: , Giacomo Leidi via Guix-patches X-Patchwork-Original-From: Giacomo Leidi via Guix-patches via From: paul Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches This patch implements restic backup with Shepherd services. It is supposed not to break any existing setup. * gnu/services/backup.scm (restic-backup-job): Add Shepherd configuration options; (restic-backup-job->mcron-job): Replace with...; (restic-job-log-file): New procedure; (restic-backup-job->shepherd-service): New procedure; (restic-backup-activation): New procedure; (restic-backup-service-type): Replace mcron with Shepherd extension and add activation extension hook. * doc/guix.texi: Document it. Change-Id: I66de3b6a1cb6177f9e4ee0c2acf3013ecbcdd338 --- doc/guix.texi | 39 +++++++++---- gnu/services/backup.scm | 122 +++++++++++++++++++++++++++++++++------- 2 files changed, 131 insertions(+), 30 deletions(-) base-commit: 5e834c220e81fddb77a26e23cf0cd5055b866844 diff --git a/doc/guix.texi b/doc/guix.texi index 0015d739bb6..33552065fe3 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -111,7 +111,7 @@ Copyright @copyright{} 2022 John Kehayias@* Copyright @copyright{} 2022⁠–⁠2023 Bruno Victal@* Copyright @copyright{} 2022 Ivan Vilata-i-Balaguer@* -Copyright @copyright{} 2023-2024 Giacomo Leidi@* +Copyright @copyright{} 2023-2025 Giacomo Leidi@* Copyright @copyright{} 2022 Antero Mejr@* Copyright @copyright{} 2023 Karl Hallsby@* Copyright @copyright{} 2023 Nathaniel Nicandro@* @@ -42709,19 +42709,16 @@ Miscellaneous Services "/etc/guix/signing-key.sec")))))))))) @end lisp -Each @code{restic-backup-job} translates to an mcron job which sets the +Each @code{restic-backup-job} translates to a Shepherd timer which sets the @env{RESTIC_PASSWORD} environment variable by reading the first line of @code{password-file} and runs @command{restic backup}, creating backups using rclone of all the files listed in the @code{files} field. -The @code{restic-backup-service-type} installs as well @code{restic-guix} -to the system profile, a @code{restic} utility wrapper that allows for easier -interaction with the Guix configured backup jobs. For example the following -could be used to instantaneusly trigger a backup for the above shown -configuration, without waiting for the scheduled job: +The @code{restic-backup-service-type} provides the ability to instantaneously +trigger a backup with the @code{trigger} Shepherd action: @example -restic-guix backup remote-ftp +sudo herd trigger remote-ftp @end example @c %start of fragment @@ -42752,6 +42749,23 @@ Miscellaneous Services @item @code{user} (default: @code{"root"}) (type: string) The user used for running the current job. +@item @code{group} (default: @code{"root"}) (type: string) +The group used for running the current job. + +@item @code{log-file} (type: maybe-string) +The file system path to the log file for this job. By default the file will +have be @file{/var/log/restic-backup/JOB-NAME.log}, where @code{JOB-NAME} is the +name defined in the @code{name} field. + +@item @code{max-duration} (type: maybe-number) +The maximum duration in seconds that a job may last. Past +@code{max-duration} seconds, the job is forcefully terminated. + +@item @code{wait-for-termination?} (default: @code{#f}) (type: boolean) +Wait until the job has finished before considering executing it again; +otherwise, perform it strictly on every occurrence of event, at the risk of +having multiple instances running concurrently. + @item @code{repository} (type: string) The restic repository target of this job. @@ -42764,9 +42778,12 @@ Miscellaneous Services for the current job. @item @code{schedule} (type: gexp-or-string) -A string or a gexp that will be passed as time specification in the -mcron job specification (@pxref{Syntax, mcron job specifications,, -mcron,GNU@tie{}mcron}). +A string or a gexp representing the frequency of the backup. Gexp must +evaluate to @code{calendar-event} records or to strings. Strings must contain +Vixie cron date lines. + +@item @code{requirement} (default: @code{'()}) (type: list-of-symbols) +The list of Shepherd services that this backup job depends upon. @item @code{files} (default: @code{'()}) (type: list-of-lowerables) The list of files or directories to be backed up. It must be a list of diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm index 555e9fc9590..3dda6ca370c 100644 --- a/gnu/services/backup.scm +++ b/gnu/services/backup.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2024 Giacomo Leidi +;;; Copyright © 2024, 2025 Giacomo Leidi ;;; ;;; This file is part of GNU Guix. ;;; @@ -18,9 +18,10 @@ (define-module (gnu services backup) #:use-module (gnu packages backup) + #:use-module (gnu packages bash) #:use-module (gnu services) #:use-module (gnu services configuration) - #:use-module (gnu services mcron) + #:use-module (gnu services shepherd) #:use-module (guix build-system copy) #:use-module (guix gexp) #:use-module ((guix licenses) @@ -33,11 +34,16 @@ (define-module (gnu services backup) restic-backup-job-fields restic-backup-job-restic restic-backup-job-user + restic-backup-job-group + restic-backup-job-log-file + restic-backup-job-max-duration + restic-backup-job-wait-for-termination? restic-backup-job-name restic-backup-job-repository restic-backup-job-password-file restic-backup-job-schedule restic-backup-job-files + restic-backup-job-requirement restic-backup-job-verbose? restic-backup-job-extra-flags @@ -64,6 +70,12 @@ (define (lowerable? value) (define list-of-lowerables? (list-of lowerable?)) +(define list-of-symbols? + (list-of symbol?)) + +(define-maybe string) +(define-maybe number) + (define-configuration/no-serialization restic-backup-job (restic (package restic) @@ -71,6 +83,23 @@ (define-configuration/no-serialization restic-backup-job (user (string "root") "The user used for running the current job.") + (group + (string "root") + "The group used for running the current job.") + (log-file + (maybe-string) + "The file system path to the log file for this job. By default the file will +have be @file{/var/log/restic-backup/JOB-NAME.log}, where @code{JOB-NAME} is the +name defined in the @code{name} field.") + (max-duration + (maybe-number) + "The maximum duration in seconds that a job may last. Past +@code{max-duration} seconds, the job is forcefully terminated.") + (wait-for-termination? + (boolean #f) + "Wait until the job has finished before considering executing it again; +otherwise, perform it strictly on every occurrence of event, at the risk of +having multiple instances running concurrently.") (name (string) "A string denoting a name for this job.") @@ -84,9 +113,12 @@ (define-configuration/no-serialization restic-backup-job current job.") (schedule (gexp-or-string) - "A string or a gexp that will be passed as time specification in the mcron -job specification (@pxref{Syntax, mcron job specifications,, mcron, -GNU@tie{}mcron}).") + "A string or a gexp representing the frequency of the backup. Gexp must +evaluate to @code{calendar-event} records or to strings. Strings must contain +Vixie cron date lines.") + (requirement + (list-of-symbols '()) + "The list of Shepherd services that this backup job depends upon.") (files (list-of-lowerables '()) "The list of files or directories to be backed up. It must be a list of @@ -175,16 +207,59 @@ (define (restic-guix jobs) (main (command-line))))) -(define (restic-backup-job->mcron-job config) - (let ((user - (restic-backup-job-user config)) - (schedule - (restic-backup-job-schedule config)) - (name - (restic-backup-job-name config))) - #~(job #$schedule - #$(string-append "restic-guix backup " name) - #:user #$user))) +(define (restic-job-log-file job) + (let ((name (restic-backup-job-name job)) + (log-file (restic-backup-job-log-file job))) + (if (maybe-value-set? log-file) + log-file + (string-append "/var/log/restic-backup/" name ".log")))) + +(define (restic-backup-job->shepherd-service config) + (let ((schedule (restic-backup-job-schedule config)) + (name (restic-backup-job-name config)) + (user (restic-backup-job-user config)) + (group (restic-backup-job-group config)) + (max-duration (restic-backup-job-max-duration config)) + (wait-for-termination? (restic-backup-job-wait-for-termination? config)) + (log-file (restic-job-log-file config)) + (requirement (restic-backup-job-requirement config))) + (shepherd-service (provision `(,(string->symbol name))) + (requirement + `(user-processes file-systems ,@requirement)) + (documentation + "Run @code{restic} backed backups on a regular basis.") + (modules '((shepherd service timer))) + (start + #~(make-timer-constructor + (if (string? #$schedule) + (cron-string->calendar-event #$schedule) + #$schedule) + (command + (list + ;; We go through bash, instead of executing + ;; restic-guix directly, because the login shell + ;; gives us the correct user environment that some + ;; backends require, such as rclone. + (string-append #+bash-minimal "/bin/bash") + "-l" "-c" + (string-append "restic-guix backup " #$name)) + #:user #$user + #:group #$group + #:environment-variables + (list + (string-append + "HOME=" (passwd:dir (getpwnam #$user))))) + #:log-file #$log-file + #:wait-for-termination? #$wait-for-termination? + #:max-duration #$(and (maybe-value-set? max-duration) + max-duration))) + (stop + #~(make-timer-destructor)) + (actions (list (shepherd-action + (name 'trigger) + (documentation "Manually trigger a backup, +without waiting for the scheduled time.") + (procedure #~trigger-timer))))))) (define (restic-guix-wrapper-package jobs) (package @@ -212,15 +287,24 @@ (define restic-backup-service-profile (restic-guix-wrapper-package jobs)) '()))) +(define (restic-backup-activation config) + #~(for-each + (lambda (log-file) + (mkdir-p (dirname log-file))) + (list #$@(map restic-job-log-file + (restic-backup-configuration-jobs config))))) + (define restic-backup-service-type (service-type (name 'restic-backup) (extensions (list + (service-extension activation-service-type + restic-backup-activation) (service-extension profile-service-type restic-backup-service-profile) - (service-extension mcron-service-type + (service-extension shepherd-root-service-type (lambda (config) - (map restic-backup-job->mcron-job + (map restic-backup-job->shepherd-service (restic-backup-configuration-jobs config)))))) (compose concatenate) @@ -232,5 +316,5 @@ (define restic-backup-service-type jobs))))) (default-value (restic-backup-configuration)) (description - "This service configures @code{mcron} jobs for running backups -with @code{restic}."))) + "This service configures @code{Shepherd} timers for running backups +with restic.")))