From patchwork Mon Mar 10 02:16:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Graves X-Patchwork-Id: 40037 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 50A5E27BBEB; Mon, 10 Mar 2025 02:19:53 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-6.4 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED, RCVD_IN_VALIDITY_CERTIFIED,RCVD_IN_VALIDITY_RPBL,RCVD_IN_VALIDITY_SAFE, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 61B9127BBE2 for ; Mon, 10 Mar 2025 02:19:50 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1trSiq-0002Mt-1Z; Sun, 09 Mar 2025 22:18:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1trSie-00029J-0l for guix-patches@gnu.org; Sun, 09 Mar 2025 22:18:19 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1trSid-0007sZ-Mc for guix-patches@gnu.org; Sun, 09 Mar 2025 22:18:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:References:In-Reply-To:Date:From:To:Subject; bh=F1rGNpJv/wQ3RbF6DRnauEIP+qMT8DNXmRkoFJkn9rk=; b=JFoBs9oPALKCvbEqI2so8R7m1F7S/vlG6P7p3q/qh8/pvYvvKrRJVvGmkTtqASU4zQvUysqtRaTk5z9fXPl3hu/ihprNemyQ2JoR7U9XNwUdLDCU4CVMBTVIDtwGqsQ6no+JWl7uLd30+e2tPVyXXI/y0MvqKVkZ2mCHlICVjCWVXq6uEC9vYUbVYbclwv0davEOAYlWOnciJzhzIkGI9NmEWuRpZfRO+Fu6a7X6fMKBrjrpQVeFZiNN4AUFmBbVs7Vvq0PzJy/BSHZl7oYF/nRWYuVmPe7VHfVndaw02sqrsJ8Oq5SC+bVQb4zXmiLTHIetpAJPXuNYmuIX58MHLg==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1trSid-0001NG-Gi for guix-patches@gnu.org; Sun, 09 Mar 2025 22:18:15 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#76835] [PATCH v3 36/40] gnu: Add snakemake. Resent-From: Nicolas Graves Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 10 Mar 2025 02:18:15 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 76835 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 76835@debbugs.gnu.org Cc: Nicolas Graves Received: via spool by 76835-submit@debbugs.gnu.org id=B76835.17415730885163 (code B ref 76835); Mon, 10 Mar 2025 02:18:15 +0000 Received: (at 76835) by debbugs.gnu.org; 10 Mar 2025 02:18:08 +0000 Received: from localhost ([127.0.0.1]:35486 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1trSiQ-0001Ja-As for submit@debbugs.gnu.org; Sun, 09 Mar 2025 22:18:08 -0400 Received: from 8.mo581.mail-out.ovh.net ([46.105.77.114]:37085) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1trShu-0001Fw-Ac for 76835@debbugs.gnu.org; Sun, 09 Mar 2025 22:17:31 -0400 Received: from director7.ghost.mail-out.ovh.net (unknown [10.108.25.152]) by mo581.mail-out.ovh.net (Postfix) with ESMTP id 4ZB0sP201Hz1HCZ for <76835@debbugs.gnu.org>; Mon, 10 Mar 2025 02:17:29 +0000 (UTC) Received: from ghost-submission-5b5ff79f4f-68m45 (unknown [10.111.182.110]) by director7.ghost.mail-out.ovh.net (Postfix) with ESMTPS id 022001FDED; Mon, 10 Mar 2025 02:17:28 +0000 (UTC) Received: from ngraves.fr ([37.59.142.112]) by ghost-submission-5b5ff79f4f-68m45 with ESMTPSA id dkyjJ7hLzmflEwAAdZqLTQ (envelope-from ); Mon, 10 Mar 2025 02:17:28 +0000 Authentication-Results: garm.ovh; auth=pass (GARM-112S0068028d7bf-c50e-4a5b-af62-e3b9a21fe067, 031A57722FDA369A3035EBF391DEEA525A5E801E) smtp.auth=ngraves@ngraves.fr X-OVh-ClientIp: 90.92.117.144 Date: Mon, 10 Mar 2025 03:16:41 +0100 Message-ID: <20250310021652.17530-36-ngraves@ngraves.fr> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250310021652.17530-1-ngraves@ngraves.fr> References: <20250310021652.17530-1-ngraves@ngraves.fr> MIME-Version: 1.0 X-Ovh-Tracer-Id: 5456392423789093602 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgdduudekuddvucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecuhedttdenucenucfjughrpefhvfevufffkffojghfgggtgfesthekredtredtjeenucfhrhhomheppfhitgholhgrshcuifhrrghvvghsuceonhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrqeenucggtffrrghtthgvrhhnpeekgeduheetffefffeuudevgfegffejkefhheegleevtefgueduuedvjeefhfegueenucffohhmrghinheprhgvrgguthhhvgguohgtshdrihhonecukfhppeduvdejrddtrddtrddupdeltddrledvrdduudejrddugeegpdefjedrheelrddugedvrdduuddvnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepuddvjedrtddrtddruddpmhgrihhlfhhrohhmpehnghhrrghvvghssehnghhrrghvvghsrdhfrhdpnhgspghrtghpthhtohepuddprhgtphhtthhopeejieekfeehseguvggssghughhsrdhgnhhurdhorhhgpdfovfetjfhoshhtpehmohehkedumgdpmhhouggvpehsmhhtphhouhht DKIM-Signature: a=rsa-sha256; bh=F1rGNpJv/wQ3RbF6DRnauEIP+qMT8DNXmRkoFJkn9rk=; c=relaxed/relaxed; d=ngraves.fr; h=From; s=ovhmo4487190-selector1; t=1741573049; v=1; b=yQqiWrdG46+/X7LbgEAp/R46wuI0u6LCnxHvwzoJONpvkW2ftFMwfvETOWdOAst+tYYczBE2 oH6Tl7cMyqxLLohpsdiZVvHvt3hiIeq450vz2NBe6E+VnZY0+g2+pH2UzpnvPbTUbohbheIlyUA L21uIt/YN7LsyX2sUwAt7MHZcZnbZWaBgYeV0BE/UKy0z7VSy1ZP7l1EJd4PgIbKQS0kXONj3wL U5T55eqE/0FtniZ22axhnMLwpyCa7p60ct6RY2Y+Lblhi6BzpPXzHhTuNMwsHsidiyS7YTtLHel 0EYwiZFetzGHO/HWv2h7l4ygH1rqzWvOFlmcfpf4dXD1w== X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Nicolas Graves X-ACL-Warn: , Nicolas Graves via Guix-patches X-Patchwork-Original-From: Nicolas Graves via Guix-patches via From: Nicolas Graves Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * gnu/packages/python-science.scm (snakemake): New variable. --- gnu/packages/python-science.scm | 122 ++++++++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) diff --git a/gnu/packages/python-science.scm b/gnu/packages/python-science.scm index ac16401254..7ccd991478 100644 --- a/gnu/packages/python-science.scm +++ b/gnu/packages/python-science.scm @@ -27,6 +27,7 @@ ;;; Copyright © 2024 Marco Baggio ;;; Copyright © 2024 Nicolas Graves ;;; Copyright © 2024 Rick Huijzer +;;; Copyright © 2025 Nicolas Graves ;;; ;;; This file is part of GNU Guix. ;;; @@ -67,6 +68,7 @@ (define-module (gnu packages python-science) #:use-module (gnu packages maths) #:use-module (gnu packages mpi) #:use-module (gnu packages pcre) + #:use-module (gnu packages package-management) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) #:use-module (gnu packages python) @@ -82,6 +84,7 @@ (define-module (gnu packages python-science) #:use-module (gnu packages ssh) #:use-module (gnu packages statistics) #:use-module (gnu packages time) + #:use-module (gnu packages version-control) #:use-module (gnu packages xdisorg) #:use-module (gnu packages xml) #:use-module (gnu packages xorg) @@ -4410,6 +4413,125 @@ (define-public python-libneuroml neuronal models") (license license:bsd-3))) +(define-public snakemake + (package + (name "snakemake") + (version "8.29.2") + (source + (origin + (method url-fetch) + (uri (pypi-uri "snakemake" version)) + (sha256 + (base32 "1ilpmrjmnc529p4gw2x23ik1d8b5pm6k1dhq08dknvfjsf3vgyjr")))) + (build-system pyproject-build-system) + (arguments + (list + #:test-flags + #~(list + ;; XXX: Unclear why these tests fail. + "--ignore=tests/test_report_href/test_script.py" + "--ignore=tests/test_script_py/scripts/test_explicit_import.py" + "--ignore=tests/test_output_index.py" + ;; We don't care about testing old python@3.7 on Guix. + "--ignore=tests/test_conda_python_3_7_script/test_script.py" + ;; Those require additional snakemake plugins. + "--ignore=tests/test_api.py" + "--ignore=tests/test_executor_test_suite.py" + ;; We don't care about lints. + "--ignore=tests/test_linting.py" + ;; These tests attempt to change S3 buckets on AWS and fail + ;; because there are no AWS credentials. + "--ignore=tests/test_tibanna" + ;; It's a similar story with this test, which requires access + ;; to the Google Storage service. + "--ignore=tests/test_google_lifesciences") + #:phases + #~(modify-phases %standard-phases + (add-after 'unpack 'avoid-assets-download + (lambda _ + (substitute* "setup.py" + (("^from assets import Assets") "") + (("^Assets\\.deploy\\(\\)") "")))) + ;; For cluster execution Snakemake will call Python. Since there is + ;; no suitable GUIX_PYTHONPATH set, cluster execution will fail. We + ;; fix this by calling the snakemake wrapper instead. + (add-after 'unpack 'call-wrapper-not-wrapped-snakemake + (lambda _ + (substitute* "snakemake/executors/__init__.py" + (("self\\.get_python_executable\\(\\),") + "") + (("\"-m snakemake\"") + (string-append "\"" #$output + "/bin/snakemake" "\"")) + ;; The snakemake command produced by format_job_exec contains + ;; references to /gnu/store. Prior to patching above that's + ;; just a reference to Python; after patching it's a reference + ;; to the snakemake executable. + ;; + ;; In Tibanna execution mode Snakemake arranges for a certain + ;; Docker image to be deployed to AWS. It then passes its own + ;; command line to Tibanna. This is misguided because it only + ;; ever works if the local Snakemake command was run inside + ;; the same Docker image. In the case of using Guix this is + ;; never correct, so we need to replace the store reference. + (("tibanna_args.command = command") + (string-append + "tibanna_args.command = command.replace('" + #$output "/bin/snakemake', 'python3 -m snakemake')"))))) + (add-after 'unpack 'patch-version + (lambda _ + (substitute* "setup.py" + (("version=versioneer.get_version\\(\\)") + (format #f "version=~s" #$version))) + (substitute* '("snakemake/_version.py" + "versioneer.py") + (("0\\+unknown") #$version)))) + (add-before 'check 'pre-check + (lambda* (#:key tests? #:allow-other-keys) + (when tests? + (setenv "HOME" "/tmp"))))))) + (inputs + (list python-appdirs + python-conda-inject + python-configargparse + python-connection-pool + python-docutils + python-dpath + python-gitpython + python-humanfriendly + python-immutables + python-jinja2 + python-jsonschema + python-nbformat + python-packaging + python-psutil + python-pulp + python-pyyaml + python-requests + python-reretry + python-smart-open + python-snakemake-interface-common + python-snakemake-interface-executor-plugins + python-snakemake-interface-report-plugins + python-snakemake-interface-storage-plugins + python-tabulate + python-throttler + python-wrapt + python-yte)) + (native-inputs + (list python-numpy + python-pandas + python-setuptools + python-tomli + python-wheel)) + (home-page "https://snakemake.readthedocs.io") + (synopsis "Python-based execution environment for make-like workflows") + (description + "Snakemake aims to reduce the complexity of creating workflows by +providing a clean and modern domain specific specification language (DSL) in +Python style, together with a fast and comfortable execution environment.") + (license license:expat))) + ;;; ;;; Avoid adding new packages to the end of this file. To reduce the chances ;;; of a merge conflict, place them above by existing packages with similar