From patchwork Fri Aug 30 14:25:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Marius Bakke X-Patchwork-Id: 15186 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 39E0517368; Fri, 30 Aug 2019 15:45:08 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id C2CD317365 for ; Fri, 30 Aug 2019 15:45:03 +0100 (BST) Received: from localhost ([::1]:32878 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i3i97-0004dG-4a for patchwork@mira.cbaines.net; Fri, 30 Aug 2019 10:45:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57645) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i3hs2-0002dS-GB for guix-patches@gnu.org; Fri, 30 Aug 2019 10:27:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i3hry-0003Aw-BV for guix-patches@gnu.org; Fri, 30 Aug 2019 10:27:21 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:45861) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1i3hrv-0002tc-3N for guix-patches@gnu.org; Fri, 30 Aug 2019 10:27:18 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1i3hrt-0003sA-On for guix-patches@gnu.org; Fri, 30 Aug 2019 10:27:13 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#37234] [PATCH 21/21] gnu: Add mlflow. Resent-From: Marius Bakke Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 30 Aug 2019 14:27:13 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 37234 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 37234@debbugs.gnu.org Received: via spool by 37234-submit@debbugs.gnu.org id=B37234.156717521914726 (code B ref 37234); Fri, 30 Aug 2019 14:27:13 +0000 Received: (at 37234) by debbugs.gnu.org; 30 Aug 2019 14:26:59 +0000 Received: from localhost ([127.0.0.1]:54663 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i3hrV-0003pB-Bl for submit@debbugs.gnu.org; Fri, 30 Aug 2019 10:26:58 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:49671) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i3hr2-0003n4-B3 for 37234@debbugs.gnu.org; Fri, 30 Aug 2019 10:26:24 -0400 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id D57664E2 for <37234@debbugs.gnu.org>; Fri, 30 Aug 2019 10:26:14 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Fri, 30 Aug 2019 10:26:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h= from:to:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=fm3; bh= oGyszr6apAkDvJT36iT1i6t/4gTJQl+nFFSThjVSJ4Y=; b=w9lY5ZN5L/CMVIeX eNDvTVI0rfhKXk9nKOQ0i8VIqs7b803CBFkR44OpUbf1Cx/CxZ7RauzhhbqFjWxI egoG0P2Z1nu3EuDgX49Amf1E5w/M9gR006q8n1XEyB4rBnt/dzYEPFsiCI8odGBd AsHiAXcWazIL3eF9MSnZehCz1iABlOX0xQeubarX2q1pi2eXWRO28BR4N0O5xpC6 pn5PDxbbwjkGcRqVep4WDB6941rJEXg7Hwj8APb6raY6BH5Av9JTitrYvoHq8Aeo NzlaL70TQQ2rWIufaVnOY/JTZ2RNls+FJRt8Z9ezdUQAd21KUrbE++RnAlGsIDcP lggD9w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=oGyszr6apAkDvJT36iT1i6t/4gTJQl+nFFSThjVSJ 4Y=; b=UQg0XhYsmVWd7y8/fi77WaloBon+841iyUtmCoxHU0WC9CQIUvofDFd+Q wjZPvwsNdT1A/r69pOKgVWJ2z6JiHXhEgfaVp5wP9ePtjZxGAyxkOpXvpIZc9pTz ok7WstRzVj0FAB997iaDMIalWAnCmRHynRcUjjuUuIvORSt8lhnn0gzJrA9XZT7D /k9zq5XT372cMDfm0W62+IBKCG5vPeSpAFkeZ+9LJxz8Hs1J1zMyGFMc19qB6Eho XZrIEjn7HUt18iEHcAXbVG2iFd9B8nNz5qBPp0K3hxW2JJb4FEuobOPpT3lJl/SF FHpEwJkrgacXWnxRb4s8InLBde+4Q== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudeigedgjeeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkofgjfhggtgfgsehtke ertdertdejnecuhfhrohhmpeforghrihhushcuuegrkhhkvgcuoehmsggrkhhkvgesfhgr shhtmhgrihhlrdgtohhmqeenucffohhmrghinhepmhhlfhhlohifrdhorhhgpdhgihhthh husgdrtghomhenucfkphepiedvrdduiedrudelvddrudehtdenucfrrghrrghmpehmrghi lhhfrhhomhepmhgsrghkkhgvsehfrghsthhmrghilhdrtghomhenucevlhhushhtvghruf hiiigvpedt X-ME-Proxy: Received: from localhost (ti0006q161-0149.bb.online.no [62.16.192.150]) by mail.messagingengine.com (Postfix) with ESMTPA id F39348005A for <37234@debbugs.gnu.org>; Fri, 30 Aug 2019 10:26:13 -0400 (EDT) From: Marius Bakke Date: Fri, 30 Aug 2019 16:25:39 +0200 Message-Id: <20190830142539.28376-21-mbakke@fastmail.com> X-Mailer: git-send-email 2.22.1 In-Reply-To: <20190830142539.28376-1-mbakke@fastmail.com> References: <20190830142539.28376-1-mbakke@fastmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/packages/machine-learning.scm (mlflow): New public variable. --- gnu/packages/machine-learning.scm | 122 +++++++++++++++++++++++++++++- 1 file changed, 121 insertions(+), 1 deletion(-) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index e48c91a2cf..19b1c7482b 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1,7 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2015, 2016, 2017, 2018, 2019 Ricardo Wurmus ;;; Copyright © 2016 Efraim Flashner -;;; Copyright © 2016, 2017 Marius Bakke +;;; Copyright © 2016, 2017, 2019 Marius Bakke ;;; Copyright © 2016 Hartmut Goebel ;;; Copyright © 2018, 2019 Tobias Geerinckx-Rice ;;; Copyright © 2018 Kei Kebreau @@ -52,6 +52,7 @@ #:use-module (gnu packages cran) #:use-module (gnu packages databases) #:use-module (gnu packages dejagnu) + #:use-module (gnu packages docker) #:use-module (gnu packages gcc) #:use-module (gnu packages glib) #:use-module (gnu packages graphviz) @@ -73,7 +74,9 @@ #:use-module (gnu packages statistics) #:use-module (gnu packages sqlite) #:use-module (gnu packages swig) + #:use-module (gnu packages time) #:use-module (gnu packages tls) + #:use-module (gnu packages version-control) #:use-module (gnu packages web) #:use-module (gnu packages xml) #:use-module (gnu packages xorg) @@ -354,6 +357,123 @@ networks) based on simulation of (stochastic) flow in graphs.") algorithm.") (license license:gpl3))) +(define-public mlflow + (package + (name "mlflow") + (version "1.2.0") + (source (origin + ;; The PyPI release lacks tests, so we take the git repository. + (method git-fetch) + (uri (git-reference + (url "https://github.com/mlflow/mlflow") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "0rn3anjkm0i6phyvh7xsrvxbhqp5dhvqf74kaa69q5hgy4y2y0py")))) + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (add-before 'check 'adjust-tests + (lambda _ + ;; TODO: It would be great to disable the individual test cases + ;; instead of deleting whole files, e.g. with + ;; (setenv "PYTEST_ADDOPTS" "-k not foo and not bar"). + (for-each delete-file + '(;; These tests require packages not yet available + ;; in Guix. + "tests/azureml/test_image_creation.py" + "tests/h2o/test_h2o_model_export.py" + "tests/onnx/test_onnx_model_export.py" + "tests/projects/test_docker_projects.py" + "tests/projects/test_kubernetes.py" + "tests/projects/test_projects.py" + "tests/pyfunc/test_spark.py" + "tests/pytorch/test_pytorch_model_export.py" + "tests/spark/test_spark_model_export.py" + "tests/store/test_azure_blob_artifact_repo.py" + "tests/store/test_gcs_artifact_repo.py" + "tests/store/test_sftp_artifact_repo.py" + "tests/tracking/test_tracking.py" + + ;; TODO: These require installing 'mlflow_test_plugin'. + "tests/server/test_handlers.py" + "tests/store/test_artifact_repository_registry.py" + "tests/tracking/test_utils.py" + "tests/tracking/context/test_registry.py" + + ;; This requires a running Docker daemon. + "tests/models/test_cli.py" + + ;; These require a standalone version of 'conda'. + "tests/pyfunc/test_scoring_server.py" + "tests/pyfunc/test_model_export_with_class_and_artifacts.py" + + ;; These tests need internet access. + "tests/projects/test_projects_cli.py" + "tests/tensorflow_autolog/test_tensorflow_autolog.py")) + + ;; The test script stores non-zero exit codes in an "$err" variable + ;; and checks it at the end of the run. We need to allow exit code 5 + ;; from pytest, meaning "no tests were collected", because the above + ;; deletions empties some of the directories given to pytest. + (substitute* "travis/run-large-python-tests.sh" + (("trap 'err=1' ERR") + "trap '[[ \"$?\" != \"5\" ]] && err=1' ERR")) + + #t)) + (delete 'check) + (add-after 'install 'check + (lambda* (#:key inputs outputs #:allow-other-keys) + (add-installed-pythonpath inputs outputs) + (setenv "PATH" + (string-append (assoc-ref outputs "out") + "/bin:" (getenv "PATH"))) + (invoke "./travis/run-large-python-tests.sh")))))) + (native-inputs + `( ;; For tests. + ("python-keras" ,python-keras) + ("python-moto" ,python-moto) + ("python-pyarrow" ,python-pyarrow) + ("python-pytest" ,python-pytest) + ("tensorflow" ,tensorflow))) + (propagated-inputs + `(("python-alembic" ,python-alembic) + ("python-boto3" ,python-boto3) + ("python-click" ,python-click) + ("python-cloudpickle" ,python-cloudpickle) + ("python-databricks-cli" ,python-databricks-cli) + ("python-dateutil" ,python-dateutil) + ("python-docker" ,python-docker-py) + ("python-entrypoints" ,python-entrypoints) + ("python-flask" ,python-flask) + ("python-gitpython" ,python-gitpython) + ("python-gorilla" ,python-gorilla) + ("python-gunicorn" ,python-gunicorn) + ("python-numpy" ,python-numpy) + ("python-pandas" ,python-pandas) + ("python-protobuf" ,python-protobuf-next) + ("python-pyyaml" ,python-pyyaml) + ("python-querystring-parser" ,python-querystring-parser) + ("python-requests" ,python-requests) + ("python-scikit-learn" ,python-scikit-learn) + ("python-simplejson" ,python-simplejson) + ("python-six" ,python-six) + ("python-sqlalchemy" ,python-sqlalchemy) + ("python-sqlparse" ,python-sqlparse) + ("python-waitress" ,python-waitress))) + (home-page "https://mlflow.org/") + (synopsis "Manage machine learning workflows") + (description + "MLflow is a platform to streamline machine learning development, +including tracking experiments, packaging code into reproducible runs, and +sharing and deploying models. MLflow offers a set of lightweight APIs that +can used with any existing machine learning application or library (TensorFlow, +PyTorch, XGBoost, etc), wherever you currently run ML code (e.g. in notebooks, +standalone applications or the cloud).") + (license license:asl2.0))) + (define-public randomjungle (package (name "randomjungle")