From patchwork Sun Aug 4 22:15:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Elsing X-Patchwork-Id: 66732 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id A9E2C27BBE2; Sun, 4 Aug 2024 23:18:41 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-7.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_VALIDITY_CERTIFIED, RCVD_IN_VALIDITY_RPBL,RCVD_IN_VALIDITY_SAFE,SPF_HELO_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 1D22527BBE9 for ; Sun, 4 Aug 2024 23:18:39 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sajY6-0007hV-7Y; Sun, 04 Aug 2024 18:17:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sajY1-0007di-FK for guix-patches@gnu.org; Sun, 04 Aug 2024 18:17:53 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sajXy-0000tp-IK for guix-patches@gnu.org; Sun, 04 Aug 2024 18:17:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:References:In-Reply-To:Date:From:To:Subject; bh=XTydPedTlydxxfYhTu+Ro3lpkbW1H9G/UBZV+NpqXF8=; b=X/hf8qy2oVll/4SsPTPYwtE+j/tAwRLMUFAl9yqNJie1eZYk9AR1FqKSCODfjqsSBhmwhcEZ0bNlaYoLbtmcnExF0dFZa6dJhRchyNbJQJ67chrPwhQQy12kdjtLkZPZuaP476qEkLKSam/SvabioDg10lXgmvXR+Qa4Dy5eCAcmiEXRhzEAcQ/aVEboAUufeawir7rR6JUP+1GvikqFV8MLJL4F0ppm4q+6q58sUQDHPK1LZc31i7dSmFIlWcugUXhbohtj4z5L0OdS7QDdVpuJkwx4XqgQ1564dfoILJGf+cqltpuTcy6ZBZUfzliQMmI20AcwP36TPpGa5S+8mg==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1sajYK-0004Uc-3i for guix-patches@gnu.org; Sun, 04 Aug 2024 18:18:12 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#72471] [PATCH 16/24] gnu: onnx: Update to 1.16.2. Resent-From: David Elsing Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 04 Aug 2024 22:18:12 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 72471 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 72471@debbugs.gnu.org Cc: David Elsing , Andy Tai Received: via spool by 72471-submit@debbugs.gnu.org id=B72471.172280987517052 (code B ref 72471); Sun, 04 Aug 2024 22:18:12 +0000 Received: (at 72471) by debbugs.gnu.org; 4 Aug 2024 22:17:55 +0000 Received: from localhost ([127.0.0.1]:57527 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sajY2-0004Qr-Cv for submit@debbugs.gnu.org; Sun, 04 Aug 2024 18:17:55 -0400 Received: from mout01.posteo.de ([185.67.36.65]:33521) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sajXf-0004Lh-64 for 72471@debbugs.gnu.org; Sun, 04 Aug 2024 18:17:38 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 9DC24240027 for <72471@debbugs.gnu.org>; Mon, 5 Aug 2024 00:17:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1722809820; bh=9O8DWgHXbqoctU66EV0kVh51IJXCUImz/VnfbXIZtZo=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding:From; b=K/xLx/4PutfWRPgXRw1k02J0QSmkmmvPVSit6nmozSHVdd9/Fa6fOVhkhL1p34Hwq kjjrVc+wFQXpLosMEVrrye59epXH6UguVCNxz02L0sHD5e/uJokxZmBEIf8gNkYUoL 4nJfDrW4r8oxKDp0xhwHCt7ygemG+JSuW9Rp8s5W4z4MSFxsNyHOkGbwYKSE46/F45 MBoRDWyKES4FTuW6aEPOFdgWR397JhfPd9af3M22A+bwgdll7jEauanr+GqDCItDc+ mSh9dRRrXYq02u8OS1bK1TSFc+skbjaxQQgngx/5Ao2EcyV5t2mgLy/ECTiq4MoI/K hBOGDPwZ8BnqQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4WcYp40VL0z6tlh; Mon, 5 Aug 2024 00:17:00 +0200 (CEST) From: David Elsing Date: Sun, 4 Aug 2024 22:15:54 +0000 Message-ID: <20240804221635.3805-16-david.elsing@posteo.net> In-Reply-To: <20240804220847.15842-1-david.elsing@posteo.net> References: <20240804220847.15842-1-david.elsing@posteo.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * gnu/packages/machine-learning.scm (onnx): Update to 1.16.2. [source]: Remove patch. [build-system]: Move to pyproject-build-system. [arguments]<#:tests?>: Disable for systems where python-nbval is not supported. <#:phases>: Remove 'relax-requirements' phase. Pass "-DONNX_BUILD_TESTS" to CMake depending on tests?. [native-inputs]: Add python-fb-re2 and python-parameterized-next. Remove python-nbval on unsupported systems. * gnu/packages/patches/onnx-shared-libraries.patch: Adjust patch. * gnu/packages/patches/onnx-skip-model-downloads.patch: Adjust patch. * gnu/packages/patches/onnx-use-system-googletest.patch: Remove file. * gnu/local.mk (dist_patch_DATA): Remove it. Co-authored-by: Andy Tai --- gnu/local.mk | 1 - gnu/packages/machine-learning.scm | 139 ++++++++++-------- .../patches/onnx-shared-libraries.patch | 18 ++- .../patches/onnx-skip-model-downloads.patch | 16 +- .../patches/onnx-use-system-googletest.patch | 57 ------- 5 files changed, 97 insertions(+), 134 deletions(-) delete mode 100644 gnu/packages/patches/onnx-use-system-googletest.patch diff --git a/gnu/local.mk b/gnu/local.mk index fd8262de55..61fb2f4816 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -1792,7 +1792,6 @@ dist_patch_DATA = \ %D%/packages/patches/onionshare-cli-async-mode.patch \ %D%/packages/patches/online-judge-tools.patch \ %D%/packages/patches/onnx-optimizer-system-library.patch \ - %D%/packages/patches/onnx-use-system-googletest.patch \ %D%/packages/patches/onnx-1.13.1-use-system-googletest.patch \ %D%/packages/patches/onnx-shared-libraries.patch \ %D%/packages/patches/onnx-skip-model-downloads.patch \ diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index b1b11b5410..fff7a8c24a 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -28,6 +28,7 @@ ;;; Copyright © 2024 Timothee Mathieu ;;; Copyright © 2024 Spencer King ;;; Copyright © 2024 David Elsing +;;; Copyright © 2024 Andy Tai ;;; ;;; This file is part of GNU Guix. ;;; @@ -1331,7 +1332,7 @@ (define (delete-ifdefs file) (define-public onnx (package (name "onnx") - (version "1.12.0") + (version "1.16.2") (source (origin (method git-fetch) (uri (git-reference @@ -1339,70 +1340,88 @@ (define-public onnx (commit (string-append "v" version)))) (sha256 (base32 - "1g9f1hviksbn7gi6fnd0dsm7nf0w3yia0mjj33d9mggklrl0db6x")) + "0f5h204ksfz4ir3qq38ckxja1jfhf1vn5xzwrj83vkkbfjq6fv16")) (file-name (git-file-name name version)) - (patches (search-patches "onnx-use-system-googletest.patch" - "onnx-shared-libraries.patch" - "onnx-skip-model-downloads.patch")) + (patches (search-patches + "onnx-shared-libraries.patch" + "onnx-skip-model-downloads.patch")) (modules '((guix build utils))) (snippet '(delete-file-recursively "third_party")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments - '(#:phases (modify-phases %standard-phases - (add-after 'unpack 'relax-requirements - (lambda _ - ;; Does this difference really matter? - (substitute* "requirements.txt" - (("3.20.1") "3.20.2")))) - (add-before 'build 'pass-cmake-arguments - (lambda* (#:key outputs #:allow-other-keys) - ;; Pass options to the CMake-based build process. - (define out - (assoc-ref outputs "out")) - - (define args - ;; Copy arguments from 'cmake-build-system', plus ask - ;; for shared libraries. - (list "-DCMAKE_BUILD_TYPE=RelWithDebInfo" - (string-append "-DCMAKE_INSTALL_PREFIX=" out) - "-DCMAKE_INSTALL_LIBDIR=lib" - "-DCMAKE_INSTALL_RPATH_USE_LINK_PATH=TRUE" - (string-append "-DCMAKE_INSTALL_RPATH=" out - "/lib") - "-DCMAKE_VERBOSE_MAKEFILE=ON" - - "-DBUILD_SHARED_LIBS=ON")) - - ;; This environment variable is honored by 'setup.py', - ;; which passes it down to 'cmake'. - (setenv "CMAKE_ARGS" (string-join args)) - - ;; This one is honored by 'setup.py' and passed to 'make - ;; -j'. - (setenv "MAX_JOBS" - (number->string (parallel-job-count))))) - (add-before 'check 'make-test-directory-writable - (lambda _ - ;; Make things writable for tests. - (setenv "HOME" (getcwd)) - (for-each make-file-writable - (find-files "onnx/examples" "." - #:directories? #t)))) - (add-after 'install 'install-from-cmake - (lambda _ - ;; Run "make install" in the build tree 'setup.py' - ;; created for CMake so that libonnx.so, - ;; libonnx_proto.so, etc. are installed. - (invoke "make" "install" - "-C" ".setuptools-cmake-build")))))) + (list + ;; python-nbval depends transitively on Rust. + #:tests? + (->bool (member (or (%current-target-system) + (%current-system)) + (package-transitive-supported-systems python-nbval))) + #:phases + #~(modify-phases %standard-phases + (add-before 'build 'pass-cmake-arguments + (lambda* (#:key outputs tests? #:allow-other-keys) + ;; For derived package use + (substitute* "CMakeLists.txt" + (("set\\(ONNX_ROOT.*") + (string-append "set(ONNX_ROOT "#$(package-source this-package) ")\n")) + (("\\$\\{ROOT_DIR\\}(/tools.*)" _ rest) + (string-append "${PROJECT_SOURCE_DIR}" rest))) + ;; Pass options to the CMake-based build process. + (define out + (assoc-ref outputs "out")) + + (define args + ;; Copy arguments from 'cmake-build-system', plus ask + ;; for shared libraries. + (list "-DCMAKE_BUILD_TYPE=RelWithDebInfo" + (string-append "-DCMAKE_INSTALL_PREFIX=" out) + "-DCMAKE_INSTALL_LIBDIR=lib" + "-DCMAKE_INSTALL_RPATH_USE_LINK_PATH=TRUE" + (string-append "-DCMAKE_INSTALL_RPATH=" out + "/lib") + "-DCMAKE_VERBOSE_MAKEFILE=ON" + (string-append "-DONNX_BUILD_TESTS=" + (if tests? "ON" "OFF")) + "-DBUILD_SHARED_LIBS=ON" + "-DONNX_USE_PROTOBUF_SHARED_LIBS=ON")) + + ;; This environment variable is honored by 'setup.py', + ;; which passes it down to 'cmake'. + (setenv "CMAKE_ARGS" (string-join args)) + + ;; This one is honored by 'setup.py' and passed to 'make + ;; -j'. + (setenv "MAX_JOBS" + (number->string (parallel-job-count))))) + (add-before 'check 'make-test-directory-writable + (lambda _ + ;; Make things writable for tests. + (setenv "HOME" (getcwd)) + (for-each make-file-writable + (find-files "onnx/examples" "." + #:directories? #t)))) + (add-after 'install 'install-from-cmake + (lambda _ + ;; Run "make install" in the build tree 'setup.py' + ;; created for CMake so that libonnx.so, + ;; libonnx_proto.so, etc. are installed. + (invoke "make" "install" + "-C" ".setuptools-cmake-build")))))) (native-inputs - (list cmake - googletest - pybind11 - python-coverage - python-nbval - python-pytest - python-pytest-runner)) + (append + (list cmake-minimal + googletest + pybind11 + python-coverage + python-fb-re2 + python-parameterized-next + python-pytest + python-pytest-runner) + (filter + (lambda (pkg) + (member (or (%current-target-system) + (%current-system)) + (package-transitive-supported-systems pkg))) + (list python-nbval)))) (inputs (list protobuf)) (propagated-inputs diff --git a/gnu/packages/patches/onnx-shared-libraries.patch b/gnu/packages/patches/onnx-shared-libraries.patch index 00583b35da..5a3fd658d0 100644 --- a/gnu/packages/patches/onnx-shared-libraries.patch +++ b/gnu/packages/patches/onnx-shared-libraries.patch @@ -1,13 +1,12 @@ -These linker options for the 'onnx_cpp2py_export.cpython-38-*-gnu.so' -(or similar) extension are meant to be used when building 'libonn.a', -a static archive. This patch adapts the link flags to linking with -'libonnx.so'. +These linker options for the 'onnx_cpp2py_export.cpython-310-*-gnu.so' (or +similar) extension are meant to be used when building 'libonn.a', a static +archive. This patch adapts the link flags to linking with 'libonnx.so'. diff --git a/CMakeLists.txt b/CMakeLists.txt -index cede3073..52f846ed 100644 +index b666eec4..1525b219 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt -@@ -475,11 +475,10 @@ if(BUILD_ONNX_PYTHON) +@@ -585,16 +585,15 @@ if(BUILD_ONNX_PYTHON) PRIVATE $) else() # Assume everything else is like gcc @@ -15,10 +14,15 @@ index cede3073..52f846ed 100644 - PRIVATE "-Wl,--whole-archive" $ - "-Wl,--no-whole-archive") + target_link_libraries(onnx_cpp2py_export PRIVATE onnx) + # Prevent "undefined symbol: _ZNSt10filesystem7__cxx114path14_M_split_cmptsEv" + # (std::filesystem::__cxx11::path::_M_split_cmpts()) on gcc 8 + if (CMAKE_CXX_STANDARD EQUAL 17 AND CMAKE_COMPILER_IS_GNUCC AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 9.0) + target_link_libraries(onnx_cpp2py_export PRIVATE "-lstdc++fs") + endif() set_target_properties(onnx_cpp2py_export - PROPERTIES LINK_FLAGS "-Wl,--exclude-libs,ALL") + PROPERTIES LINK_FLAGS -+ "-Wl,-rpath=${CMAKE_INSTALL_PREFIX}/lib") ++ "-Wl,-rpath=${CMAKE_INSTALL_PREFIX}/lib") endif() target_link_libraries(onnx_cpp2py_export PRIVATE onnx) diff --git a/gnu/packages/patches/onnx-skip-model-downloads.patch b/gnu/packages/patches/onnx-skip-model-downloads.patch index 4ab55b4ceb..55925a3ebf 100644 --- a/gnu/packages/patches/onnx-skip-model-downloads.patch +++ b/gnu/packages/patches/onnx-skip-model-downloads.patch @@ -1,16 +1,14 @@ -A few tests require downloading models from URLs such as - . -Skip those. +A few tests require downloading models. Skip those. diff --git a/onnx/backend/test/runner/__init__.py b/onnx/backend/test/runner/__init__.py -index 049ed57b..f236f1bf 100644 +index 5b60e7c0..838c7ba5 100644 --- a/onnx/backend/test/runner/__init__.py +++ b/onnx/backend/test/runner/__init__.py -@@ -202,6 +202,7 @@ class Runner(object): - print('Start downloading model {} from {}'.format( - model_test.model_name, - model_test.url)) +@@ -236,6 +236,7 @@ class Runner: + print( + f"Start downloading model {model_test.model_name} from {model_test.url}" + ) + raise unittest.SkipTest('Skipping download') urlretrieve(model_test.url, download_file.name) - print('Done') + print("Done") with tarfile.open(download_file.name) as t: diff --git a/gnu/packages/patches/onnx-use-system-googletest.patch b/gnu/packages/patches/onnx-use-system-googletest.patch deleted file mode 100644 index 5dfcbc6dc3..0000000000 --- a/gnu/packages/patches/onnx-use-system-googletest.patch +++ /dev/null @@ -1,57 +0,0 @@ -ONNX will build googletest from a Git checkout. Patch CMake to use our -googletest package and enable tests by default. - -diff --git a/CMakeLists.txt b/CMakeLists.txt -index 0aa9fda2..a573170c 100644 ---- a/CMakeLists.txt -+++ b/CMakeLists.txt -@@ -22,7 +22,7 @@ option(BUILD_ONNX_PYTHON "Build Python binaries" OFF) - option(ONNX_GEN_PB_TYPE_STUBS "Generate protobuf python type stubs" ON) - option(ONNX_WERROR "Build with Werror" OFF) - option(ONNX_COVERAGE "Build with coverage instrumentation" OFF) --option(ONNX_BUILD_TESTS "Build ONNX C++ APIs Tests" OFF) -+option(ONNX_BUILD_TESTS "Build ONNX C++ APIs Tests" ON) - option(ONNX_USE_LITE_PROTO "Use lite protobuf instead of full." OFF) - option(ONNXIFI_ENABLE_EXT "Enable onnxifi extensions." OFF) - if(NOT DEFINED ONNX_ML) -@@ -82,8 +82,10 @@ if(CMAKE_SYSTEM_NAME STREQUAL "AIX") - endif() - - if(ONNX_BUILD_TESTS) -- list(APPEND CMAKE_MODULE_PATH ${ONNX_ROOT}/cmake/external) -- include(googletest) -+ find_package(GTest REQUIRED) -+ if(NOT GTest_FOUND) -+ message(FATAL_ERROR "cannot find googletest") -+ endif() - endif() - - if((ONNX_USE_LITE_PROTO AND TARGET protobuf::libprotobuf-lite) OR ((NOT ONNX_USE_LITE_PROTO) AND TARGET protobuf::libprotobuf)) -diff --git a/cmake/unittest.cmake b/cmake/unittest.cmake -index e29a93ff..ae146390 100644 ---- a/cmake/unittest.cmake -+++ b/cmake/unittest.cmake -@@ -6,8 +6,8 @@ include(${ONNX_ROOT}/cmake/Utils.cmake) - - find_package(Threads) - --set(${UT_NAME}_libs ${googletest_STATIC_LIBRARIES}) --set(${ONNXIFI_TEST_DRIVER}_libs ${googletest_STATIC_LIBRARIES}) -+set(${UT_NAME}_libs ${GTEST_LIBRARIES}) -+set(${ONNXIFI_TEST_DRIVER}_libs ${GTEST_LIBRARIES}) - - list(APPEND ${UT_NAME}_libs onnx) - list(APPEND ${UT_NAME}_libs onnx_proto) -@@ -31,10 +31,10 @@ function(AddTest) - list(REMOVE_DUPLICATES _UT_SOURCES) - - add_executable(${_UT_TARGET} ${_UT_SOURCES}) -- add_dependencies(${_UT_TARGET} onnx onnx_proto googletest) -+ add_dependencies(${_UT_TARGET} onnx onnx_proto) - - target_include_directories(${_UT_TARGET} -- PUBLIC ${googletest_INCLUDE_DIRS} -+ PUBLIC ${GTEST_INCLUDE_DIRS} - ${ONNX_INCLUDE_DIRS} - ${PROTOBUF_INCLUDE_DIRS} - ${ONNX_ROOT}