From patchwork Sun Aug 4 22:15:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Elsing X-Patchwork-Id: 66739 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 8076E27BBE9; Sun, 4 Aug 2024 23:19:03 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-7.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_VALIDITY_CERTIFIED, RCVD_IN_VALIDITY_RPBL,RCVD_IN_VALIDITY_SAFE,SPF_HELO_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 5615827BBEB for ; Sun, 4 Aug 2024 23:19:01 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sajY5-0007gm-Ei; Sun, 04 Aug 2024 18:17:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sajY1-0007dg-Ex for guix-patches@gnu.org; Sun, 04 Aug 2024 18:17:53 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sajXx-0000ti-Kg for guix-patches@gnu.org; Sun, 04 Aug 2024 18:17:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:References:In-Reply-To:Date:From:To:Subject; bh=m+eMODtXRZG2InATlVA2JSoN0i+nEVRZ1zp1Su2FkgU=; b=FKKRZpq6PREIsLQevu1WlJNHUUuDS2yPvs0aN2zYytSQBuKiIvKfWvgoncE4rRcoiLQkroqAfMwI5rYmmTJRGsUfyX8QRHdkkkA2vSeHlA4EiIH0FEIE75+lpg1ThIJA1HSUF5VemXDhm8Lly7Wa77Gi4gtz6apseeNar6lj7vS46zOnxncln+lrSCyNoSGexy0c+tMNMs49EiFUjPqQeOjnpz1DxEHQevMw1R4MAKjGAVw+JA+GT5mAnuv5v/vk869QyUuPcS0gfVVJA1jH63eQuB6y5rs6lUo7/qgG4a95rrEKgCzxkX/q4i+ZiYrOtbFBvZmc/IY+vmytoTCWLg==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1sajYJ-0004UL-5H for guix-patches@gnu.org; Sun, 04 Aug 2024 18:18:11 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#72471] [PATCH 17/24] gnu: onnx-optimizer: Update to 0.3.19. Resent-From: David Elsing Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 04 Aug 2024 22:18:11 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 72471 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 72471@debbugs.gnu.org Cc: David Elsing , Andy Tai Received: via spool by 72471-submit@debbugs.gnu.org id=B72471.172280987417037 (code B ref 72471); Sun, 04 Aug 2024 22:18:11 +0000 Received: (at 72471) by debbugs.gnu.org; 4 Aug 2024 22:17:54 +0000 Received: from localhost ([127.0.0.1]:57523 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sajY1-0004Qc-7J for submit@debbugs.gnu.org; Sun, 04 Aug 2024 18:17:53 -0400 Received: from mout01.posteo.de ([185.67.36.65]:33057) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sajXf-0004Li-5k for 72471@debbugs.gnu.org; Sun, 04 Aug 2024 18:17:36 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 2CC84240027 for <72471@debbugs.gnu.org>; Mon, 5 Aug 2024 00:17:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1722809821; bh=WLL3PaAPyxeKPsvaL8abFSnuTssAMOCYT0PGof8pp2M=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version: Content-Transfer-Encoding:From; b=MKUlfblDNUb+RWu9OjnnkBgpAUgP0laKDwOPKeU4WqNVmbfeNdoxR7i79sqcPWq5A zU3f/kelz9SKgRJ5bjBrbNhziZHZy11lz13sfig2huSZbseNyHrqRPJ2F3y6956SwB CdnVKiSK0ioz7PihnTSWmKVNKe1rglOZSrQPINMpj6ruYaYj/PqQx7zosSqM6tcl3S LB8U+xoPXJ84gr95P/822JqUoY2d4eFOBpQ/X9XbRJgW+V6it1H8yzvGMVskY5+abI SLsRYYZUXyRprCy6U8aFHe6dElH2LHQmjBLbvXmClmQiwdTvjkEs5rKpdLBOs7W2Un zLxAMgypi/tMw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4WcYp45nMHz6tlh; Mon, 5 Aug 2024 00:17:00 +0200 (CEST) From: David Elsing Date: Sun, 4 Aug 2024 22:15:55 +0000 Message-ID: <20240804221635.3805-17-david.elsing@posteo.net> In-Reply-To: <20240804220847.15842-1-david.elsing@posteo.net> References: <20240804220847.15842-1-david.elsing@posteo.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * gnu/packages/machine-learning.scm (onnx-optimizer): Update to 0.3.19. * gnu/packages/patches/onnx-optimizer-system-library.patch: Adjust patch. Co-authored-by: Andy Tai --- gnu/packages/machine-learning.scm | 47 +++++++++++---- .../onnx-optimizer-system-library.patch | 60 +++++++++---------- 2 files changed, 66 insertions(+), 41 deletions(-) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index fff7a8c24a..eb7315730a 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1361,8 +1361,7 @@ (define-public onnx (lambda* (#:key outputs tests? #:allow-other-keys) ;; For derived package use (substitute* "CMakeLists.txt" - (("set\\(ONNX_ROOT.*") - (string-append "set(ONNX_ROOT "#$(package-source this-package) ")\n")) + (("set\\(ONNX_ROOT.*") "") (("\\$\\{ROOT_DIR\\}(/tools.*)" _ rest) (string-append "${PROJECT_SOURCE_DIR}" rest))) ;; Pass options to the CMake-based build process. @@ -1382,14 +1381,15 @@ (define args (string-append "-DONNX_BUILD_TESTS=" (if tests? "ON" "OFF")) "-DBUILD_SHARED_LIBS=ON" - "-DONNX_USE_PROTOBUF_SHARED_LIBS=ON")) + "-DONNX_USE_PROTOBUF_SHARED_LIBS=ON" + (string-append + "-DONNX_ROOT=" #$(package-source this-package)))) ;; This environment variable is honored by 'setup.py', ;; which passes it down to 'cmake'. (setenv "CMAKE_ARGS" (string-join args)) - ;; This one is honored by 'setup.py' and passed to 'make - ;; -j'. + ;; This one is honored by 'setup.py' and passed to 'make -j'. (setenv "MAX_JOBS" (number->string (parallel-job-count))))) (add-before 'check 'make-test-directory-writable @@ -1445,8 +1445,8 @@ (define-public python-onnx (define-public onnx-optimizer (package (name "onnx-optimizer") - ;; Note: 0.2.x is *more* recent than 1.5.0. - (version "0.2.6") + ;; Note: 0.3.x is *more* recent than 1.5.0. + (version "0.3.19") (home-page "https://github.com/onnx/optimizer") (source (origin (method git-fetch) @@ -1455,7 +1455,7 @@ (define-public onnx-optimizer (commit (string-append "v" version)))) (sha256 (base32 - "1wkqqdxcxpfbf8zpbdfdd3zz5jkw775g31gyykj11z4y6pp659l6")) + "1mx3hsl42na6fr05nh2x3j9kxm56cpfmwk6lwl2cfq9zs3gv929w")) (file-name (git-file-name name version)) (patches (search-patches "onnx-optimizer-system-library.patch")) (modules '((guix build utils))) @@ -1466,10 +1466,35 @@ (define-public onnx-optimizer (substitute-keyword-arguments (package-arguments onnx) ((#:phases phases) #~(modify-phases #$phases - (delete 'relax-requirements))))) + (add-after 'pass-cmake-arguments + 'pass-onnx-optimizer-cmake-arguments + (lambda _ + (setenv + "CMAKE_ARGS" + (string-append + (getenv "CMAKE_ARGS") + " -DONNX_OPT_USE_SYSTEM_PROTOBUF=ON" + " -DCMAKE_CXX_FLAGS=\"-DONNX_ML=1 -DONNX_NAMESPACE=onnx\"")))) + (replace 'check + (lambda* (#:key tests? #:allow-other-keys) + (if tests? + (invoke "pytest" "-vv" "-k" + ;; These tests fail with upstream ONNX: + ;; https://github.com/onnx/optimizer/issues/138 + (string-append + "not test_fuse_matmul" + " and not test_fuse_consecutive" + " and not test_fuse_transpose"))))))))) (native-inputs - (list cmake python-pytest python-pytest-runner python-nbval - python-coverage)) + (append + (list cmake-minimal python-pytest python-pytest-runner + python-coverage) + (filter + (lambda (pkg) + (member (or (%current-target-system) + (%current-system)) + (package-transitive-supported-systems pkg))) + (list python-nbval)))) (inputs (list onnx protobuf pybind11)) (propagated-inputs diff --git a/gnu/packages/patches/onnx-optimizer-system-library.patch b/gnu/packages/patches/onnx-optimizer-system-library.patch index 5c592597e0..513623fe05 100644 --- a/gnu/packages/patches/onnx-optimizer-system-library.patch +++ b/gnu/packages/patches/onnx-optimizer-system-library.patch @@ -1,53 +1,53 @@ -Arrange so that onnx-optimizer (1) uses our own ONNX build, -(2) builds as a shared library, and (3) links against the shared -libraries of ONNX. +Arrange so that onnx-optimizer uses our own ONNX build, and links against the +shared libraries of ONNX. diff --git a/CMakeLists.txt b/CMakeLists.txt -index c2e48b35..8af51076 100644 +index 00633856..76a73853 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt -@@ -9,8 +9,6 @@ endif(NOT MSVC) +@@ -27,8 +27,6 @@ if(NOT ONNX_OPT_USE_SYSTEM_PROTOBUF) + endif() - set(CMAKE_POSITION_INDEPENDENT_CODE ON) -set(ONNX_ROOT ${PROJECT_SOURCE_DIR}/third_party/onnx) --add_subdirectory(${ONNX_ROOT}) +-add_subdirectory_if_no_target(${ONNX_ROOT} ${ONNX_TARGET_NAME}) file(READ "${PROJECT_SOURCE_DIR}/VERSION_NUMBER" ONNX_OPTIMIZER_VERSION) string(STRIP "${ONNX_OPTIMIZER_VERSION}" ONNX_OPTIMIZER_VERSION) -@@ -21,14 +19,18 @@ file(GLOB_RECURSE onnx_opt_srcs "onnxoptimizer/*.cc" +@@ -41,14 +39,14 @@ file(GLOB onnx_opt_srcs "onnxoptimizer/*.cc" list(REMOVE_ITEM onnx_opt_srcs "${PROJECT_SOURCE_DIR}/onnxoptimizer/cpp2py_export.cc") - add_library(onnx_optimizer ${onnx_opt_srcs}) --target_link_libraries(onnx_optimizer PUBLIC onnx) -+target_link_libraries(onnx_optimizer PUBLIC onnx onnx_proto) + onnxopt_add_library(onnx_optimizer ${onnx_opt_srcs}) +-target_link_libraries(onnx_optimizer PUBLIC ${ONNX_TARGET_NAME}) ++target_link_libraries(onnx_optimizer PUBLIC ${ONNX_TARGET_NAME} onnx_proto) target_include_directories(onnx_optimizer PUBLIC $ $ ) -+# These cpp macros must be defined so the ONNX headers behave -+# correctly. -+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -DONNX_ML=1 -DONNX_NAMESPACE=onnx") -+ - add_executable(onnx_optimizer_exec examples/onnx_optimizer_exec.cpp) + onnxopt_add_executable(onnx_optimizer_exec examples/onnx_optimizer_exec.cpp) -target_link_libraries(onnx_optimizer_exec onnx_optimizer) -+target_link_libraries(onnx_optimizer_exec onnx_optimizer protobuf) - - if(BUILD_ONNX_PYTHON) - if("${PY_EXT_SUFFIX}" STREQUAL "") -@@ -79,11 +81,10 @@ if(BUILD_ONNX_PYTHON) - PRIVATE $) - else() - # Assume everything else is like gcc -- target_link_libraries(onnx_opt_cpp2py_export -- PRIVATE "-Wl,--whole-archive" $ -- "-Wl,--no-whole-archive") -+ target_link_libraries(onnx_opt_cpp2py_export PRIVATE onnx_optimizer) ++target_link_libraries(onnx_optimizer_exec onnx_optimizer onnx_proto) + + + file(GLOB onnx_opt_c_api_srcs "onnxoptimizer/c_api/*.cc" +@@ -116,7 +114,8 @@ if(BUILD_ONNX_PYTHON) + PRIVATE "-Wl,--whole-archive" $ + "-Wl,--no-whole-archive") set_target_properties(onnx_opt_cpp2py_export - PROPERTIES LINK_FLAGS "-Wl,--exclude-libs,ALL") -+ PROPERTIES LINK_FLAGS -+ "-Wl,-rpath=${CMAKE_INSTALL_PREFIX}/lib") ++ PROPERTIES LINK_FLAGS "-Wl,--exclude-libs,ALL,-rpath=${CMAKE_INSTALL_PREFIX}/lib") ++ endif() target_link_libraries(onnx_opt_cpp2py_export PRIVATE onnx_optimizer) +diff --git a/cmake/utils.cmake b/cmake/utils.cmake +index 6cca9f36..8e39d5c4 100644 +--- a/cmake/utils.cmake ++++ b/cmake/utils.cmake +@@ -1,4 +1,4 @@ +-include(${PROJECT_SOURCE_DIR}/third_party/onnx/cmake/Utils.cmake) ++include(${ONNX_ROOT}/cmake/Utils.cmake) + + # Poor man's FetchContent + function(add_subdirectory_if_no_target dir target)