From patchwork Fri Feb 21 18:24:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Graves X-Patchwork-Id: 38921 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 590B027BBEA; Fri, 21 Feb 2025 19:54:30 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-6.4 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED, RCVD_IN_VALIDITY_CERTIFIED,RCVD_IN_VALIDITY_RPBL,RCVD_IN_VALIDITY_SAFE, SPF_HELO_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 9409427BBE2 for ; Fri, 21 Feb 2025 19:54:28 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tlZ66-000775-5U; Fri, 21 Feb 2025 14:54:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlZ62-00076a-Qt for guix-patches@gnu.org; Fri, 21 Feb 2025 14:54:03 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tlZ62-0001re-Hs for guix-patches@gnu.org; Fri, 21 Feb 2025 14:54:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:From:To:Subject; bh=lH4V6QKlfoGJEGkEq2t9wL9euOD4aOJnuLouYLhEc9w=; b=o/JZQRVCIQgyXpgKv7qLik6vMnbRuEUhgoAJJok7uslF4ZqF09SjZg1uR95WIwXMaXTwgfYj6ibjsLxgOTdfDadHlvgze0UDRkLO7fQo1z2ovvKuGaTry5lc7fhfI0XUQP+hlHupyjTdDBDB4DXOe0N61ukkGGwCLO0CWqvBVKutM4fAXDkSnMDeeFvRW+IJMDxjhUPJGBJ9ey0XHmsVNSJjkqkkv5YBUxQrO8lGgaZ0I84AGDObBQxDBiVC4Y1bod2yVEiPaBRZDetHTKfuncFt7eHrXuiw7scDyJkFIOg49lRcpjrTKHtGuKtGnkX8dMq2X31PkAxGOWCECVYW9Q==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1tlZ62-0004Sd-2g for guix-patches@gnu.org; Fri, 21 Feb 2025 14:54:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#76477] [PATCH] gnu: faiss: Update to 1.10.0. Resent-From: Nicolas Graves Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 21 Feb 2025 19:54:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 76477 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 76477@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.174016758916720 (code B ref -1); Fri, 21 Feb 2025 19:54:01 +0000 Received: (at submit) by debbugs.gnu.org; 21 Feb 2025 19:53:09 +0000 Received: from localhost ([127.0.0.1]:36728 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tlZ59-0004LR-LL for submit@debbugs.gnu.org; Fri, 21 Feb 2025 14:53:09 -0500 Received: from lists.gnu.org ([2001:470:142::17]:45070) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tlZ54-0004Jb-3W for submit@debbugs.gnu.org; Fri, 21 Feb 2025 14:53:04 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlZ4x-0006zU-HI for guix-patches@gnu.org; Fri, 21 Feb 2025 14:52:55 -0500 Received: from 15.mo582.mail-out.ovh.net ([188.165.39.161]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tlZ4t-0001mC-P8 for guix-patches@gnu.org; Fri, 21 Feb 2025 14:52:55 -0500 Received: from director4.ghost.mail-out.ovh.net (unknown [10.108.9.71]) by mo582.mail-out.ovh.net (Postfix) with ESMTP id 4Z014m2C2Jz1ZXW for ; Fri, 21 Feb 2025 19:52:40 +0000 (UTC) Received: from ghost-submission-5b5ff79f4f-7bt47 (unknown [10.110.96.132]) by director4.ghost.mail-out.ovh.net (Postfix) with ESMTPS id E3A811FDF8 for ; Fri, 21 Feb 2025 19:52:39 +0000 (UTC) Received: from ngraves.fr ([37.59.142.112]) by ghost-submission-5b5ff79f4f-7bt47 with ESMTPSA id o7PHKofZuGdzHgAAQxmiBA (envelope-from ) for ; Fri, 21 Feb 2025 19:52:39 +0000 Authentication-Results: garm.ovh; auth=pass (GARM-112S006411cf750-77b0-459f-a05c-368427ddf392, 94F4052D2D9A3500C11B15ADD6AE584288BF003F) smtp.auth=ngraves@ngraves.fr X-OVh-ClientIp: 193.51.120.250 Date: Fri, 21 Feb 2025 19:24:28 +0100 Message-ID: <875xl3krco.fsf@ngraves.fr> MIME-Version: 1.0 X-Ovh-Tracer-Id: 15674778503076307706 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgdejtdeltdcutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecunecujfgurhephffvufffkfggtgesthdtredttddttdenucfhrhhomheppfhitgholhgrshcuifhrrghvvghsuceonhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrqeenucggtffrrghtthgvrhhnpeetfeeukeehledtffdvieeuleejkefhudeuhfdukefgtdevgeetgedtkeehueeivdenucffohhmrghinhepghhithhhuhgsrdgtohhmnecukfhppeduvdejrddtrddtrddupdduleefrdehuddruddvtddrvdehtddpfeejrdehledrudegvddrudduvdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpeduvdejrddtrddtrddupdhmrghilhhfrhhomhepnhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrpdhnsggprhgtphhtthhopedupdhrtghpthhtohepghhuihigqdhprghttghhvghssehgnhhurdhorhhgpdfovfetjfhoshhtpehmohehkedvmgdpmhhouggvpehsmhhtphhouhht DKIM-Signature: a=rsa-sha256; bh=lH4V6QKlfoGJEGkEq2t9wL9euOD4aOJnuLouYLhEc9w=; c=relaxed/relaxed; d=ngraves.fr; h=From; s=ovhmo4487190-selector1; t=1740167560; v=1; b=uSIidQZLC3FyALMtRDKuL4+i2BP2PxE5TXHaGil091ZM9ERBtUzUpQ6+4mBEGUct2mqB5Ks/ 4zZUi+RhDteLw4BBV7Dzxk8FkRlLPm+sbyMmXRSUytw76KtwUE74UYeicqu3OUTC1+j8Pz20Ghs IwrmDPkmBshsRBICTJRCCChSiT5LGIgafCCPbXRhzHwuqY5oj6zCtLatpO9xOTM7/Jph/kvG8nm tBb7+5R7OkZfumPMscgLVasWRnhZiTDfaRPaH0QOvtO+3E28QJo8zpp5XbTbDjiFdwbzPH6pO4j UCDHeMAX4RmOy+T9l27N0SbAqoWs2Xi2ri5ytYEuNXp2w== Received-SPF: pass client-ip=188.165.39.161; envelope-from=ngraves@ngraves.fr; helo=15.mo582.mail-out.ovh.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Nicolas Graves X-ACL-Warn: , Nicolas Graves via Guix-patches X-Patchwork-Original-From: Nicolas Graves via Guix-patches via From: Nicolas Graves Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * gnu/packages/graph.scm (faiss): Update to 1.10.0. [source]: Adapt snippet. Add patch. [arguments]: Delete likely uneeded {phases}. Adapt {configure-flags}. [native-inputs]: Add openmpi. (python-faiss): Update to 1.10.0. [build-system]: Switch to cmake-build-system. [arguments]: Rewrite all arguments. [inputs]: Improve-style. Switch python for python-wrapper. [native-inputs]: Add python-scipy. [propagated-inputs]: Remove python-matplotlib. * gnu/packages/patches/faiss-tests-CMakeLists-find-googletest.patch: Add patch. * gnu/local.mk: Likewise. * gnu/packages/graph.scm --- gnu/local.mk | 1 + gnu/packages/graph.scm | 202 ++++++++---------- ...iss-tests-CMakeLists-find-googletest.patch | 53 +++++ 3 files changed, 146 insertions(+), 110 deletions(-) create mode 100644 gnu/packages/patches/faiss-tests-CMakeLists-find-googletest.patch diff --git a/gnu/local.mk b/gnu/local.mk index c7f63c458a..464e3896b6 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -1258,6 +1258,7 @@ dist_patch_DATA = \ %D%/packages/patches/fail2ban-python310-server-action.patch \ %D%/packages/patches/fail2ban-python310-server-actions.patch \ %D%/packages/patches/fail2ban-python310-server-jails.patch \ + %D%/packages/patches/faiss-tests-CMakeLists-find-googletest.patch \ %D%/packages/patches/falcosecurity-libs-shared-build.patch \ %D%/packages/patches/farstream-gupnp.patch \ %D%/packages/patches/farstream-make.patch \ diff --git a/gnu/packages/graph.scm b/gnu/packages/graph.scm index f29a71c0d7..467a7f661b 100644 --- a/gnu/packages/graph.scm +++ b/gnu/packages/graph.scm @@ -62,6 +62,7 @@ (define-module (gnu packages graph) #:use-module (gnu packages linux) #:use-module (gnu packages machine-learning) #:use-module (gnu packages maths) + #:use-module (gnu packages mpi) #:use-module (gnu packages multiprecision) #:use-module (gnu packages ncurses) #:use-module (gnu packages pkg-config) @@ -582,85 +583,36 @@ (define-public python-pygsp (define-public faiss (package (name "faiss") - (version "1.5.0") - (source (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/facebookresearch/faiss") - (commit (string-append "v" version)))) - (file-name (git-file-name name version)) - (sha256 - (base32 - "0pk15jfa775cy2pqmzq62nhd6zfjxmpvz5h731197c28aq3zw39w")) - (modules '((guix build utils))) - (snippet - '(begin - (substitute* "utils.cpp" - (("#include ") - "#ifdef __SSE__\n#include \n#endif")) - #t)))) + (version "1.10.0") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/facebookresearch/faiss") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "1x6z94f6vhh7ppsn7wll46k7i63lzcnc3r3rv5zfarljybqhrsjd")) + ;; Including but skipping perf_tests requires to patch + ;; perf_tests/CMakeLists.txt. KISS: Remove it instead. + (modules '((guix build utils))) + (snippet #~(begin + (delete-file-recursively "perf_tests") + (substitute* "CMakeLists.txt" + (("add_subdirectory\\(perf_tests\\)") "")))) + (patches + (search-patches "faiss-tests-CMakeLists-find-googletest.patch")))) (build-system cmake-build-system) (arguments - `(#:configure-flags - (list "-DBUILD_WITH_GPU=OFF" ; thanks, but no thanks, CUDA. - "-DBUILD_TUTORIAL=OFF") ; we don't need those - #:phases - (modify-phases %standard-phases - (add-after 'unpack 'prepare-build - (lambda _ - (let ((features (list ,@(let ((system (or (%current-target-system) - (%current-system)))) - (cond - ((string-prefix? "x86_64" system) - '("-mavx" "-msse2" "-mpopcnt")) - ((string-prefix? "i686" system) - '("-msse2" "-mpopcnt")) - (else - '())))))) - (substitute* "CMakeLists.txt" - (("-m64") "") - (("-mpopcnt") "") ; only some architectures - (("-msse4") - (string-append - (string-join features) - " -I" (getcwd))) - ;; Build also the shared library - (("ARCHIVE DESTINATION lib") - "LIBRARY DESTINATION lib") - (("add_library.*" m) - "\ -add_library(objlib OBJECT ${faiss_cpu_headers} ${faiss_cpu_cpp}) -set_property(TARGET objlib PROPERTY POSITION_INDEPENDENT_CODE 1) -add_library(${faiss_lib}_static STATIC $) -add_library(${faiss_lib} SHARED $) -install(TARGETS ${faiss_lib}_static ARCHIVE DESTINATION lib) -\n"))) - - ;; See https://github.com/facebookresearch/faiss/issues/520 - (substitute* "IndexScalarQuantizer.cpp" - (("#define USE_AVX") "")) - - ;; Make header files available for compiling tests. - (mkdir-p "faiss") - (for-each (lambda (file) - (mkdir-p (string-append "faiss/" (dirname file))) - (copy-file file (string-append "faiss/" file))) - (find-files "." "\\.h$")) - #t)) - (replace 'check - (lambda _ - (invoke "make" "-C" "tests" - (format #f "-j~a" (parallel-job-count))))) - (add-after 'install 'remove-tests - (lambda* (#:key outputs #:allow-other-keys) - (delete-file-recursively - (string-append (assoc-ref outputs "out") - "/test")) - #t))))) + (list #:configure-flags + #~'("-DFAISS_ENABLE_GPU=OFF" ; thanks, but no thanks, CUDA. + "-DFAISS_ENABLE_PYTHON=OFF" + "-DBUILD_TESTING=ON"))) (inputs (list openblas)) (native-inputs - (list googletest)) + (list googletest openmpi)) (home-page "https://github.com/facebookresearch/faiss") (synopsis "Efficient similarity search and clustering of dense vectors") (description "Faiss is a library for efficient similarity search and @@ -672,45 +624,75 @@ (define-public faiss (define-public python-faiss (package (inherit faiss) (name "python-faiss") - (build-system python-build-system) + (build-system cmake-build-system) (arguments - `(#:phases - (modify-phases %standard-phases - (add-after 'unpack 'chdir - (lambda _ (chdir "python") #t)) - (add-after 'chdir 'build-swig - (lambda* (#:key inputs #:allow-other-keys) - (with-output-to-file "../makefile.inc" - (lambda () - (let ((python-version ,(version-major+minor (package-version python)))) - (format #t "\ -PYTHONCFLAGS =-I~a/include/python~a/ -I~a/lib/python~a/site-packages/numpy/core/include -LIBS = -lpython~a -lfaiss -SHAREDFLAGS = -shared -fopenmp -CXXFLAGS = -fpermissive -fopenmp -fPIC -CPUFLAGS = ~{~a ~}~%" - (assoc-ref inputs "python*") python-version - (assoc-ref inputs "python-numpy") python-version - python-version - (list ,@(let ((system (or (%current-target-system) - (%current-system)))) - (cond - ((string-prefix? "x86_64" system) - '("-mavx" "-msse2" "-mpopcnt")) - ((string-prefix? "i686" system) - '("-msse2" "-mpopcnt")) - (else - '())))))))) - (substitute* "Makefile" - (("../libfaiss.a") "")) - (invoke "make" "cpu")))))) + (list + #:imported-modules `(,@%cmake-build-system-modules + (guix build gremlin) + (guix build python-build-system)) + #:modules '((guix build cmake-build-system) + ((guix build python-build-system) #:prefix python:) + (guix build utils) + (guix build gremlin) + (ice-9 match) + (srfi srfi-1) + (srfi srfi-26)) + #:phases + #~(modify-phases %standard-phases + (add-after 'unpack 'chdir + (lambda _ (chdir "faiss/python"))) + (add-before 'install 'python-build + (lambda _ + ((assoc-ref python:%standard-phases 'build) + #:use-setuptools? #t))) + (replace 'install + (lambda args + (apply + (assoc-ref python:%standard-phases 'install) + #:use-setuptools? #t + #:configure-flags ''() + args) + (for-each + delete-file + (find-files #$output + "_*faiss_example_external_module\\.(so|py)$")))) + ;; Move check phase after 'install. + (delete 'check) + (add-after 'install 'check + (lambda* (#:key inputs outputs tests? #:allow-other-keys) + (if tests? + (with-directory-excursion "../../tests" + (let* ((version #$(version-major+minor + (package-version + (this-package-input "python-wrapper")))) + (destination (string-append "/lib/python" version + "/site-packages/"))) + (setenv + "PYTHONPATH" + (string-join + (filter + directory-exists? + (map (match-lambda + ((name . directory) + (string-append directory destination))) + (append outputs inputs))) + ":"))) + (for-each + (lambda (file) + (invoke "python" file)) + (remove (cut member <> '(;; External module removed + "./external_module_test.py" + ;; Avoid torch dependency + "./torch_test_contrib.py" + "./torch_test_neural_net.py")) + (find-files "." "\\.py$")))) + (format #t "test suite not run~%"))))))) + (native-inputs + (list python-scipy)) (inputs - `(("faiss" ,faiss) - ("openblas" ,openblas) - ("python*" ,python) - ("swig" ,swig))) + (list faiss openblas python-wrapper swig)) (propagated-inputs - (list python-matplotlib python-numpy)) + (list python-numpy)) (description "Faiss is a library for efficient similarity search and clustering of dense vectors. This package provides Python bindings to the Faiss library."))) diff --git a/gnu/packages/patches/faiss-tests-CMakeLists-find-googletest.patch b/gnu/packages/patches/faiss-tests-CMakeLists-find-googletest.patch new file mode 100644 index 0000000000..5183bac08a --- /dev/null +++ b/gnu/packages/patches/faiss-tests-CMakeLists-find-googletest.patch @@ -0,0 +1,53 @@ +From 9f845321de3bdd2b840c42b977c1c092ac553cc2 Mon Sep 17 00:00:00 2001 +From: Nicolas Graves +Date: Fri, 21 Feb 2025 12:05:59 +0100 +Subject: [PATCH] tests/CMakeLists.txt: Find googletest package from inputs + +--- + tests/CMakeLists.txt | 29 +---------------------------- + 1 file changed, 1 insertion(+), 28 deletions(-) + +diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt +index dfab76e0..16a51b0b 100644 +--- a/tests/CMakeLists.txt ++++ b/tests/CMakeLists.txt +@@ -48,35 +48,8 @@ if (FAISS_ENABLE_PYTHON) + target_link_libraries(faiss_test PUBLIC faiss_example_external_module) + endif() + +-include(FetchContent) +-FetchContent_Declare( +- googletest +- GIT_REPOSITORY https://github.com/google/googletest.git +- GIT_TAG 58d77fa8070e8cec2dc1ed015d66b454c8d78850 # release-1.12.1 +- OVERRIDE_FIND_PACKAGE) +-set(BUILD_GMOCK CACHE BOOL OFF) +-set(INSTALL_GTEST CACHE BOOL OFF) +-FetchContent_MakeAvailable(googletest) +- +-if(NOT EXISTS ${CMAKE_FIND_PACKAGE_REDIRECTS_DIR}/gtest-config.cmake +- AND NOT EXISTS ${CMAKE_FIND_PACKAGE_REDIRECTS_DIR}/GTestConfig.cmake) +- file( +- WRITE ${CMAKE_FIND_PACKAGE_REDIRECTS_DIR}/gtest-config.cmake +- [=[ +-include(CMakeFindDependencyMacro) +-find_dependency(googletest) +-if(NOT TARGET GTest::GTest) +- add_library(GTest::GTest INTERFACE IMPORTED) +- target_link_libraries(GTest::GTest INTERFACE GTest::gtest) +-endif() +-if(NOT TARGET GTest::Main) +- add_library(GTest::Main INTERFACE IMPORTED) +- target_link_libraries(GTest::Main INTERFACE GTest::gtest_main) +-endif() +-]=]) +-endif() +- + find_package(OpenMP REQUIRED) ++find_package(GTest REQUIRED) + find_package(GTest CONFIG REQUIRED) + + target_link_libraries(faiss_test PRIVATE +-- +2.48.1 +