From patchwork Thu Apr 4 03:46:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Fremlin X-Patchwork-Id: 62690 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id DDABF27BBE9; Thu, 4 Apr 2024 04:49:23 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 9FC6027BBE2 for ; Thu, 4 Apr 2024 04:49:22 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rsE5z-0002Iv-P5; Wed, 03 Apr 2024 23:48:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rsE5x-0002IR-Nw for guix-patches@gnu.org; Wed, 03 Apr 2024 23:48:57 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rsE5x-0003bQ-GE for guix-patches@gnu.org; Wed, 03 Apr 2024 23:48:57 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rsE61-0003nH-Tk for guix-patches@gnu.org; Wed, 03 Apr 2024 23:49:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#70175] [PATCH] gnu: llama-cpp: support OpenBLAS for faster prompt processing Resent-From: John Fremlin Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 04 Apr 2024 03:49:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 70175 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 70175@debbugs.gnu.org Cc: John Fremlin X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.171220252114415 (code B ref -1); Thu, 04 Apr 2024 03:49:01 +0000 Received: (at submit) by debbugs.gnu.org; 4 Apr 2024 03:48:41 +0000 Received: from localhost ([127.0.0.1]:60100 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rsE5f-0003kB-F0 for submit@debbugs.gnu.org; Wed, 03 Apr 2024 23:48:40 -0400 Received: from lists.gnu.org ([2001:470:142::17]:46118) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rsE5c-0003ib-2w for submit@debbugs.gnu.org; Wed, 03 Apr 2024 23:48:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rsE5R-0002DY-Ht for guix-patches@gnu.org; Wed, 03 Apr 2024 23:48:25 -0400 Received: from spribog.eb.pe ([5.9.0.72]) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rsE5E-0003Db-MW for guix-patches@gnu.org; Wed, 03 Apr 2024 23:48:25 -0400 Received: by spribog.eb.pe (Postfix, from userid 1000) id 03154162B75E; Thu, 4 Apr 2024 05:48:05 +0200 (CEST) Date: Wed, 3 Apr 2024 23:46:25 -0400 Message-ID: <7f9fe9d92aff101a12057e8571772428028dc6ae.1712202385.git.john@fremlin.org> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 Received-SPF: none client-ip=5.9.0.72; envelope-from=john@spribog.eb.pe; helo=spribog.eb.pe X-Spam_score_int: -15 X-Spam_score: -1.6 X-Spam_bar: - X-Spam_report: (-1.6 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: John Fremlin X-ACL-Warn: , John Fremlin via Guix-patches X-Patchwork-Original-From: John Fremlin via Guix-patches via From: John Fremlin Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches OpenBLAS is recommended by https://github.com/ggerganov/llama.cpp Change-Id: Iaf6f22252da13e2d6f503992878b35b0da7de0aa --- gnu/packages/machine-learning.scm | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) base-commit: 1441a205b1ebb610ecfae945b5770734cbe8478c diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 225bff0ca2..ea3674ce3e 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -542,6 +542,8 @@ (define-public llama-cpp (build-system cmake-build-system) (arguments (list + #:configure-flags + '(list "-DLLAMA_BLAS=ON" "-DLLAMA_BLAS_VENDOR=OpenBLAS") #:modules '((ice-9 textual-ports) (guix build utils) ((guix build python-build-system) #:prefix python:) @@ -576,8 +578,9 @@ (define-public llama-cpp (lambda _ (copy-file "bin/main" (string-append #$output "/bin/llama"))))))) (inputs (list python)) + (native-inputs (list pkg-config)) (propagated-inputs - (list python-numpy python-pytorch python-sentencepiece)) + (list python-numpy python-pytorch python-sentencepiece openblas)) (home-page "https://github.com/ggerganov/llama.cpp") (synopsis "Port of Facebook's LLaMA model in C/C++") (description "This package provides a port to Facebook's LLaMA collection