From patchwork Thu Feb 27 20:41:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Arun Isaac X-Patchwork-Id: 20460 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 2BF3927BBEB; Thu, 27 Feb 2020 20:43:21 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id F0FF327BBF0 for ; Thu, 27 Feb 2020 20:43:18 +0000 (GMT) Received: from localhost ([::1]:38034 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j7Q06-0005Fv-HV for patchwork@mira.cbaines.net; Thu, 27 Feb 2020 15:43:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59513) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j7Pzv-0005FF-53 for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:08 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j7Pzr-0008Vo-Uu for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:05 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:54386) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1j7Pzr-0008Vj-Rm for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1j7Pzr-0008Ks-Pf for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#39258] [PATCH 1/4] gnu: Add guile-xapian. Resent-From: Arun Isaac Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 27 Feb 2020 20:43:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 39258 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: 39258@debbugs.gnu.org Cc: Arun Isaac , ludo@gnu.org, zimon.toutoune@gmail.com Received: via spool by 39258-submit@debbugs.gnu.org id=B39258.158283614531958 (code B ref 39258); Thu, 27 Feb 2020 20:43:03 +0000 Received: (at 39258) by debbugs.gnu.org; 27 Feb 2020 20:42:25 +0000 Received: from localhost ([127.0.0.1]:60354 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j7PzA-0008JD-J5 for submit@debbugs.gnu.org; Thu, 27 Feb 2020 15:42:25 -0500 Received: from mugam.systemreboot.net ([139.59.75.54]:37304) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j7Pz6-0008IN-LY for 39258@debbugs.gnu.org; Thu, 27 Feb 2020 15:42:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=systemreboot.net; s=default; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=PvLHNWGiLSPRr+w84SBaVoddHcDGgiKIDnzT9hWFMVA=; b=C9dtcu9OhaIzJuaJbl+WCAdkfB I2c0TImaQT9nBRXd2Wy1H8+1horlIoXo6c1yq13GJ/YokGc0Fj2ZDD5DL5nCrXhQrW1f2SQILiyUo Owtd5nH0kq39SKPltOU3XmBGCH6I8CzRa3Gysjm9aqthHKkdXJdLsWE7qqg/CyxkEzR8=; Received: from [192.168.2.1] (helo=steel.lan) by systemreboot.net with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.93) (envelope-from ) id 1j7Pyz-001ckB-EN; Fri, 28 Feb 2020 02:12:09 +0530 From: Arun Isaac Date: Fri, 28 Feb 2020 02:11:47 +0530 Message-Id: <20200227204150.30985-2-arunisaac@systemreboot.net> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20200227204150.30985-1-arunisaac@systemreboot.net> References: <20200227204150.30985-1-arunisaac@systemreboot.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/packages/guile-xyz.scm (guile-xapian, guile3.0-xapian): New variables. --- gnu/packages/guile-xyz.scm | 50 +++++++++++++++++++++++++++++++++++++- 1 file changed, 49 insertions(+), 1 deletion(-) diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm index 37a5198e4e..75aba83593 100644 --- a/gnu/packages/guile-xyz.scm +++ b/gnu/packages/guile-xyz.scm @@ -17,7 +17,7 @@ ;;; Copyright © 2017 ng0 ;;; Copyright © 2017, 2018 Tobias Geerinckx-Rice ;;; Copyright © 2018 Maxim Cournoyer -;;; Copyright © 2018, 2019 Arun Isaac +;;; Copyright © 2018, 2019, 2020 Arun Isaac ;;; Copyright © 2018 Pierre-Antoine Rouby ;;; Copyright © 2018 Eric Bavier ;;; Copyright © 2019 swedebugia @@ -80,8 +80,10 @@ #:use-module (gnu packages python) #:use-module (gnu packages readline) #:use-module (gnu packages sdl) + #:use-module (gnu packages search) #:use-module (gnu packages slang) #:use-module (gnu packages sqlite) + #:use-module (gnu packages swig) #:use-module (gnu packages tex) #:use-module (gnu packages texinfo) #:use-module (gnu packages tls) @@ -3109,3 +3111,49 @@ currently a re-implementation of the lentes library for Clojure. Lenses provide composable procedures, which can be used to focus, apply functions over, or update a value in arbitrary data structures.") (license license:gpl3+)))) + +(define-public guile-xapian + (let ((commit "bfad1b0e2a88bfe1d4c100046da0d585b96d2a73") + (revision "1")) + (package + (name "guile-xapian") + (version (git-version "0.1.0" revision commit)) + (home-page "https://git.systemreboot.net/guile-xapian") + (source + (origin + (method git-fetch) + (uri (git-reference (url home-page) + (commit commit))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "1nrs23abb0lxx7gw14jw5k8jgbma0gi21gzahw0jgv6b25d9jdwp")))) + (build-system gnu-build-system) + (arguments + '(#:make-flags '("GUILE_AUTO_COMPILE=0"))) ; to prevent guild warnings + (inputs + `(("guile" ,guile-2.2) + ("xapian" ,xapian) + ("zlib" ,zlib))) + (native-inputs + `(("autoconf" ,autoconf) + ("autoconf-archive" ,autoconf-archive) + ("automake" ,automake) + ("libtool" ,libtool) + ("pkg-config" ,pkg-config) + ("swig" ,swig))) + (synopsis "Guile bindings for Xapian") + (description "@code{guile-xapian} provides Guile bindings for Xapian, a +search engine library. Xapian is a highly adaptable toolkit which allows +developers to easily add advanced indexing and search facilities to their own +applications. It has built-in support for several families of weighting +models and also supports a rich set of boolean query operators.") + (license license:gpl2+)))) + +(define-public guile3.0-xapian + (package + (inherit guile-xapian) + (name "guile3.0-xapian") + (inputs + `(("guile" ,guile-next) + ,@(alist-delete "guile" (package-inputs guile-xapian)))))) From patchwork Thu Feb 27 20:41:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Arun Isaac X-Patchwork-Id: 20461 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 8824927BBEB; Thu, 27 Feb 2020 20:43:24 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 4C5A027BBE4 for ; Thu, 27 Feb 2020 20:43:24 +0000 (GMT) Received: from localhost ([::1]:38040 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j7Q0B-0005VT-RX for patchwork@mira.cbaines.net; Thu, 27 Feb 2020 15:43:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59515) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j7Pzv-0005FH-Ej for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:10 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j7Pzs-0008W9-D5 for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:07 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:54387) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1j7Pzs-0008W4-A4 for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:04 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1j7Pzs-0008Kz-7f for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:04 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#39258] [PATCH 2/4] build-self: Add guile-xapian to Guix dependencies. Resent-From: Arun Isaac Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 27 Feb 2020 20:43:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 39258 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: 39258@debbugs.gnu.org Cc: Arun Isaac , ludo@gnu.org, zimon.toutoune@gmail.com Received: via spool by 39258-submit@debbugs.gnu.org id=B39258.158283614631967 (code B ref 39258); Thu, 27 Feb 2020 20:43:04 +0000 Received: (at 39258) by debbugs.gnu.org; 27 Feb 2020 20:42:26 +0000 Received: from localhost ([127.0.0.1]:60356 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j7PzG-0008JW-76 for submit@debbugs.gnu.org; Thu, 27 Feb 2020 15:42:26 -0500 Received: from mugam.systemreboot.net ([139.59.75.54]:37312) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j7Pz9-0008Iq-Qk for 39258@debbugs.gnu.org; Thu, 27 Feb 2020 15:42:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=systemreboot.net; s=default; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=+qfmxXIuB58cypMqiYHpA/+3NJ3qUzwieaNRxXHkquQ=; b=HLAxgPtntu2rhEJ63A3jYxzkKN ku6985TvNdc4Ql4xBCP7pLL18vHpXzsThc+7XQMLNcVzgH2VM1FzJcq9hWIZEJaY6WLhBmbpbyngF qhGe+MNTvKvqMnAkKe21VZDHBiusvwYB1TwREWxV9KEE27tO+W4RZS6CohO4LRjawJB4=; Received: from [192.168.2.1] (helo=steel.lan) by systemreboot.net with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.93) (envelope-from ) id 1j7Pyz-001ckB-KX; Fri, 28 Feb 2020 02:12:09 +0530 From: Arun Isaac Date: Fri, 28 Feb 2020 02:11:48 +0530 Message-Id: <20200227204150.30985-3-arunisaac@systemreboot.net> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20200227204150.30985-1-arunisaac@systemreboot.net> References: <20200227204150.30985-1-arunisaac@systemreboot.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * build-aux/build-self.scm (build-program): Import fake guile-xapian module. * guix/self.scm (compiled-guix): Add guile-xapian to Guix dependencies. --- build-aux/build-self.scm | 11 +++++++++++ guix/self.scm | 7 ++++++- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm index f2e785b7f1..05d0353ccf 100644 --- a/build-aux/build-self.scm +++ b/build-aux/build-self.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2014, 2016, 2017, 2018, 2019, 2020 Ludovic Courtès +;;; Copyright © 2020 Arun Isaac ;;; ;;; This file is part of GNU Guix. ;;; @@ -261,6 +262,10 @@ interface (FFI) of Guile.") #~(define-module (gcrypt hash) #:export (sha1 sha256)))) + (define fake-xapian-hash + ;; Fake (xapian xapian) module; see below. + (scheme-file "xapian.scm" #~(define-module (xapian xapian)))) + (define fake-git (scheme-file "git.scm" #~(define-module (git)))) @@ -273,6 +278,12 @@ interface (FFI) of Guile.") ;; adjust %LOAD-PATH later on. ((gcrypt hash) => ,fake-gcrypt-hash) + ;; To avoid relying on 'with-extensions', which was + ;; introduced in 0.15.0, provide a fake (xapian + ;; xapian) just so that we can build modules, and + ;; adjust %LOAD-PATH later on. + ((xapian xapian) => ,fake-xapian-hash) + ;; (guix git-download) depends on (git) but only ;; for peripheral functionality. Provide a dummy ;; (git) to placate it. diff --git a/guix/self.scm b/guix/self.scm index 6b633f9bc0..a4f40574d1 100644 --- a/guix/self.scm +++ b/guix/self.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2017, 2018, 2019, 2020 Ludovic Courtès +;;; Copyright © 2020 Arun Isaac ;;; ;;; This file is part of GNU Guix. ;;; @@ -54,6 +55,7 @@ ("guile-git" (ref '(gnu packages guile) 'guile3.0-git)) ("guile-sqlite3" (ref '(gnu packages guile) 'guile3.0-sqlite3)) ("guile-gcrypt" (ref '(gnu packages gnupg) 'guile3.0-gcrypt)) + ("guile-xapian" (ref '(gnu packages guile-xyz) 'guile3.0-xapian)) ("gnutls" (ref '(gnu packages tls) 'guile3.0-gnutls)) ("zlib" (ref '(gnu packages compression) 'zlib)) ("lzlib" (ref '(gnu packages compression) 'lzlib)) @@ -682,6 +684,9 @@ Info manual." (define guile-gcrypt (specification->package "guile-gcrypt")) + (define guile-xapian + (specification->package "guile-xapian")) + (define gnutls (specification->package "gnutls")) @@ -690,7 +695,7 @@ Info manual." (cons (list "x" package) (package-transitive-propagated-inputs package))) (list guile-gcrypt gnutls guile-git guile-json - guile-ssh guile-sqlite3)) + guile-ssh guile-sqlite3 guile-xapian)) (((labels packages _ ...) ...) packages))) From patchwork Thu Feb 27 20:41:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Arun Isaac X-Patchwork-Id: 20462 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 00EC327BBE4; Thu, 27 Feb 2020 20:43:24 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 6B47827BBEA for ; Thu, 27 Feb 2020 20:43:24 +0000 (GMT) Received: from localhost ([::1]:38038 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j7Q0B-0005VR-UV for patchwork@mira.cbaines.net; Thu, 27 Feb 2020 15:43:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59489) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j7Pzs-0005F4-4e for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j7Pzr-0008VI-0X for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:54384) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1j7Pzq-0008VE-U1 for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1j7Pzq-0008Kd-Rr for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#39258] [PATCH 3/4] gnu: Generate xapian package search index. Resent-From: Arun Isaac Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 27 Feb 2020 20:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 39258 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: 39258@debbugs.gnu.org Cc: Arun Isaac , ludo@gnu.org, zimon.toutoune@gmail.com Received: via spool by 39258-submit@debbugs.gnu.org id=B39258.158283614031934 (code B ref 39258); Thu, 27 Feb 2020 20:43:02 +0000 Received: (at 39258) by debbugs.gnu.org; 27 Feb 2020 20:42:20 +0000 Received: from localhost ([127.0.0.1]:60349 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j7Pz9-0008It-J1 for submit@debbugs.gnu.org; Thu, 27 Feb 2020 15:42:19 -0500 Received: from mugam.systemreboot.net ([139.59.75.54]:37300) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j7Pz5-0008IH-HA for 39258@debbugs.gnu.org; Thu, 27 Feb 2020 15:42:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=systemreboot.net; s=default; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=SX6el7Fgocm4MXmlCLaq3jNgIZifmGn86XxWLvpUS/E=; b=YpPvLc1uI7RCn0XyQdwSVLArSP wM2dvmmUKLal+6JVkofvThIgJu56cahVD4rsXbFb7zePWFPr8M26wadyb+8Rw+NfVNSOnkzJphQI9 ZZoZl6w6sj/n13AXvD1lfCp4WE5wO1BYJqmTNszqrd3NRnY3lqQl6jystDVl9Km/ymsM=; Received: from [192.168.2.1] (helo=steel.lan) by systemreboot.net with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.93) (envelope-from ) id 1j7Pyz-001ckB-QI; Fri, 28 Feb 2020 02:12:09 +0530 From: Arun Isaac Date: Fri, 28 Feb 2020 02:11:49 +0530 Message-Id: <20200227204150.30985-4-arunisaac@systemreboot.net> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20200227204150.30985-1-arunisaac@systemreboot.net> References: <20200227204150.30985-1-arunisaac@systemreboot.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/packages.scm (%package-search-index): New variable. (generate-package-search-index): New function. * guix/channels.scm (package-search-index): New function. (%channel-profile-hooks): Add package-search-index. --- gnu/packages.scm | 29 ++++++++++++++++++++++++++++- guix/channels.scm | 34 +++++++++++++++++++++++++++++++++- 2 files changed, 61 insertions(+), 2 deletions(-) diff --git a/gnu/packages.scm b/gnu/packages.scm index d22c992bb1..e91753e2a8 100644 --- a/gnu/packages.scm +++ b/gnu/packages.scm @@ -4,6 +4,7 @@ ;;; Copyright © 2014 Eric Bavier ;;; Copyright © 2016, 2017 Alex Kost ;;; Copyright © 2016 Mathieu Lirzin +;;; Copyright © 2020 Arun Isaac ;;; ;;; This file is part of GNU Guix. ;;; @@ -43,6 +44,7 @@ #:use-module (srfi srfi-34) #:use-module (srfi srfi-35) #:use-module (srfi srfi-39) + #:use-module (xapian xapian) #:export (search-patch search-patches search-auxiliary-file @@ -64,7 +66,8 @@ specification->location specifications->manifest - generate-package-cache)) + generate-package-cache + generate-package-search-index)) ;;; Commentary: ;;; @@ -426,6 +429,30 @@ reducing the memory footprint." #:opts '(#:to-file? #t))))) cache-file) +(define %package-search-index + ;; Location of the package search-index + "/lib/guix/package-search.index") + +(define (generate-package-search-index directory) + "Generate under DIRECTORY a xapian index of all the available packages." + (define db-path + (string-append directory %package-search-index)) + + (mkdir-p (dirname db-path)) + (call-with-writable-database db-path + (lambda (db) + (fold-packages (lambda (package _) + (let* ((idterm (string-append "Q" (package-name package))) + (doc (make-document #:data (package-name package) + #:terms `((,idterm . 0)))) + (term-generator (make-term-generator #:stem (make-stem "en") + #:document doc))) + (index-text! term-generator (package-description package)) + (replace-document! db idterm doc))) + #f))) + + db-path) + (define %sigint-prompt ;; The prompt to jump to upon SIGINT. diff --git a/guix/channels.scm b/guix/channels.scm index f0261dc2da..c70c70938c 100644 --- a/guix/channels.scm +++ b/guix/channels.scm @@ -2,6 +2,7 @@ ;;; Copyright © 2018, 2019, 2020 Ludovic Courtès ;;; Copyright © 2018 Ricardo Wurmus ;;; Copyright © 2019 Jan (janneke) Nieuwenhuizen +;;; Copyright © 2020 Arun Isaac ;;; ;;; This file is part of GNU Guix. ;;; @@ -581,9 +582,40 @@ be used as a profile hook." (hook . package-cache)) #:local-build? #t))) +(define (package-search-index manifest) + "Build a package search index for the instance in MANIFEST. This is meant +to be used as a profile hook." + (mlet %store-monad ((profile (profile-derivation manifest + #:hooks '()))) + + (define build + #~(begin + (use-modules (gnu packages)) + + (if (defined? 'generate-package-search-index) + (begin + ;; Delegate package search index generation to the inferior. + (format (current-error-port) + "Generating package search index for '~a'...~%" + #$profile) + (generate-package-search-index #$output)) + (mkdir #$output)))) + + (gexp->derivation-in-inferior "guix-package-search-index" build + profile + + ;; If the Guix in PROFILE is too old and + ;; lacks 'guix repl', don't build the cache + ;; instead of failing. + #:silent-failure? #t + + #:properties '((type . profile-hook) + (hook . package-search-index)) + #:local-build? #t))) + (define %channel-profile-hooks ;; The default channel profile hooks. - (cons package-cache-file %default-profile-hooks)) + (cons* package-cache-file package-search-index %default-profile-hooks)) (define (channel-instances->derivation instances) "Return the derivation of the profile containing INSTANCES, a list of From patchwork Thu Feb 27 20:41:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Arun Isaac X-Patchwork-Id: 20459 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 0015A27BBE4; Thu, 27 Feb 2020 20:43:19 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id D660527BBEE for ; Thu, 27 Feb 2020 20:43:18 +0000 (GMT) Received: from localhost ([::1]:38032 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j7Q06-0005Fi-Cx for patchwork@mira.cbaines.net; Thu, 27 Feb 2020 15:43:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59501) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j7Pzs-0005F8-Mz for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j7Pzr-0008Vb-G5 for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:54385) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1j7Pzr-0008VU-Cl for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1j7Pzr-0008Kl-AZ for guix-patches@gnu.org; Thu, 27 Feb 2020 15:43:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#39258] [PATCH 4/4] gnu: Use xapian index for package search. Resent-From: Arun Isaac Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 27 Feb 2020 20:43:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 39258 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: 39258@debbugs.gnu.org Cc: Arun Isaac , ludo@gnu.org, zimon.toutoune@gmail.com Received: via spool by 39258-submit@debbugs.gnu.org id=B39258.158283614031946 (code B ref 39258); Thu, 27 Feb 2020 20:43:03 +0000 Received: (at 39258) by debbugs.gnu.org; 27 Feb 2020 20:42:20 +0000 Received: from localhost ([127.0.0.1]:60351 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j7PzA-0008J1-0f for submit@debbugs.gnu.org; Thu, 27 Feb 2020 15:42:20 -0500 Received: from mugam.systemreboot.net ([139.59.75.54]:37294) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j7Pz5-0008II-Gz for 39258@debbugs.gnu.org; Thu, 27 Feb 2020 15:42:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=systemreboot.net; s=default; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=sQEq9FYeMCTpeCbSa0BT6wsOvHihjAdebVbxE4b6Xv0=; b=bKhUzRcGGgmRLO/tm9x6FZnZcW bThUjGRDJfczKHWi5PQTz+OMXPgjEoQz6nd8m0hlmkhQmv24gRbDHZAM7YWZW/oeMGiQxw7YsFMx0 Nxi30o2OIP32Xu1mksoUUGmUhblw11KvXPZh8tDxgt0tXEjlNFRtLJjS8EcF8siBXvW0=; Received: from [192.168.2.1] (helo=steel.lan) by systemreboot.net with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.93) (envelope-from ) id 1j7Pyz-001ckB-W4; Fri, 28 Feb 2020 02:12:10 +0530 From: Arun Isaac Date: Fri, 28 Feb 2020 02:11:50 +0530 Message-Id: <20200227204150.30985-5-arunisaac@systemreboot.net> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20200227204150.30985-1-arunisaac@systemreboot.net> References: <20200227204150.30985-1-arunisaac@systemreboot.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/packages.scm (search-package-index): New function. * guix/scripts/package.scm (find-packages-by-description): Search using the xapian package index if search patterns are literal strings. Else, search using fold-packages. --- gnu/packages.scm | 17 +++++++++++- guix/scripts/package.scm | 57 +++++++++++++++++++++++----------------- 2 files changed, 49 insertions(+), 25 deletions(-) diff --git a/gnu/packages.scm b/gnu/packages.scm index e91753e2a8..5b5b29bf84 100644 --- a/gnu/packages.scm +++ b/gnu/packages.scm @@ -67,7 +67,8 @@ specifications->manifest generate-package-cache - generate-package-search-index)) + generate-package-search-index + search-package-index)) ;;; Commentary: ;;; @@ -453,6 +454,20 @@ reducing the memory footprint." db-path) +(define (search-package-index profile querystring) + (let ((offset 0) + (pagesize 10)) + (call-with-database (string-append profile %package-search-index) + (lambda (db) + (let ((query (parse-query querystring #:stemmer (make-stem "en")))) + (mset-fold (lambda (item result) + (match (find-packages-by-name + (document-data (mset-item-document item))) + ((package _ ...) + (append result `((,package . ,(mset-item-weight item))))))) + '() + (enquire-mset (enquire db query) offset pagesize))))))) + (define %sigint-prompt ;; The prompt to jump to upon SIGINT. diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm index 1cb0d382bf..6a3b9002dd 100644 --- a/guix/scripts/package.scm +++ b/guix/scripts/package.scm @@ -7,6 +7,7 @@ ;;; Copyright © 2016 Benz Schenk ;;; Copyright © 2016 Chris Marusich ;;; Copyright © 2019 Tobias Geerinckx-Rice +;;; Copyright © 2020 Arun Isaac ;;; ;;; This file is part of GNU Guix. ;;; @@ -178,31 +179,40 @@ hooks\" run when building the profile." ;;; Package specifications. ;;; -(define (find-packages-by-description regexps) +(define (find-packages-by-description patterns) "Return a list of pairs: packages whose name, synopsis, description, or output matches at least one of REGEXPS sorted by relevance, and its non-zero relevance score." - (let ((matches (fold-packages (lambda (package result) - (if (package-superseded package) - result - (match (package-relevance package - regexps) - ((? zero?) - result) - (score - (cons (cons package score) - result))))) - '()))) - (sort matches - (lambda (m1 m2) - (match m1 - ((package1 . score1) - (match m2 - ((package2 . score2) - (if (= score1 score2) - (string>? (package-full-name package1) - (package-full-name package2)) - (> score1 score2)))))))))) + (define (regexp? str) + (string-any + (char-set #\. #\[ #\{ #\} #\( #\) #\\ #\* #\+ #\? #\| #\^ #\$) + str)) + + (if (and (current-profile) + (not (any regexp? patterns))) + (search-package-index (current-profile) (string-join patterns " ")) + (let* ((regexps (map (cut make-regexp* <> regexp/icase) patterns)) + (matches (fold-packages (lambda (package result) + (if (package-superseded package) + result + (match (package-relevance package + regexps) + ((? zero?) + result) + (score + (cons (cons package score) + result))))) + '()))) + (sort matches + (lambda (m1 m2) + (match m1 + ((package1 . score1) + (match m2 + ((package2 . score2) + (if (= score1 score2) + (string>? (package-full-name package1) + (package-full-name package2)) + (> score1 score2))))))))))) (define (transaction-upgrade-entry store entry transaction) "Return a variant of TRANSACTION that accounts for the upgrade of ENTRY, a @@ -777,8 +787,7 @@ processed, #f otherwise." (('query 'search rx) rx) (_ #f)) opts)) - (regexps (map (cut make-regexp* <> regexp/icase) patterns)) - (matches (find-packages-by-description regexps))) + (matches (find-packages-by-description patterns))) (leave-on-EPIPE (display-search-results matches (current-output-port))) #t))