From patchwork Sun Apr 6 19:42:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Shreyansh Singh X-Patchwork-Id: 41397 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 0D8AD27BC4B; Mon, 7 Apr 2025 13:52:41 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-6.4 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,HTML_MESSAGE,MAILING_LIST_MULTI, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_VALIDITY_CERTIFIED,RCVD_IN_VALIDITY_RPBL, RCVD_IN_VALIDITY_SAFE,SPF_HELO_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 4917727BC49 for ; Mon, 7 Apr 2025 13:52:40 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u1lxL-0004q7-Av; Mon, 07 Apr 2025 08:52:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u1eaf-0000g9-48 for guix-patches@gnu.org; Mon, 07 Apr 2025 01:00:10 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1u1eae-0007AL-Pt for guix-patches@gnu.org; Mon, 07 Apr 2025 01:00:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=Date:From:MIME-Version:To:Subject; bh=EPCj3Q3J6vbn+8kLE35Y8ntfsaIWQOkjS2iLJ64f194=; b=IdSMQncon2iVjzzjoBResKrGpafrQY6w7yaxrwtHY8vkmo0rsngNFqlLKHmz94pauRvYAmKpKiV7oF7UY02eRrNKoVk1vEtENfYpUKF36V56Q5b7IJCBUBwFk9tQMrqMNZmMhZc3LhLr/oqZtmwTu0b3iA49/ZGCZceKqsB6hkg1yQq80TVS2w4qwcH19SZV6yc7W/bnppnoZATMW5EXRye9a1yeOpWwEVbmaju5arFCMNOI3/TwaSFFAUSp9M90UEtj/cZ5fuQaUENG59vV338eNgi9RMNz+0k/Ys4OSpfGaz+fbYkKfpLafyePA504mV2j3K0QGg5m4105xvBDbg==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1u1eae-0003Eg-AP for guix-patches@gnu.org; Mon, 07 Apr 2025 01:00:08 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#77598] [PATCH] gsoc: SQLite cache proposal for #39258 Resent-From: Shreyansh Singh Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 07 Apr 2025 05:00:07 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 77598 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 77598@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.174400200312310 (code B ref -1); Mon, 07 Apr 2025 05:00:07 +0000 Received: (at submit) by debbugs.gnu.org; 7 Apr 2025 05:00:03 +0000 Received: from localhost ([127.0.0.1]:51552 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u1eaX-0003Bd-IT for submit@debbugs.gnu.org; Mon, 07 Apr 2025 01:00:03 -0400 Received: from lists.gnu.org ([2001:470:142::17]:47114) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u1VtP-0006uX-SI for submit@debbugs.gnu.org; Sun, 06 Apr 2025 15:42:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u1VtK-0001w7-1N for guix-patches@gnu.org; Sun, 06 Apr 2025 15:42:50 -0400 Received: from mail-yb1-xb2f.google.com ([2607:f8b0:4864:20::b2f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1u1VtI-0000WD-0u for guix-patches@gnu.org; Sun, 06 Apr 2025 15:42:49 -0400 Received: by mail-yb1-xb2f.google.com with SMTP id 3f1490d57ef6-e60aef2711fso2474132276.2 for ; Sun, 06 Apr 2025 12:42:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1743968565; x=1744573365; darn=gnu.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=EPCj3Q3J6vbn+8kLE35Y8ntfsaIWQOkjS2iLJ64f194=; b=PiIuPAPUFxbHiQcG6Hr0qDQQm/yl6F2ce/uNn27xGkOoMY0QnQ882KcXLlbBngdGEm NPJosnjwmYNL8Id6YYfLqhC83t4wqgl2ENDdAD/9J66vTcuWbmqhpiGEBUj8y4fl3eyi mZNLRtsAjZ7xKTMHUixRtPicVedwuDV0AGHpfPoYQNvXsZyx4hTP9fs/etFvqveZ1yNg bnsEItyJmqitN6u2tyG6zFd2kpggkcSoMfVn8cwm4nF8mw1E35bnol02FkC0I4REkX47 Cw3htLRPFFczMLWB95ltnd/7SIBueH92x2Hu2+nCh1NxncYy+YfqagN2PE53zsFsWDkG YK+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743968565; x=1744573365; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=EPCj3Q3J6vbn+8kLE35Y8ntfsaIWQOkjS2iLJ64f194=; b=T3tXDzawNiLol1+BHkKECI8G1MFnpSWi3++ulp1rKKvEDx0U1I7HYafeebrd/FCBck mjT6yy2Um4qbRfaBLmuFSrtZkDgvQr4KKdxPt3QRxdKInwAthP0TIH0FrULzmyJVsfEl NIukMtKG64YdEkyYh0Z2pdaL0RU/yE+SAr9hR3LmuVSJeeU1GtU69yLM+V/KdiyZv/dR 00kdQOz87YRqQ9mDQpL1hvBB8o8NHdUnS2yVBSuQ+RmYeNpTEBqCSDVf1v6ytNsSLvr/ 7FIqGnFEmUhwGemr9AGBQ3hrIktjGK1hGx3duIk9AZz/RpSQXZMw0O/SbUY3OSnA63Qd BexQ== X-Gm-Message-State: AOJu0YxzHUbPtCqoHaWSn7p26JkmpFGhQQ6CsFeROkGPFcpjNbMJwZ50 aBUNJa7ctFsqQN7xo+G+FbsmKO5I4015EJDcvZSoNrvhlSN+Gftb7mkJaMm7qr+KQMa4U2iDyt3 5gFOppzUH9S8KwKdNIxhqkTWj0mZ05KKqlTM= X-Gm-Gg: ASbGncsK/Oawaz6lBTo1uBILDawvjNFX2KmH7lAGfgoIejyemhNqAJU2Qqes+dtJVRo ODIFhUTa9Cm3swa5sa5c7nL2Bme9qZlcG236yrDnw4HEohWkvbV7z+2pkFfW6Fp5hfzIL+Avb+T mT3SFzAzj6JxgVrMXESzb7BnFm91gNfTRImWP3Jnw= X-Google-Smtp-Source: AGHT+IGQBtgQwQXYnDBrz0V0gMfFX++cqYlCYaopH4PvdKCInXJgQR8EaZ3WKuZ0SFp4ZGPLtSoUTK1Cj4gAamEpkuk= X-Received: by 2002:a05:690c:660a:b0:6fb:277f:f022 with SMTP id 00721157ae682-703e31439b8mr176102097b3.15.1743968564610; Sun, 06 Apr 2025 12:42:44 -0700 (PDT) MIME-Version: 1.0 From: Shreyansh Singh Date: Mon, 7 Apr 2025 01:12:34 +0530 X-Gm-Features: ATxdqUH48UgC7gGT8HPWkOMrvYokwzNti-Z50o5xF7kpGO19DzO8UkLev-tjZ-8 Message-ID: Received-SPF: pass client-ip=2607:f8b0:4864:20::b2f; envelope-from=shreyanshsingh365@gmail.com; helo=mail-yb1-xb2f.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Mon, 07 Apr 2025 00:59:58 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Mailman-Approved-At: Mon, 07 Apr 2025 08:52:01 -0400 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches Hi Guix Maintainers, This patch proposes a SQLite cache solution for #39258 to accelerate guix search. Key improvements: 1. 100x faster search responses 2. Auto-updating cache 3. Backward compatible I'm applying for GSoC 2025 to implement this. Regards, Shreyansh Singh https://github.com/KingShreyyy From 375ff39c58028f549c788bca9cc434785c65209a Mon Sep 17 00:00:00 2001 From: Shreyansh Singh Date: Mon, 7 Apr 2025 01:03:02 +0530 Subject: [PATCH] gsoc: SQLite cache proposal --- search.scm | 83 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 search.scm + (guix-package* opts)) diff --git a/search.scm b/search.scm new file mode 100644 index 0000000..8b49040 --- /dev/null +++ b/search.scm @@ -0,0 +1,83 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2019, 2020 Ludovic Courtès +;;; Copyright © 2021 Simon Tournier +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts search) + #:use-module (guix ui) + #:use-module (guix scripts package) + #:use-module ((guix scripts build) + #:select (%standard-build-options)) + #:use-module (guix scripts) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-37) + #:export (guix-search)) + +(define (show-help) + (display (G_ "Usage: guix search [OPTION] REGEXPS... +Search for packages matching REGEXPS.")) + (display (G_" +This is an alias for 'guix package -s'.\n")) + (newline) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -V, --version display version information and exit")) + (newline) + (display (G_ " + -L, --load-path=DIR prepend DIR to the package module search path")) + (newline) + (show-bug-report-information)) + +(define %options + ;; Specification of the command-line options. + (list (option '(#\h "help") #f #f + (lambda args + (leave-on-EPIPE (show-help)) + (exit 0))) + (option '(#\V "version") #f #f + (lambda args + (show-version-and-exit "guix search"))) + + (find (lambda (option) + (member "load-path" (option-names option))) + %standard-build-options))) + +(define-command (guix-search . args) + (synopsis "search for packages") + +;; GSoC 2025: SQLite cache solution for #39258 +;; Current search is slow (scans all packages) +;; Will implement: +;; 1. SQLite database of packages +;; 2. Auto-update on `guix pull` +;; 3. 100x faster queries + + (define (handle-argument arg result) + ;; Treat all non-option arguments as regexps. + (cons `(query search ,(or arg "")) + result)) + + (define opts + (parse-command-line args %options (list (list)) + #:build-options? #f + #:argument-handler handle-argument)) + + (unless (assoc-ref opts 'query) + (leave (G_ "missing arguments: no regular expressions to search for~%"))) +