From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 11 11:29:42 2020 Received: (at 39258) by debbugs.gnu.org; 11 Feb 2020 16:29:42 +0000 Received: from localhost ([127.0.0.1]:57411 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j1YPu-0004RI-Ij for submit@debbugs.gnu.org; Tue, 11 Feb 2020 11:29:42 -0500 Received: from eggs.gnu.org ([209.51.188.92]:40322) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j1YPs-0004R6-FH for 39258@debbugs.gnu.org; Tue, 11 Feb 2020 11:29:41 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:46770) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1j1YPm-0002am-Gq; Tue, 11 Feb 2020 11:29:34 -0500 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=34126 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1j1YPl-000741-De; Tue, 11 Feb 2020 11:29:34 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Arun Isaac Subject: Re: [bug#39258] Faster guix search using an sqlite cache References: Date: Tue, 11 Feb 2020 17:29:30 +0100 In-Reply-To: (Arun Isaac's message of "Thu, 06 Feb 2020 07:28:22 +0530") Message-ID: <8736bhytn9.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 39258 Cc: 39258@debbugs.gnu.org, zimoun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hello Arun! Arun Isaac skribis: > From 4c883fcff1f44339b28df6ccdb2b10c906439e3d Mon Sep 17 00:00:00 2001 > From: Arun Isaac > Date: Tue, 21 Jan 2020 20:45:43 +0530 > Subject: [PATCH] fast search [...] > --- a/gnu/packages.scm > +++ b/gnu/packages.scm > @@ -43,6 +43,7 @@ > #:use-module (srfi srfi-34) > #:use-module (srfi srfi-35) > #:use-module (srfi srfi-39) > + #:use-module (sqlite3) > #:export (search-patch > search-patches > search-auxiliary-file > @@ -204,10 +205,8 @@ PROC is called along these lines: > PROC can use #:allow-other-keys to ignore the bits it's not interested i= n. > When a package cache is available, this procedure does not actually load= any > package module." > - (define cache > - (load-package-cache (current-profile))) > - > - (if (and cache (cache-is-authoritative?)) > + (if (and (cache-is-authoritative?) > + (current-profile)) > (vhash-fold (lambda (name vector result) > (match vector > (#(name version module symbol outputs > @@ -220,7 +219,7 @@ package module." > #:supported? supported? > #:deprecated? deprecated?)))) > init > - cache) > + (cache-lookup (current-profile))) > (fold-packages (lambda (package result) > (proc (package-name package) > (package-version package) > @@ -252,31 +251,7 @@ is guaranteed to never traverse the same package twi= ce." >=20=20 > (define %package-cache-file > ;; Location of the package cache. > - "/lib/guix/package.cache") > - > -(define load-package-cache [...] > +(define* (cache-lookup profile #:optional name) > "Lookup package NAME in CACHE. Return a list sorted in increasing ver= sion > order." > (define (package-version (version>? (vector-ref v2 1) (vector-ref v1 1))) >=20=20 > - (sort (vhash-fold* cons '() name cache) > - package-version + (define (int->boolean n) > + (case n > + ((0) #f) > + ((1) #t))) > + > + (define (string->list str) > + (call-with-input-string str read)) > + > + (define select-statement > + (string-append > + "SELECT name, version, module, symbol, outputs, supported, supersed= ed, locationFile, locationLine, locationColumn from packages" > + (if name " WHERE name =3D :name" ""))) I would rather keep the current package cache as-is instead of inserting sqlite in here. I don=E2=80=99t expect it to bring much compared performance-wise to the current simple cache (especially if we look at load time), and it does increase complexity quite a bit. However, using sqlite for keyword search as you initially proposed on guix-devel does sound like a great idea to me. WDYT? Thanks, Ludo=E2=80=99.