The bug is reproducible. On Guix commit aa7cdc57dc28673dedfc6ec210974aaa0099a419, a search for keyword "cargo" does not find the "cargo" output of the "rust" package:
--8<---------------cut here---------------start------------->8--- $ guix package --search=cargo name: chromium-bsu version: 0.9.16.1 outputs: out systems: x86_64-linux i686-linux armhf-linux aarch64-linux mips64el-linux dependencies: gettext-minimal@0.19.8.1 glu@9.0.0 pkg-config@0.29.2 + quesoglc@0.7.2 sdl-union@1.2.15 location: gnu/packages/games.scm:3295:2 homepage: http://chromium-bsu.sourceforge.net/ license: Clarified Artistic, Expat synopsis: Fast-paced, arcade-style, top-scrolling space shooter description: In this game you are the captain of the cargo ship Chromium + B.S.U. and are responsible for delivering supplies to the troops on the front + line. Your ship has a small fleet of robotic fighters which you control from + the relative safety of the Chromium vessel. relevance: 2 --8<---------------cut here---------------end--------------->8---
It should have printed all the "rust" packages, and the "mrustc" package, since each of them has a "cargo" output. Example:
--8<---------------cut here---------------start------------->8--- name: rust version: 1.34.0 outputs: out doc cargo systems: x86_64-linux i686-linux armhf-linux aarch64-linux mips64el-linux dependencies: bison@3.0.5 cmake@3.13.1 curl@7.63.0 flex@2.6.4 gdb@8.2.1 + jemalloc@5.1.0 libssh2@1.8.2 llvm@6.0.1 openssl@1.0.2p pkg-config@0.29.2 + procps@3.3.15 python2@2.7.15 rust@1.33.0 which@2.21 location: gnu/packages/rust.scm:1026:4 homepage: https://www.rust-lang.org license: ASL 2.0, Expat synopsis: Compiler for the Rust progamming language description: Rust is a systems programming language that provides memory + safety and thread safety guarantees. --8<---------------cut here---------------end--------------->8---
The general problem here is that Guix does not include outputs whose name matches the given regex. One might argue that we should simply add the output's name to a searchable field, like the description, in cases like this, but if it's easy to fix, we should just fix it.
* guix/ui.scm (%package-metrics): Add a PACKAGE-OUTPUTS metric with a relevance of 1. * guix/scripts/package.scm (process-query)<search>: Add the REGEXP/NEWLINE flag. ---
mikadoZero, Guix,
Here's a patch to match package outputs (except ‘out’, since it can't affect the relative score) in ‘guix search’.
diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm index aa27984ea2..a31e78484e 100644 --- a/guix/scripts/package.scm +++ b/guix/scripts/package.scm @@ -751,7 +751,8 @@ processed, #f otherwise." (('query 'search rx) rx) (_ #f)) opts)) - (regexps (map (cut make-regexp* <> regexp/icase) patterns))) + (regexps (map (cut make-regexp* <> regexp/icase regexp/newline) + patterns))) (leave-on-EPIPE (let-values (((packages scores) (find-packages-by-description regexps))) diff --git a/guix/ui.scm b/guix/ui.scm index 92c845e944..f2466b605b 100644 --- a/guix/ui.scm +++ b/guix/ui.scm @@ -1404,6 +1404,12 @@ score, the more relevant OBJ is to REGEXPS." ;; of regexps. `((,package-name . 4) + ;; Separate package outputs by newlines to match regexps like "^tools$". + ;; Hard-codedly ignore ‘out’ since it presumably exists for every package. + (,(lambda (package) + (string-join (delete "out" (package-outputs package)) + "\n")) . 1) + ;; Match regexps on the raw Texinfo since formatting it is quite expensive ;; and doesn't have much of an effect on search results. (,(lambda (package) -- 2.21.0
> Here's a patch to match package outputs (except ‘out’, since it can't affect the relative score) in ‘guix search’.
Wow, thank you for the quick patch! Indeed, it works as advertised.
That said, I see that your patch only omits the "out" output, and it also changes the way the regular expression matching works for all fields. Previously, a regex like "foo$" would have matched against "foo" at the end of a package's description string, but now it matches "foo" right before any newline in the description. In practice, it seems we have many newlines in the descriptions, since people wrap the lines in the source code with non-escaped newlines. I doubt anyone is relying on the use of $ or ^ to match at the start of end of the description, so this probably isn't a big deal, but it made me wonder how else we might be able to accomplish the same task without changing the way the regexes already work.
I've attached a patch that attempts to do that. Basically, this patch is like yours in spirit, but it excludes a few more "common" outputs, and it splits the outputs into a list, rather than separating them by newlines in a single string. The downside is that I had to update all the other "metrics" procedures so they would always return a list. I kind of feel like returning the same type of object is an upside, not a downside, but I guess the pattern of returning #f for special cases is pretty common in Guile, so maybe my change isn't very idiomatic.
What do you think of this solution? I think it's a little more drastic, but it feels cleaner to me. If I'm bike shedding, feel free to call me out on that! ;-)
From 91d06d8950e700a8a3c3479c3613b579c53411c9 Mon Sep 17 00:00:00 2001 From: Chris Marusich <cmmarusich@gmail.com> Date: Mon, 6 May 2019 01:51:30 -0700 Subject: [PATCH] ui: Make package outputs searchable.
* guix/ui.scm (relevance): Expect the "field" procedure to always return a (possibly empty) list of strings, thereby eliminating the case in which (before this commit) "field" might return #f. Add up the score of each string in the list, and use that as the overall score when folding over the metrics. Update docstring. (%package-metrics): Return a list in every case. * guix/scripts/system/search.scm (process-query) <%service-type-metrics>: Return a list in every case. * guix/scripts/package.scm (find-packages-by-description): Update docstring. --- guix/scripts/package.scm | 6 ++--- guix/scripts/system/search.scm | 17 +++++++++++---- guix/ui.scm | 40 ++++++++++++++++++++++++---------- 3 files changed, 44 insertions(+), 19 deletions(-)
> Tobias Geerinckx-Rice <me@tobias.gr> writes: > >> Here's a patch to match package outputs (except ‘out’, since it can't affect the relative score) in ‘guix search’. > > Wow, thank you for the quick patch! Indeed, it works as advertised. > > That said, I see that your patch only omits the "out" output, and it > also changes the way the regular expression matching works for all > fields.
Yeah, how does this regexp/newline related to matching outputs, Tobias?
> What do you think of this solution? I think it's a little more drastic, > but it feels cleaner to me. If I'm bike shedding, feel free to call me > out on that! ;-)
FWIW I find it a bit too drastic. :-) It leads to much verbosity and I’m not sure this is warranted.
> (define %package-metrics > ;; Metrics used to compute the "relevance score" of a package against a set > ;; of regexps. > - `((,package-name . 4) > - > + `((,(lambda (package) > + (list (package-name package))) > + . 4) > + ;; Match against uncommon outputs. > + (,(lambda (package) > + (filter (lambda (output) > + (not (member output > + ;; Some common outpus shared by many packages. > + '("out" "debug" "doc" "static")))) > + (package-outputs package))) > + . 1)
Could we have just this hunk or is there something I’m missing that would make it insufficient?
> Could we have just this hunk or is there something I’m missing that > would make it insufficient?
That hunk alone is not enough, but I think the attached patch would do the trick. We just need to allow for the new possibility that the "field" procedure returns a list of strings.
From c1150a217a416ef4ceccf87c56e36e8e921f873a Mon Sep 17 00:00:00 2001 From: Chris Marusich <cmmarusich@gmail.com> Date: Mon, 6 May 2019 01:51:30 -0700 Subject: [PATCH] ui: Make package outputs searchable.
* guix/ui.scm (relevance): Allow the "field" procedure of a metric to return a list, and handle that case appropriately. Update docstring. (%package-metrics): Add a metric for package outputs. * guix/scripts/package.scm (find-packages-by-description): Update docstring.
Works for me! I've changed the match clause accordingly in the attached patch.
swedebugia <swedebugia@riseup.net> writes:
> On 2019-05-07 02:57, Chris Marusich wrote: >> + ;; Match against uncommon outputs. >> + (,(lambda (package) >> + (filter (lambda (output) >> + (not (member output >> + ;; Some common outpus shared by many packages. >> + '("out" "debug" "doc" "static")))) > > I suggest we add "gui" and "lib" to this list.
Actually, I was curious about this, so I checked how many outputs are being used by all our packages today. Here are the results:
From dedea618bdd0d557e3e1183c403ff4e11d1757f8 Mon Sep 17 00:00:00 2001 From: Chris Marusich <cmmarusich@gmail.com> Date: Mon, 6 May 2019 01:51:30 -0700 Subject: [PATCH] ui: Make package outputs searchable.
* guix/ui.scm (relevance): Allow the "field" procedure of a metric to return a list, and handle that case appropriately. Update docstring. (%package-metrics): Add a metric for package outputs. * guix/scripts/package.scm (find-packages-by-description): Update docstring. * tests/guix-package.sh: Add a test case to verify that package outputs are included in search results.
> In light of that, I've chosen to exclude all of the following outputs: > > '("out" "doc" "debug" "lib" "static" "bin" "examples" "gui" "tests" > "utils" "include") > > I've also added a test to verify that package outputs are included in > search results.
I would exclude only the standard output names used by (guix build gnu-build-system), so:
("out" "doc" "debug" "lib" "include" "bin")
That way one can still do ‘guix search git gui’ or ‘guix search bind utils’.
> I would exclude only the standard output names used by (guix build > gnu-build-system), so: > > ("out" "doc" "debug" "lib" "include" "bin") > > That way one can still do ‘guix search git gui’ or ‘guix search bind utils’.
I didn't know you could do that! Your suggestion makes sense. I've done as you suggest in commit 387e6b9e340ce4b401f220f72881415623a466f7.