guix package --search does not search output names

DoneSubmitted by Chris Marusich.
C
Chris Marusich wrote on Sun May 05 12:39:05-0700 2019
(address . bug-guix@gnu.org)
87r29czo6e.fsf@gmail.com
Hi,

mikadoZero reported a bug here:

https://lists.gnu.org/archive/html/help-guix/2019-04/msg00206.html

The bug is reproducible. On Guix commit
aa7cdc57dc28673dedfc6ec210974aaa0099a419, a search for keyword "cargo"
does not find the "cargo" output of the "rust" package:

--8<---------------cut here---------------start------------->8---
$ guix package --search=cargo
name: chromium-bsu
version: 0.9.16.1
outputs: out
systems: x86_64-linux i686-linux armhf-linux aarch64-linux mips64el-linux
dependencies: gettext-minimal@0.19.8.1 glu@9.0.0 pkg-config@0.29.2
+ quesoglc@0.7.2 sdl-union@1.2.15
location: gnu/packages/games.scm:3295:2
homepage: http://chromium-bsu.sourceforge.net/
license: Clarified Artistic, Expat
synopsis: Fast-paced, arcade-style, top-scrolling space shooter
description: In this game you are the captain of the cargo ship Chromium
+ B.S.U. and are responsible for delivering supplies to the troops on the front
+ line. Your ship has a small fleet of robotic fighters which you control from
+ the relative safety of the Chromium vessel.
relevance: 2
--8<---------------cut here---------------end--------------->8---

It should have printed all the "rust" packages, and the "mrustc"
package, since each of them has a "cargo" output. Example:

--8<---------------cut here---------------start------------->8---
name: rust
version: 1.34.0
outputs: out doc cargo
systems: x86_64-linux i686-linux armhf-linux aarch64-linux mips64el-linux
dependencies: bison@3.0.5 cmake@3.13.1 curl@7.63.0 flex@2.6.4 gdb@8.2.1
+ jemalloc@5.1.0 libssh2@1.8.2 llvm@6.0.1 openssl@1.0.2p pkg-config@0.29.2
+ procps@3.3.15 python2@2.7.15 rust@1.33.0 which@2.21
location: gnu/packages/rust.scm:1026:4
homepage: https://www.rust-lang.org
license: ASL 2.0, Expat
synopsis: Compiler for the Rust progamming language
description: Rust is a systems programming language that provides memory
+ safety and thread safety guarantees.
--8<---------------cut here---------------end--------------->8---

The general problem here is that Guix does not include outputs whose
name matches the given regex. One might argue that we should simply add
the output's name to a searchable field, like the description, in cases
like this, but if it's easy to fix, we should just fix it.

--
Chris
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAlzPO9kACgkQ3UCaFdgi
Rp1Hdg//bDcqZLlY1wzBNhWaQek/zM1m44KKllUgWW3sMAZ2YR7EO14bq+h9IlOS
KIfDVb1WVnjV3T+vY7MVhuUqCrnonyRYlKnidP2rWFf3fpVB+9aA7FuI0s8V5AWM
hFg9KOZvYGZx3ECMvfWqNutvnftbRMgjy0zIBePvY+1IWn8P3NGYxKAEMjYm+F1W
Irieg+MABrRGShPbcI5L0GEsn8TvNhQE5o1V2LS9RKqvohAgL7FSe6co5SkaezKv
xO9bULtFyI9HIgzKRezhwIW8Dslp2oF4BaExRl4E6ft5Wa+BKzFdWom17POSWbMS
erpNA5pmZNOHphkndb0ydQOn4ykopD1V/w6p/4X62KNieFgS4JPjy8Vwwj8y4RH4
MQGdWwujU/9+SCHOy7m72F1WTqheA5uvReVwHHJhMlpxS9n+by+0mnxPyc+51Kya
8Tl2MgrX4STZqcR+N0sSya3RMVpTSPWqp2D0URlb0TfZmnU78dJuoYqMJ3+gNRuF
nShog/90gYe5Tcx7oU8xqwEUOCBGI2W7FPoQFMPp5JHR12EdWJnm5kYd/vOODSDL
Au4riebspMRi2qfxP/+3v37v7TtKRGOSzWRJBbLJFMmPSvHJZ3gr8iCdplepxiFr
LbiNMOTnL9v72zlibkfCSkcignV6BRyaO2w6k/tI3Hq9TEybOoI=
=vlAX
-----END PGP SIGNATURE-----
T
Tobias Geerinckx-Rice wrote on Sun May 05 23:41:53+0200 2019
[PATCH] ui: Search matches additional package outputs.
(address . 35588@debbugs.gnu.org)
20190505214153.32372-1-me@tobias.gr
* guix/ui.scm (%package-metrics): Add a PACKAGE-OUTPUTS metric with a
relevance of 1.
* guix/scripts/package.scm (process-query)<search>: Add the
REGEXP/NEWLINE flag.
---

mikadoZero, Guix,

Here's a patch to match package outputs (except ‘out’, since it can't affect the relative score) in ‘guix search’.

Before:

~ λ guix search ernel-patch
# nothing

After:

~ λ guix search ernel-patch
name: wireguard
version: 0.0.20190406
outputs: out kernel-patch


~ λ guix search ^ernel-patch
# nothing

While the new REGEXP/NEWLINE flag affects how all fields are matched, I don't think it actually changes anything in practice without the second hunk.

If there's a possibility that it might, I'd split this into two commits.

Kind regards,

T G-R

guix/scripts/package.scm | 3 ++-
guix/ui.scm | 6 ++++++
2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm
index aa27984ea2..a31e78484e 100644
--- a/guix/scripts/package.scm
+++ b/guix/scripts/package.scm
@@ -751,7 +751,8 @@ processed, #f otherwise."
(('query 'search rx) rx)
(_ #f))
opts))
- (regexps (map (cut make-regexp* <> regexp/icase) patterns)))
+ (regexps (map (cut make-regexp* <> regexp/icase regexp/newline)
+ patterns)))
(leave-on-EPIPE
(let-values (((packages scores)
(find-packages-by-description regexps)))
diff --git a/guix/ui.scm b/guix/ui.scm
index 92c845e944..f2466b605b 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -1404,6 +1404,12 @@ score, the more relevant OBJ is to REGEXPS."
;; of regexps.
`((,package-name . 4)

+ ;; Separate package outputs by newlines to match regexps like "^tools$".
+ ;; Hard-codedly ignore ‘out’ since it presumably exists for every package.
+ (,(lambda (package)
+ (string-join (delete "out" (package-outputs package))
+ "\n")) . 1)
+
;; Match regexps on the raw Texinfo since formatting it is quite expensive
;; and doesn't have much of an effect on search results.
(,(lambda (package)
--
2.21.0



C
Chris Marusich wrote on Mon May 06 02:08:58-0700 2019
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)(address . 35588@debbugs.gnu.org)
87bm0g6jbp.fsf@garuda.local.i-did-not-set--mail-host-address--so-tickle-me
Hi Tobias!

Tobias Geerinckx-Rice <me@tobias.gr> writes:

> Here's a patch to match package outputs (except ‘out’, since it can't affect the relative score) in ‘guix search’.

Wow, thank you for the quick patch! Indeed, it works as advertised.

That said, I see that your patch only omits the "out" output, and it
also changes the way the regular expression matching works for all
fields. Previously, a regex like "foo$" would have matched against
"foo" at the end of a package's description string, but now it matches
"foo" right before any newline in the description. In practice, it
seems we have many newlines in the descriptions, since people wrap the
lines in the source code with non-escaped newlines. I doubt anyone is
relying on the use of $ or ^ to match at the start of end of the
description, so this probably isn't a big deal, but it made me wonder
how else we might be able to accomplish the same task without changing
the way the regexes already work.

I've attached a patch that attempts to do that. Basically, this patch
is like yours in spirit, but it excludes a few more "common" outputs,
and it splits the outputs into a list, rather than separating them by
newlines in a single string. The downside is that I had to update all
the other "metrics" procedures so they would always return a list. I
kind of feel like returning the same type of object is an upside, not a
downside, but I guess the pattern of returning #f for special cases is
pretty common in Guile, so maybe my change isn't very idiomatic.

What do you think of this solution? I think it's a little more drastic,
but it feels cleaner to me. If I'm bike shedding, feel free to call me
out on that! ;-)

--
Chris
From 91d06d8950e700a8a3c3479c3613b579c53411c9 Mon Sep 17 00:00:00 2001
From: Chris Marusich <cmmarusich@gmail.com>
Date: Mon, 6 May 2019 01:51:30 -0700
Subject: [PATCH] ui: Make package outputs searchable.

* guix/ui.scm (relevance): Expect the "field" procedure to always return
a (possibly empty) list of strings, thereby eliminating the case in
which (before this commit) "field" might return #f. Add up the score of
each string in the list, and use that as the overall score when folding
over the metrics. Update docstring.
(%package-metrics): Return a list in every case.
* guix/scripts/system/search.scm (process-query)
<%service-type-metrics>: Return a list in every case.
* guix/scripts/package.scm (find-packages-by-description): Update
docstring.
---
guix/scripts/package.scm | 6 ++---
guix/scripts/system/search.scm | 17 +++++++++++----
guix/ui.scm | 40 ++++++++++++++++++++++++----------
3 files changed, 44 insertions(+), 19 deletions(-)

diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm
index aa27984ea2..06e4cf5b9c 100644
--- a/guix/scripts/package.scm
+++ b/guix/scripts/package.scm
@@ -180,9 +180,9 @@ hooks\" run when building the profile."
;;;

(define (find-packages-by-description regexps)
- "Return two values: the list of packages whose name, synopsis, or
-description matches at least one of REGEXPS sorted by relevance, and the list
-of relevance scores."
+ "Return two values: the list of packages whose name, synopsis, description,
+or output matches at least one of REGEXPS sorted by relevance, and the list of
+relevance scores."
(let ((matches (fold-packages (lambda (package result)
(if (package-superseded package)
result
diff --git a/guix/scripts/system/search.scm b/guix/scripts/system/search.scm
index 955cdd1e95..733fa22614 100644
--- a/guix/scripts/system/search.scm
+++ b/guix/scripts/system/search.scm
@@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2017, 2018 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2018 Clément Lassieur <clement@lassieur.org>
+;;; Copyright © 2019 Chris Marusich <cmmarusich@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -128,14 +129,22 @@ columns."

(define %service-type-metrics
;; Metrics used to estimate the relevance of a search result.
- `((,service-type-name* . 3)
- (,service-type-description-string . 2)
+ `((,(lambda (type)
+ (match (service-type-name* type)
+ (#f '())
+ (name (list name))))
+ . 3)
+ (,(lambda (type)
+ (match (service-type-description-string type)
+ (#f '())
+ (description (list description))))
+ . 2)
(,(lambda (type)
(match (and=> (service-type-location type) location-file)
((? string? file)
- (basename file ".scm"))
+ (list (basename file ".scm")))
(#f
- "")))
+ '())))
. 1)))

(define (find-service-types regexps)
diff --git a/guix/ui.scm b/guix/ui.scm
index 92c845e944..9121e3daee 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -11,6 +11,7 @@
;;; Copyright © 2016 Benz Schenk <benz.schenk@uzh.ch>
;;; Copyright © 2018 Kyle Meyer <kyle@kyleam.com>
;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2019 Chris Marusich <cmmarusich@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -1370,7 +1371,7 @@ WIDTH columns. EXTRA-FIELDS is a list of symbol/value pairs to emit."
(define (relevance obj regexps metrics)
"Compute a \"relevance score\" for OBJ as a function of its number of
matches of REGEXPS and accordingly to METRICS. METRICS is list of
-field/weight pairs, where FIELD is a procedure that returns a string
+field/weight pairs, where FIELD is a procedure that returns a list of strings
describing OBJ, and WEIGHT is a positive integer denoting the weight of this
field in the final score.

@@ -1392,29 +1393,44 @@ score, the more relevant OBJ is to REGEXPS."
(fold (lambda (metric relevance)
(match metric
((field . weight)
- (match (field obj)
- (#f relevance)
- (str (+ relevance
- (* (score str) weight)))))))
+ (let ((strings (field obj)))
+ (+ relevance
+ (* weight
+ ;; Evaluates to 0 when strings is the empty list.
+ (apply + (map score strings))))))))
0
metrics))

(define %package-metrics
;; Metrics used to compute the "relevance score" of a package against a set
;; of regexps.
- `((,package-name . 4)
-
+ `((,(lambda (package)
+ (list (package-name package)))
+ . 4)
+ ;; Match against uncommon outputs.
+ (,(lambda (package)
+ (filter (lambda (output)
+ (not (member output
+ ;; Some common outpus shared by many packages.
+ '("out" "debug" "doc" "static"))))
+ (package-outputs package)))
+ . 1)
;; Match regexps on the raw Texinfo since formatting it is quite expensive
;; and doesn't have much of an effect on search results.
(,(lambda (package)
- (and=> (package-synopsis package) P_)) . 3)
+ (match (and=> (package-synopsis package) P_)
+ (#f '())
+ (synopsis (list synopsis))))
+ . 3)
(,(lambda (package)
- (and=> (package-description package) P_)) . 2)
-
+ (match (and=> (package-description package) P_)
+ (#f '())
+ (description (list description))))
+ . 2)
(,(lambda (type)
(match (and=> (package-location type) location-file)
- ((? string? file) (basename file ".scm"))
- (#f "")))
+ ((? string? file) (list (basename file ".scm")))
+ (#f '())))
. 1)))

(define (package-relevance package regexps)
--
2.20.1
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAlzP+aoACgkQ3UCaFdgi
Rp2vHBAAt0ZFgM8T0NjBBlgrBuuYMH3Rx6Nk5fQt/9qzNQNMo0t2BeRFwHjZLmST
/bKFpQi8H3/u/wRVr0bKcwM8P5Iu9C+swX6AJckt0h1XO85J/EOI4THCjCIAIx+/
vWMTNfaUhT1XEp3VLU/IpbyNyM7ZEEXL5PoE1UD0ZkL2ODuXHK1x4qO9lOApFvTq
FRqveKBez1Qbj+jpWGRqGKbH4x0APg8Cm9M+jlGuzgw89lu0D44h0OB0A+s0GbMi
4OOZCug9eYKdeMQkpj4gv7HFjSSirrsCJnDOv+uK3y65oYUxfhV+nmSKIxshotr5
kElJn1UlJt7Eq0crqZ56Q1cnARNQKzOLiNleKZuzKSZf0n/3fsSTBvzEtajJmSqr
vUSPB1ySWlLjva98azP0M8HxDE/X4xNyqgUlC9iKPZsi+nq+lFMbbzj1ljE4Upy9
c6ZaDL2hM71OqY3GUQYimeksaaKRdXexuO/LttV98L1zAiI494ZNoJ9hG7KHByd3
eJVhEtpGar8PynjapQhQPXqPwziEiqF/kcIUJ199BG44B11Sq3/8hDjcvcOiUEC8
D12vmC1BEvdIM3b/LJPFVQMSV/cxo47IXM3wGAwN3hp7CxWN1kyFZPY9LN7FdiQv
WxaeOSmZGn9HN4AYSmLaSaF79+L5EYSjS4gAG14wdO0gYU+sMR4=
=PVgI
-----END PGP SIGNATURE-----
L
Ludovic Courtès wrote on Mon May 06 11:32:12+0200 2019
(name . Chris Marusich)(address . cmmarusich@gmail.com)
87muk0kjxf.fsf@gnu.org
Hello,

Chris Marusich <cmmarusich@gmail.com> skribis:

> Tobias Geerinckx-Rice <me@tobias.gr> writes:
>
>> Here's a patch to match package outputs (except ‘out’, since it can't affect the relative score) in ‘guix search’.
>
> Wow, thank you for the quick patch! Indeed, it works as advertised.
>
> That said, I see that your patch only omits the "out" output, and it
> also changes the way the regular expression matching works for all
> fields.

Yeah, how does this regexp/newline related to matching outputs, Tobias?

> What do you think of this solution? I think it's a little more drastic,
> but it feels cleaner to me. If I'm bike shedding, feel free to call me
> out on that! ;-)

FWIW I find it a bit too drastic. :-) It leads to much verbosity and
I’m not sure this is warranted.

> (define %package-metrics
> ;; Metrics used to compute the "relevance score" of a package against a set
> ;; of regexps.
> - `((,package-name . 4)
> -
> + `((,(lambda (package)
> + (list (package-name package)))
> + . 4)
> + ;; Match against uncommon outputs.
> + (,(lambda (package)
> + (filter (lambda (output)
> + (not (member output
> + ;; Some common outpus shared by many packages.
> + '("out" "debug" "doc" "static"))))
> + (package-outputs package)))
> + . 1)

Could we have just this hunk or is there something I’m missing that
would make it insufficient?

Thank you,
Ludo’.


C
Chris Marusich wrote on Mon May 06 17:57:46-0700 2019
(name . Ludovic Courtès)(address . ludo@gnu.org)
87zhnzxer9.fsf@gmail.com
Ludovic Courtès <ludo@gnu.org> writes:

> Could we have just this hunk or is there something I’m missing that
> would make it insufficient?

That hunk alone is not enough, but I think the attached patch would do
the trick. We just need to allow for the new possibility that the
"field" procedure returns a list of strings.

What do you think?

--
Chris
From c1150a217a416ef4ceccf87c56e36e8e921f873a Mon Sep 17 00:00:00 2001
From: Chris Marusich <cmmarusich@gmail.com>
Date: Mon, 6 May 2019 01:51:30 -0700
Subject: [PATCH] ui: Make package outputs searchable.

* guix/ui.scm (relevance): Allow the "field" procedure of a metric to
return a list, and handle that case appropriately. Update docstring.
(%package-metrics): Add a metric for package outputs.
* guix/scripts/package.scm (find-packages-by-description): Update
docstring.

Co-authored-by: Tobias Geerinckx-Rice <me@tobias.gr>
---
guix/scripts/package.scm | 6 +++---
guix/ui.scm | 24 +++++++++++++++++++-----
2 files changed, 22 insertions(+), 8 deletions(-)

diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm
index aa27984ea2..06e4cf5b9c 100644
--- a/guix/scripts/package.scm
+++ b/guix/scripts/package.scm
@@ -180,9 +180,9 @@ hooks\" run when building the profile."
;;;

(define (find-packages-by-description regexps)
- "Return two values: the list of packages whose name, synopsis, or
-description matches at least one of REGEXPS sorted by relevance, and the list
-of relevance scores."
+ "Return two values: the list of packages whose name, synopsis, description,
+or output matches at least one of REGEXPS sorted by relevance, and the list of
+relevance scores."
(let ((matches (fold-packages (lambda (package result)
(if (package-superseded package)
result
diff --git a/guix/ui.scm b/guix/ui.scm
index 92c845e944..b7ccd8312a 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -11,6 +11,8 @@
;;; Copyright © 2016 Benz Schenk <benz.schenk@uzh.ch>
;;; Copyright © 2018 Kyle Meyer <kyle@kyleam.com>
;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2019 Chris Marusich <cmmarusich@gmail.com>
+;;; Copyright © 2019 Tobias Geerinckx-Rice <me@tobias.gr>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -1370,9 +1372,9 @@ WIDTH columns. EXTRA-FIELDS is a list of symbol/value pairs to emit."
(define (relevance obj regexps metrics)
"Compute a \"relevance score\" for OBJ as a function of its number of
matches of REGEXPS and accordingly to METRICS. METRICS is list of
-field/weight pairs, where FIELD is a procedure that returns a string
-describing OBJ, and WEIGHT is a positive integer denoting the weight of this
-field in the final score.
+field/weight pairs, where FIELD is a procedure that returns a string or list
+of strings describing OBJ, and WEIGHT is a positive integer denoting the
+weight of this field in the final score.

A score of zero means that OBJ does not match any of REGEXPS. The higher the
score, the more relevant OBJ is to REGEXPS."
@@ -1394,8 +1396,11 @@ score, the more relevant OBJ is to REGEXPS."
((field . weight)
(match (field obj)
(#f relevance)
- (str (+ relevance
- (* (score str) weight)))))))
+ ((? string? str) (+ relevance
+ (* (score str) weight)))
+ ((? list? lst) (+ relevance
+ (* weight
+ (apply + (map score lst)))))))))
0
metrics))

@@ -1404,6 +1409,15 @@ score, the more relevant OBJ is to REGEXPS."
;; of regexps.
`((,package-name . 4)

+ ;; Match against uncommon outputs.
+ (,(lambda (package)
+ (filter (lambda (output)
+ (not (member output
+ ;; Some common outpus shared by many packages.
+ '("out" "debug" "doc" "static"))))
+ (package-outputs package)))
+ . 1)
+
;; Match regexps on the raw Texinfo since formatting it is quite expensive
;; and doesn't have much of an effect on search results.
(,(lambda (package)
--
2.20.1
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAlzQ2AoACgkQ3UCaFdgi
Rp1pqxAAlaBwzbBdkq1QL3T+eJusUlwMF3PV/2SPXlnGxO90Hl5Vlpcxx5v1CN2S
ZCwQR5twjWhG6LuwuTCpaJ8XDUcseJIXY335Sq2TCRTbkYxugFltWGONpXBTERIm
KlpGW2cRUsO+NWgi3Hdy/oC5PcY7kS/z2S74002M9Z0qcx4LuiKIG06hIsi2yMxn
j3tBKcqk+DSr3PjG/HghSvb1W9Bisltq0S7NKOOWL1o1TFVGnsRScRFsCQVNn8Cp
jwpc10TSVW02xdfIJwJcmQxE4nI4GtMTBGPzn5Bg/myuOlhgRk6N+YIannQ8EHhx
G1otQDSoCoLsEiuX+a0SUaEKQ4LpNO0HjbgucsFDqJhbcTHLYZtx22AbvR6ckpiK
F82mU8Oh2fBGLLnh1siNI7WXoFp/iGN2oFHcRqgLVqFRScvz9kDS/HKtnS+eQsbU
fLBCFJ6YkaLJ2XqXVdOr6CrwC4UGEW52EBIPn361VAB5/4HKZQw71LFxICDDkyrB
P5PJuTUS30SwnU+IG3Pm1xHA/PiWYE/EKc2ipZOPLLE7cT4iG3x1IMhvvmHFGawC
/HhzMTjmwLw3Qn7kvhphKUAOWfNv5ZV1l8BtIqRBqK9mh1Kl6OLUkai8kSLjfF5/
xQgeI9A8WRRP4D0VK+oy9LnESG6i5yOJpP+mVEuTHXVTHg5YpZU=
=0Ax+
-----END PGP SIGNATURE-----
L
Ludovic Courtès wrote on Tue May 07 10:25:39+0200 2019
(name . Chris Marusich)(address . cmmarusich@gmail.com)
87mujy8yd8.fsf@gnu.org
Hello,

Chris Marusich <cmmarusich@gmail.com> skribis:

> From c1150a217a416ef4ceccf87c56e36e8e921f873a Mon Sep 17 00:00:00 2001
> From: Chris Marusich <cmmarusich@gmail.com>
> Date: Mon, 6 May 2019 01:51:30 -0700
> Subject: [PATCH] ui: Make package outputs searchable.
>
> * guix/ui.scm (relevance): Allow the "field" procedure of a metric to
> return a list, and handle that case appropriately. Update docstring.
> (%package-metrics): Add a metric for package outputs.
> * guix/scripts/package.scm (find-packages-by-description): Update
> docstring.
>
> Co-authored-by: Tobias Geerinckx-Rice <me@tobias.gr>

[...]

> (match (field obj)
> (#f relevance)
> - (str (+ relevance
> - (* (score str) weight)))))))
> + ((? string? str) (+ relevance
> + (* (score str) weight)))
> + ((? list? lst) (+ relevance
> + (* weight
> + (apply + (map score lst)))))))))

Nitpick: it’s a bit subjective, but I think this clause might be
slightly nicer like this:

((lst ...)
(+ relevance (* weight (reduce + 0 (map score lst)))))

Anyway, LGTM!

Thanks,
Ludo’.


S
swedebugia wrote on Wed May 08 00:24:39+0200 2019
(address . 35588@debbugs.gnu.org)
7951ec53-8eee-04a3-9be4-7501d46eceb8@riseup.net
On 2019-05-07 02:57, Chris Marusich wrote:
> + ;; Match against uncommon outputs.
> + (,(lambda (package)
> + (filter (lambda (output)
> + (not (member output
> + ;; Some common outpus shared by many packages.
> + '("out" "debug" "doc" "static"))))

I suggest we add "gui" and "lib" to this list.

--
Cheers
Swedebugia


C
Chris Marusich wrote on Tue May 07 23:50:29-0700 2019
Re: bug#35588: [PATCH] ui: Search matches additional package outputs., bug#35588: [PATCH] ui: Search matches additional package outputs.
878svhv3re.fsf@gmail.com
Ludovic Courtès <ludo@gnu.org> writes:

>> (match (field obj)
>> (#f relevance)
>> - (str (+ relevance
>> - (* (score str) weight)))))))
>> + ((? string? str) (+ relevance
>> + (* (score str) weight)))
>> + ((? list? lst) (+ relevance
>> + (* weight
>> + (apply + (map score lst)))))))))
>
> Nitpick: it’s a bit subjective, but I think this clause might be
> slightly nicer like this:
>
> ((lst ...)
> (+ relevance (* weight (reduce + 0 (map score lst)))))

Works for me! I've changed the match clause accordingly in the attached
patch.

swedebugia <swedebugia@riseup.net> writes:

> On 2019-05-07 02:57, Chris Marusich wrote:
>> + ;; Match against uncommon outputs.
>> + (,(lambda (package)
>> + (filter (lambda (output)
>> + (not (member output
>> + ;; Some common outpus shared by many packages.
>> + '("out" "debug" "doc" "static"))))
>
> I suggest we add "gui" and "lib" to this list.

Actually, I was curious about this, so I checked how many outputs are
being used by all our packages today. Here are the results:

--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> ,use (gnu packages) (guix packages)
scheme@(guix-user)> (define (increment table key) (hash-set! table key (+ 1 (hash-ref table key 0))))
scheme@(guix-user)> (define (increment-outputs package table) (for-each (lambda (output) (increment table output)) (package-outputs package)) table)
scheme@(guix-user)> (define outputs-to-count (fold-packages increment-outputs (make-hash-table)))
scheme@(guix-user)> ,pp (sort (hash-map->list cons outputs-to-count) (lambda (a b) (< (cdr a) (cdr b))))
$1 = (("kernel-patch" . 1)
("pcf-8bit" . 1)
("python2" . 1)
("schema" . 1)
("octave" . 1)
("jp" . 1)
("db" . 1)
("gtk3" . 1)
("kr" . 1)
("subtree" . 1)
("pulseaudio" . 1)
("cn" . 1)
("ruby" . 1)
("fbgrab" . 1)
("credential-netrc" . 1)
("front-end" . 1)
("psf" . 1)
("tw" . 1)
("libedataserverui" . 1)
("gtk2" . 1)
("pcf" . 1)
("send-email" . 1)
("jack" . 1)
("python3" . 1)
("ndiff" . 1)
("installer" . 1)
("svn" . 1)
("image" . 1)
("tiles" . 2)
("fortran" . 3)
("include" . 3)
("utils" . 3)
("tests" . 3)
("python" . 4)
("tk" . 4)
("metis" . 4)
("gui" . 4)
("examples" . 5)
("jdk" . 6)
("bin" . 8)
("cargo" . 16)
("static" . 33)
("lib" . 38)
("debug" . 87)
("doc" . 134)
("out" . 9811))
scheme@(guix-user)>
--8<---------------cut here---------------end--------------->8---

In light of that, I've chosen to exclude all of the following outputs:

'("out" "doc" "debug" "lib" "static" "bin" "examples" "gui" "tests"
"utils" "include")

I've also added a test to verify that package outputs are included in
search results.

What do you all think of this latest version?

--
Chris
From dedea618bdd0d557e3e1183c403ff4e11d1757f8 Mon Sep 17 00:00:00 2001
From: Chris Marusich <cmmarusich@gmail.com>
Date: Mon, 6 May 2019 01:51:30 -0700
Subject: [PATCH] ui: Make package outputs searchable.

* guix/ui.scm (relevance): Allow the "field" procedure of a metric to
return a list, and handle that case appropriately. Update docstring.
(%package-metrics): Add a metric for package outputs.
* guix/scripts/package.scm (find-packages-by-description): Update
docstring.
* tests/guix-package.sh: Add a test case to verify that package outputs are
included in search results.

Co-authored-by: Tobias Geerinckx-Rice <me@tobias.gr>
---
guix/scripts/package.scm | 6 +++---
guix/ui.scm | 25 ++++++++++++++++++++-----
tests/guix-package.sh | 25 +++++++++++++++++++++++++
3 files changed, 48 insertions(+), 8 deletions(-)

diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm
index aa27984ea2..06e4cf5b9c 100644
--- a/guix/scripts/package.scm
+++ b/guix/scripts/package.scm
@@ -180,9 +180,9 @@ hooks\" run when building the profile."
;;;

(define (find-packages-by-description regexps)
- "Return two values: the list of packages whose name, synopsis, or
-description matches at least one of REGEXPS sorted by relevance, and the list
-of relevance scores."
+ "Return two values: the list of packages whose name, synopsis, description,
+or output matches at least one of REGEXPS sorted by relevance, and the list of
+relevance scores."
(let ((matches (fold-packages (lambda (package result)
(if (package-superseded package)
result
diff --git a/guix/ui.scm b/guix/ui.scm
index 92c845e944..8f95c00361 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -11,6 +11,8 @@
;;; Copyright © 2016 Benz Schenk <benz.schenk@uzh.ch>
;;; Copyright © 2018 Kyle Meyer <kyle@kyleam.com>
;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2019 Chris Marusich <cmmarusich@gmail.com>
+;;; Copyright © 2019 Tobias Geerinckx-Rice <me@tobias.gr>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -1370,9 +1372,9 @@ WIDTH columns. EXTRA-FIELDS is a list of symbol/value pairs to emit."
(define (relevance obj regexps metrics)
"Compute a \"relevance score\" for OBJ as a function of its number of
matches of REGEXPS and accordingly to METRICS. METRICS is list of
-field/weight pairs, where FIELD is a procedure that returns a string
-describing OBJ, and WEIGHT is a positive integer denoting the weight of this
-field in the final score.
+field/weight pairs, where FIELD is a procedure that returns a string or list
+of strings describing OBJ, and WEIGHT is a positive integer denoting the
+weight of this field in the final score.

A score of zero means that OBJ does not match any of REGEXPS. The higher the
score, the more relevant OBJ is to REGEXPS."
@@ -1394,8 +1396,10 @@ score, the more relevant OBJ is to REGEXPS."
((field . weight)
(match (field obj)
(#f relevance)
- (str (+ relevance
- (* (score str) weight)))))))
+ ((? string? str)
+ (+ relevance (* (score str) weight)))
+ ((lst ...)
+ (+ relevance (* weight (apply + (map score lst)))))))))
0
metrics))

@@ -1404,6 +1408,17 @@ score, the more relevant OBJ is to REGEXPS."
;; of regexps.
`((,package-name . 4)

+ ;; Match against uncommon outputs.
+ (,(lambda (package)
+ (filter (lambda (output)
+ (not (member output
+ ;; Some common outpus shared by many packages.
+ '("out" "doc" "debug" "lib" "static" "bin"
+ "examples" "gui" "tests" "utils"
+ "include"))))
+ (package-outputs package)))
+ . 1)
+
;; Match regexps on the raw Texinfo since formatting it is quite expensive
;; and doesn't have much of an effect on search results.
(,(lambda (package)
diff --git a/tests/guix-package.sh b/tests/guix-package.sh
index 0d60481895..262d29bd59 100644
--- a/tests/guix-package.sh
+++ b/tests/guix-package.sh
@@ -398,3 +398,28 @@ else
grep "manifest.scm:[1-3]:.*wonderful-package.*: unbound variable" \
"$module_dir/stderr"
fi
+
+# Verify that package outputs are included in search results.
+rm -rf "$module_dir"
+mkdir "$module_dir"
+cat > "$module_dir/foo.scm"<<EOF
+(define-module (foo)
+ #:use-module (guix packages)
+ #:use-module (guix build-system trivial))
+
+(define-public dummy-package
+ (package
+ (name "dummy-package")
+ (version "dummy-version")
+ (outputs '("out" "dummy-output"))
+ (source #f)
+ ;; Without a real build system, the "guix pacakge -s" command will fail.
+ (build-system trivial-build-system)
+ (synopsis "dummy-synopsis")
+ (description "dummy-description")
+ (home-page "https://dummy-home-page")
+ (license #f)))
+EOF
+guix package -L "$module_dir" -s dummy-output > /tmp/out
+test "`guix package -L "$module_dir" -s dummy-output | grep ^name:`" = "name: dummy-package"
+rm -rf "$module_dir"
--
2.20.1
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAlzSfDUACgkQ3UCaFdgi
Rp1xwg/+NTUP1dIArj6abV7Sdr+sKDaOPxKDxPcMjSeHvgYDoPtA9LKyEcJdwqnN
SPcEgHsUMLdjiKSlkzrOlP+TFIPpLOI6Ttl8g7DSzk8nunUbzo4/GtnrACQB0+PM
x+rr20a2PNqtZslmqicIJ8S3i63v+8b7K9+4CUuN/uQzNenufk5+XW2Fk6T0G4/e
1EKEiayOSDy/yZ4R177X1wlr+MPrwc7+M7OfavTAf0VPJ6EBUpAO1PltuJGSR9oY
LVTcIUOligLrcguYE6Ftc5wCwC7tV7V81cQj8FwudR/EIB3qGU2NZQi1866SiPoq
+poynWV7ZB03BD6ttossmyM3ZTrF2PzrGTADajHCwHalZf6Sc49ZHV5JXXyEcYKL
Y0h+tdMInF4JhMm+5b3lbDPQ1zCyukABiEdg2RGzH3SL8HRfzDbPfMKq4f9YrMxw
WfcmC3abU8d7upU0mtYXTTOWRoRLwV01lDHYq0wJGfotudcoaZUCX30iSl1/4dOT
8EQSSNhudqr3uqYcEWoo8cqY6e3if+JxMwaNkngsb5bEyg2ed3eykjBNA3wvNnk4
FihDjqpvT4JQOry/c8Wg+sUR6pgaKhanhQQT+1PCSuMYbGUi7ImBRrN/g/NdEA+Q
TwDrLV8++2gRwqX8A65RycsADgrVC+d3PhSwT8g4wSDQejmLTFQ=
=imKk
-----END PGP SIGNATURE-----
L
Ludovic Courtès wrote on Wed May 08 12:39:44+0200 2019
(name . Chris Marusich)(address . cmmarusich@gmail.com)
87pnotz0un.fsf@gnu.org
Hi!

Chris Marusich <cmmarusich@gmail.com> skribis:

> In light of that, I've chosen to exclude all of the following outputs:
>
> '("out" "doc" "debug" "lib" "static" "bin" "examples" "gui" "tests"
> "utils" "include")
>
> I've also added a test to verify that package outputs are included in
> search results.

I would exclude only the standard output names used by (guix build
gnu-build-system), so:

("out" "doc" "debug" "lib" "include" "bin")

That way one can still do ‘guix search git gui’ or ‘guix search bind utils’.

> What do you all think of this latest version?

Thanks for the new test! LGTM.

Ludo’.


C
Chris Marusich wrote on Thu May 09 00:22:03-0700 2019
Re: bug#35588: [PATCH] ui: Search matches additional package outputs.
(name . Ludovic Courtès)(address . ludo@gnu.org)
87lfzgcctg.fsf_-_@gmail.com
Ludovic Courtès <ludo@gnu.org> writes:

> I would exclude only the standard output names used by (guix build
> gnu-build-system), so:
>
> ("out" "doc" "debug" "lib" "include" "bin")
>
> That way one can still do ‘guix search git gui’ or ‘guix search bind utils’.

I didn't know you could do that! Your suggestion makes sense. I've
done as you suggest in commit 387e6b9e340ce4b401f220f72881415623a466f7.

Thanks everyone! I'm closing this bug report.

--
Chris
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAlzT1RsACgkQ3UCaFdgi
Rp2r0A/9GHKitI2Oid54NoL3eBuh4K6NgLeDG4nRVDOoJsRn4M+LJVqiDZtggIeQ
fKaffTR6oBZtGIeqbLz1A9Op+2+k6IuzABDPlAze3G+E8iZ7h1SJHlmhDng6Rlav
C85YoGKpITHekWXxXibdl2MjuYKPI75ZtA+WcWU/x+ofWOx7KCdgp068nbSZchpd
sIMNjylyDtQadLkBfAFlJHhn0ev1+ksQ1qPSnempZMpr/H4IeMLY4O43BdLdjpix
vkijxShjZP1BM0S3qFRJMcqcKoZyX0sJwCMYmV4YMbkO5p1fKAYxkLcwLeqMLL9g
tSHGIv9PGyEv/3H0rBQwwSc2oD26PrgocA6FsQbfmx+9GQSd5OzQWJzC1oRTwg2J
fPGxZmBf2LfHE3LYuNnlHi7JMrntjHOfIH0Zo684nSan8jm7c8MOcc8uypxULcSS
/pmCBolo2sQCtlvrL2v7lxrQXfVgQsn9YmksrTWQRgT4YkN2ECPcsjp109/TYrne
lfGWFXBw1nTtifmEdSvAjr2Rygk9FRcVGtLkuT6cMJ9a08Xr+62B6jiMPXPxCp/F
TrIVJN9BCG1MUvLU9Jk3qRE2GsB3C3SmsLTCCDwhbvTLJZ0FB5WoJ8903ee6tN51
M9WgQ3VvwXod7QujNwsAAi+chKNX8JnF07axG0ZDLkJct1HEiWA=
=gsX2
-----END PGP SIGNATURE-----
L
Ludovic Courtès wrote on Thu May 09 12:18:36+0200 2019
control message for bug #35588
(address . control@debbugs.gnu.org)
87d0ksexs3.fsf@gnu.org
tags 35588 fixed
close 35588


?
Your comment

Comments via the web interface are not currently supported. To comment on this conversation send email to 35588@debbugs.gnu.org

  • Chris Marusich
  • Ludovic Courtès
  • Tobias Geerinckx-Rice
  • swedebugia
unassigned
normal
Done