On Sun, 8 Mar 2020 at 12:33, Ludovic Courtès wrote: > Arun Isaac skribis: > > This is a problem, but I would see it as a necessary "compilation" > > step. :-P In fact, this whole patchset speeds up `guix search` by doing > > part of the work of `guix search` ahead of time. So, some such cost is > > unavoidable. > > Yeah. I think we need to take the whole user experience into account, > not just ‘guix search’. ‘guix pull’ already feels very slow, and it’s a > fairly common operation. Conversely, ‘guix search’ takes roughly > between 0.5 and 2 seconds and is an uncommon operation on a “slow path” > (in the sense that when you’re searching for software, you’ll probably > have to spend more than a couple of seconds to find what you’re looking > for.) We could imagine something doing the job of indexing in the background; using the daemon or whatever. > >> What I like about the recutils format in this context is that it’s both > >> human- and machine-readable. The examples in the manual show how it can > >> be useful to select the information displayed or to refine the search > >> (info "(guix) Invoking guix package"). > > > > Xapian's query language is much more natural (as in natural language) > > than the regexp based techniques we need to use with recutils. I have > > hardly ever used the regexp based search and I suspect many others > > haven't either. Also, refining the search query should be easier to do > > with Xapian. We could even use Xapian's query expansion feature to > > suggest improved queries to the user. > > I’m not sufficiently familiar with Xapian’s query language. The > examples I had in mind were: > > guix search malloc | recsel -p name,version,relevance > guix search | recsel -p name -e 'license ~ "LGPL 3"' > guix search crypto library | \ > recsel -e '! (name ~ "^(ghc|perl|python|ruby)")' -p name,synopsis I think these examples are good ones to benchmark the different approaches. Because the speed is one thing, the accuracy is another one. Let cut the "slow path" by providing a better experience when searching. ;-) > It’s not so much about regexps than it is about selecting individual > fields. The regexp should be provided directly to "guix search" actually and 'recsel' is only a "filter" allowing to deal differently with the fields. > To me, adding 20–50 seconds on ‘guix pull’ would be undesirable. :-/ Ok, at least it is clear. :-) And computing in the background? All the best, simon