cuirass: Search queries are too slow.

  • Done
  • quality assurance status badge
Details
One participant
  • Mathieu Othacehe
Owner
unassigned
Submitted by
Mathieu Othacehe
Severity
normal
M
M
Mathieu Othacehe wrote on 4 Oct 2020 13:17
(address . bug-guix@gnu.org)
87mu12p7ec.fsf@gnu.org
Hello,

Search queries can take a long time to complete.

This query took 658.67 seconds to complete:

Toggle snippet (3 lines)
SELECT * FROM ( SELECT Builds.rowid, Builds.timestamp, Builds.starttime, Builds.stoptime, Builds.log, Builds.status, Builds.job_name, Builds.system, Builds.nix_name, Specifications.name FROM Builds INNER JOIN Evaluations ON Builds.evaluation = Evaluations.id INNER JOIN Specifications ON Evaluations.specification = Specifications.name WHERE (Builds.nix_name LIKE '%hurd-barebones.qcow2%') AND (0 IS NULL OR (Builds.status = 0)) AND ('guix-master' IS NULL OR (Specifications.name = 'guix-master')) AND ('x86_64-linux' IS NULL OR (Builds.system = 'x86_64-linux')) AND (NULL IS NULL OR (NULL < Builds.rowid)) AND (NULL IS NULL OR (NULL > Builds.rowid)) ORDER BY CASE WHEN NULL IS NULL THEN Builds.rowid ELSE -Builds.rowid END DESC LIMIT 1) ORDER BY rowid DESC 658.67

and this one:

Toggle snippet (3 lines)
SELECT MIN(Builds.rowid) FROM Builds INNER JOIN Evaluations ON Builds.evaluation = Evaluations.id INNER JOIN Specifications ON Evaluations.specification = Specifications.name WHERE (Builds.nix_name LIKE '%ganeti%') AND (NULL IS NULL OR (Builds.status = NULL)) AND (NULL IS NULL OR (Specifications.name = NULL)) AND (NULL IS NULL OR (Builds.system = NULL)) 146.70

took 146.70 seconds.

The naive solution of adding an index on nix_name column does not work
as it is still ignored for some reason.

According to https://www.sqlite.org/optoverview.html,starting the LIKE
search pattern with '%' prevent the index use.

However, searching for 'hurd-barebones.qcow2%' also skips the index.

Thanks,

Mathieu
M
M
Mathieu Othacehe wrote on 5 Oct 2020 15:02
(address . 43791-done@debbugs.gnu.org)
87r1qc7rmk.fsf@gnu.org
Toggle quote (2 lines)
> However, searching for 'hurd-barebones.qcow2%' also skips the index.

That's because the index was not created in case insensitive mode.

I dropped support for searching with "^" and "$" characters as this is
not compatible with using an index and hence way too slow.

The new search behaviour is to remove any "%" that could be added by the
user, and add one at the end of the query. It should be enough for the
modest search needs in Cuirass and reasonable in term of performance
(< 10 ms).

Anyway, fixed with: 930c2f315c6a9768d31a80d35d2a2550b588deaa.

Thanks,

Mathieu
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 43791@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 43791
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch