[PATCH] substitute: Add a ‘--clear-cache’ option.

  • Open
  • quality assurance status badge
Details
4 participants
  • Ludovic Courtès
  • Tobias Geerinckx-Rice
  • Mathieu Othacehe
  • zimoun
Owner
unassigned
Submitted by
Tobias Geerinckx-Rice
Severity
normal
T
T
Tobias Geerinckx-Rice wrote on 26 Dec 2021 17:21
[PATCH] substitute: Add a ‘--clear-cache ’ option.
(address . guix-patches@gnu.org)
20211226162126.23749-1-me@tobias.gr
Unlike ‘rm -rf ~/.cache/guix/substitute’ this respects XDG_CACHE_HOME.

Use the --OPTION form for consistency with the existing ‘guix substitute’
interface, where other guix subcommands would prefer an ACTION. This
subcommand really supports only one OPTION at a time, anyway.

* guix/scripts/substitute.scm (guix-substitute):
Add a new ‘--clear-cache’ option argument.
(show-help): Document it.
---
guix/scripts/substitute.scm | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)

Toggle diff (43 lines)
diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm
index c044e1d47a..1e71b4848b 100755
--- a/guix/scripts/substitute.scm
+++ b/guix/scripts/substitute.scm
@@ -3,6 +3,7 @@
;;; Copyright © 2014 Nikita Karetnikov <nikita@karetnikov.org>
;;; Copyright © 2018 Kyle Meyer <kyle@kyleam.com>
;;; Copyright © 2020 Christopher Baines <mail@cbaines.net>
+;;; Copyright © 2021 Tobias Geerinckx-Rice <me@tobias.gr>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -40,7 +41,7 @@ (define-module (guix scripts substitute)
#:use-module (guix cache)
#:use-module (gcrypt pk-crypto)
#:use-module (guix pki)
- #:use-module ((guix build utils) #:select (mkdir-p))
+ #:use-module ((guix build utils) #:select (delete-file-recursively mkdir-p))
#:use-module ((guix build download)
#:select (uri-abbreviation nar-uri-abbreviation
(open-connection-for-uri
@@ -239,6 +240,8 @@ (define-syntax with-networking
(define (show-help)
(display (G_ "Usage: guix substitute [OPTION]...
Internal tool to substitute a pre-built binary to a local build.\n"))
+ (display (G_ "
+ --clear-cache forget previously cached availability information"))
(display (G_ "
--query report on the availability of substitutes for the
store file names passed on the standard input"))
@@ -749,6 +752,10 @@ (define reply-port
(with-networking
(with-error-handling ; for signature errors
(match args
+ (("--clear-cache")
+ ;; This is meant to be a big hammer, so don't pussyfoot around with
+ ;; maybe-remove-expired-cache-entries.
+ (delete-file-recursively %narinfo-cache-directory))
(("--query")
(let ((acl (current-acl)))
(let loop ((command (read-line)))
--
2.34.0
M
M
Mathieu Othacehe wrote on 27 Dec 2021 22:12
Re: bug#52805: [PATCH] substitute: Add a ‘--cle ar-cache’ option.
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)(address . 52805@debbugs.gnu.org)
87ee5x7nwj.fsf@gnu.org
Hello Tobias,

Toggle quote (2 lines)
> Unlike ‘rm -rf ~/.cache/guix/substitute’ this respects XDG_CACHE_HOME.

This script is invoked by the daemon as root. In that context,
%narinfo-cache-directory is "/var/guix/substitute/cache", whereas when
run as an user it is likely: "~/.cache/guix/substitute".

That would mean that users need to run "sudo guix substitute
--clear-cache". Moreover this script is an internal undocumented tool,
so I'm not in favor of adding user commands to it.

Maybe "guix gc" would be a better place?

Thanks,

Mathieu
T
T
Tobias Geerinckx-Rice wrote on 27 Dec 2021 22:20
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 52805@debbugs.gnu.org)
87pmphhg8q.fsf@nckx
Hi Mathieu,

Thanks for the review!

Mathieu Othacehe ???
Toggle quote (5 lines)
> This script is invoked by the daemon as root. In that context,
> %narinfo-cache-directory is "/var/guix/substitute/cache",
> whereas when
> run as an user it is likely: "~/.cache/guix/substitute".

Correct.

Toggle quote (3 lines)
> That would mean that users need to run "sudo guix substitute
> --clear-cache".

If they want to clear /var instead of ~/.cache, sure.

That sounds like a sane interface, if clearing ‘root’'s cache were
the intention? How would you even do that without sudo,
privilege-wise?

Toggle quote (3 lines)
> Moreover this script is an internal undocumented tool,
> so I'm not in favor of adding user commands to it.

OK.

The point was to have a little command I could ask other users to
run, that's guaranteed to clear the correct directory, and doesn't
involve a spooky rm (which saves some back-and-forth :-).

Toggle quote (2 lines)
> Maybe "guix gc" would be a better place?

No, I'd rather not add yet another option to ‘guix gc’ that has
nothing to do with GC.

Kind regards,

T G-R
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCYco0hQ0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW15d+4BAPkuQWWC+qZ1C/dkdaYx/U1PC85Yn8QdGWLVDgED
STCNAQClkSgUbCbBmujSQMiP17E0ovE+2PBVu7VDCLaNfj0DAQ==
=rAZY
-----END PGP SIGNATURE-----

Z
Z
zimoun wrote on 29 Dec 2021 17:49
Re: [bug#52805] [PATCH] substitute: Add a ‘--cl ear-cache’ option.
(address . 52805@debbugs.gnu.org)
864k6rjqzr.fsf@gmail.com
Hi Tobias,

On Mon, 27 Dec 2021 at 22:20, Tobias Geerinckx-Rice via Guix-patches via <guix-patches@gnu.org> wrote:

Toggle quote (5 lines)
>> Maybe "guix gc" would be a better place?
>
> No, I'd rather not add yet another option to ‘guix gc’ that has
> nothing to do with GC.

Why? Well, garbage collection is about automatic memory management and
cache is about efficiency of memory usage. Therefore, it is appears to
me relevant to have an option “guix gc --clear-cache” to force the clean
of various caches.

Cheers,
simon
L
L
Ludovic Courtès wrote on 5 Jan 2022 21:59
Re: bug#52805: [PATCH] substitute: Add a ‘--cle ar-cache’ option.
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)
87czl5kif3.fsf_-_@gnu.org
Hello,

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

Toggle quote (2 lines)
> Mathieu Othacehe ???

[...]

Toggle quote (3 lines)
>> Moreover this script is an internal undocumented tool,
>> so I'm not in favor of adding user commands to it.

+1

Toggle quote (4 lines)
> The point was to have a little command I could ask other users to run,
> that's guaranteed to clear the correct directory, and doesn't involve
> a spooky rm (which saves some back-and-forth :-).

In which case do users have to ‘rm -rf ~/.cache/guix/substitute’?

That directory is only used by ‘guix weather’. In that case,
‘--clear-cache’ could be added to ‘guix weather’ maybe?

Now, ideally there would not be any cache-related option because the
cache would always be valid. If it’s not, it would be nice to see how
that can be improved or mitigated.

Thoughts?

Ludo’.
T
T
Tobias Geerinckx-Rice wrote on 5 Jan 2022 23:41
(name . Ludovic Courtès)(address . ludo@gnu.org)
87o84phjpo.fsf@nckx
Hi Ludo'!

Ludovic Courtès ???
Toggle quote (2 lines)
> That directory is only used by ‘guix weather’.

Let me back up then: why does ‘guix weather’ cache responses at
all?

Kind regards,

T G-R
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCYdYi8w0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW15RyoBAJXhrl/4QD7nxisNCDj34S446bMNV1+eEIvr6gcG
2905AP0QuZVQwkNjMGPKWQZ11iS6Q4fYbTSyhEwYO82X2MfNCg==
=1n7x
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 11 Jan 2022 13:47
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)
87sftuv3q2.fsf@gnu.org
Hi,

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

Toggle quote (5 lines)
> Ludovic Courtès ???
>> That directory is only used by ‘guix weather’.
>
> Let me back up then: why does ‘guix weather’ cache responses at all?

For the same reason as ‘guix substitute’: caching allows it to send only
requests for which it doesn’t already know the answer.

Ludo’.
T
T
Tobias Geerinckx-Rice wrote on 11 Jan 2022 19:00
(name . Ludovic Courtès)(address . ludo@gnu.org)
87a6g2qgo0.fsf@nckx
Hiya Ludo',

Ludovic Courtès ???
Toggle quote (4 lines)
> For the same reason as ‘guix substitute’: caching allows it to
> send only
> requests for which it doesn’t already know the answer.

Thanks! The problem is this reason doesn't make sense.

N.B. I agree it's an acceptable trade-off for ‘guix substitute’,
which has different operating parameters.

When I run ‘guix weather’ twice in a row, I am *never* interested
in reviewing the stale numbers I got the first time (or worse: a
mix of stale and new numbers). I don't care if the data is very
fresh: I am asking for fresher.

Caching is never to my advantage: it is ‘saving’ only the work I
asked it to perform, against my wishes. ‘Look how fast I fed you
possibly bogus data against your will!’ is not an impressive
boast.

Is my concern clear? Reasonable?

Kind regards,

T G-R
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCYd3KHw0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW15/JsBAMQ2T769JEx/ZicPgMp8iMj99AH0oY3bXRseZiIL
L7uAAP9s6l7GmK4J/hEi3cgfN3lvAsorfsH8VIPNGFzMUdifCw==
=RT/V
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 12 Jan 2022 22:04
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)
87v8yopswq.fsf@gnu.org
Hi,

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

Toggle quote (6 lines)
> Caching is never to my advantage: it is ‘saving’ only the work I asked
> it to perform, against my wishes. ‘Look how fast I fed you possibly
> bogus data against your will!’ is not an impressive boast.
>
> Is my concern clear? Reasonable?

It’s clear, but I’m not sure it’s reasonable. :-)

When do you have to worry about caches?

‘guix weather’ can cache negative replies (404s) for
%narinfo-negative-ttl = 10mn, unless the server provides a different
‘Cache-Control’ TTL. So there’s a possibility that during that time
window it’ll tell you “nope” when in fact the thing has just arrived.

I think that’s acceptable (there’s a slight delay but nothing’s
“bogus”), and it’s kind to server admins.

(It’s possible for admins to run ‘guix publish --negative-ttl=0’ if they
don’t mind extra traffic from users.)

That said, if you find it sufficiently annoying, what about adding
‘--clear-cache’ to ‘guix weather’?

HTH!

Ludo’.
?