[PATCH 0/2] scripts: weather: Provide more representative request statistics.

Done

Details

2 participants

Ludovic Courtès
Christopher Baines

Owner: unassigned

Submitted by: Christopher Baines

Severity: normal

Debbugs page

Christopher Baines wrote 4 years ago

Recipients:(address . guix-patches@gnu.org)

Message-ID:877dm8fpdg.fsf@cbaines.net

Christopher Baines (2):
  substitutes: lookup-narinfos: Return the number of requests made.
  scripts: weather: Provide more representative request statistics.

 guix/scripts/weather.scm | 19 ++++++++++---------
 guix/substitutes.scm     | 15 ++++++++-------
 2 files changed, 18 insertions(+), 16 deletions(-)

Christopher Baines wrote 4 years ago

[PATCH 2/2] scripts: weather: Provide more representative request statistics.

Recipients:(address . 47158@debbugs.gnu.org)

Message-ID:20210315151257.17384-2-mail@cbaines.net

Previously, the "seconds per request" and "requests per second" statistics
really reported (cache lookups + requests) per second.  By looking at the
actual number of requests made within lookup-narinfos, a more representative
value can be reported.

* guix/scripts/weather.scm (let/time): Allow for multiple return values.
(report-server-coverage): Alter the reporting of request statistics.
---
 guix/scripts/weather.scm | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)

Toggle diff (47 lines)diff --git a/guix/scripts/weather.scm b/guix/scripts/weather.scm
index 26ec543211..349052459c 100644
--- a/guix/scripts/weather.scm
+++ b/guix/scripts/weather.scm
@@ -117,8 +117,8 @@ values."
          (end    (current-time time-monotonic)))
     (apply kont (time-difference end start) result)))
 
-(define-syntax-rule (let/time ((time result exp)) body ...)
-  (call-with-time (lambda () exp) (lambda (time result) body ...)))
+(define-syntax-rule (let/time ((time result ... exp)) body ...)
+  (call-with-time (lambda () exp) (lambda (time result ...) body ...)))
 
 (define (histogram field proc seed lst)
   "Return an alist giving a histogram of all the values of FIELD for elements
@@ -181,11 +181,12 @@ Return the coverage ratio, an exact number between 0 and 1."
   (format #t (G_ "looking for ~h store items on ~a...~%")
           (length items) server)
 
-  (let/time ((time narinfos (lookup-narinfos
-                             server items
-                             #:make-progress-reporter
-                             (lambda* (total #:key url #:allow-other-keys)
-                               (progress-reporter/bar total)))))
+  (let/time ((time narinfos requests-made
+                   (lookup-narinfos
+                    server items
+                    #:make-progress-reporter
+                    (lambda* (total #:key url #:allow-other-keys)
+                      (progress-reporter/bar total)))))
     (format #t "~a~%" server)
     (let ((obtained  (length narinfos))
           (requested (length items))
@@ -212,9 +213,9 @@ Return the coverage ratio, an exact number between 0 and 1."
       (format #t (G_ "  ~,1h MiB on disk (uncompressed)~%")
               (/ (reduce + 0 (map narinfo-size narinfos)) MiB))
       (format #t (G_ "  ~,3h seconds per request (~,1h seconds in total)~%")
-              (/ time requested 1.) time)
+              (/ time requests-made 1.) time)
       (format #t (G_ "  ~,1h requests per second~%")
-              (/ requested time 1.))
+              (/ requests-made time 1.))
 
       (guard (c ((http-get-error? c)
                  (if (= 404 (http-get-error-code c))
-- 
2.30.1

Christopher Baines wrote 4 years ago

[PATCH 1/2] substitutes: lookup-narinfos: Return the number of requests made.

Recipients:(address . 47158@debbugs.gnu.org)

Message-ID:20210315151257.17384-1-mail@cbaines.net

As an additional value, in addition to the narinfos. This value is useful in

the weather script for reporting how many requests to the substitute server

were made.

* guix/substitutes.scm (lookup-narinfos): Additionally return the number of

requests made.

---

guix/substitutes.scm | 15 ++++++++-------

1 file changed, 8 insertions(+), 7 deletions(-)

Toggle diff (28 lines)diff --git a/guix/substitutes.scm b/guix/substitutes.scm
index ef78013659..08f8c24efd 100644
--- a/guix/substitutes.scm
+++ b/guix/substitutes.scm
@@ -310,13 +310,14 @@ information is available locally."
                        '()
                        '()
                        paths)))
-    (if (null? missing)
-        cached
-        (let ((missing (fetch-narinfos cache missing
-                                       #:open-connection open-connection
-                                       #:make-progress-reporter
-                                       make-progress-reporter)))
-          (append cached (or missing '()))))))
+    (values (if (null? missing)
+                cached
+                (let ((missing (fetch-narinfos cache missing
+                                               #:open-connection open-connection
+                                               #:make-progress-reporter
+                                               make-progress-reporter)))
+                  (append cached (or missing '()))))
+            (length missing))))
 
 (define* (lookup-narinfos/diverse caches paths authorized?
                                   #:key (open-connection
-- 
2.30.1

Ludovic Courtès wrote 4 years ago

Re: bug#47158: [PATCH 0/2] scripts: weather: Provide more representative request statistics.

Recipients:(name . Christopher Baines)(address . mail@cbaines.net)(address . 47158@debbugs.gnu.org)

Message-ID:874kha59rv.fsf@gnu.org

Hi,

Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (8 lines)

> Christopher Baines (2):

> substitutes: lookup-narinfos: Return the number of requests made.

> scripts: weather: Provide more representative request statistics.

> guix/scripts/weather.scm | 19 ++++++++++---------

> guix/substitutes.scm | 15 ++++++++-------

> 2 files changed, 18 insertions(+), 16 deletions(-)

LGTM, thanks!

This had been bothering me for some time, I’m glad this is fixed!

Ludo’.

Christopher Baines wrote 4 years ago

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 47158-done@debbugs.gnu.org)

Message-ID:87wnu5crbb.fsf@cbaines.net

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (16 lines)> Hi,
>
> Christopher Baines <mail@cbaines.net> skribis:
>
>> Christopher Baines (2):
>>   substitutes: lookup-narinfos: Return the number of requests made.
>>   scripts: weather: Provide more representative request statistics.
>>
>>  guix/scripts/weather.scm | 19 ++++++++++---------
>>  guix/substitutes.scm     | 15 ++++++++-------
>>  2 files changed, 18 insertions(+), 16 deletions(-)
>
> LGTM, thanks!
>
> This had been bothering me for some time, I’m glad this is fixed!

Great, I've pushed this as c37f78a9f55874015d10f081be36104bb33a3ae1.

Closed

Your comment

This issue is archived.

To comment on this conversation send an email to 47158@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 47158

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags

mumi command -t +confirmed -t +easy

Or, remove the moreinfo tag and set the help tag

mumi command -t -moreinfo -t +help

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date