Constructor created by define-configuration is slow (>25s) when many fields are defined

  • Open
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Tomas Volf
Owner
unassigned
Submitted by
Tomas Volf
Severity
normal
T
T
Tomas Volf wrote on 9 Dec 16:13 +0100
(address . bug-guix@gnu.org)
87msh499f7.fsf@wolfsden.cz
Hello,

I am writing a configuration for mpv and hit an issue of the generated
constructor (`home-mpv-configuration') is extremely slow. While the
almost 800 fields is unusually high, initializing the instances of the
record should *not* take almost 30 seconds. The full file is attached
to this email (as `mpv.scm').

Toggle snippet (6 lines)
scheme@(guix-user)> ,use (gnu home services mpv)
scheme@(guix-user)> ,t (and (home-mpv-configuration (global (mpv-profile-configuration (fullscreen #t)))) #t)
$1 = #t
;; 29.615315s real time, 50.896973s run time. 24.031723s spent in GC.

Interestingly, profiling does not give us anything useful:

Toggle snippet (4 lines)
scheme@(guix-user)> ,pr (and (home-mpv-configuration (global (mpv-profile-configuration (fullscreen #t)))) #t)
No samples recorded.

I also tried to remove all my custom sanitizers, to make sure the issue
is not in my code, but that did not help.

I took a short look, but code of both the define-configuration and of
Guix records is complex, full of syntax magic and I did not have any
luck trying to debug it myself. Hence this report.

Have a nice day,
Tomas
Attachment: mpv.scm
--
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.
-----BEGIN PGP SIGNATURE-----

iQJCBAEBCgAsFiEEt4NJs4wUfTYpiGikL7/ufbZ/wakFAmdXCQwOHH5Ad29sZnNk
ZW4uY3oACgkQL7/ufbZ/wamT4hAAg8D+H05v8RtRA4NjEP/K429gwVFI+rwBZ8Qh
kHr9JRHcUyeW6KoWXaE+ugRIqFALTv9wDj8LlZeA6XMKVar55ZvJN42JNKr9Vxi9
Su3aXSwZ8ou8NwiVs/2bkS9PpEM2yEA1vclQ5R1w4MybuJ3OCXkDNZ/st++CzKiJ
ydZLy4bkxwvpX4KG0L1m0V9cA0rEn8lo09TpGLdwyptJd4OJMZ691MoFEnwZUVlx
3O7JyO3mh7ZeaGZvH3y8g1qG65/f3HAtk9Af3nr1ukHJ1NEu4czb9vSE8DL9mTKx
Je9mOUqKn6e6r8oYfpgk1KcANpiv+s4nXSx4vtmcWstVKpkss0vH/FuYepvN2eUp
70EyMSDA3MoZ6Af9+sS3XFx8J8nNfacqkvqWS1QV1jykEk++s4xRjsZkb+ki56sj
jj4O3wF1PwzZZg4tKgJvOtCnwGZt59XOK8ZpM5KdwNhdIeMsVu6/9wVB+bWoXdC3
HX2XoDw1KugQXgFr6xKfoyvktOkxYfCN7TTGiWhYxOGK2GBUlnVki4J5VYfsokqx
u18wjBYxQQZkKz196UBPRAW/SM0tjXbj6xGr0yePPQVpChxANvZRqiEoxjEYZ4sH
TMbAlfEwUTfgS20yv5p9Bssw4d4p7CY9FUS4pLiiXOMlJ9APGM2DiKE8jnFsTvQh
oTgtbD4=
=4w6D
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote 19 hours ago
(name . Tomas Volf)(address . ~@wolfsden.cz)(address . 74748@debbugs.gnu.org)
87cyhh5eop.fsf@gnu.org
Hi,

Tomas Volf <~@wolfsden.cz> skribis:

Toggle quote (6 lines)
> I am writing a configuration for mpv and hit an issue of the generated
> constructor (`home-mpv-configuration') is extremely slow. While the
> almost 800 fields is unusually high, initializing the instances of the
> record should *not* take almost 30 seconds. The full file is attached
> to this email (as `mpv.scm').

AFAICS, time is spent in one of the compilation steps:

Toggle snippet (13 lines)
scheme@(gnu home services mpv)> ,o interp #t
scheme@(gnu home services mpv)> ,t (lambda () (mpv-profile-configuration))
$13 = #<procedure 7f6ed3796700 at ice-9/eval.scm:330:13 ()>
;; 0.376347s real time, 0.375402s run time. 0.000000s spent in GC.
scheme@(gnu home services mpv)> ,o interp #f
scheme@(gnu home services mpv)> ,t (lambda () (mpv-profile-configuration))
$14 = #<procedure 7f6ec32088f8 at <unknown port>:40:3 ()>
;; 11.149828s real time, 12.052915s run time. 1.581736s spent in GC.
scheme@(gnu home services mpv)> ,t (->bool (macroexpand '(mpv-profile-configuration)))
$15 = #t
;; 0.373865s real time, 0.372698s run time. 0.000000s spent in GC.

If we look more closely, it’s the optimizing compiler that takes time;
the baseline compile (-O1) is doing okay:

Toggle snippet (16 lines)
scheme@(gnu home services mpv)> ,use(system base compile)
scheme@(gnu home services mpv)> ,t (->bool (compile '(mpv-profile-configuration) #:to 'tree-il #:env (current-module)))
$20 = #t
;; 0.378741s real time, 0.377043s run time. 0.000000s spent in GC.
scheme@(gnu home services mpv)> ,t (->bool (compile '(mpv-profile-configuration) #:to 'bytecode #:env (current-module)))
$21 = #t
;; 11.946879s real time, 12.961704s run time. 1.777478s spent in GC.
scheme@(gnu home services mpv)> (define til (compile '(mpv-profile-configuration) #:to 'tree-il #:env (current-module)))
scheme@(gnu home services mpv)> ,t (->bool (compile til #:from 'tree-il #:to 'bytecode))
$22 = #t
;; 11.403420s real time, 12.317183s run time. 1.581972s spent in GC.
scheme@(gnu home services mpv)> ,t (->bool (compile til #:from 'tree-il #:to 'bytecode #:optimization-level 1))
$23 = #t
;; 0.225455s real time, 0.156137s run time. 0.000000s spent in GC.

Currently (guix build compile) applies these options to services:

((string-contains file "gnu/services/")
;; '-O2 -Ono-letrectify' compiles about ~20% faster than '-O2' for
;; large files like gnu/services/mail.scm.
(override-option #:letrectify? #f
(optimizations-for-level 2)))

I think we should at least apply this patch:
Toggle diff (14 lines)
diff --git a/guix/build/compile.scm b/guix/build/compile.scm
index 5b27b55d02..f90016b9ae 100644
--- a/guix/build/compile.scm
+++ b/guix/build/compile.scm
@@ -127,7 +127,8 @@ (define (optimization-options file)
;; Use '-O1' to have partial evaluation and primitive inlining so we
;; can honor the "macro writer's bill of rights".
(optimizations-for-level 1))
- ((string-contains file "gnu/services/")
+ ((or (string-contains file "gnu/services/")
+ (string-contains file "gnu/home/services/"))
;; '-O2 -Ono-letrectify' compiles about ~20% faster than '-O2' for
;; large files like gnu/services/mail.scm.
(override-option #:letrectify? #f
Would that be enough for the home-mpv configuration records you wrote?

Thanks,
Ludo’.
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 74748@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 74748
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch