‘guix-package-cache.drv’ eats all your memory with ‘guix-cran’

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Ludovic Courtès
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
important
L
L
Ludovic Courtès wrote on 2 Jun 2023 18:22
(address . bug-guix@gnu.org)
87sfb93kca.fsf@inria.fr
Consider this:

Toggle snippet (47 lines)
$ cat /tmp/cran.scm
(cons (channel
(name 'guix-cran)
(url "https://github.com/guix-science/guix-cran.git"))
%default-channels)
$ guix time-machine -C /tmp/cran.scm -- describe
Updating channel 'guix-cran' from Git repository at 'https://github.com/guix-science/guix-cran.git'...
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...

[...]

The following derivations will be built:
/gnu/store/90qiphp9048zgvi96lj8zqg8zhwi4js0-profile.drv
/gnu/store/2lv56bl8ghyf2kvfkwj5iy3mrlfik4la-inferior-script.scm.drv
/gnu/store/vnjf8gf6yf5gjm1syvz7yyfn71mmvy27-profile.drv
/gnu/store/8x49757i1s0r6igrah8ramn9zc1v577b-guix-69dfdb7bd.drv
/gnu/store/ll5x27npc3znyqv0rkjh945x9cr41k9q-guix-command.drv
/gnu/store/m03hp1n62bv8gkid53g8igizbw012gai-guix-module-union.drv
/gnu/store/pw8k2jwwyj2nwvqjcd1w3xjvz2n5zpbg-guix-69dfdb7bd-modules.drv
/gnu/store/a6si5zgzsxyha2466v4q45hqimg31mjg-guix-config-modules.drv
/gnu/store/3pm8aivzg3hfv95zlffxjrxvf8hfbg5n-guix-config-source.drv
/gnu/store/rffsvls17s21v2h6gw7agah5vczwi3rm-config.scm.drv
/gnu/store/v61jlksbgfslh27y8dbaj1jcajxz5c8z-guix-config.drv
/gnu/store/z6r6xdcq9dyxlk8ln8cq1ka2gspxp83i-guix-daemon.drv
/gnu/store/fshds3i961v1p3x8f7awh55lix0f7vkl-guix-cran.drv
/gnu/store/xsnp3znb0h630kipjxd198qfd3jr2ww5-inferior-script.scm.drv

building /gnu/store/xsnp3znb0h630kipjxd198qfd3jr2ww5-inferior-script.scm.drv...
building /gnu/store/rffsvls17s21v2h6gw7agah5vczwi3rm-config.scm.drv...
building /gnu/store/3pm8aivzg3hfv95zlffxjrxvf8hfbg5n-guix-config-source.drv...
building /gnu/store/v61jlksbgfslh27y8dbaj1jcajxz5c8z-guix-config.drv...
building /gnu/store/a6si5zgzsxyha2466v4q45hqimg31mjg-guix-config-modules.drv...
building /gnu/store/pw8k2jwwyj2nwvqjcd1w3xjvz2n5zpbg-guix-69dfdb7bd-modules.drv...
building /gnu/store/m03hp1n62bv8gkid53g8igizbw012gai-guix-module-union.drv...
building /gnu/store/ll5x27npc3znyqv0rkjh945x9cr41k9q-guix-command.drv...
building /gnu/store/z6r6xdcq9dyxlk8ln8cq1ka2gspxp83i-guix-daemon.drv...
building /gnu/store/8x49757i1s0r6igrah8ramn9zc1v577b-guix-69dfdb7bd.drv...
building /gnu/store/fshds3i961v1p3x8f7awh55lix0f7vkl-guix-cran.drv...
building CA certificate bundle...
listing Emacs sub-directories...
building fonts directory...
building directory of Info manuals...
building profile with 2 packages...
building /gnu/store/2lv56bl8ghyf2kvfkwj5iy3mrlfik4la-inferior-script.scm.drv...
building package cache...

It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-guix-package-cache.drv, which
consumes memory until it gets OOM-killed.

It used to work fine a few months ago.

Ludo’.
L
L
Ludovic Courtès wrote on 2 Jun 2023 22:55
control message for bug #63852
(address . control@debbugs.gnu.org)
87r0qt37pu.fsf@gnu.org
severity 63852 important
quit
L
L
Ludovic Courtès wrote on 3 Jun 2023 16:05
Re: bug#63852: ‘guix-package-cache.drv ’ eats all your memory with ‘guix-cran’
(address . 63852@debbugs.gnu.org)(name . Lars-Dominik Braun)(address . lars@6xq.net)
875y843aln.fsf@gnu.org
Ludovic Courtès <ludovic.courtes@inria.fr> skribis:

Toggle quote (3 lines)
> It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-guix-package-cache.drv, which
> consumes memory until it gets OOM-killed.

The culprit is easily found (here on just the 22K packages of ‘guix’, so
less than half of what you get with ‘guix-cran’):

Toggle snippet (52 lines)
$ guix repl
GNU Guile 3.0.9
Copyright (C) 1995-2023 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guix-user)> ,m(gnu packages)
scheme@(gnu packages)> ,use(statprof)
scheme@(gnu packages)> (gcprof (lambda () (generate-package-cache "/tmp/t")))
% cumulative self
time seconds seconds procedure
50.00 13.77 13.77 ice-9/boot-9.scm:2498:4
6.94 1.91 1.91 display
5.56 1.53 1.53 object->string
2.78 0.76 0.76 reverse
2.78 0.76 0.76 gensym
2.78 0.76 0.76 ice-9/boot-9.scm:2217:0:%load-announce
1.39 6.88 0.38 gnu/packages.scm:388:4:expand-cache
1.39 0.76 0.38 srfi/srfi-1.scm:1028:0:lset-intersection
1.39 0.38 0.38 number->string
1.39 0.38 0.38 gnu/packages/crates-io.scm:17783:2:arguments
1.39 0.38 0.38 open-output-string
1.39 0.38 0.38 gnu/packages/crates-io.scm:18450:2:arguments
1.39 0.38 0.38 gnu/packages/crates-io.scm:36208:2:arguments
1.39 0.38 0.38 guix/packages.scm:1320:8:mproc
1.39 0.38 0.38 gnu/packages/crates-io.scm:25101:2:arguments
1.39 0.38 0.38 gnu/packages/crates-io.scm:29972:2:arguments
1.39 0.38 0.38 gnu/packages/crates-io.scm:883:2:arguments
1.39 0.38 0.38 ice-9/vlist.scm:449:0:vhash-cons
1.39 0.38 0.38 append
1.39 0.38 0.38 string-append
1.39 0.38 0.38 system/vm/assembler.scm:2175:8
1.39 0.38 0.38 guix/build/syscalls.scm:1168:0:read-dirent-header/linux
1.39 0.38 0.38 srfi/srfi-1.scm:1033:17
1.39 0.38 0.38 gnu/packages/crates-io.scm:9336:2:arguments
1.39 0.38 0.38 gnu/packages/crates-io.scm:64009:2:arguments
1.39 0.38 0.38 guix/build-system/cargo.scm:229:4
1.39 0.38 0.38 ice-9/boot-9.scm:2759:0:module-make-local-var!
0.00 7328.64 0.00 system/vm/assembler.scm:1258:0:intern-constant
0.00 1423.97 0.00 ice-9/boot-9.scm:220:5:map1
0.00 201.90 0.00 ice-9/threads.scm:388:4

[...]

---
Sample count: 72
Total time: 27.531145665 seconds (14.056589389 seconds in GC)

The assembler allocates a huge constant table in ‘intern-constant’
(‘generate-package-cache’ produces one literal list containing one small
vector per package, each of which contains strings and other constants).

The attached file emulates what ‘generate-package-cache’ does. Each
vector in the list looks like this:

#(0 "xyzxyz-0" "1.0.0" (gnu packages xyz0) xyzxyz-0 ("out") #t #f "gnu/packages/xyz0" 0 1)

The program terminates with roughly a 250 MiB heap for 22K entries—about
10 MiB per vector.

The heap profile on completion looks like this:

Toggle snippet (26 lines)
% type self avg obj size
57.5 bytevector 8,205,632 455868.4
23.9 vector 3,413,857 549.3
7.7 struct 1,093,840 40.8
5.3 pair 753,376 16.0
2.7 stringbuf 385,856 51.1
1.1 symbol 151,424 32.0
0.4 program 59,424 47.3
0.3 string 49,152 32.0
0.2 heap-number 23,520 32.0
0.2 variable 21,696 30.3
0.1 smob 21,056 32.0
0.1 vm-continuation 20,864 32.0
0.1 atomic-box 20,736 32.0
0.1 weak-table 19,584 31.8
0.1 unknown 19,040 32.0
0.0 pointer 2,000 16.0
0.0 hash-table 896 32.0
0.0 primitive 144 16.0
0.0 weak-vector 96 16.0
0.0 frame 96 32.0
0.0 primitive-generic 64 32.0
0.0 keyword 16 16.0
sampled heap: 13.60166 MiB (heap size: 233.16016 MiB)

It’s likely that the ‘buf’ field of <asm> is what’s taking space

On that topic, see also:


Ludo’.
;; https://issues.guix.gnu.org/63852 (use-modules (system base compile) (language tree-il) (srfi srfi-1) (srfi srfi-26) (ice-9 match) (ice-9 time) (statprof)) (define lst (unfold (cut > <> 22000) (lambda (i) ;; Mimic 'generate-package-cache'. (define name (string-append "xyzxyz-" (number->string i 8))) (vector i name (string-append "1.0." (number->string (modulo i 300))) (list 'gnu 'packages (string->symbol (string-append "xyz" (number->string i)))) (string->symbol name) '("out") #t #f (string-append "gnu/packages/xyz" (number->string (modulo i 300))) (modulo i 1000) 1)) 1+ 0)) (pk 'go! (gc-stats)) (let ((exp (match (or (getenv "VARIANT") "0") ("0" `',lst) ("1" `(list ,@lst)) ("2" (make-const #f lst)) ;avoid psyntax overhead ("3" (make-primcall #f 'list (map (lambda (vector) (make-const #f vector)) lst)))))) (compile exp #:to 'bytecode #:from (if (pair? exp) 'scheme 'tree-il) #:optimization-level 0 #:opts '(#:to-file? #t))) (pk (gc-stats))
L
L
Ludovic Courtès wrote on 5 Jun 2023 18:34
(address . 63852@debbugs.gnu.org)(name . Lars-Dominik Braun)(address . lars@6xq.net)
87ttvlhnps.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (8 lines)
> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>
>> It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-guix-package-cache.drv, which
>> consumes memory until it gets OOM-killed.
>
> The culprit is easily found (here on just the 22K packages of ‘guix’, so
> less than half of what you get with ‘guix-cran’):

(Beware of “easily” when the guy doesn’t know what he’s talking about.)

Turns out the problem is more pedestrian: ‘r-future-tests’ depends on
itself.

Toggle snippet (34 lines)
$ guix import cran future.tests

Starting download of /tmp/guix-file.TOBjcf
From http://cran.r-project.org/src/contrib/future.tests_0.7.0.tar.gz...
…_0.7.0.tar.gz 186KiB 1.2MiB/s 00:00 ???????????????????? 100.0%
(package
(name "r-future-tests")
(version "0.7.0")
(source (origin
(method url-fetch)
(uri (cran-uri "future.tests" version))
(sha256
(base32
"10g1w99xqr7l0cn27642aphqcvfidgpas38f84r815yy6k1ryrlx"))))
(properties `((upstream-name . "future.tests")))
(build-system r-build-system)
(propagated-inputs (list r-cli r-crayon r-future r-prettyunits r-sessioninfo))
(native-inputs (list r-future-tests))
(home-page "https://future.tests.futureverse.org")
(synopsis "Test Suite for 'Future API' Backends")
(description
"Backends implementing the Future API, as defined by the future package, should
use the tests provided by this package to validate that they meet the minimal
requirements of the Future API. The tests can be performed easily from within R
or from outside of R from the command line making it straightforward to include
them in package tests and in Continuous Integration (CI) pipelines.")
$ guix describe
Generation 265 Jun 04 2023 23:48:31 (current)
guix eed55a6
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: eed55a6544d5bda2245ec853e5fa4b28e1865bea

To be fixed in the importer.

Ludo’.
L
L
Ludovic Courtès wrote on 5 Jun 2023 23:58
(address . 63852@debbugs.gnu.org)(name . Lars-Dominik Braun)(address . lars@6xq.net)
87pm69h8r9.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (15 lines)
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>>
>>> It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-guix-package-cache.drv, which
>>> consumes memory until it gets OOM-killed.
>>
>> The culprit is easily found (here on just the 22K packages of ‘guix’, so
>> less than half of what you get with ‘guix-cran’):
>
> (Beware of “easily” when the guy doesn’t know what he’s talking about.)
>
> Turns out the problem is more pedestrian: ‘r-future-tests’ depends on
> itself.

Here’s a patch to prevent that from happening again:


(We also need to fix the importer.)

Ludo’.
L
L
Ludovic Courtès wrote on 15 Jun 2023 15:13
(address . 63852-done@debbugs.gnu.org)(name . Lars-Dominik Braun)(address . lars@6xq.net)
87edmcx3zy.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (15 lines)
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>>
>>> It goes OK until building /gnu/store/6rfaqfq693vda59a55asc4wjjg52ilns-guix-package-cache.drv, which
>>> consumes memory until it gets OOM-killed.
>>
>> The culprit is easily found (here on just the 22K packages of ‘guix’, so
>> less than half of what you get with ‘guix-cran’):
>
> (Beware of “easily” when the guy doesn’t know what he’s talking about.)
>
> Turns out the problem is more pedestrian: ‘r-future-tests’ depends on
> itself.

With commit e4259d4e9e3251e4c4b45d1cce4008ac32b504c8, such cycles are
detected early on.

Closing this bug.

We should still improve the memory footprint of the assembler though.

Ludo’.
Closed
?