All build users used up when using '--check' with many packages

OpenSubmitted by Maxim Cournoyer.
Details
3 participants
  • Ludovic Courtès
  • Maxim Cournoyer
  • zimoun
Owner
unassigned
Severity
normal
M
M
Maxim Cournoyer wrote on 15 Jul 18:13 +0200
Grafts max out build slots without being queued
(name . bug-guix)(address . bug-guix@gnu.org)
878sfkkb8m.fsf@gmail.com
Hello,
While working on ruby-asciidoctor-pdf, the following command failedbecause of maxing out build users:
./pre-inst-env guix build --quiet --keep-going --check ruby_version@1.0.2 gnome-shell-extension-gsconnect@33 numix-gtk-theme@2.6.7 arc-theme@20190917 gnome@3.34.2 ruby-fakefs@1.2.2 ruby-protobuf@3.10.3 ruby-fast-gettext@2.0.3 ruby-sass-rails@5.0.7 mailcatcher@0.7.1 ruby-rspec-its@1.3.0 warzone2100@3.4.0 keepassxc@2.6.0 wsjtx@2.2.2 js8call@2.2.0 ruby-yard-with-tests@0.9.25 ruby-asciidoctor-pdf@1.5.3-1.f25ea78 ruby-web-console@3.7.0 ruby-autoprefixer-rails@9.4.7 ruby-jekyll-paginate-v2@2.0.0 ruby-solargraph@0.36.0 --max-jobs=1The following derivations will be built: /gnu/store/w5qba9nzswv7ysrlxfwcqdvrf9vsgbk6-ruby-jekyll-paginate-v2-2.0.0.drv /gnu/store/4r1ziw3cb5n5jgknd1jfsvdl46cz56f9-ruby-autoprefixer-rails-9.4.7.drv /gnu/store/2f674244hhb399p86z15zgv246sj78gr-ruby-web-console-3.7.0.drv /gnu/store/sqy6f63vzgbyk6jg1zigxvks9g365wfs-ruby-rspec-its-1.3.0.drv /gnu/store/zz34dh2q37r21r2d26k7z5rpryysc3km-ruby-sass-rails-5.0.7.drv /gnu/store/62nlbfwd2jf01nm0p0wlc9sgvn5yp6gc-ruby-fakefs-1.2.2.drv /gnu/store/pjdh08jh7pxlfm4dcz86bg5xqbjyf4lm-gnome-3.34.2.drv /gnu/store/xqdvar6gmkz28003md4w9c1mqvg0slas-arc-theme-20190917.drv /gnu/store/qwmj9b493ays4dvgkmniq61fnxg3claw-numix-gtk-theme-2.6.7.drv /gnu/store/s112ayw6qg86y7j45r5yslfl151ngvgn-ruby_version-1.0.2.drvThe following grafts will be made: /gnu/store/ww8blyc1nkf788jjb3anh22dylibvbah-ruby-solargraph-0.36.0.drv /gnu/store/c7bfyqfq8d7imgy8il7wzqpx3qpx4zpw-ruby-asciidoctor-pdf-1.5.3-1.f25ea78.drv /gnu/store/cf501w2kdgw6k2gv6n24ikhzww8f51x7-ruby-yard-with-tests-0.9.25.drv /gnu/store/da8hhcaj20z06f2kww0sg4y24ma034dy-js8call-2.2.0.drv /gnu/store/zn1jkg86fpji54s0pa0lfnf1dy00s2nh-wsjtx-2.2.2.drv /gnu/store/77g250mf14rskilpsggn070y4fjrbngx-keepassxc-2.6.0.drv /gnu/store/5k393vfm5qw442ya7v061q5fmjvhhcs5-warzone2100-3.4.0.drv /gnu/store/5d4h76qsr0rmbkn2c0zilyjr7mpzk5pd-mailcatcher-0.7.1.drv /gnu/store/mxm64sm569fyhdmxh6jlvdl18fwdb6f6-ruby-fast-gettext-2.0.3.drv /gnu/store/wglga3r1j6x87hqksrh5d1ra70kl729l-ruby-protobuf-3.10.3.drv /gnu/store/73nw9dxmmp8jykhmzwdpc3wk7mn81yal-gnome-shell-extension-gsconnect-33.drvguix build: error: all build users are currently in use; considercreating additional users and adding them to the `guixbuild' group
It seems that grafts trigger the problem, as passing the --no-graftsoption works around it. Could it be that grafts can't somehow be queuedlike normal builds, adapting to fit to the available build slots?
Maxim
M
M
Maxim Cournoyer wrote on 15 Jul 19:13 +0200
(address . 42371@debbugs.gnu.org)
87zh80itv9.fsf@gmail.com
Hello,
Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
Toggle quote (4 lines)> It seems that grafts trigger the problem, as passing the --no-grafts> option works around it. Could it be that grafts can't somehow be queued> like normal builds, adapting to fit to the available build slots?
I sent this message too early; the '--no-grafts' option does *not* workaround the issue... I wonder what triggers this here? I commonly buildthings (sometimes hundreds of packages at the same time) and never raninto this issue.
Maxim
M
M
Maxim Cournoyer wrote on 15 Jul 19:15 +0200
control message for bug #42371
(address . control@debbugs.gnu.org)
87y2nkitt7.fsf@gmail.com
retitle 42371 Error maxing out build slots on a build commandquit
L
L
Ludovic Courtès wrote on 16 Jul 12:18 +0200
Re: bug#42371: Grafts max out build slots without being queued
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 42371@debbugs.gnu.org)
871rlb7ogs.fsf@gnu.org
Hi,
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
Toggle quote (30 lines)> While working on ruby-asciidoctor-pdf, the following command failed> because of maxing out build users:>> ./pre-inst-env guix build --quiet --keep-going --check ruby_version@1.0.2 gnome-shell-extension-gsconnect@33 numix-gtk-theme@2.6.7 arc-theme@20190917 gnome@3.34.2 ruby-fakefs@1.2.2 ruby-protobuf@3.10.3 ruby-fast-gettext@2.0.3 ruby-sass-rails@5.0.7 mailcatcher@0.7.1 ruby-rspec-its@1.3.0 warzone2100@3.4.0 keepassxc@2.6.0 wsjtx@2.2.2 js8call@2.2.0 ruby-yard-with-tests@0.9.25 ruby-asciidoctor-pdf@1.5.3-1.f25ea78 ruby-web-console@3.7.0 ruby-autoprefixer-rails@9.4.7 ruby-jekyll-paginate-v2@2.0.0 ruby-solargraph@0.36.0 --max-jobs=1> The following derivations will be built:> /gnu/store/w5qba9nzswv7ysrlxfwcqdvrf9vsgbk6-ruby-jekyll-paginate-v2-2.0.0.drv> /gnu/store/4r1ziw3cb5n5jgknd1jfsvdl46cz56f9-ruby-autoprefixer-rails-9.4.7.drv> /gnu/store/2f674244hhb399p86z15zgv246sj78gr-ruby-web-console-3.7.0.drv> /gnu/store/sqy6f63vzgbyk6jg1zigxvks9g365wfs-ruby-rspec-its-1.3.0.drv> /gnu/store/zz34dh2q37r21r2d26k7z5rpryysc3km-ruby-sass-rails-5.0.7.drv> /gnu/store/62nlbfwd2jf01nm0p0wlc9sgvn5yp6gc-ruby-fakefs-1.2.2.drv> /gnu/store/pjdh08jh7pxlfm4dcz86bg5xqbjyf4lm-gnome-3.34.2.drv> /gnu/store/xqdvar6gmkz28003md4w9c1mqvg0slas-arc-theme-20190917.drv> /gnu/store/qwmj9b493ays4dvgkmniq61fnxg3claw-numix-gtk-theme-2.6.7.drv> /gnu/store/s112ayw6qg86y7j45r5yslfl151ngvgn-ruby_version-1.0.2.drv> The following grafts will be made:> /gnu/store/ww8blyc1nkf788jjb3anh22dylibvbah-ruby-solargraph-0.36.0.drv> /gnu/store/c7bfyqfq8d7imgy8il7wzqpx3qpx4zpw-ruby-asciidoctor-pdf-1.5.3-1.f25ea78.drv> /gnu/store/cf501w2kdgw6k2gv6n24ikhzww8f51x7-ruby-yard-with-tests-0.9.25.drv> /gnu/store/da8hhcaj20z06f2kww0sg4y24ma034dy-js8call-2.2.0.drv> /gnu/store/zn1jkg86fpji54s0pa0lfnf1dy00s2nh-wsjtx-2.2.2.drv> /gnu/store/77g250mf14rskilpsggn070y4fjrbngx-keepassxc-2.6.0.drv> /gnu/store/5k393vfm5qw442ya7v061q5fmjvhhcs5-warzone2100-3.4.0.drv> /gnu/store/5d4h76qsr0rmbkn2c0zilyjr7mpzk5pd-mailcatcher-0.7.1.drv> /gnu/store/mxm64sm569fyhdmxh6jlvdl18fwdb6f6-ruby-fast-gettext-2.0.3.drv> /gnu/store/wglga3r1j6x87hqksrh5d1ra70kl729l-ruby-protobuf-3.10.3.drv> /gnu/store/73nw9dxmmp8jykhmzwdpc3wk7mn81yal-gnome-shell-extension-gsconnect-33.drv> guix build: error: all build users are currently in use; consider> creating additional users and adding them to the `guixbuild' group
Presumably here it’s running one job at a time, so I don’t think it canend up using more than one build user.
What does “guix processes” return when that happens? Could it be thatsome other Guix clients are running?
Thanks,Ludo’.
Z
Z
zimoun wrote on 16 Jul 13:04 +0200
86365rzpol.fsf@gmail.com
Dear Maxim,

On Wed, 15 Jul 2020 at 12:13, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
Toggle quote (10 lines)> ./pre-inst-env guix build --quiet --keep-going --check> ruby_version@1.0.2 gnome-shell-extension-gsconnect@33> numix-gtk-theme@2.6.7 arc-theme@20190917 gnome@3.34.2> ruby-fakefs@1.2.2 ruby-protobuf@3.10.3 ruby-fast-gettext@2.0.3> ruby-sass-rails@5.0.7 mailcatcher@0.7.1 ruby-rspec-its@1.3.0> warzone2100@3.4.0 keepassxc@2.6.0 wsjtx@2.2.2 js8call@2.2.0> ruby-yard-with-tests@0.9.25 ruby-asciidoctor-pdf@1.5.3-1.f25ea78> ruby-web-console@3.7.0 ruby-autoprefixer-rails@9.4.7> ruby-jekyll-paginate-v2@2.0.0 ruby-solargraph@0.36.0 --max-jobs=1
Instead, I have run:
Toggle snippet (8 lines)guix build --keep-going --check ruby_versiongnome-shell-extension-gsconnect numix-gtk-theme arc-theme gnomeruby-fakefs git-minimal ruby-fast-gettext ruby-sass-rails mailcatcherruby-rspec-its warzone2100 keepassxc wsjtx js8call ruby-web-consolehello ruby-autoprefixer-rails ruby-jekyll-paginate-v2 ruby-solargraph--max-jobs=1
replacing the 2 unknown packages ruby-yard-with-tests and ruby-protobufby git-minimal and hello.
And it passes. I have 10 guixbuilder users, as it is by default. :-)

Toggle quote (26 lines)> The following derivations will be built:> /gnu/store/w5qba9nzswv7ysrlxfwcqdvrf9vsgbk6-ruby-jekyll-paginate-v2-2.0.0.drv> /gnu/store/4r1ziw3cb5n5jgknd1jfsvdl46cz56f9-ruby-autoprefixer-rails-9.4.7.drv> /gnu/store/2f674244hhb399p86z15zgv246sj78gr-ruby-web-console-3.7.0.drv> /gnu/store/sqy6f63vzgbyk6jg1zigxvks9g365wfs-ruby-rspec-its-1.3.0.drv> /gnu/store/zz34dh2q37r21r2d26k7z5rpryysc3km-ruby-sass-rails-5.0.7.drv> /gnu/store/62nlbfwd2jf01nm0p0wlc9sgvn5yp6gc-ruby-fakefs-1.2.2.drv> /gnu/store/pjdh08jh7pxlfm4dcz86bg5xqbjyf4lm-gnome-3.34.2.drv> /gnu/store/xqdvar6gmkz28003md4w9c1mqvg0slas-arc-theme-20190917.drv> /gnu/store/qwmj9b493ays4dvgkmniq61fnxg3claw-numix-gtk-theme-2.6.7.drv> /gnu/store/s112ayw6qg86y7j45r5yslfl151ngvgn-ruby_version-1.0.2.drv> The following grafts will be made:> /gnu/store/ww8blyc1nkf788jjb3anh22dylibvbah-ruby-solargraph-0.36.0.drv> /gnu/store/c7bfyqfq8d7imgy8il7wzqpx3qpx4zpw-ruby-asciidoctor-pdf-1.5.3-1.f25ea78.drv> /gnu/store/cf501w2kdgw6k2gv6n24ikhzww8f51x7-ruby-yard-with-tests-0.9.25.drv> /gnu/store/da8hhcaj20z06f2kww0sg4y24ma034dy-js8call-2.2.0.drv> /gnu/store/zn1jkg86fpji54s0pa0lfnf1dy00s2nh-wsjtx-2.2.2.drv> /gnu/store/77g250mf14rskilpsggn070y4fjrbngx-keepassxc-2.6.0.drv> /gnu/store/5k393vfm5qw442ya7v061q5fmjvhhcs5-warzone2100-3.4.0.drv> /gnu/store/5d4h76qsr0rmbkn2c0zilyjr7mpzk5pd-mailcatcher-0.7.1.drv> /gnu/store/mxm64sm569fyhdmxh6jlvdl18fwdb6f6-ruby-fast-gettext-2.0.3.drv> /gnu/store/wglga3r1j6x87hqksrh5d1ra70kl729l-ruby-protobuf-3.10.3.drv> /gnu/store/73nw9dxmmp8jykhmzwdpc3wk7mn81yal-gnome-shell-extension-gsconnect-33.drv> guix build: error: all build users are currently in use; consider> creating additional users and adding them to the `guixbuild' group
How many users do you have? What are they doing?

All the best,simon
Z
Z
zimoun wrote on 16 Jul 13:09 +0200
(address . 42371@debbugs.gnu.org)
86zh7zyavz.fsf@gmail.com
On Thu, 16 Jul 2020 at 12:18, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (6 lines)> Presumably here it’s running one job at a time, so I don’t think it can> end up using more than one build user.>> What does “guix processes” return when that happens? Could it be that> some other Guix clients are running?
Is it possible that the "lockFile()" is busy for another reason?
Cheers,simon
M
M
Maxim Cournoyer wrote on 17 Jul 00:06 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 42371@debbugs.gnu.org)
87tuy7i07z.fsf@gmail.com
Hi Ludo :-)
Ludovic Courtès <ludo@gnu.org> writes:
Toggle quote (51 lines)> Hi,>> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:>>> While working on ruby-asciidoctor-pdf, the following command failed>> because of maxing out build users:>>>> ./pre-inst-env guix build --quiet --keep-going --check>> ruby_version@1.0.2 gnome-shell-extension-gsconnect@33>> numix-gtk-theme@2.6.7 arc-theme@20190917 gnome@3.34.2>> ruby-fakefs@1.2.2 ruby-protobuf@3.10.3 ruby-fast-gettext@2.0.3>> ruby-sass-rails@5.0.7 mailcatcher@0.7.1 ruby-rspec-its@1.3.0>> warzone2100@3.4.0 keepassxc@2.6.0 wsjtx@2.2.2 js8call@2.2.0>> ruby-yard-with-tests@0.9.25 ruby-asciidoctor-pdf@1.5.3-1.f25ea78>> ruby-web-console@3.7.0 ruby-autoprefixer-rails@9.4.7>> ruby-jekyll-paginate-v2@2.0.0 ruby-solargraph@0.36.0 --max-jobs=1>> The following derivations will be built:>> /gnu/store/w5qba9nzswv7ysrlxfwcqdvrf9vsgbk6-ruby-jekyll-paginate-v2-2.0.0.drv>> /gnu/store/4r1ziw3cb5n5jgknd1jfsvdl46cz56f9-ruby-autoprefixer-rails-9.4.7.drv>> /gnu/store/2f674244hhb399p86z15zgv246sj78gr-ruby-web-console-3.7.0.drv>> /gnu/store/sqy6f63vzgbyk6jg1zigxvks9g365wfs-ruby-rspec-its-1.3.0.drv>> /gnu/store/zz34dh2q37r21r2d26k7z5rpryysc3km-ruby-sass-rails-5.0.7.drv>> /gnu/store/62nlbfwd2jf01nm0p0wlc9sgvn5yp6gc-ruby-fakefs-1.2.2.drv>> /gnu/store/pjdh08jh7pxlfm4dcz86bg5xqbjyf4lm-gnome-3.34.2.drv>> /gnu/store/xqdvar6gmkz28003md4w9c1mqvg0slas-arc-theme-20190917.drv>> /gnu/store/qwmj9b493ays4dvgkmniq61fnxg3claw-numix-gtk-theme-2.6.7.drv>> /gnu/store/s112ayw6qg86y7j45r5yslfl151ngvgn-ruby_version-1.0.2.drv>> The following grafts will be made:>> /gnu/store/ww8blyc1nkf788jjb3anh22dylibvbah-ruby-solargraph-0.36.0.drv>> /gnu/store/c7bfyqfq8d7imgy8il7wzqpx3qpx4zpw-ruby-asciidoctor-pdf-1.5.3-1.f25ea78.drv>> /gnu/store/cf501w2kdgw6k2gv6n24ikhzww8f51x7-ruby-yard-with-tests-0.9.25.drv>> /gnu/store/da8hhcaj20z06f2kww0sg4y24ma034dy-js8call-2.2.0.drv>> /gnu/store/zn1jkg86fpji54s0pa0lfnf1dy00s2nh-wsjtx-2.2.2.drv>> /gnu/store/77g250mf14rskilpsggn070y4fjrbngx-keepassxc-2.6.0.drv>> /gnu/store/5k393vfm5qw442ya7v061q5fmjvhhcs5-warzone2100-3.4.0.drv>> /gnu/store/5d4h76qsr0rmbkn2c0zilyjr7mpzk5pd-mailcatcher-0.7.1.drv>> /gnu/store/mxm64sm569fyhdmxh6jlvdl18fwdb6f6-ruby-fast-gettext-2.0.3.drv>> /gnu/store/wglga3r1j6x87hqksrh5d1ra70kl729l-ruby-protobuf-3.10.3.drv>> /gnu/store/73nw9dxmmp8jykhmzwdpc3wk7mn81yal-gnome-shell-extension-gsconnect-33.drv>> guix build: error: all build users are currently in use; consider>> creating additional users and adding them to the `guixbuild' group>> Presumably here it’s running one job at a time, so I don’t think it can> end up using more than one build user.>> What does “guix processes” return when that happens? Could it be that> some other Guix clients are running?>> Thanks,> Ludo’.
So, I'm afraid this isn't very useful, but:
for the most part of the process, I had something like:
Toggle snippet (18 lines)SessionPID: 12549ClientPID: 12540ClientCommand: /gnu/store/0w76khfspfy8qmcpjya41chj3bgfcy0k-guile-3.0.4/bin/guile --no-auto-compile /home/maxim/.config/guix/current/bin/guix environment --pure gcc-toolchain@10 --ad-hoc coreutils strace the-silver-searcher findutils
SessionPID: 1876ClientPID: 1868ClientCommand: /gnu/store/6ms9q743slbyx4zj7p97sqrwx8x3dgkm-profile/bin/guile \ /home/maxim/src/guix/scripts/guix build --no-grafts --max-jobs=1 --quiet --keep-going --check ruby_version@1.0.2 gnome-shell-extension-gsconnect@33 numix-gtk-theme@2.6.7 arc-theme@20190917 gnome@3.34.2 ruby-fakefs@1.2.2 ruby-protobuf@3.10.3 ruby-fast-gettext@2.0.3 ruby-sass-rails@5.0.7 mailcatcher@0.7.1 ruby-rspec-its@1.3.0 warzone2100@3.4.0 keepassxc@2.6.0 wsjtx@2.2.2 js8call@2.2.0 ruby-yard-with-tests@0.9.25 ruby-asciidoctor-pdf@1.5.3-1.f25ea78 ruby-web-console@3.7.0 ruby-autoprefixer-rails@9.4.7 ruby-jekyll-paginate-v2@2.0.0 ruby-solargraph@0.36.0
SessionPID: 447ClientPID: 441ClientCommand: /gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2/bin/guile \ /gnu/store/3f4xln2mii6cmc4c6x9138wl0bsh0fs7-guix-1.1.0-7.d445c30/bin/.guix-real publish -u guix-publish -p 80 -C gzip:3 --nar-path=nar --listen=0.0.0.0
SessionPID: 7060ClientPID: 7039ClientCommand: /gnu/store/0w76khfspfy8qmcpjya41chj3bgfcy0k-guile-3.0.4/bin/guile --no-auto-compile /home/maxim/.config/guix/current/bin/guix environment guix

And after the same error was printed (guix build: error: all build users arecurrently in use; consider creating additional users and adding them tothe `guixbuild' group), then I had:
Toggle snippet (14 lines)
SessionPID: 12549ClientPID: 12540ClientCommand: /gnu/store/0w76khfspfy8qmcpjya41chj3bgfcy0k-guile-3.0.4/bin/guile --no-auto-compile /home/maxim/.config/guix/current/bin/guix environment --pure gcc-toolchain@10 --ad-hoc coreutils strace the-silver-searcher findutils
SessionPID: 447ClientPID: 441ClientCommand: /gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2/bin/guile \ /gnu/store/3f4xln2mii6cmc4c6x9138wl0bsh0fs7-guix-1.1.0-7.d445c30/bin/.guix-real publish -u guix-publish -p 80 -C gzip:3 --nar-path=nar --listen=0.0.0.0
SessionPID: 7060ClientPID: 7039ClientCommand: /gnu/store/0w76khfspfy8qmcpjya41chj3bgfcy0k-guile-3.0.4/bin/guile --no-auto-compile /home/maxim/.config/guix/current/bin/guix environment guix
I didn't see the amount of build process going higher than than,monitoring it every second.
I've pushed the branch I'm using to test locally here [0], hoping it'llhelp to reproduce the issue somewhere else.
The command to run on this branch is:
./pre-inst-env guix build --no-grafts --max-jobs=1 --quiet --keep-going--check ruby_version@1.0.2 gnome-shell-extension-gsconnect@33numix-gtk-theme@2.6.7 arc-theme@20190917 gnome@3.34.2 ruby-fakefs@1.2.2ruby-protobuf@3.10.3 ruby-fast-gettext@2.0.3 ruby-sass-rails@5.0.7mailcatcher@0.7.1 ruby-rspec-its@1.3.0 warzone2100@3.4.0 keepassxc@2.6.0wsjtx@2.2.2 js8call@2.2.0 ruby-yard-with-tests@0.9.25ruby-asciidoctor-pdf@1.5.3-1.f25ea78 ruby-web-console@3.7.0ruby-autoprefixer-rails@9.4.7 ruby-jekyll-paginate-v2@2.0.0ruby-solargraph@0.36.0
The bug is only triggered when I also pass the "--check" option. Soyou'll probably have to build the stack once without "--check", then asecond time with "--check" passed to trigger the issue (if it'sreproducible).
Thank you!
Maxim
[0] https://gitlab.com/Apteryks/guix/-/tree/add-ruby-asciidoctor-pdf
M
M
Maxim Cournoyer wrote on 17 Jul 04:38 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 42371@debbugs.gnu.org)
87o8oej27f.fsf@gmail.com
Hey Simon,
zimoun <zimon.toutoune@gmail.com> writes:
Toggle quote (27 lines)> Dear Maxim,>>> On Wed, 15 Jul 2020 at 12:13, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:>>> ./pre-inst-env guix build --quiet --keep-going --check>> ruby_version@1.0.2 gnome-shell-extension-gsconnect@33>> numix-gtk-theme@2.6.7 arc-theme@20190917 gnome@3.34.2>> ruby-fakefs@1.2.2 ruby-protobuf@3.10.3 ruby-fast-gettext@2.0.3>> ruby-sass-rails@5.0.7 mailcatcher@0.7.1 ruby-rspec-its@1.3.0>> warzone2100@3.4.0 keepassxc@2.6.0 wsjtx@2.2.2 js8call@2.2.0>> ruby-yard-with-tests@0.9.25 ruby-asciidoctor-pdf@1.5.3-1.f25ea78>> ruby-web-console@3.7.0 ruby-autoprefixer-rails@9.4.7>> ruby-jekyll-paginate-v2@2.0.0 ruby-solargraph@0.36.0 --max-jobs=1>> Instead, I have run:>> guix build --keep-going --check ruby_version> gnome-shell-extension-gsconnect numix-gtk-theme arc-theme gnome> ruby-fakefs git-minimal ruby-fast-gettext ruby-sass-rails mailcatcher> ruby-rspec-its warzone2100 keepassxc wsjtx js8call ruby-web-console> hello ruby-autoprefixer-rails ruby-jekyll-paginate-v2 ruby-solargraph> --max-jobs=1>> replacing the 2 unknown packages ruby-yard-with-tests and ruby-protobuf> by git-minimal and hello.
Thank you for your tests! This makes me wonder whether these are theones causing the problem... I've uploaded the branch I'm working on sothat you could try to reproduce [0] if you are so inclined :-).
Toggle quote (2 lines)> [...] I have 10 guixbuilder users, as it is by default. :-)
Same here.
[...]
Toggle quote (5 lines)>> guix build: error: all build users are currently in use; consider>> creating additional users and adding them to the `guixbuild' group>> How many users do you have? What are they doing?
10 users, not doing anything special, as I found out looking at guixprocesses, so I don't know what is causing the exhaustion of buildusers.
Thanks,
Maxim
[0] https://gitlab.com/Apteryks/guix/-/tree/add-ruby-asciidoctor-pdf
M
M
Maxim Cournoyer wrote on 17 Jul 05:06 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
87zh7yhmbi.fsf@gmail.com
Hello everyone,
I did a couple more tests, and there's no need to get fancy withthe reproduction steps; any number greater of packages greater than yournumber of build slots/users should suffice:
Toggle snippet (17 lines)guix build --quiet --check $(guix package -A | head -n 11 | cut -f1)The following derivations will be built: /gnu/store/l7hrbi8cikf4c0calvqwd2n99v51ljzb-aalib-1.4rc5.drv /gnu/store/bijqg7w67rq1lr20hqbk18rq73963xfr-a2ps-4.14.drv /gnu/store/vh283nmxk08sl0m89s1fn07m432c9dg0-4ti2-1.6.9.drv /gnu/store/davmjim4nfvbhimpim329p642p9mg50d-0xffff-0.8.drv /gnu/store/7qaf91fmf42c8vp3fcysvqpkg4zahl1j-0ad-data-0.0.23b-alpha.drvThe following grafts will be made: /gnu/store/hjr33zqgx46ldswbqlmvgz7g37l7hrxl-abbaye-2.0.1.drv /gnu/store/71rdqs3hldmy1d92742camwc97l3pb3w-7kaa-2.15.3.drv /gnu/store/0b92jff2kfmd6l23y8v9xcprk8l7casw-4store-1.1.6.drv /gnu/store/82pj69v420fjmkqjxjxix3cckwcfdwg2-4dtris-0.4.3.drv /gnu/store/gamc676fdg1p7qzvzwfq7rp6qn4agss1-389-ds-base-1.4.0.21.drv /gnu/store/p310ysfwix0ci2594x78vv4x5isqcxix-0ad-0.0.23b-alpha.drvguix build: error: all build users are currently in use; consider creating additional users and adding them to the `guixbuild' group
The key is passing more than 10 packages (I have 10 build users) as anargument when using the '--check' argument.
I'll update the title to reflect the actual bug.
Maxim
M
M
Maxim Cournoyer wrote on 17 Jul 05:08 +0200
control message for bug #42371
(address . control@debbugs.gnu.org)
87y2nihm8u.fsf@gmail.com
retitle 42371 All build users used up when using '--check' with many packagesquit
Z
Z
zimoun wrote on 17 Jul 11:23 +0200
Re: bug#42371: Grafts max out build slots without being queued
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
86ft9qxzpj.fsf@gmail.com
Hi Maxim,
Toggle quote (3 lines)> --8<---------------cut here---------------start------------->8---> guix build --quiet --check $(guix package -A | head -n 11 | cut -f1)
[...]
Toggle quote (3 lines)> guix build: error: all build users are currently in use; consider creating additional users and adding them to the `guixbuild' group> --8<---------------cut here---------------end--------------->8---
In my case, it starts to download substitutes (but I have remove--quiet), something like:
Toggle snippet (71 lines)substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% 11.3 MB will be downloaded: /gnu/store/…-jansson-2.12 /gnu/store/…-jemalloc-5.2.1 /gnu/store/…-libev-4.31 /gnu/store/…-glibc-2.31 /gnu/store/…-glibc-2.31-static /gnu/store/…-c-ares-1.16.0 /gnu/store/…-nghttp2-1.41.0-lib /gnu/store/…-nghttp2-1.41.0 substituting /gnu/store/…-c-ares-1.16.0... downloading from https://ci.guix.gnu.org/nar/lzip/84hjhzjypgp5pgh6r5y7m561lk625\pn0-c-ares-1.16.0 ... c-ares-1.16.0 123KiB 1.6MiB/s 00:00 [##################] 100.0% substituting /gnu/store/…-jansson-2.12... downloading from https://ci.guix.gnu.org/nar/lzip/v5rgf8v6cjxjbngkzjcznj98dkxj8\svg-jansson-2.12 ... jansson-2.12 27KiB 4.3MiB/s 00:00 [##################] 100.0% […]13.2 MB will be downloaded: /gnu/store/…-cyrus-sasl-2.1.27 /gnu/store/…-glibc-2.31 /gnu/store/…-glibc-2.31-static /gnu/store/…-gnutls-3.6.12 /gnu/store/…-openldap-2.4.50 substituting /gnu/store/…-cyrus-sasl-2.1.27... downloading from https://ci.guix.gnu.org/nar/lzip/xp8545c74xkkj12q41d1zl5wngb47\7gj-cyrus-sasl-2.1.27 ... cyrus-sasl-2.1.27 234K[…] 28.0 MB will be downloaded: /gnu/store/…-pcre-8.44-bin /gnu/store/…-pcre-8.44 /gnu/store/…-mpfr-4.0.2 /gnu/store/…-rasqal-0.9.33 /gnu/store/…-libxslt-1.1.34 /gnu/store/…-raptor2-2.0.15 /gnu/store/…-glib-2.62.6 /gnu/store/…-dbus-1.12.16 /gnu/store/…-glib-2.62.6-bin /gnu/store/…-glib-2.62.6 /gnu/store/…-avahi-0.8 /gnu/store/…-4store-1.1.6 /gnu/store/…-a2ps-4.14 /gnu/store/…-mesa-20.0.8-bin /gnu/store/…-mesa-20.0.8 […] The following derivations will be built: /gnu/store/…-abbaye-2.0.1.drv /gnu/store/…-sdl-union-1.2.15.drv /gnu/store/…-0ad-0.0.23b-alpha.drv 899.1 MB will be downloaded: /gnu/store/…-libsodium-1.0.18 /gnu/store/…-mozjs-38.2.1.rc0[…] /gnu/store/…-ed-1.16 /gnu/store/…-patch-2.7.6 /gnu/store/…-diffutils-3.7[…]successfully built /gnu/store/…-sdl-union-1.2.15.drv building /gnu/store/…-0ad-0.0.23b-alpha.drv... WARNING: (guile-user): imported module (guix build utils) overrides core bindin\g `delete' starting phase `set-SOURCE-DATE-EPOCH' phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds starting phase `set-paths' environment variable `PATH' …[…]
And I even tried with "-n21" and it seems ok.
I am running Guix on the top of Debian with ext4.
Toggle quote (3 lines)> The key is passing more than 10 packages (I have 10 build users) as an> argument when using the '--check' argument.
Well, I am not sure to understand, but in nix/libstore/build.cc,"void UserLock::acquire()", there is:
Toggle snippet (13 lines) foreach (Strings::iterator, i, users) {[…] if (lockFile(fd, ltWrite, false)) {[…] return; } }
throw Error(format("all build users are currently in use; " "consider creating additional users and adding them to the `%1%' group") % settings.buildUsersGroup);
So maybe the file is locked for whatever other reason on your machine?

All the best,simon
Z
Z
zimoun wrote on 21 Jul 12:40 +0200
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
86365ldubo.fsf@gmail.com
Dear,
On my laptop and my desktop, both “cat /proc/cpuinfo |grep processor|wc-l” returning 4, and with 10 guixbuild users, I get these results.
On Thu, 16 Jul 2020 at 23:06, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
Toggle quote (2 lines)> --8<---------------cut here---------------start------------->8---> guix build --quiet --check $(guix package -A | head -n 11 | cut -f1)
[…]guix build: error: derivation `/gnu/store/…-4ti2-1.6.9.drv' may not be determin\istic: output `/gnu/store/…-4ti2-1.6.9' differs
Toggle quote (2 lines)> --8<---------------cut here---------------end--------------->8---
Then, I run this tiny script fixing --max-jobs and --cores on my desktopmachine only:
Toggle snippet (20 lines)for npkg in 4 5 6 7 8 9 10 11 12 do for JOBS in 1 2 3 4 do for CORES in 1 2 4 8 16 32 do echo "# num.of pkg=$npkg" echo "# --max-jobs=$JOBS" echo "# --cores=$CORES" guix build --quiet --check -k \ --max-jobs=$JOBS \ --cores=$CORES \ $(guix package -A | head -n $npkg | cut -f1) echo " " sleep 2 done done done
And it hits the non-deterministic error for all the cases until the casenum.of pkg=11 --max-jobs=2 --cores=1 where it errors "all build usersare currently in use; consider creating additional users and adding themto the `guixbuild' group".
Then npkg=11 --max-jobs=2 --cores=2 fails with non-deterministic.
(I reproduce on my laptop the result for npkg=11 max-jobs=2 cores=1,2.)
The next case, npkg=11 --max-jobs=2 --cores=4 does not seem reproducibleon my machine; similarly for cores=8,16,32.
Then I have not checked the reproducibility of the cases below.
Interestingly, npkg=11 --max-jobs=3 --cores=1,2,4,8,16 fails with notenough builders. But not for --cores=32 which fails withnon-deterministic.
All the values for cores fails with not enough builder for npkg=11 and--max-jobs=4.
Another surprise, npkg=12 --max-jobs=1 --cores=1,2,4,8,16,32 fails withnon-deterministic. And npkg=12 --max-jobs=2 --cores=2,4,8,16,32 too.

All the best,simon
?