"error: connect*: Connection timed out" when using 6+ jobs to fetch substitutes

  • Open
  • quality assurance status badge
Details
3 participants
  • Ludovic Courtès
  • Maxim Cournoyer
  • Maxime Devos
Owner
unassigned
Submitted by
Maxim Cournoyer
Severity
normal
M
M
Maxim Cournoyer wrote on 24 Aug 2022 22:42
Download
(name . bug-guix)(address . bug-guix@gnu.org)
87fshls78z.fsf@gmail.com
Download
Hello,

On a machine configured with 20 builders:

Toggle snippet (10 lines)
(guix-service-type config =>
(guix-configuration
(inherit config)
(build-accounts 20)
(discover? #t)
(log-compression 'none)
(extra-options '("--max-jobs=20"))))


Running 'guix package -m manifest.scm' with hundreds of packages/MiB to
fetch from the substitute servers, it fails like:

Toggle snippet (43 lines)
$ guix package -m ~/stow/guix/manifest.scm -M7
The following packages will be installed:
acpi 1.7
adb 7.1.2_r36
adwaita-icon-theme 40.1.1
alsa-utils 1.2.4
ansible 4.7.0
anthy 9100h
arc-icon-theme 20161122
arc-theme 20210412
[...]
xournal 0.4.8.2016
xrandr 1.5.1
xrdb 1.2.1
xsetroot 1.1.2
yelp 41.0

substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
924.3 MB will be downloaded

substitution of /gnu/store/1w54x244cykra3p2bmmxv9jn12q92l0r-gimp-2.10.32-doc complete

[...]

substitution of /gnu/store/adq43ck5b12sx0n9hy4hgay4gk7spy5b-libxml++-2.40.1 complete

substitution of /gnu/store/i379f1vadva2lnk1jh60c03s26m6rj7b-ibus-1.5.24-doc complete

substitution of /gnu/store/9j1hfzgpllgshy6qn82pl8zkkgn4058w-hackneyed-x11-cursors-0.8.2 complete

substitution of /gnu/store/bl9y57k0wg3aqxb8xiwddagj3i4dk036-perl-b-hooks-endofscope-0.24 complete

substitution of /gnu/store/qpc614b2hbzcsjhac38m9av3cqnrwjqi-perl-datetime-1.54 complete

substitution of /gnu/store/r07xvd187my02gg5smlw4pis90x7m3ah-opensp-1.5.2 complete
guix substitute: warning: while fetching https://ci.guix.gnu.org/nar/lzip/q7hiikl3q3dk1r46rlyyzd90yaddydrd-gnucash-docs-4.10: server is somewhat slow
guix substitute: warning: try `--no-substitutes' if the problem persists
guix substitute: error: connect*: Connection timed out
substitution of /gnu/store/q7hiikl3q3dk1r46rlyyzd90yaddydrd-gnucash-docs-4.10 failed
guix package: error: corrupt input while restoring archive from socket

It fails like the above when the number of jobs is 6 or higher, but
proceeds without error when the number of jobs is 5 or less.

I do not think the network to be at caused, as everything is cabled and
the link is healthy and fast.

Thanks,

Maxim
M
M
Maxime Devos wrote on 24 Aug 2022 23:52
Download
892c76c1-87a3-bb8f-d74d-f887f4fc34ac@telenet.be
Download
On 24-08-2022 22:42, Maxim Cournoyer wrote:
Toggle quote (7 lines)
> It fails like the above when the number of jobs is 6 or higher, but
> proceeds without error when the number of jobs is 5 or less.
>
> I do not think the network to be at caused, as everything is cabled and
> the link is healthy and fast.
>
> Thanks,
I was getting a lot of those errors recently -- I have "--max-jobs=14"
in the extra-options of guix-configuration.
It's not exactly the same situation though (WiFi, albeit usually not too
bad ...).
I'll give "-M5" a try in the future, maybe I can reproduce the 5/6 boundary.
Greetings,
Maxime.
Attachment: OpenPGP_0x49E3EE22191725EE.asc
Attachment: OpenPGP_signature
M
M
Maxime Devos wrote on 25 Aug 2022 14:56
Download
056cf7ca-57ff-3f1f-3b06-3b8c3e4db307@telenet.be
Download
On 24-08-2022 22:42, Maxim Cournoyer wrote:
Toggle quote (5 lines)
> It fails like the above when the number of jobs is 6 or higher, but
> proceeds without error when the number of jobs is 5 or less.
>
> I do not think the network to be at caused, as everything is cabled and
> the link is healthy and fast.
I encounter less "connect*: Connection timed out" after setting -M5, but
it still sometimes happens:
Toggle quote (12 lines)
> /gnu/store/vifx9ajaa65624f412bqqcdk4k83v461-flite-2.2-checkout
> vervangen...
> guix substitute: waarschuwing: tijdens het binnenhalen van
> https://ci.guix.gnu.org/nar/lzip/vifx9ajaa65624f412bqqcdk4k83v461-flite-2.2-checkout:
> de server is een beetje traag
> guix substitute: waarschuwing: probeer ‘--no-substitutes’ als het
> probleem hardnekkig is
> guix substitute: fout: connect*: Verbinding is verlopen
> vervanging van
> /gnu/store/vifx9ajaa65624f412bqqcdk4k83v461-flite-2.2-checkout mislukt
> guix build: fout: beschadigde invoer tijdens het terugplaatsen van het
> archief uit #<closed: file 7efc38785e00>
Greetings,
Maxime.
Attachment: OpenPGP_0x49E3EE22191725EE.asc
Attachment: OpenPGP_signature
L
L
Ludovic Courtès wrote on 5 May 10:54 +0200
Download
Re: bug#63024: Crash during `guix import pypi -r'
(name . Greg Hogan)(address . code@greghogan.com)
87h6srnop6.fsf@gnu.org
Download
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (12 lines)
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> First, the person who wrote ‘connect*’ *cough* hadn’t carefully read the
>> Guile manual, which reads (info "(guile) Ports and File Descriptors"):
>>
>> Note that ‘select’ may return early for other reasons, for example due
>> to pending interrupts.
>>
>> Clearly, ‘select’ is returning early, so we should check that and loop.
>
> Pushed a fix for that in fc6c96c88a0e4ad0b9e48272e5f97ffaa6eec36e.

… and updated the ‘guix’ package in
3c91f4ca490a7ac56dc0aebbca9c4bf4df201877, which should address
‘connect*’ timeout issues that were coming from ‘guix substitute’.

Ludo’.
?