Guix assumes ideal network exists, does not

  • Open
  • quality assurance status badge
Details
4 participants
  • Leo Famulari
  • Ludovic Courtès
  • Tobias Geerinckx-Rice
  • zimoun
Owner
unassigned
Submitted by
Tobias Geerinckx-Rice
Severity
normal
T
T
Tobias Geerinckx-Rice wrote on 2 Mar 2021 17:32
(address . bug-guix@gnu.org)
87eegxms7h.fsf@nckx
Guixfriends,

Errors like the one below are omnipresent. It's a problem.

Expecting an ever-increasing (16k+?) number of requests to finish
flawlessly or aborting a very long (20+ minutes here) process is
doomed.

We don't seem to have a plan forward (I include myself!). Looking
at other package managers, a blunt ‘retry a finite number of
times’ seems common. Guix's ‘pretend that "transient" is a magic
powerword’ approach less so.

What do people think? Does the heavy pipelining complicate
matters?

Toggle snippet (37 lines)
~ $ guix weather --substitute-urls=https://guix.tobias.gr
computing 16,745 package derivations for x86_64-linux...
looking for 18,127 store items on https://guix.tobias.gr...
updating substitutes from 'https://guix.tobias.gr'...
41.6%Backtrace:
11 (primitive-load
"/home/nckx/.config/guix/current/bin/guix")
In guix/ui.scm:
2164:12 10 (run-guix-command _ . _)
In ice-9/boot-9.scm:
1736:10 9 (with-exception-handler _ _ #:unwind? _
#:unwind-for-type _)
1731:15 8 (with-exception-handler #<procedure 7fe8a173d7e0 at
ice-9/boot-9.scm:1815:7 (exn)> _ #:unwind? _ …)
In guix/scripts/weather.scm:
546:9 7 (_)
In guix/build/utils.scm:
569:23 6 (every* #<procedure 7fe875fdd4e0 at
guix/scripts/weather.scm:546:17 (server)> _)
In guix/scripts/weather.scm:
547:19 5 (_ "https://guix.tobias.gr")
116:17 4 (report-server-coverage _ _ #:display-missing? _)
In unknown file:
3 (_ #<procedure 7fe86e3cbbe0 at
guix/scripts/weather.scm:184:2 ()> #<procedure list _>
. #w())
In guix/scripts/substitute.scm:
411:23 2 (lookup-narinfos _ _ #:open-connection _)
371:26 1 (fetch-narinfos _ _ #:open-connection _)
In ice-9/boot-9.scm:
1669:16 0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1669:16: In procedure raise-exception:
Throw to key `gnutls-error' with args `(#<gnutls-error-enum Error
in the pull function.> read_from_session_record_port)'.

Kind regards,

T G-R
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCYD5osg0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW15JcsA/jJ+ct5hDD+9ORqA0GwsvGNyspvrdqP+RpNWXrdf
afuoAQCIvpujM9vA26Stvfu/VxN3MVCM6Z/gAYIaBT61qTaqDQ==
=9dJQ
-----END PGP SIGNATURE-----

L
L
Leo Famulari wrote on 2 Mar 2021 20:55
(name . Tobias Geerinckx-Rice via Bug reports for GNU Guix)(address . bug-guix@gnu.org)(address . 46877@debbugs.gnu.org)
YD6YFsxzzX7Jy6Ay@jasmine.lan
On Tue, Mar 02, 2021 at 05:32:50PM +0100, Tobias Geerinckx-Rice via Bug reports for GNU Guix wrote:
Toggle quote (16 lines)
> Guixfriends,
>
> Errors like the one below are omnipresent. It's a problem.
>
> Expecting an ever-increasing (16k+?) number of requests to finish flawlessly
> or aborting a very long (20+ minutes here) process is doomed.
>
> We don't seem to have a plan forward (I include myself!). Looking at other
> package managers, a blunt ‘retry a finite number of times’ seems common.
> Guix's ‘pretend that "transient" is a magic powerword’ approach less so.
>
> What do people think? Does the heavy pipelining complicate matters?
>
> --8<---------------cut here---------------start------------->8---
> ~ $ guix weather --substitute-urls=https://guix.tobias.gr

I sometimes see similar failures with `guix weather`.

To clarify, do you also see them when substituting? Or just when
checking for substitutes?
Z
Z
zimoun wrote on 2 Mar 2021 21:00
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)(address . 46877@debbugs.gnu.org)
CAJ3okZ2v=c=zFM-k3BUntvkEpwsq4D8cHQk3DBbhrTLY+Tirmw@mail.gmail.com
Hi Tobias,

On Tue, 2 Mar 2021 at 17:33, Tobias Geerinckx-Rice via Bug reports for
GNU Guix <bug-guix@gnu.org> wrote:

Toggle quote (5 lines)
> We don't seem to have a plan forward (I include myself!). Looking
> at other package managers, a blunt ‘retry a finite number of
> times’ seems common. Guix's ‘pretend that "transient" is a magic
> powerword’ approach less so.

If I read correctly, the issue comes from (guix scripts substitutes)
in 'fetch-narinfos':

Toggle snippet (11 lines)
(call-with-connection-error-handling
uri
(lambda ()
(http-multiple-get uri
handle-narinfo-response '()
requests
#:open-connection
open-connection
#:verify-certificate? #f))))))

where 'requests' can be really large (all the missing information from
local cache).

Instead of processing the list in one go, we could imagine split it
into several chunks, say X, and so process only X requests by X
requests.

WDYT?


Cheers,
simon
Z
Z
zimoun wrote on 2 Mar 2021 21:18
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)(address . 46877@debbugs.gnu.org)
CAJ3okZ0hmCcdObysufmWXL2=xB02mXHzB9D0Xz=MF33bFgH+iA@mail.gmail.com
On Tue, 2 Mar 2021 at 21:00, zimoun <zimon.toutoune@gmail.com> wrote:

Toggle quote (4 lines)
> Instead of processing the list in one go, we could imagine split it
> into several chunks, say X, and so process only X requests by X
> requests.

My bad, it is already the case where X=1000 in (guix http-client).
Well, the error / retry seems poorly handled in 'http-multiple-get'.

Cheers,
simon
L
L
Ludovic Courtès wrote on 10 Mar 2021 12:17
(name . Tobias Geerinckx-Rice via Bug reports for GNU Guix)(address . bug-guix@gnu.org)
87o8fr2r6y.fsf@gnu.org
Hi,

Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix@gnu.org>
skribis:

Toggle quote (36 lines)
> ~ $ guix weather --substitute-urls=https://guix.tobias.gr
> computing 16,745 package derivations for x86_64-linux...
> looking for 18,127 store items on https://guix.tobias.gr...
> updating substitutes from 'https://guix.tobias.gr'...
> 41.6%Backtrace:
> 11 (primitive-load
> "/home/nckx/.config/guix/current/bin/guix")
> In guix/ui.scm:
> 2164:12 10 (run-guix-command _ . _)
> In ice-9/boot-9.scm:
> 1736:10 9 (with-exception-handler _ _ #:unwind? _
> #:unwind-for-type _)
> 1731:15 8 (with-exception-handler #<procedure 7fe8a173d7e0 at
> ice-9/boot-9.scm:1815:7 (exn)> _ #:unwind? _ …)
> In guix/scripts/weather.scm:
> 546:9 7 (_)
> In guix/build/utils.scm:
> 569:23 6 (every* #<procedure 7fe875fdd4e0 at
> guix/scripts/weather.scm:546:17 (server)> _)
> In guix/scripts/weather.scm:
> 547:19 5 (_ "https://guix.tobias.gr")
> 116:17 4 (report-server-coverage _ _ #:display-missing? _)
> In unknown file:
> 3 (_ #<procedure 7fe86e3cbbe0 at
> guix/scripts/weather.scm:184:2 ()> #<procedure list _>
> . #w())
> In guix/scripts/substitute.scm:
> 411:23 2 (lookup-narinfos _ _ #:open-connection _)
> 371:26 1 (fetch-narinfos _ _ #:open-connection _)
> In ice-9/boot-9.scm:
> 1669:16 0 (raise-exception _ #:continuable? _)
>
> ice-9/boot-9.scm:1669:16: In procedure raise-exception:
> Throw to key `gnutls-error' with args `(#<gnutls-error-enum Error
> in the pull function.> read_from_session_record_port)'.

Hmm isn’t it a regression?

First, in (guix scripts substitute) there’s a ‘with-networking’ macro to
nicely present networking errors; perhaps we’re missing it here?

Second, is it a genuine bug? Do you see the same thing from (guix
scripts substitute)?

Thanks,
Ludo’.
Z
Z
zimoun wrote on 10 Mar 2021 23:04
86k0qebr78.fsf@gmail.com
Hi,

On Wed, 10 Mar 2021 at 12:17, Ludovic Courtès <ludo@gnu.org> wrote:


Toggle quote (18 lines)
>> In guix/scripts/substitute.scm:
>> 411:23 2 (lookup-narinfos _ _ #:open-connection _)
>> 371:26 1 (fetch-narinfos _ _ #:open-connection _)
>> In ice-9/boot-9.scm:
>> 1669:16 0 (raise-exception _ #:continuable? _)
>>
>> ice-9/boot-9.scm:1669:16: In procedure raise-exception:
>> Throw to key `gnutls-error' with args `(#<gnutls-error-enum Error
>> in the pull function.> read_from_session_record_port)'.
>
> Hmm isn’t it a regression?
>
> First, in (guix scripts substitute) there’s a ‘with-networking’ macro to
> nicely present networking errors; perhaps we’re missing it here?
>
> Second, is it a genuine bug? Do you see the same thing from (guix
> scripts substitute)?

I do not know if it is related, the bug#47055 seems “similar”:



Cheers,
simon
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 46877@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 46877
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch