Latest binary tarball & Hurd qcow2 image unavailable (502)

  • Open
  • quality assurance status badge
Details
5 participants
  • Bengt Richter
  • Ludovic Courtès
  • Tobias Geerinckx-Rice
  • Mathieu Othacehe
  • zimoun
Owner
unassigned
Submitted by
Tobias Geerinckx-Rice
Severity
normal
T
T
Tobias Geerinckx-Rice wrote on 18 Nov 2020 17:39
(name . Bug Guix)(address . bug-guix@gnu.org)
87y2iyvcti.fsf@nckx
Guix,

The binary tarball and Hurd qcow2 links on
https://ci.guix.gnu.org/download/1450 respectively, return 502.

Here's why:

2020-11-18T17:31:24 GET /download/1450
Uncaught exception in fiber ##f:
In ice-9/boot-9.scm:
1669:16 0 (raise-exception _ #:continuable? _)
ice-9/boot-9.scm:1669:16: In procedure raise-exception:
In procedure open-file: No such file or directory:
"/gnu/store/vd5y7hmvgqc1hwmqyiwqa171r0iz1wib-hurd-barebones.qcow2"

The file really is gone. It seems that Cuirass is no longer
creating sufficient GC roots. I vaguely remember some changes to
image GC'ing being discussed but not what (if anything) changed.

Thanks to zimoun for pointing out the broken links.

Kind regards,

T G-R
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCX7VOWQ0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW15C7wBAMBSliYVOfM/xiE/1XSBxMHNq86C2kD1GzqK7cp4
lLHKAQC04LAqqwJLLYBsIdkdYAyrdyJg053xzo+qjO7wruiYCA==
=pqAB
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 18 Nov 2020 18:45
(name . Tobias Geerinckx-Rice via Bug reports for GNU Guix)(address . bug-guix@gnu.org)
875z62muea.fsf@gnu.org
Hi,

Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix@gnu.org>
skribis:

Toggle quote (19 lines)
> The binary tarball and Hurd qcow2 links on
> <http://guix.gnu.org/en/download/latest/>, which resolve to
> <https://ci.guix.gnu.org/download/1385> and
> <https://ci.guix.gnu.org/download/1450> respectively, return 502.
>
> Here's why:
>
> 2020-11-18T17:31:24 GET /download/1450
> Uncaught exception in fiber ##f:
> In ice-9/boot-9.scm:
> 1669:16 0 (raise-exception _ #:continuable? _)
> ice-9/boot-9.scm:1669:16: In procedure raise-exception:
> In procedure open-file: No such file or directory:
> "/gnu/store/vd5y7hmvgqc1hwmqyiwqa171r0iz1wib-hurd-barebones.qcow2"
>
> The file really is gone. It seems that Cuirass is no longer creating
> sufficient GC roots. I vaguely remember some changes to image GC'ing
> being discussed but not what (if anything) changed.

QCOW2 images get GC’d rather quickly, because they take up a lot of
space; see in particular:


I guess it’s OK for Guix System ISO images because a fresh one gets
built at each evaluation, roughly, but perhaps the Hurd images are not
rebuilt frequently enough and the changes that they’re GC’d are higher.

I don’t have a good solution for that, we’re consuming storage space too
quickly with all these images currently. :-/

Thanks,
Ludo’.
Z
Z
zimoun wrote on 18 Nov 2020 18:50
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 44723@debbugs.gnu.org)
CAJ3okZ3fawBARjyc+0q-hG468MooXEk2Gy5c38YG89VqErhM7A@mail.gmail.com
Hi,

On Wed, 18 Nov 2020 at 18:46, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (7 lines)
> I guess it’s OK for Guix System ISO images because a fresh one gets
> built at each evaluation, roughly, but perhaps the Hurd images are not
> rebuilt frequently enough and the changes that they’re GC’d are higher.
>
> I don’t have a good solution for that, we’re consuming storage space too
> quickly with all these images currently. :-/

Does it make sense to only keep the latest? And GC all the older one
except the stable one (pinned).

Cheers,
simon
T
T
Tobias Geerinckx-Rice wrote on 18 Nov 2020 20:16
(name . zimoun)(address . zimon.toutoune@gmail.com)
87sg96v5l8.fsf@nckx
Simon,

zimoun ???
Toggle quote (4 lines)
> Does it make sense to only keep the latest? And GC all the
> older one
> except the stable one (pinned).

It absolutely makes sense. But knowing how much of a baroque
spaghetti painting some of this stuff is behind the scenes, it
might not be as trivial as it sounds.

Kind regards,

T G-R
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCX7Vy8w0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW15P9QBAJJsgqgqjdizF2KF/XQx3cj+Sl6tWTxJUwsilmTO
hu8wAP9yF2+7gFwl69NlLvezkE1PTS+vKI+gNFiXlsGTGpO8AQ==
=Nvtn
-----END PGP SIGNATURE-----

B
B
Bengt Richter wrote on 18 Nov 2020 21:13
(name . zimoun)(address . zimon.toutoune@gmail.com)
20201118201355.GA2833@LionPure
Hi,

On +2020-11-18 18:50:03 +0100, zimoun wrote:
Toggle quote (11 lines)
> Hi,
>
> On Wed, 18 Nov 2020 at 18:46, Ludovic Courtès <ludo@gnu.org> wrote:
>
> > I guess it’s OK for Guix System ISO images because a fresh one gets
> > built at each evaluation, roughly, but perhaps the Hurd images are not
> > rebuilt frequently enough and the changes that they’re GC’d are higher.
> >
> > I don’t have a good solution for that, we’re consuming storage space too
> > quickly with all these images currently. :-/
>
Eric Blake from Red Hat is IIUC working on qemu features to share clean parts
of images without duplicating [1] and LWN has an article re his work [2]
(I hope posting this [2] subsriberlink here is Ok with LWN -- I would think so)

Perhaps their incremental chaining technology can help consolidate "all these images" ?

Toggle quote (8 lines)
> Does it make sense to only keep the latest? And GC all the older one
> except the stable one (pinned).
>
> Cheers,
> simon
>
>
>
([2] should be available freely to all as [3] in less than a week)
--
Regards,
Bengt Richter
L
L
Ludovic Courtès wrote on 19 Nov 2020 09:05
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 44723@debbugs.gnu.org)
878saxixf9.fsf@gnu.org
Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (12 lines)
> On Wed, 18 Nov 2020 at 18:46, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> I guess it’s OK for Guix System ISO images because a fresh one gets
>> built at each evaluation, roughly, but perhaps the Hurd images are not
>> rebuilt frequently enough and the changes that they’re GC’d are higher.
>>
>> I don’t have a good solution for that, we’re consuming storage space too
>> quickly with all these images currently. :-/
>
> Does it make sense to only keep the latest? And GC all the older one
> except the stable one (pinned).

We could change ‘cleanup-cuirass-roots’ (the mcron job I showed) to keep
at least one ‘hurd-barebones.qcow2’ root. Help welcome!

Ludo’.
Z
Z
zimoun wrote on 19 Nov 2020 09:43
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 44723@debbugs.gnu.org)
86blft3fev.fsf@gmail.com
Hi,
On Thu, 19 Nov 2020 at 09:05, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (6 lines)
>> Does it make sense to only keep the latest? And GC all the older one
>> except the stable one (pinned).
>
> We could change ‘cleanup-cuirass-roots’ (the mcron job I showed) to keep
> at least one ‘hurd-barebones.qcow2’ root. Help welcome!

Where is the configuration? Repo maintenance? Elsewhere?

All the beest,
simon
M
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 44723@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 44723
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch