linux-libre-guix.tar.xz CI times out on aarch64-linux

Done

Details

2 participants

Christopher Baines
Richard Sent

Owner: unassigned

Submitted by: Richard Sent

Severity: normal

Richard Sent wrote on 23 May 00:06 +0200

Recipients:(address . bug-guix@gnu.org)(address . mail@cbaines.net)

Message-ID:878r01wlgf.fsf@freakingpenguin.com

Hi Guix!

On aarch64 platforms, the linux-libre-guix.tar.xz.drv build times out

[1].

This package is known to have a long build time [2].

Users can work around this by running the build locally with a (very,

very large) max-silent-time (~28800 seconds on my machine). Given the

impact of this issue (unable to build linux-libre on 64-bit Arm without

running a multi-hour build), I think it's worth revisiting if the

substitute server timeout should be increased again.

Alternatively, perhaps we could fetch the officially released

Linux-libre tarballs instead of computing them ourselves [3].

[1]: https://ci.guix.gnu.org/build/4711550/log/raw

[2]: https://mail.gnu.org/archive/html/guix-devel/2021-08/msg00077.html

[3]: http://linux-libre.fsfla.org/pub/linux-libre/releases/

CC'ing Christopher Baines since this deals with substitutes.

Take it easy,

Richard Sent

Making my computer weirder one commit at a time.

Christopher Baines wrote on 26 May 20:26 +0200

Recipients:(name . Richard Sent)(address . richard@freakingpenguin.com)(address . bug-guix@gnu.org)

Message-ID:87ttike8fp.fsf@cbaines.net

Richard Sent <richard@freakingpenguin.com> writes:

Toggle quote (20 lines)> On aarch64 platforms, the linux-libre-guix.tar.xz.drv build times out
> [1].
>
> This package is known to have a long build time [2].
>
> Users can work around this by running the build locally with a (very,
> very large) max-silent-time (~28800 seconds on my machine). Given the
> impact of this issue (unable to build linux-libre on 64-bit Arm without
> running a multi-hour build), I think it's worth revisiting if the
> substitute server timeout should be increased again.
>
> Alternatively, perhaps we could fetch the officially released
> Linux-libre tarballs instead of computing them ourselves [3].
>
> [1]: https://ci.guix.gnu.org/build/4711550/log/raw
> [2]: https://mail.gnu.org/archive/html/guix-devel/2021-08/msg00077.html
> [3]: http://linux-libre.fsfla.org/pub/linux-libre/releases/
>
> CC'ing Christopher Baines since this deals with substitutes.

This derivation seems to have been built fine by the bordeaux build

farm:

https://data.guix.gnu.org/gnu/store/ny56fdcig9cd9bd3pssmlraz2c1q10q8-linux-libre-6.8.10-guix.tar.xz.drv

This is the timeout configuration for the bordeaux ARM build machines:

(max-silent-time (* 12 3600))

(timeout (* 72 3600))

e.g. https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/hatysa.scm#n262

The ci.guix.gnu.org Honeycomb timeouts are different, but don't look too

short. Maybe this build happened on an Overdrive system though.

Richard Sent wrote on 26 May 20:34 +0200

Recipients:(name . Christopher Baines)(address . mail@cbaines.net)(address . bug-guix@gnu.org)

Message-ID:87wmngs9q4.fsf@freakingpenguin.com

Christopher Baines <mail@cbaines.net> writes:

Toggle quote (5 lines)

> This derivation seems to have been built fine by the bordeaux build

> farm:

> https://data.guix.gnu.org/gnu/store/ny56fdcig9cd9bd3pssmlraz2c1q10q8-linux-libre-6.8.10-guix.tar.xz.drv

You're right. It looks like the derivation itself built fine based on

that, but something seems off when the nar is sent. Assuming I

understand the following correctly:

Toggle snippet (16 lines)~/code/cloned/guix/guix $ guix build linux-libre --system=aarch64-linux
substitute: updating substitutes from 'http://10.1.2.2:80'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
The following derivations will be built:
  /gnu/store/v471590wpsw1fcnqrrr9bwh52skbb5rn-linux-libre-6.8.10.drv
  /gnu/store/1wi10rg7236ck8k5vdrdfap5l7a9s9z0-linux-libre-6.8.10-guix.tar.xz.drv
143.1 MB will be downloaded:
  /gnu/store/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz
substituting /gnu/store/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz...
downloading from https://bordeaux.guix.gnu.org/nar/none/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz ...
 linux-libre-6.8.10-guix.tar.xz  136.5MiB                                                                          23.9MiB/s 00:06 ????????????????????  98.0%guix substitute: error: corrupt input while restoring '/gnu/store/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz' from #<input: string 7f9644c23b60>
substitution of /gnu/store/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz failed
guix build: error: some substitutes for the outputs of derivation `/gnu/store/ny56fdcig9cd9bd3pssmlraz2c1q10q8-linux-libre-6.8.10-guix.tar.xz.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source 

Even though the derivation was built, the substitution fails.

Just for fun, here's the output on a ARM64-native system:

Toggle snippet (24 lines)root@caustic ~# guix build linux-libre  
substitute: updating substitutes from 'http://10.1.2.2:80'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
The following derivations will be built:
  /gnu/store/v471590wpsw1fcnqrrr9bwh52skbb5rn-linux-libre-6.8.10.drv
  /gnu/store/1wi10rg7236ck8k5vdrdfap5l7a9s9z0-linux-libre-6.8.10-guix.tar.xz.drv
143.1 MB will be downloaded:
  /gnu/store/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz
substituting /gnu/store/0jfsx4hljddyand45z7i77ynpvr0mhb5-module-import-compiled...
downloading from http://10.1.2.2/nar/zstd/0jfsx4hljddyand45z7i77ynpvr0mhb5-module-import-compiled ...
 module-import-compiled                                                                                                    4.2MiB/s 00:00 | 244KiB transferred

substituting /gnu/store/fwqy5d6xyar9x5yksny79r2d519s25cx-100gnu+freedo.patch...
downloading from http://10.1.2.2/nar/zstd/fwqy5d6xyar9x5yksny79r2d519s25cx-100gnu%2Bfreedo.patch ...
 100gnu%2Bfreedo.patch                                                                                                      1.8MiB/s 00:00 | 46KiB transferred

substituting /gnu/store/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz...
downloading from https://bordeaux.guix.gnu.org/nar/none/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz ...
 linux-libre-6.8.10-guix.tar.xz  136.5MiB                                                                           1.7MiB/s 01:21 ????????????????????  99.9%guix substitute: error: corrupt input while restoring '/gnu/store/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz' from #<input: string ffff7daf3460>
substitution of /gnu/store/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz failed
guix build: error: corrupt input while restoring archive from #<closed: file ffff977abb60>

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Christopher Baines wrote on 26 May 20:46 +0200

Recipients:(name . Richard Sent)(address . richard@freakingpenguin.com)(address . bug-guix@gnu.org)

Message-ID:87o78se7is.fsf@cbaines.net

Richard Sent <richard@freakingpenguin.com> writes:

Toggle quote (30 lines)> Christopher Baines <mail@cbaines.net> writes:
>
>> This derivation seems to have been built fine by the bordeaux build
>> farm:
>>
>>   https://data.guix.gnu.org/gnu/store/ny56fdcig9cd9bd3pssmlraz2c1q10q8-linux-libre-6.8.10-guix.tar.xz.drv
>
> You're right. It looks like the derivation itself built fine based on
> that, but something seems off when the nar is sent. Assuming I
> understand the following correctly:
>
> --8<---------------cut here---------------start------------->8---
> ~/code/cloned/guix/guix $ guix build linux-libre --system=aarch64-linux
> substitute: updating substitutes from 'http://10.1.2.2:80'... 100.0%
> substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
> The following derivations will be built:
>   /gnu/store/v471590wpsw1fcnqrrr9bwh52skbb5rn-linux-libre-6.8.10.drv
>   /gnu/store/1wi10rg7236ck8k5vdrdfap5l7a9s9z0-linux-libre-6.8.10-guix.tar.xz.drv
> 143.1 MB will be downloaded:
>   /gnu/store/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz
> substituting /gnu/store/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz...
> downloading from https://bordeaux.guix.gnu.org/nar/none/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz ...
>  linux-libre-6.8.10-guix.tar.xz  136.5MiB                                                                          23.9MiB/s 00:06 ????????????????????  98.0%guix substitute: error: corrupt input while restoring '/gnu/store/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz' from #<input: string 7f9644c23b60>
> substitution of /gnu/store/y813phs2n9xnb7zbcr07g0j9509bzbsb-linux-libre-6.8.10-guix.tar.xz failed
> guix build: error: some substitutes for the outputs of derivation `/gnu/store/ny56fdcig9cd9bd3pssmlraz2c1q10q8-linux-libre-6.8.10-guix.tar.xz.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source 
> --8<---------------cut here---------------end--------------->8---
>
> Even though the derivation was built, the substitution fails.

Yeah, something's up here specifically with the bordeaux build farm,

feel free to open a new issue.

You can tell something is wrong just by looking at the narinfo:

https://bordeaux.guix.gnu.org/y813phs2n9xnb7zbcr07g0j9509bzbsb.narinfo

Given there's no compression here, the file size should be the same as

the nar size, but it's not, it's missing a few bytes.

Richard Sent wrote on 26 May 23:15 +0200

Recipients:(name . Christopher Baines)(address . mail@cbaines.net)(address . bug-guix@gnu.org)

Message-ID:87jzjgs2ae.fsf@freakingpenguin.com

New issue opened at https://issues.guix.gnu.org/71214.I'l close this
one. Thanks!

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Richard Sent wrote on 26 May 23:16 +0200

control message for bug #71133

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87ikz0s28h.fsf@freakingpenguin.com

tags 71133 notabug
close 71133
quit

Your comment

This issue is archived.

To comment on this conversation send an email to 71133@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 71133

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date