"build-system compiler failures are not cached as expected by Cuirass"

> I noticed that many Common Lisp or SBCL-related packages are failing to

> build on the aarc64 platform on our build farm, due the failure to build

> SBCL:

>

> From the log of <https://ci.guix.gnu.org/build/180326/details>:

>

> ------

> //entering make-target-2.sh

> //doing warm init - compilation phase

> This is SBCL 2.1.0, an implementation of ANSI Common Lisp.

> More information about SBCL is available at <http://www.sbcl.org/>.

>

> SBCL is free software, provided as is, with absolutely no warranty.

> It is mostly in the public domain; some portions are provided under

> BSD-style licenses. See the CREDITS and COPYING files in the

> distribution for more information.

> Initial page table:

> Gen Boxed Code Raw LgBox LgCode LgRaw Pin Alloc Waste Trig WP GCs Mem-age

> 6 397 250 0 0 0 0 0 42335440 66352 2000000 647 0 0.0000

> Total bytes allocated = 42335440

> Dynamic-space-size bytes = 3221225472

> COLD-INIT... (Length(TLFs)= 9736)

> Disassembler: 72 printers, 0 prefilters, 4 labelers

> CORRUPTION WARNING in SBCL pid 1774 tid 1774:

> Memory fault at 0xfffffffffffffffa (pc=0x1002199f70)

> The integrity of this image is possibly compromised.

> Exiting.

> Error opening /dev/tty: No such device or address

> Welcome to LDB, a low-level debugger for the Lisp runtime environment.

> ldb>

> real 0m6.120s

> user 0m5.958s

> sys 0m0.137s

> command "sh" "make.sh" "clisp" "--prefix=/gnu/store/j1ciw4dc8iskd5fdcw0s1ba08kkg7vx6-sbcl-2.1.0" "--dynamic-space-size=3072" "--with-sb-core-compression" "--with-sb-xref-for-internals" failed with status 1

> ------

>

> It appears that SBCL can support this platform. However, until we make

> it work, I plan to remove aarch64 from the "supported-systems" of sbcl,

> to avoid attempting these builds.

> I tried to bootstrap sbcl using ecl instead of clisp, using

> "guix build -s aarch64-linux sbcl" on a x86-64 machine because I don't

> have any arm64 hardware, but it failed with the same memory fault.

> On Wed, Jan 13, 2021 at 10:03:47PM +0100, Guillaume Le Vaillant wrote:

>> I tried to bootstrap sbcl using ecl instead of clisp, using

>> "guix build -s aarch64-linux sbcl" on a x86-64 machine because I don't

>> have any arm64 hardware, but it failed with the same memory fault.

>

> Thanks! On #guix, Efraim reported that the builds were succeeding on his

> aarch64 hardware. So, I'm not sure what's going on :/

> When taking a look at the logs of failing builds of sbcl-* packages on

> aarch64-linux (for example at [1] for master or at [2] for staging),

> I saw that the build jobs try to build sbcl (which is currently failing)

> for every package.

>

> I would have expected the builds for sbcl-* packages to be marked as

> "failed because missing dependency" given that the main dependency of

> the asdf-build-system/sbcl failed to build. Instead the build farm is

> trying to rebuild sbcl over and over, which wastes quite some time and

> resources.

>

> It looks like the dependencies of the build system are not considered as

> dependencies for the packages that use this build system.

> Am I missing something?

> That's a good observation. I hadn't thought of it.

>

> I'm CC-ing Mathieu Othacehe and guix-sysadmin so that we can disable

> these builds until we can fix the bug for real. Mathieu: this might

> explain why the build farm is spending all its effort on aarch64.

> Hello Leo & Guillaume,

>

>> That's a good observation. I hadn't thought of it.

>>

>> I'm CC-ing Mathieu Othacehe and guix-sysadmin so that we can disable

>> these builds until we can fix the bug for real. Mathieu: this might

>> explain why the build farm is spending all its effort on aarch64.

>

> If we want to disable SBCL builds temporarily we can do something

> similar to what I did to disable Rust builds on non-x86_64 architectures

> here: 0ed631866cc0b7cece2b0a0b50e39b37ae91bb67.

>

> Regarding the rebuilding that is a limitation of the new Cuirass remote

> building mechanism I am aware of and I need to solve. As build failures

> are only cached of the machine performing the build and the builds are

> now distributed to all the machines across the build farm, we need to

> find a way to centralize the builds failure cache.

>

> It would also be nice to optionally publish the build failures cache so

> that the user doesn't try to build a package that is known to be failing

> on the build farm. Chris, have you encountered this issue with the Build

> Coordinator?

> If we want to disable SBCL builds temporarily we can do something

> similar to what I did to disable Rust builds on non-x86_64 architectures

> here: 0ed631866cc0b7cece2b0a0b50e39b37ae91bb67.

> On Sun, Jan 17, 2021 at 10:29:49AM +0100, Mathieu Othacehe wrote:

> > If we want to disable SBCL builds temporarily we can do something

> > similar to what I did to disable Rust builds on non-x86_64 architectures

> > here: 0ed631866cc0b7cece2b0a0b50e39b37ae91bb67.

>

> ------

> diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm

> index 35a96b5754..91b5d6b6ec 100644

> --- a/gnu/packages/rust.scm

> +++ b/gnu/packages/rust.scm

> @@ -452,6 +452,7 @@ test = { path = \"../libtest\" }

> (variable "LIBRARY_PATH")

> (files '("lib" "lib64")))))

>

> + (supported-systems '("x86_64-linux"))

> (synopsis "Compiler for the Rust programming language")

> (description "Rust is a systems programming language that provides memory

> safety and thread safety guarantees.")

> ------

>

> Hm, this would also prevent users from building the packages on their

> own machines. We need a way to make changes like this but limit them to

> the build farm.

> The behaviour is configurable, but the default missing inputs hook will

> submit a new build for a missing input, but only if one doesn't already

> exist. Because of this, you don't get the behaviour where some missing

> prerequisite that fails to built is built over and over again, every

> time you try and build a derivation that uses it.

> On Sun, Jan 17, 2021 at 02:11:11PM -0500, Leo Famulari wrote:

>> On Sun, Jan 17, 2021 at 10:29:49AM +0100, Mathieu Othacehe wrote:

>> > If we want to disable SBCL builds temporarily we can do something

>> > similar to what I did to disable Rust builds on non-x86_64 architectures

>> > here: 0ed631866cc0b7cece2b0a0b50e39b37ae91bb67.

>>

>> ------

>> diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm

>> index 35a96b5754..91b5d6b6ec 100644

>> --- a/gnu/packages/rust.scm

>> +++ b/gnu/packages/rust.scm

>> @@ -452,6 +452,7 @@ test = { path = \"../libtest\" }

>> (variable "LIBRARY_PATH")

>> (files '("lib" "lib64")))))

>>

>> + (supported-systems '("x86_64-linux"))

>> (synopsis "Compiler for the Rust programming language")

>> (description "Rust is a systems programming language that provides memory

>> safety and thread safety guarantees.")

>> ------

>>

>> Hm, this would also prevent users from building the packages on their

>> own machines. We need a way to make changes like this but limit them to

>> the build farm.

>

> Would marking it unsubstitutable work? Or would we need build-local

> Now, I think we should avoid papering over CI configuration issues (did

> I get that right?) by changing package definitions.

> On Tue, Jan 19, 2021 at 02:13:58PM +0100, Ludovic Courtès wrote:

>> Now, I think we should avoid papering over CI configuration issues (did

>> I get that right?) by changing package definitions.

>

> Yes, that's idiomatic English, if that is what you were asking about.

>

> And I agree, the package definitions shouldn't include workarounds for

> CI problems.

> This time I wasn’t asking about English :-), but rather about whether

> the patch Efraim provided was indeed to paper over our CI problems.

> I'm testing building sbcl-cambl on it now. That is the package that

> spurred this bug report.