[rust-team] rust-bootstrap fails parallel build on x86-64

Open

Details

2 participants

Greg Hogan
Efraim Flashner

Owner: unassigned

Submitted by: Greg Hogan

Severity: normal

Greg Hogan wrote on 31 Oct 20:13 +0100

Recipients:(address . bug-guix@gnu.org)(name . Efraim Flashner)(address . efraim@flashner.co.il)

Message-ID:CA+3U0ZkyT4R5qbouEZqRPDg5s_hnoKPrwGdh+qU5BtsF4W8mew@mail.gmail.com

derivation: 83qbfg0xxsfr5lsc81m5a770c7c7b55k-rust-1.54.0.drv

commit: 6e50b0c56a8cc767bd3acb26638f78c450bde718

Build log attached, tail portion below. Has consistently failed to

build unless cores=1.

The package definition has "#:parallel-build? ,(target-x86-64?)" but

the noted build issue ("race conditions between various dependent

crates") is also seen on x86-64.

It is only the 1.54 build with mrustc that fails in this manner, the

following rust packages in the chain build fine with parallelism

enabled.

Greg

Toggle snippet (57 lines)(36/175) BUILDING autocfg v1.0.1
> /tmp/guix-build-rust-1.54.0.drv-0/mrustc/bin/mrustc rustc-1.54.0-src/vendor/autocfg/src/lib.rs -o output/rustc-build/libautocfg-1_0_1.rlib --crate-name autocfg --crate-type rlib -C emit-depfile=output/rustc-build/libautocfg-1_0_1.rlib.d --crate-tag 1_0_1 --cfg debug_assertions -O -L output -L output/rustc-build
> /tmp/guix-build-rust-1.54.0.drv-0/mrustc/output/rustc-build/build_bitflags-1_2_1_run
> /tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/bitflags/output/rustc-build/build_memchr-2_4_0_H6_run
Unable to run process
'/tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/bitflags/output/rustc-build/build_memchr-2_4_0_H6_run'
- No such file or directory
Calling /tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/bitflags/output/rustc-build/build_memchr-2_4_0_H6_run
failed (see /tmp/guix-build-rust-1.54.0.drv-0/mrustc/output/rustc-build/build_memchr-2_4_0_H6.txt_failed.txt
for stdout)
(37/175) BUILDING bitflags v1.2.1
> /tmp/guix-build-rust-1.54.0.drv-0/mrustc/bin/mrustc rustc-1.54.0-src/vendor/bitflags/src/lib.rs -o output/rustc-build/libbitflags-1_2_1.rlib --crate-name bitflags --crate-type rlib -C emit-depfile=output/rustc-build/libbitflags-1_2_1.rlib.d --crate-tag 1_2_1 --cfg debug_assertions -O -L output -L output/rustc-build --cfg bitflags_const_fn
Unable to run process
'/tmp/guix-build-rust-1.54.0.drv-0/mrustc/bin/mrustc' - No such file
or directory
> /tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/memchr/output/rustc-build/build_ryu-1_0_5_run
Unable to run process
'/tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/memchr/output/rustc-build/build_ryu-1_0_5_run'
- No such file or directory
Calling /tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/memchr/output/rustc-build/build_ryu-1_0_5_run
failed (see /tmp/guix-build-rust-1.54.0.drv-0/mrustc/output/rustc-build/build_ryu-1_0_5.txt_failed.txt
for stdout)
> /tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/memchr/output/rustc-build/build_snap-1_0_5_run
Unable to run process
'/tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/memchr/output/rustc-build/build_snap-1_0_5_run'
- No such file or directory
Calling /tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/memchr/output/rustc-build/build_snap-1_0_5_run
failed (see /tmp/guix-build-rust-1.54.0.drv-0/mrustc/output/rustc-build/build_snap-1_0_5.txt_failed.txt
for stdout)
> /tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/memchr/output/rustc-build/build_maybe-uninit-2_0_0_run
Unable to run process
'/tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/memchr/output/rustc-build/build_maybe-uninit-2_0_0_run'
- No such file or directory
Calling /tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/memchr/output/rustc-build/build_maybe-uninit-2_0_0_run
failed (see /tmp/guix-build-rust-1.54.0.drv-0/mrustc/output/rustc-build/build_maybe-uninit-2_0_0.txt_failed.txt
for stdout)
> /tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/memchr/output/rustc-build/build_typenum-1_12_0_run
Unable to run process
'/tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/memchr/output/rustc-build/build_typenum-1_12_0_run'
- No such file or directory
Calling /tmp/guix-build-rust-1.54.0.drv-0/rustc-1.54.0-src/vendor/memchr/output/rustc-build/build_typenum-1_12_0_run
failed (see /tmp/guix-build-rust-1.54.0.drv-0/mrustc/output/rustc-build/build_typenum-1_12_0.txt_failed.txt
for stdout)
BUILD FAILED
make: *** [minicargo.mk:228: output/rustc] Error 1
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("-f"
"minicargo.mk" "output/rustc" "RUSTC_TARGET=x86_64-unknown-linux-gnu"
"RUSTC_VERSION=1.54.0" "MRUSTC_TARGET_VER=1.54" "CFLAGS=-std=gnu11"
"OUTDIR_SUF=") exit-status: 2 term-signal: #f stop-signal: #f>
phase `build' failed after 617.7 seconds
command "make" "-f" "minicargo.mk" "output/rustc"
"RUSTC_TARGET=x86_64-unknown-linux-gnu" "RUSTC_VERSION=1.54.0"
"MRUSTC_TARGET_VER=1.54" "CFLAGS=-std=gnu11" "OUTDIR_SUF=" failed with
status 2
build process 18 exited with status 256

Attachment: qbfg0xxsfr5lsc81m5a770c7c7b55k-rust-1.54.0.drv.gz

Greg Hogan wrote on 31 Oct 21:26 +0100

[PATCH rust-team] gnu: rust-bootstrap: Disable parallel builds.

Recipients:(address . 74142@debbugs.gnu.org)(name . Greg Hogan)(address . code@greghogan.com)

Message-ID:16b103aec59e189d5bbff2aec69ff63f11459eeb.1730404732.git.code@greghogan.com

* gnu/packages/rust.scm (rust-bootstrap): Disable parallel builds for

all architectures.

Change-Id: Icafbbacd30200fad457b6a70730acd466524485b

---

gnu/packages/rust.scm | 6 +++---

1 file changed, 3 insertions(+), 3 deletions(-)

Toggle diff (21 lines)diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index cb58a4b4b3..4f74fc7571 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -208,9 +208,9 @@ (define-public rust-bootstrap
        ;; that doesn't seem to cause issues.
        #:validate-runpath? #f
        ;; Most of the build is single-threaded. This also improves the
-       ;; build time on machines with "only" 8GB of RAM.
-       ;; ppc64le regularly sees race conditions between various dependant crates.
-       #:parallel-build? ,(target-x86-64?)
+       ;; build time on machines with "only" 8GB of RAM and prevents
+       ;; regularly seen race conditions between various dependant crates.
+       #:parallel-build? #f
        #:make-flags
        (list ,(string-append "RUSTC_TARGET="
                              (platform-rust-target

base-commit: 6e50b0c56a8cc767bd3acb26638f78c450bde718
-- 
2.46.0

Efraim Flashner wrote on 3 Nov 08:49 +0100

Re: [rust-team] rust-bootstrap fails parallel build on x86-64

Recipients:(name . Greg Hogan)(address . code@greghogan.com)(address . bug-guix@gnu.org)

Message-ID:ZycrGulTfQfbaciu@3900XT

On Thu, Oct 31, 2024 at 03:13:40PM -0400, Greg Hogan wrote:

Toggle quote (16 lines)> derivation: 83qbfg0xxsfr5lsc81m5a770c7c7b55k-rust-1.54.0.drv
> commit: 6e50b0c56a8cc767bd3acb26638f78c450bde718
> 
> Build log attached, tail portion below. Has consistently failed to
> build unless cores=1.
> 
> The package definition has "#:parallel-build? ,(target-x86-64?)" but
> the noted build issue ("race conditions between various dependent
> crates") is also seen on x86-64.
> 
> It is only the 1.54 build with mrustc that fails in this manner, the
> following rust packages in the chain build fine with parallelism
> enabled.
> 
> Greg

I see the note there says something about ppc64le having a race
condition too. My machine has "only" 24 cores, so I didn't run into
anything, but I see you're building with 32 cores. IIRC the ppc64le
machine was building with at least 32 cores also.

Can you try building rust-1.54 with only 24 cores and see if that works
for you? If it does then I think we can switch the (job-count) variable
to (min 24 (parallel-job-count))

-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Greg Hogan wrote on 4 Nov 00:06 +0100

Recipients:

Message-ID:CA+3U0ZnTgcfUAo16sbWFZEgSkUTuQKn+MV0B35AQe-KAB474uw@mail.gmail.com

On Sun, Nov 3, 2024 at 2:49?AM Efraim Flashner <efraim@flashner.co.il> wrote:

Toggle quote (27 lines)>
> On Thu, Oct 31, 2024 at 03:13:40PM -0400, Greg Hogan wrote:
> > derivation: 83qbfg0xxsfr5lsc81m5a770c7c7b55k-rust-1.54.0.drv
> > commit: 6e50b0c56a8cc767bd3acb26638f78c450bde718
> >
> > Build log attached, tail portion below. Has consistently failed to
> > build unless cores=1.
> >
> > The package definition has "#:parallel-build? ,(target-x86-64?)" but
> > the noted build issue ("race conditions between various dependent
> > crates") is also seen on x86-64.
> >
> > It is only the 1.54 build with mrustc that fails in this manner, the
> > following rust packages in the chain build fine with parallelism
> > enabled.
> >
> > Greg
>
> I see the note there says something about ppc64le having a race
> condition too. My machine has "only" 24 cores, so I didn't run into
> anything, but I see you're building with 32 cores. IIRC the ppc64le
> machine was building with at least 32 cores also.
>
> Can you try building rust-1.54 with only 24 cores and see if that works
> for you? If it does then I think we can switch the (job-count) variable
> to (min 24 (parallel-job-count))

Log attached from what looks to be a similar failure on an 8 core instance.

Attachment: qbfg0xxsfr5lsc81m5a770c7c7b55k-rust-1.54.0.drv.gz

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 74142@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 74142

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date