[PATCH] gnu: rust: Update rust from 1.22.1 release to 1.24.1

(%rust-bootstrap-binaries): Update hashsums.

(cargo-bootstrap): Update minor version.

(rust): Rename "rustc" to "rust". Update sha256 for tarball. Add cargo related

dependencies. Install "cargo" as separate "rust" output. Remove substitutes

for bugs that was fixed in "rustc" source code. Install documentation to

separate "doc" output. Update configuration according to changes in "rustc"

source code.

"default-rustc" functions with "default-rust" function.

(cargo-build-system): Take only one package as "rust" source. Use "rustc" from

default output of "rust" package, use "cargo" as "cargo" output of "rust"

package.

Cargo was moved from separate package to "rust" output because starting from

0.25.0 release cargo have no "Cargo.lock" anymore. According to

commit message "cargo" used as "rust" dependency and stable version for

"cargo" provided through "Cargo.lock" file from "rust-lang/rust"

repository. As result only way to get stable version of "cargo" is build it from

"rust" sources.

---

gnu/packages/rust.scm | 1146 ++-----------------------------------------

guix/build-system/cargo.scm | 21 +-

2 files changed, 37 insertions(+), 1130 deletions(-)

Hi Nikolai,

Great piece of work!

Nikolai Merinov <nikolai.merinov@member.fsf.org> skribis:

OK.

Instead of incrementing the version number of the bootstrap binaries, do

you think it would be an option to keep 1.22 and build 1.23 from it, or

does that cargo/rust merge prevent it?

That would allow us to build Rust independently from what upstream did,

thereby increasing binary diversity, which I think is a good thing

security-wise.

Thoughts?

Ludo’.

Hi,

Rust after this patch is not reproducible.

Binary files /gnu/store/nbyxci7zgv9z5zrkzw9c5wppgv3bk1rd-rust-1.24.1/lib/librustc-7b5ea53516e6ab59.so and /gnu/store/nbyxci7zgv9z5zrkzw9c5wppgv3bk1rd-rust-1.24.1-check/lib/librustc-7b5ea53516e6ab59.so differ

Binary files /gnu/store/nbyxci7zgv9z5zrkzw9c5wppgv3bk1rd-rust-1.24.1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-7b5ea53516e6ab59.so and /gnu/store/nbyxci7zgv9z5zrkzw9c5wppgv3bk1rd-rust-1.24.1-check/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc-7b5ea53516e6ab59.so differ

diffoscope says:

? ? ? - [26] .rustc PROGBITS 0000000000000000 be3030 6e5dd5 00 0 0 16

? ? ? - [27] .symtab SYMTAB 0000000000000000 12c8e08 158340 18 28 54351 8

? ? ? - [28] .strtab STRTAB 0000000000000000 1421148 3b00e3 00 0 0 1

? ? ? - [29] .shstrtab STRTAB 0000000000000000 17d122b 0000fe 00 0 0 1

? ? ? + [26] .rustc PROGBITS 0000000000000000 be3030 6e6b05 00 0 0 16

? ? ? + [27] .symtab SYMTAB 0000000000000000 12c9b38 158340 18 28 54351 8

? ? ? + [28] .strtab STRTAB 0000000000000000 1421e78 3b00e3 00 0 0 1

? ? ? + [29] .shstrtab STRTAB 0000000000000000 17d1f5b 0000fe 00 0 0 1

? ? ? Key to Flags:

[...]

? ? ? - 1603: 0000000000000000 0x6e5dd5 OBJECT GLOBAL DEFAULT 26 rust_metadata_rustc_d87c35d223a98218627136b41e7e5b87

? ? ? + 1603: 0000000000000000 0x6e6b05 OBJECT GLOBAL DEFAULT 26 rust_metadata_rustc_d87c35d223a98218627136b41e7e5b87

And a metadata body part is missing in the second round.

Hi Ludovic,

In rust 1.22 there already was possibility to build "cargo" from "rust"

repository, so suggested packages structures change can be used either

for old "rust" releases.

By default rust should be built with

https://github.com/rust-lang/rust/blob/1.24.1/src/stage0.txtversions of

rust and cargo. In most of cases this for "1.X.Y" release it want to use

"1.(X-1).0" version of rustc and "0.X.0" version of cargo. As result to

build release version of Rust compiler we should have previous release

of rust compiler.

Actually there is two ways to achieve this:

1. Constantly update bootsrap binaries version.

2. For each new release create new package. As result we'll be able to

use old rust release to build each new rust release. E.g. we can use

1.21.0 bootstrap binaries, then build 1.22.0 rust and use it to build

1.23.0 rust and then use it to build 1.24.1 rust.

Which way is preferable?

Please do not merge my patch until I'll check Danny's report about

reproducability.

Regards,

Nikola

Hello,

Nikolai Merinov <nikolai.merinov@member.fsf.org> skribis:

Like I wrote, I would prefer option #2, so as to increase “binary

diversity” and not rely on builds made by upstream.

This obviously relates to http://bootstrappable.org/. Ricardo, what’s

your take on this?

Sure!

Thanks,

Ludo’.

Ludovic Courtès <ludo@gnu.org> writes:

I agree. In the long run, however, I’d prefer for Rust to be

bootstrapped through one of the alternative implementations. Then we

don’t need to keep a long chain of older versions.

Currently, however, I don’t see a way around it.

--

Ricardo

GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC

Hi Danny,

I checked several times with "guix build --no-grafts --rounds=2 rust@1.24.1"

with "guix gc" between tries, but it worked correctly. I tried on x86-64

architecture with my patch over 79f450a120068ac1d2fc4e5b4b33ae938b604f95

guix commit.

Do you know how I can get same result as in your build?

Regards,

Nikolai

Danny Milosavljevic <dannym@scratchpost.org> writes:

I tested the patch on my aarch64 box and with the following patch I was

able to start the building process. Currently rust fails its test stage

with the error:

test result: ok. 193 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out

finished in 267.991

Testing libstd stage1 (aarch64-unknown-linux-gnu -> aarch64-unknown-linux-gnu)

Compiling libc v0.2.34

Compiling core v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-0/rustc-1.24.1-src/src/libcore)

Compiling std_unicode v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-0/rustc-1.24.1-src/src/libstd_unicode)

Compiling rand v0.3.19

Compiling std v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-0/rustc-1.24.1-src/src/libstd)

Compiling alloc v0.0.0 (file:///tmp/guix-build-rust-1.24.1.drv-0/rustc-1.24.1-src/src/liballoc)

error: unused attribute

--> src/libstd/sys/unix/process/process_common.rs:449:5

|

449 | #[ignore]

| ^^^^^^^^^

|

note: lint level defined here

--> src/libstd/lib.rs:232:9

|

232 | #![deny(warnings)]

| ^^^^^^^^

= note: #[deny(unused_attributes)] implied by #[deny(warnings)]

error: aborting due to previous error

error: Could not compile `std`.

warning: build failed, waiting for other jobs to finish...

error: build failed

command did not execute successfully: "/gnu/store/3vvhw5xpbcym5l3xk2pszzzyx4m7k8l6-cargo-bootstrap-0.24.0/bin/cargo" "test" "--target" "aarch64-unknown-linux-gnu" "-j" "6" "--release" "--frozen" "--features" "panic-unwind jemalloc backtrace" "--manifest-p

ath" "/tmp/guix-build-rust-1.24.1.drv-0/rustc-1.24.1-src/src/libstd/Cargo.toml" "-p" "std:0.0.0" "-p" "libc:0.0.0" "-p" "alloc_system:0.0.0" "-p" "panic_abort:0.0.0" "-p" "compiler_builtins:0.0.0" "-p" "alloc:0.0.0" "-p" "std_unicode:0.0.0" "-p" "unwind:0

.0.0" "-p" "core:0.0.0" "--"

expected success, got: exit code: 101

failed to run: /tmp/guix-build-rust-1.24.1.drv-0/rustc-1.24.1-src/build/bootstrap/debug/bootstrap test

Build completed unsuccessfully in 1:53:33

phase `check' failed after 6813.3 seconds

This is ultimately no different from master, where rust currently fails

somewhere during the test stage.

--

Efraim Flashner <efraim@flashner.co.il> ????? ?????

GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351

Confidentiality cannot be guaranteed on emails sent or received unencrypted

From d3f6c14535163be65aedfc761b747c4386387537 Mon Sep 17 00:00:00 2001

---

gnu/packages/rust.scm | 5 +++++

1 file changed, 5 insertions(+)

Ricardo Wurmus <rekado@elephly.net> writes:

Hi, I prepared proof-of-concept solution with rust-bootstrap frozed on

1.22.1 release. Do you think suggested code with "split all code to

small steps and remove fixed issues in new releases" is correct way to

provide series of releases? Or it will be better to copy builder code to

state it directly that this specific modification is tested on each rust

release?

Regards,

Nikolai

Hi Nikolai,

On Thu, 22 Mar 2018 00:02:46 +0500

Nikolai Merinov <nikolai.merinov@member.fsf.org> wrote:

Weird, works fine for me with 79f450a120068ac1d2fc4e5b4b33ae938b604f95.

But with the current master, 2cac8891510533338f6214fbd5ab9c2f6204e856, it fails

as describes (I tried again, fails).

Hello Nikolai,

Nikolai Merinov <nikolai.merinov@member.fsf.org> skribis:

Would it be an option to stick to 1.21? Or is it already too difficult?

(Apologies if this was already answered previously.) I’m asking because

I wonder how big the temptation will be to upgrade ‘rust-bootstrap’

again next time.

I’m not sure what you mean.

Danny, you probably have more experience than I do with Rust. :-)

Thoughts?

Thanks,

Ludo’.

Hi Ludovic,

In attachement you can find updated patch with rust-bootstrap 1.22.1 and

rust packages of 1.23.0 and 1.24.1 releases. I made small changes like

replacement (zero? (system* ...)) with (invoke ...) and checked

reproducability.

Please find my comments below.

Regards,

Nikolai

ludo@gnu.org (Ludovic Courtès) writes:

It should be easy enough to stick with 1.21 bootstrap (still we already

have package for 1.22), but:

1. We still should change code for bootstrap package to match design

where "rustc" and "cargo" installed from one package.

2. Rust 1.23 and 1.24 can use same "config.toml", while rust 1.22 have

different "config.toml". I do not know how to make it to look pretty in

first time. For additional information look below about package code

design.

3. My machine is pretty slow and testing reproducablity for three

versions in a row will take two days :-)

I talking about how to write code for packages. e.g. if we have:

(package

(name "rust")

(version "1.23.0")

...

(arguments

...

#:phases (modify-phases %standard-phases

...

(add-after ... 'disable-test-1)

(add-after ... 'disable-test-2)

(add-after ... 'fix-source-bug-1)

(add-after ... 'fix-source-bug-2)

(add-after ... 'fix-source-bug-3)

(replace 'configure ...)

(replace 'check ...)

...)))

Then how should look package for 1.24 release?

It can be inherited from previous one and replace some of old steps

(let ((base-rust rust-1.23))

(package

(inherit base-rust)

(version "1.24.1")

...

(native-inputs

(alist-replace "cargo-bootstrap" (list base-rust "cargo")

(alist-replace "rustc-bootstrap" (list base-rust)

(package-native-inputs base-rust))))

(arguments (substitute-keyword-arguments (package-arguments base-rust)

((#:phases phases)

`(modify-phases ,phases

(delete 'fix-source-bug-1)

(add-after ... 'disable-test-3)

(replace 'fix-source-bug-2

(lambda* ....))))))))

Or we should copy all steps required for each new release:

(package

(name "rust")

(version "1.24.0")

...

(arguments

...

#:phases (modify-phases %standard-phases

...

(add-after ... 'disable-test-1)

(add-after ... 'disable-test-2)

(add-after ... 'disable-test-3)

(add-after ... 'fix-source-bug-1)

(add-after ... 'fix-source-bug-3)

(replace 'configure ...)

(replace 'check ...)

...)))

Second solution looks too verbose for me, but with first solution on

long chain of versions it will be very hard to manage which changes we

have in newest package.

Hi Danny,

I checked last version of my patch over

d591989b42088a358f33d6cf68f12efc2ad4403c (master as on March 24) and

looks that build is reproducable. It will be helpful if you'll be able

to check this patch also:

Regards,

Nikolai

Danny Milosavljevic <dannym@scratchpost.org> writes:

Note: I hope that mrust (A Rust compiler written in C++) is released soon

so we won't have these bootstrap binaries any longer.

Hi Nikolai,

I think the incremental (first) version is the best one since you can see

which bugs are worked around by us and which are fixed per release at a glance.

But really it doesn't matter much which.

I hope we won't have a long chain of cumulative Rust versions in Guix. I know

that Rust upstream likes to do this chain of Rust1 -> Rust2 -> Rust3 -> Rust4

but that's not really scalable - especially since even one Rust takes a day

to compile. We should try to get mrust [1] to work and use it to compile just

the newest Rust. If it doesn't work we can still fall back to one of the

other ways later.

(What Mozilla recommends is we compile Rust1, use Rust2 to compile Rust3, use

Rust3 to compile Rust4, likewise for each new release)

[1] https://github.com/thepowersgang/mrustc

Danny Milosavljevic <dannym@scratchpost.org> writes:

I think we should heed upstreams advice in that case. A long bootstrap

chain is really only a problem during 'core-updates', no?

IIUC mrustc only targets x86_64 currently, so I don't expect it to

become viable for Guix in a good while.

Marius Bakke <mbakke@fastmail.com> writes:

It’s a problem for people building from source and it makes for a

brittle bootstrap because any of the packages could break. Having a

long bootstrap chain to debug on core-updates is no fun — I’m saying

this as someone who has just fixed problems in the relatively simple

bootstrap chain for Java.

This is a problem, but I do hope that eventually we will be able to use

mrust to bypass an ever-growing chain.

--

Ricardo

Hi Nikolai,

On Tue, 27 Mar 2018 18:02:58 +0500

Nikolai Merinov <nikolai.merinov@member.fsf.org> wrote:

I did now - it works for me and is reproducible.

I've thus fixed up the commit message a little and pushed this patch to master.

Thanks!

Hi Marius,

On Tue, 27 Mar 2018 20:08:01 +0200

Marius Bakke <mbakke@fastmail.com> wrote:

I've checked the source code and it seems there's support for ARM in the works.

There's an old issue report at https://github.com/thepowersgang/mrustc/issues/39.

I've commented there - let's see.

close 30831