[PATCH] gnu: rust: make rust 1.25.0 build reproducible

  • Done
  • quality assurance status badge
Details
3 participants
  • Jakub K?dzio?ka
  • Ludovic Courtès
  • Nikolai Merinov
Owner
unassigned
Submitted by
Nikolai Merinov
Severity
normal
N
N
Nikolai Merinov wrote on 13 Jul 2018 22:15
(address . guix-patches@gnu.org)
87a7qlk6tx.fsf@member.fsf.org
* gnu/packages/rust.scm (rust-1.25): switch back to llvm@3.9 still builds with
llvm@6.0 is not reproducible. Remove all llvm@6.0 related twicks.
(rust-1.26): switch to llvm@6.0 still rust-1.26 can build reproducible
binaries with new llvm release. Enable llvm@5.0 related twicks.
---
gnu/packages/rust.scm | 39 ++++++++++++++++++---------------------
1 file changed, 18 insertions(+), 21 deletions(-)

Toggle diff (73 lines)
diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index 28bffb44b..1095070eb 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -545,10 +545,6 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
"0baxjr99311lvwdq0s38bipbnj72pn6fgbk6lcq7j555xq53mxpf")))
(package
(inherit base-rust)
- (inputs
- ;; Use LLVM 6.0
- (alist-replace "llvm" (list llvm)
- (package-inputs base-rust)))
(arguments
(substitute-keyword-arguments (package-arguments base-rust)
((#:phases phases)
@@ -559,23 +555,6 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
;; This test wants to update the crate index.
(("fn no_index_update") "#[ignore]\nfn no_index_update"))
#t))
- (add-after 'configure 'enable-codegen-tests
- (lambda _
- (substitute* "config.toml"
- (("codegen-tests = false") ""))
- #t))
- (replace 'patch-aarch64-test
- (lambda _
- (substitute* "src/librustc_metadata/dynamic_lib.rs"
- ;; This test is known to fail on aarch64 and powerpc64le:
- ;; https://github.com/rust-lang/rust/issues/45410
- (("fn test_loading_cosine") "#[ignore]\nfn test_loading_cosine"))
- ;; This test fails on aarch64 with llvm@6.0:
- ;; https://github.com/rust-lang/rust/issues/49807
- ;; other possible solution:
- ;; https://github.com/rust-lang/rust/pull/47688
- (delete-file "src/test/debuginfo/by-value-self-argument-in-trait-impl.rs")
- #t))
(delete 'ignore-glibc-2.27-incompatible-test))))))))
(define-public rust-1.26
@@ -585,6 +564,10 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
#:patches '("rust-coresimd-doctest.patch"))))
(package
(inherit base-rust)
+ (inputs
+ ;; Use LLVM 6.0
+ (alist-replace "llvm" (list llvm)
+ (package-inputs base-rust)))
(arguments
(substitute-keyword-arguments (package-arguments base-rust)
((#:phases phases)
@@ -624,6 +607,20 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
(substitute* "src/tools/cargo/tests/testsuite/generate_lockfile.rs"
;; This test wants to update the crate index.
(("fn no_index_update") "#[ignore]\nfn no_index_update"))
+ #t))
+ (add-after 'patch-aarch64-test 'patch-aarch64-llvm6-test
+ (lambda _
+ ;; This test fails on aarch64 with llvm@6.0:
+ ;; https://github.com/rust-lang/rust/issues/49807
+ ;; other possible solution:
+ ;; https://github.com/rust-lang/rust/pull/47688
+ (delete-file "src/test/debuginfo/by-value-self-argument-in-trait-impl.rs")
+ #t))
+ (add-after 'configure 'enable-codegen-tests
+ (lambda* _
+ (substitute* "config.toml"
+ ;; codegen test should pass with LLVM 6
+ (("codegen-tests = false") ""))
#t)))))))))
(define-public rust
--
2.17.1
N
N
Nikolai Merinov wrote on 20 Jul 2018 22:21
(address . 32227@debbugs.gnu.org)
876019k698.fsf_-_@member.fsf.org
Hi,

I tried to configure how rust use llvm, but I did not succeed in
it. Looks like llvm@6.0 usage was fixed fully only in rust 1.26.

I already checked once that 1.25 and 1.26 rust releases is reproducible
in my environment with llvm@3.9 and llvm@6.0 respectively. When I posted
this patch I restarted my builds with "--rounds=2".

Regards,
Nikolai
N
N
Nikolai Merinov wrote on 22 Jul 2018 11:55
(address . 32227@debbugs.gnu.org)
877eln61cv.fsf@member.fsf.org
Hi,

Please, do not merge. Change rejected. With this change 1.27.0 build
still not reproducible. I should try to figure out which llvm
compilation options make builds non-reproducible.

Regards,
Nikolai

Nikolai Merinov <nikolai.merinov@member.fsf.org> writes:

Toggle quote (11 lines)
> Hi,
>
> I tried to configure how rust use llvm, but I did not succeed in
> it. Looks like llvm@6.0 usage was fixed fully only in rust 1.26.
>
> I already checked once that 1.25 and 1.26 rust releases is reproducible
> in my environment with llvm@3.9 and llvm@6.0 respectively. When I posted
> this patch I restarted my builds with "--rounds=2".
>
> Regards,
> Nikolai
L
L
Ludovic Courtès wrote on 21 Aug 2018 23:19
Re: [bug#32227] [PATCH] gnu: rust: make rust 1.25.0 build reproducible
(name . Nikolai Merinov)(address . nikolai.merinov@member.fsf.org)(address . 32227@debbugs.gnu.org)
87ftz7mn89.fsf@gnu.org
Hello Nikolai,

Nikolai Merinov <nikolai.merinov@member.fsf.org> skribis:

Toggle quote (4 lines)
> Please, do not merge. Change rejected. With this change 1.27.0 build
> still not reproducible. I should try to figure out which llvm
> compilation options make builds non-reproducible.

Does the problem still applies to 1.27.0 as currently available in
master?

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 21 Aug 2018 23:19
(name . Nikolai Merinov)(address . nikolai.merinov@member.fsf.org)(address . 32227@debbugs.gnu.org)
87efermn81.fsf@gnu.org
Hello Nikolai,

Nikolai Merinov <nikolai.merinov@member.fsf.org> skribis:

Toggle quote (4 lines)
> Please, do not merge. Change rejected. With this change 1.27.0 build
> still not reproducible. I should try to figure out which llvm
> compilation options make builds non-reproducible.

Does the problem still apply to 1.27.0 as currently available in master?

Thanks,
Ludo’.
N
N
Nikolai Merinov wrote on 23 Sep 2018 23:38
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 32227@debbugs.gnu.org)
8736tzlws1.fsf@member.fsf.org
Hello Ludo,

Sorry for long answer. I have no chance to continue ivestigation last
month, but I'll want to provide my old finding:

1. Rust 1.25 build cargo binary (but not rustc itself) in
non-reproducible manner when compiled with llvm-6.0 and llvm-6.1. In
both cases we got non-reproducible code for next functions:
"<env_logger::Logger as log::Log>::log", "git2::panic::check",
"curl::easy::handler::ssl_ctx_cb".
2. Rust 1.26 with llvm-6.0 build is reproducible regardless of which
rustc 1.25 (with llvm-6.0 or llvm-3.9) was used to bootstrap it.
3. Rust 1.27.0 and Rust 1.27.2 again build rustc itself in reproducible
manner, while "cargo" binary differ build to build. I observed
non-reproducible code in "git2::panic::check" function and in
".gcc_except_table" section of resulting binary.

In attachements you can find patch to move rust-1.25 to llvm-3.9, and
patch to update Rust 1.27 to 1.27.2.
From: Nikolai Merinov <nikolai.merinov@member.fsf.org>
To: guix-patches@gnu.org
Subject: [PATCH] gnu: rust: make rust 1.25.0 build reproducible

* gnu/packages/rust.scm (rust-1.25): switch back to llvm@3.9 still builds with
llvm@6.0 is not reproducible. Remove all llvm@6.0 related twicks.
(rust-1.26): switch to llvm@6.0 still rust-1.26 can build reproducible
binaries with new llvm release. Enable llvm@5.0 related twicks.
---
gnu/packages/rust.scm | 39 ++++++++++++++++++---------------------
1 file changed, 18 insertions(+), 21 deletions(-)

Toggle diff (73 lines)
diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index 28bffb44b..1095070eb 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -545,10 +545,6 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
"0baxjr99311lvwdq0s38bipbnj72pn6fgbk6lcq7j555xq53mxpf")))
(package
(inherit base-rust)
- (inputs
- ;; Use LLVM 6.0
- (alist-replace "llvm" (list llvm)
- (package-inputs base-rust)))
(arguments
(substitute-keyword-arguments (package-arguments base-rust)
((#:phases phases)
@@ -559,23 +555,6 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
;; This test wants to update the crate index.
(("fn no_index_update") "#[ignore]\nfn no_index_update"))
#t))
- (add-after 'configure 'enable-codegen-tests
- (lambda _
- (substitute* "config.toml"
- (("codegen-tests = false") ""))
- #t))
- (replace 'patch-aarch64-test
- (lambda _
- (substitute* "src/librustc_metadata/dynamic_lib.rs"
- ;; This test is known to fail on aarch64 and powerpc64le:
- ;; https://github.com/rust-lang/rust/issues/45410
- (("fn test_loading_cosine") "#[ignore]\nfn test_loading_cosine"))
- ;; This test fails on aarch64 with llvm@6.0:
- ;; https://github.com/rust-lang/rust/issues/49807
- ;; other possible solution:
- ;; https://github.com/rust-lang/rust/pull/47688
- (delete-file "src/test/debuginfo/by-value-self-argument-in-trait-impl.rs")
- #t))
(delete 'ignore-glibc-2.27-incompatible-test))))))))
(define-public rust-1.26
@@ -585,6 +564,10 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
#:patches '("rust-coresimd-doctest.patch"))))
(package
(inherit base-rust)
+ (inputs
+ ;; Use LLVM 6.0
+ (alist-replace "llvm" (list llvm)
+ (package-inputs base-rust)))
(arguments
(substitute-keyword-arguments (package-arguments base-rust)
((#:phases phases)
@@ -624,6 +607,20 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
(substitute* "src/tools/cargo/tests/testsuite/generate_lockfile.rs"
;; This test wants to update the crate index.
(("fn no_index_update") "#[ignore]\nfn no_index_update"))
+ #t))
+ (add-after 'patch-aarch64-test 'patch-aarch64-llvm6-test
+ (lambda _
+ ;; This test fails on aarch64 with llvm@6.0:
+ ;; https://github.com/rust-lang/rust/issues/49807
+ ;; other possible solution:
+ ;; https://github.com/rust-lang/rust/pull/47688
+ (delete-file "src/test/debuginfo/by-value-self-argument-in-trait-impl.rs")
+ #t))
+ (add-after 'configure 'enable-codegen-tests
+ (lambda* _
+ (substitute* "config.toml"
+ ;; codegen test should pass with LLVM 6
+ (("codegen-tests = false") ""))
#t)))))))))
(define-public rust
--
2.17.1
Toggle diff (15 lines)
diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index 0695f8c7d..4bc8d1d88 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -631,8 +631,8 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
(define-public rust
(let ((base-rust
- (rust-bootstrapped-package rust-1.26 "1.27.0"
- "089d7rhw55zpvnw71dj8vil6qrylvl4xjr4m8bywjj83d4zq1f9c"
+ (rust-bootstrapped-package rust-1.26 "1.27.2"
+ "0pg1s37bhx9zqbynxyydq5j6q7kij9vxkcv8maz0m25prm88r0cs"
#:patches
'("rust-coresimd-doctest.patch"
"rust-bootstrap-stage0-test.patch"))))
Regards,
Nikolai

ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (12 lines)
> Hello Nikolai,
>
> Nikolai Merinov <nikolai.merinov@member.fsf.org> skribis:
>
>> Please, do not merge. Change rejected. With this change 1.27.0 build
>> still not reproducible. I should try to figure out which llvm
>> compilation options make builds non-reproducible.
>
> Does the problem still apply to 1.27.0 as currently available in master?
>
> Thanks,
> Ludo’.
J
J
Jakub K?dzio?ka wrote on 23 Feb 2020 15:27
closing
(address . 32227-done@debbugs.gnu.org)
20200223142717.h3mvval7iru4o4h4@gravity
These patches seem to have been made obsolete by commit
e027a494fd5e6025e645ce9f36cccca39a77d327 - closing.
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEE5Xa/ss9usT31cTO54xWnWEYTFWQFAl5Si8EACgkQ4xWnWEYT
FWQI7Q/+M007DbVDJ6FqFbPuOuUw5RDCFCayPunUSWfQWi2VLILfjdHjPktj2u7t
uR1ckzk2WTSchFjtZFhOnQHhO6h1EBtQMkIKpzyTYDxn6I4PbskPMY2SQniWCmUB
wg5mFXhczDRFa8k0s3/wNR7RGrEjOZHczUrbRweAV8D4FuK75GtNLWqHUc2qqLI3
N8WB8GeugjoKK4kSsOynqU/KRLA4f1A4gT15J41CXJv80nJQwLFHnJS7e+dUZ3ma
C21EQy887T+Hk1cUclXTvikZyyyn3O/eZzgEHKMA6PdCsvFyUyuxtGXmPEnxnr7m
BcVKZs4Ge2vu3/n7afC32llsuH25KmM+/S41IDw9Tgqi0Z7LY9Hdb6n0jMnMHjVd
GSVGzw+y8Mp+ywawqr5m+lEfpLji6aLp2o/ZUvtkcYbUullpB7q0Xgzw3vA0UmPD
4M94Jwufk9/9hRcaSmYiJ+9aa4+Fgu+MSIHHVfB/YVqDk7pN807h8BqK5HqMFzYz
/KSj9blASeyu/cy03HJDZybL3yHTJ8N/MjFwyQY1Uj6RqfvdNnKMqijbYUhpUGOU
FghNvopM+OIJnrGznltqb5QMn10rJAWupTXLk1soM52NVt9hOs+8R99BrNjlqSdF
480ocmpDyx8psSzjMV/yZTxSLryBtyCO8Y97qw197HZ2Ou7kJY8=
=dgmf
-----END PGP SIGNATURE-----


Closed
J
J
Jakub K?dzio?ka wrote on 23 Feb 2020 15:29
add wontfix tag
(address . control@debbugs.gnu.org)
20200223142934.hwsktvj6alf64cpo@gravity
tags 32227 wontfix
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEE5Xa/ss9usT31cTO54xWnWEYTFWQFAl5SjE4ACgkQ4xWnWEYT
FWRdXQ/+JZZzUY0LZ9pao9XlnMH0L5SlIhUusn9oUzcksgOpjhsxjzCAym7QPW75
hCbOPGEFyZHY8l8ou9iDA2lcM7xOfLjV7CYWZiify2W9nQSyiRFGFva1T12RhfnN
mootvUEb7mhdWz0DKoG4jkd6ljNZ3mHfeBMpZuOncjFDezXuI3DOwZJy2ykmWHcv
AcpIMKtlns0qVG7csL/zJ18C5M5UBs84Rbkmvla6qzkwqjVY2DndFcEWB+xsLcYT
s0zqRCj2HGgkyQTiBkM/ybd5MLbj1tu5oyBq5bBoXHDxkkbBne6HBFHR+xWYu+7E
iVk65cHapIucYNHjweJzkYVA8H1yHFkBdqqjhXejHDSaQFpQceYwNYCnGkEERJdr
HoKn7oxtPk/rN5H61HDpbGR7cDI3PJcsahdA1RL2A7Q4aBE+PWZ0HeEK171L8l+I
wrEhaXo0x4F6k7yOw8ndPIgJEdDmoK1doZ3Cdzlv4wc8TRwagLsDP0EMRCWSt7r8
HPEiyCzMCaVXUh15ZtffCqae5CWlo5e5H8SU5Zzz26udW8eHUdO+dA/ePDdlS3Hj
YdOcdiPonHZu9H1tx2bPyMfC7uZ99HkmxiLkdadIFDSChU8jJCY+Q5FrUHxTlqJa
JM6/wjiPk3JyzoZ1Z6/HiLt37CxD2/NWoSSaPQwJIHn3ICBIyYk=
=mlXs
-----END PGP SIGNATURE-----


?