[PATCH] staging gnu: Add more tools to rust outputs.

OpenSubmitted by John Soo.
Details
3 participants
  • John Soo
  • Jakub Kądziołka
  • Maxim Cournoyer
Owner
unassigned
Severity
normal
J
J
John Soo wrote on 28 Jan 2021 23:00
(address . guix-patches@gnu.org)
87eei4k9hn.fsf@asu.edu
Hi Guix!

I work with rust for my job and I would love to be able to use clippy
and the other tools that are in the rustc tree. This patch installs
those tools as other outputs in the same way we have cargo and rustfmt
now. Thanks!

All the best,

John
From 2f381b5af04a2e487242b3661edf088fecb6c484 Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Wed, 25 Nov 2020 06:25:43 -0800
Subject: [PATCH] gnu: Add more tools to rust outputs.

The goal is to provide standard rust tools as outputs of rustc. The tools we
were missing were rls, clippy, src, and rust-analyzer.

* gnu/packages/rust.scm (rust-1.46): [outputs] add rls, clippy, src, and
rust-analyzer, [arguments] alter phases to build, test, and install each new
output, [arguments] patch RUNPATHS of outputs that require it, [arguments]
delete all uninstall scripts from outputs, [arguments] delete all the install
logs and manifests.
---
gnu/packages/rust.scm | 133 +++++++++++++++++++++++++++++++-----------
1 file changed, 99 insertions(+), 34 deletions(-)

Toggle diff (184 lines)
diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index e646cd0091..98ae770745 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -11,6 +11,7 @@
 ;;; Copyright © 2020, 2021 Jakub Kądziołka <kuba@kadziolka.net>
 ;;; Copyright © 2020 Pierre Langlois <pierre.langlois@gmx.com>
 ;;; Copyright © 2020 Matthew Kraai <kraai@ftbfs.org>
+;;; Copyright © 2021 John Soo <jsoo1@asu.edu>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -41,6 +42,7 @@
   #:use-module (gnu packages jemalloc)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages llvm)
+  #:use-module (gnu packages node)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
   #:use-module (gnu packages ssh)
@@ -1322,19 +1324,42 @@ move around."
            "0a17jby2pd050s24cy4dfc0gzvgcl585v3vvyfilniyvjrqknsid")))
     (package
       (inherit base-rust)
-      (outputs (cons "rustfmt" (package-outputs base-rust)))
+      (outputs (append '("rustfmt" "rls" "src" "clippy" "rust-analyzer")
+                       (package-outputs base-rust)))
+      (inputs
+       `(("gcc-lib" ,gcc "lib")
+         ,@(package-inputs base-rust)))
+      (native-inputs
+       `(("node" ,node)
+         ("patchelf" ,patchelf)
+         ,@(package-native-inputs base-rust)))
       (arguments
        (substitute-keyword-arguments (package-arguments base-rust)
          ((#:phases phases)
           `(modify-phases ,phases
+             (replace 'patch-cargo-checksums
+               ;; Generate checksums after patching generated files (in
+               ;; particular, vendor/jemalloc/rep/Makefile).
+               (lambda* _
+                 (use-modules (guix build cargo-utils))
+                 (substitute* '("Cargo.lock"
+                                "src/tools/rust-analyzer/Cargo.lock")
+                   (("(checksum = )\".*\"" all name)
+                    (string-append name "\"" ,%cargo-reference-hash "\"")))
+                 (generate-all-checksums "vendor")
+                 #t))
              (replace 'build
                (lambda* _
                  (invoke "./x.py" "build")
                  (invoke "./x.py" "build" "src/tools/cargo")
-                 (invoke "./x.py" "build" "src/tools/rustfmt")))
+                 (invoke "./x.py" "build" "src/tools/rustfmt")
+                 (invoke "./x.py" "build" "src/tools/clippy")
+                 (invoke "./x.py" "build" "src/tools/rls")
+                 (invoke "./x.py" "build"
+                         "src/tools/rust-analyzer/crates/rust-analyzer")))
              (replace 'check
                (lambda* _
-                 ;; Test rustfmt.
+                 ;; Enable parallel execution.
                  (let ((parallel-job-spec
                         (string-append "-j" (number->string
                                              (min 4
@@ -1343,44 +1368,84 @@ move around."
                    (invoke "./x.py" parallel-job-spec "test"
                            "src/tools/cargo")
                    (invoke "./x.py" parallel-job-spec "test"
-                           "src/tools/rustfmt"))))
+                           "src/tools/rustfmt")
+                   ;; Clippy tests do not work. See
+                   ;; https://github.com/rust-lang/rust/issues/78717
+                   ;; Even with --stage 1, they fail to compile
+                   ;; (invoke "./x.py" parallel-job-spec "test" "--stage" "1"
+                   ;;         "src/tools/clippy")
+                   (substitute* "src/tools/rls/tests/client.rs"
+                     (("fn client_dependency_typo_and_fix" all)
+                      (string-append "#[ignore]\n" all)))
+                   (invoke "./x.py" parallel-job-spec "test"
+                           "src/tools/rls"))))
              (replace 'install
                (lambda* (#:key outputs #:allow-other-keys)
                  (invoke "./x.py" "install")
+                 (for-each delete-file-recursively
+                           (find-files (assoc-ref outputs "out")
+                                       "^uninstall\\.sh$"))
                  (substitute* "config.toml"
                    ;; replace prefix to specific output
-                   (("prefix = \"[^\"]*\"")
-                    (string-append "prefix = \"" (assoc-ref outputs "cargo") "\"")))
-                 (invoke "./x.py" "install" "cargo")
-                 (substitute* "config.toml"
-                   ;; replace prefix to specific output
-                   (("prefix = \"[^\"]*\"")
-                    (string-append "prefix = \"" (assoc-ref outputs "rustfmt") "\"")))
-                 (invoke "./x.py" "install" "rustfmt")))
+                   (("\\[build\\]" all)
+                    (string-append all "
+extended = true
+tools =
+")))
+                 (define (install-component component)
+                   (substitute* "config.toml"
+                     ;; replace prefix to specific output
+                     (("(tools =).*" all tools)
+                      (string-append tools " [\"" component "\"]\n"))
+                     (("prefix = \"[^\"]*\"")
+                      (string-append
+                       "prefix = \"" (assoc-ref outputs component) "\"")))
+                   (mkdir-p (assoc-ref outputs component))
+                   (invoke "./x.py" "install" component)
+                   (for-each delete-file-recursively
+                             (find-files (assoc-ref outputs component)
+                                         "uninstall\\.sh")))
+                 (for-each install-component
+                           '("cargo"
+                             "rustfmt"
+                             "clippy"
+                             "rls"
+                             "src"
+                             "rust-analyzer"))
+                 #t))
+             (add-after 'install 'patch-tools-runpaths
+               (lambda* (#:key outputs inputs #:allow-other-keys)
+                 (use-modules (ice-9 popen)
+                              (ice-9 textual-ports))
+                 (define (patch-path path)
+                   (let* ((read-rpath
+                           (string-append
+                            "patchelf --print-rpath " path))
+                          (pipe (open-input-pipe read-rpath))
+                          (current-rpath (get-string-all pipe))
+                          (out (assoc-ref outputs "out"))
+                          (libc (assoc-ref inputs "libc"))
+                          (gcc-lib (assoc-ref inputs "gcc-lib")))
+                     (close-pipe pipe)
+                     (invoke "patchelf" "--set-rpath"
+                             (string-append current-rpath
+                                            ":" out "/lib"
+                                            ":" libc "/lib"
+                                            ":" gcc-lib "/lib")
+                             path)))
+                 (define (patch-component component)
+                   (for-each patch-path
+                             (find-files (assoc-ref outputs component)
+                                         (lambda (p s) (executable-file? p)))))
+                 (for-each patch-component '("clippy" "rls"))))
              (replace 'delete-install-logs
                (lambda* (#:key outputs #:allow-other-keys)
-                 (define (delete-manifest-file out-path file)
-                   (delete-file (string-append out-path "/lib/rustlib/" file)))
-
-                 (let ((out (assoc-ref outputs "out"))
-                       (cargo-out (assoc-ref outputs "cargo"))
-                       (rustfmt-out (assoc-ref outputs "rustfmt")))
-                   (for-each
-                     (lambda (file) (delete-manifest-file out file))
-                     '("install.log"
-                       "manifest-rust-docs"
-                       ,(string-append "manifest-rust-std-"
-                                       (nix-system->gnu-triplet-for-rust))
-                       "manifest-rustc"))
-                   (for-each
-                     (lambda (file) (delete-manifest-file cargo-out file))
-                     '("install.log"
-                       "manifest-cargo"))
-                   (for-each
-                     (lambda (file) (delete-manifest-file rustfmt-out file))
-                     '("install.log"
-                       "manifest-rustfmt-preview"))
-                   #t))))))))))
+                 (define log-manifest-re
+                   "^install\\.log$|^manifest-([a-z]|[0-9]|_|-)+(-preview)?$")
+                 (define (delete-install-log output)
+                   (for-each delete-file-recursively
+                             (find-files output log-manifest-re)))
+                 (for-each delete-install-log (map cdr outputs)))))))))))
 
 (define-public rust-1.47
   (let ((base-rust
-- 
2.30.0
-----BEGIN PGP SIGNATURE-----

iQJCBAEBCAAsFiEEWhWPr0BqdIqBqdxOT0N6drRIojsFAmATM/QOHGpzb28xQGFz
dS5lZHUACgkQT0N6drRIojtLBBAA0tIdWUdry9+VKn67NlIMjvxOAvN8krw9VSzd
z5+APOhmtscN956V0lkBKgc1ME1nA+iE643lnmKfK+DWnr621afd52JJAMrmG4xv
Ds7XOGGfANxHDRNqxqrEPZUEN1vL7eUC0z1hsl11hCOkjL6Ka56/DUIerPViOMLW
AZEPBhx6eDsKPLckRFPGWZUbB/O6ejLtpmou38IeYGAMlgJVop7Uk4JGzo6h9ZHD
c2bkoEofi8DgtBoYuZ4RmCSGFw1JrqIFnZtXzrqqSdjaFA/GBV5ozGRtfXLlfxZN
qB4ZEvgJRnvGOWELKJVuyk9oeD4BqpYYva3Xqf3zlp3jCNHLa17ZC1EKlyt/WaOm
r7EorYpJdIuc1kpAwVzz3ns/LatdmnMZX3aWyifAU6jJWDH8xl1TPAiKTo8hRil/
RhEtqfUBegV/qIn/hdidKWXEnWlEtEgU3gRl1KTrZ5+f/hy2eSj3T7RFPkdQmH3i
qhyF8HFGmlejjX2IXN5/UB0bVBWVnR3PF7cjAE+D3yJdJ/PwGVetPlydsL6nFISL
CVhlDsjFiNiv6Ry1evYEI9NYhl+jLTRt4eygSk9iP7zFFq0bk3aib6F2VX83gdXM
fVceT5S+SiE32w7OYhn8A0+4+yywKciasJZTV1zBP86YYGb993Y0bvctKujiD33p
rEOqDQA=
=V2V8
-----END PGP SIGNATURE-----

J
J
John Soo wrote on 29 Jan 2021 00:29
(address . 46162@debbugs.gnu.org)
875z3gk5cd.fsf@asu.edu
Here is an edited patch. I had rebased incorrectly, my apologies.

- John
From cd3bddf4785f9efac275424c5c735092600f45c5 Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Wed, 25 Nov 2020 06:25:43 -0800
Subject: [PATCH] gnu: Add more tools to rust outputs.

The goal is to provide standard rust tools as outputs of rustc. The tools we
were missing were rls, clippy, src, and rust-analyzer.

* gnu/packages/rust.scm (rust-1.46): [outputs] add rls, clippy, src, and
rust-analyzer, [arguments] alter phases to build, test, and install each new
output, [arguments] patch RUNPATHS of outputs that require it, [arguments]
delete all uninstall scripts from outputs, [arguments] delete all the install
logs and manifests.
---
gnu/packages/rust.scm | 131 +++++++++++++++++++++++++++++++-----------
1 file changed, 97 insertions(+), 34 deletions(-)

Toggle diff (175 lines)
diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index e646cd0091..b59fe98546 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -11,6 +11,7 @@
 ;;; Copyright © 2020, 2021 Jakub Kądziołka <kuba@kadziolka.net>
 ;;; Copyright © 2020 Pierre Langlois <pierre.langlois@gmx.com>
 ;;; Copyright © 2020 Matthew Kraai <kraai@ftbfs.org>
+;;; Copyright © 2021 John Soo <jsoo1@asu.edu>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -1322,19 +1323,41 @@ move around."
            "0a17jby2pd050s24cy4dfc0gzvgcl585v3vvyfilniyvjrqknsid")))
     (package
       (inherit base-rust)
-      (outputs (cons "rustfmt" (package-outputs base-rust)))
+      (outputs (append '("rustfmt" "rls" "src" "clippy" "rust-analyzer")
+                       (package-outputs base-rust)))
+      (inputs
+       `(("gcc-lib" ,gcc "lib")
+         ,@(package-inputs base-rust)))
+      (native-inputs
+       `(("patchelf" ,patchelf)
+         ,@(package-native-inputs base-rust)))
       (arguments
        (substitute-keyword-arguments (package-arguments base-rust)
          ((#:phases phases)
           `(modify-phases ,phases
+             (replace 'patch-cargo-checksums
+               ;; Generate checksums after patching generated files (in
+               ;; particular, vendor/jemalloc/rep/Makefile).
+               (lambda* _
+                 (use-modules (guix build cargo-utils))
+                 (substitute* '("Cargo.lock"
+                                "src/tools/rust-analyzer/Cargo.lock")
+                   (("(checksum = )\".*\"" all name)
+                    (string-append name "\"" ,%cargo-reference-hash "\"")))
+                 (generate-all-checksums "vendor")
+                 #t))
              (replace 'build
                (lambda* _
                  (invoke "./x.py" "build")
                  (invoke "./x.py" "build" "src/tools/cargo")
-                 (invoke "./x.py" "build" "src/tools/rustfmt")))
+                 (invoke "./x.py" "build" "src/tools/rustfmt")
+                 (invoke "./x.py" "build" "src/tools/clippy")
+                 (invoke "./x.py" "build" "src/tools/rls")
+                 (invoke "./x.py" "build"
+                         "src/tools/rust-analyzer/crates/rust-analyzer")))
              (replace 'check
                (lambda* _
-                 ;; Test rustfmt.
+                 ;; Enable parallel execution.
                  (let ((parallel-job-spec
                         (string-append "-j" (number->string
                                              (min 4
@@ -1343,44 +1366,84 @@ move around."
                    (invoke "./x.py" parallel-job-spec "test"
                            "src/tools/cargo")
                    (invoke "./x.py" parallel-job-spec "test"
-                           "src/tools/rustfmt"))))
+                           "src/tools/rustfmt")
+                   ;; Clippy tests do not work. See
+                   ;; https://github.com/rust-lang/rust/issues/78717
+                   ;; Even with --stage 1, they fail to compile
+                   ;; (invoke "./x.py" parallel-job-spec "test" "--stage" "1"
+                   ;;         "src/tools/clippy")
+                   (substitute* "src/tools/rls/tests/client.rs"
+                     (("fn client_dependency_typo_and_fix" all)
+                      (string-append "#[ignore]\n" all)))
+                   (invoke "./x.py" parallel-job-spec "test"
+                           "src/tools/rls"))))
              (replace 'install
                (lambda* (#:key outputs #:allow-other-keys)
                  (invoke "./x.py" "install")
+                 (for-each delete-file-recursively
+                           (find-files (assoc-ref outputs "out")
+                                       "^uninstall\\.sh$"))
                  (substitute* "config.toml"
                    ;; replace prefix to specific output
-                   (("prefix = \"[^\"]*\"")
-                    (string-append "prefix = \"" (assoc-ref outputs "cargo") "\"")))
-                 (invoke "./x.py" "install" "cargo")
-                 (substitute* "config.toml"
-                   ;; replace prefix to specific output
-                   (("prefix = \"[^\"]*\"")
-                    (string-append "prefix = \"" (assoc-ref outputs "rustfmt") "\"")))
-                 (invoke "./x.py" "install" "rustfmt")))
+                   (("\\[build\\]" all)
+                    (string-append all "
+extended = true
+tools =
+")))
+                 (define (install-component component)
+                   (substitute* "config.toml"
+                     ;; replace prefix to specific output
+                     (("(tools =).*" all tools)
+                      (string-append tools " [\"" component "\"]\n"))
+                     (("prefix = \"[^\"]*\"")
+                      (string-append
+                       "prefix = \"" (assoc-ref outputs component) "\"")))
+                   (mkdir-p (assoc-ref outputs component))
+                   (invoke "./x.py" "install" component)
+                   (for-each delete-file-recursively
+                             (find-files (assoc-ref outputs component)
+                                         "uninstall\\.sh")))
+                 (for-each install-component
+                           '("cargo"
+                             "rustfmt"
+                             "clippy"
+                             "rls"
+                             "src"
+                             "rust-analyzer"))
+                 #t))
+             (add-after 'install 'patch-tools-runpaths
+               (lambda* (#:key outputs inputs #:allow-other-keys)
+                 (use-modules (ice-9 popen)
+                              (ice-9 textual-ports))
+                 (define (patch-path path)
+                   (let* ((read-rpath
+                           (string-append
+                            "patchelf --print-rpath " path))
+                          (pipe (open-input-pipe read-rpath))
+                          (current-rpath (get-string-all pipe))
+                          (out (assoc-ref outputs "out"))
+                          (libc (assoc-ref inputs "libc"))
+                          (gcc-lib (assoc-ref inputs "gcc-lib")))
+                     (close-pipe pipe)
+                     (invoke "patchelf" "--set-rpath"
+                             (string-append current-rpath
+                                            ":" out "/lib"
+                                            ":" libc "/lib"
+                                            ":" gcc-lib "/lib")
+                             path)))
+                 (define (patch-component component)
+                   (for-each patch-path
+                             (find-files (assoc-ref outputs component)
+                                         (lambda (p s) (executable-file? p)))))
+                 (for-each patch-component '("clippy" "rls"))))
              (replace 'delete-install-logs
                (lambda* (#:key outputs #:allow-other-keys)
-                 (define (delete-manifest-file out-path file)
-                   (delete-file (string-append out-path "/lib/rustlib/" file)))
-
-                 (let ((out (assoc-ref outputs "out"))
-                       (cargo-out (assoc-ref outputs "cargo"))
-                       (rustfmt-out (assoc-ref outputs "rustfmt")))
-                   (for-each
-                     (lambda (file) (delete-manifest-file out file))
-                     '("install.log"
-                       "manifest-rust-docs"
-                       ,(string-append "manifest-rust-std-"
-                                       (nix-system->gnu-triplet-for-rust))
-                       "manifest-rustc"))
-                   (for-each
-                     (lambda (file) (delete-manifest-file cargo-out file))
-                     '("install.log"
-                       "manifest-cargo"))
-                   (for-each
-                     (lambda (file) (delete-manifest-file rustfmt-out file))
-                     '("install.log"
-                       "manifest-rustfmt-preview"))
-                   #t))))))))))
+                 (define log-manifest-re
+                   "^install\\.log$|^manifest-([a-z]|[0-9]|_|-)+(-preview)?$")
+                 (define (delete-install-log output)
+                   (for-each delete-file-recursively
+                             (find-files output log-manifest-re)))
+                 (for-each delete-install-log (map cdr outputs)))))))))))
 
 (define-public rust-1.47
   (let ((base-rust
-- 
2.30.0
-----BEGIN PGP SIGNATURE-----

iQJCBAEBCAAsFiEEWhWPr0BqdIqBqdxOT0N6drRIojsFAmATSPIOHGpzb28xQGFz
dS5lZHUACgkQT0N6drRIojthdw//b0ZQ+OGlEdBbInkATFXl0e+AMw77cttcJO2y
iYE95ZiK2H7HprM36x9cdTTZfXyGnOp1rqe3G/U2245U7TvaSymqlPWsSGEIZ2Ut
gVSrFFOulF9DwsUuBPvCxcPHFvHLZYTQtmcp8fUkHvVYswOhwkZ9ir11bkfffpXw
zjzEWeZDVhN8lv+iqvHSkX8qrOjn8YNX3vvI71urbtFuLc5K3EQH01ZIlso6v290
rLc7hGqJW+Y6dh4FmEorVDvCbr/EWo4AJkTic4odcew6PTb7XcFZi+VqQdyW6IxS
d8eGghlOzi6zi+X2t4n84nZWCgdjjaG+qsj60oaMC3qTUAH87CRy/EObCHSckRcv
7CZWjyKrXH7ZTQ3p0me+HMc7lLUGWS9DiMZxYGXHColWNYDtt73hlyVE4fSkRlXk
GZsDPDtaabim9hs9sBHwQhxNYWvK3Stx722bhKC8i7KKfXOaVvhNdcpJM0aBE+fD
Aj4qMuN5O4PejOaWt0pR1hzrucd4THnzNTN1dzQUc1hVa1P1HRCJlNGddg/NlT7p
I2SYc6aDbE+GBTo3mRibYMBdV2zhqJNw1GRBInnKdk/bFoTu/F/1WSTk9QMP1ehK
wFOr8MLcG+3yiq2zl6svXVpXVjLyM0NmVvRmjiHINB4E8d3tvcTVkILk+vtAa+Z/
XtLziOY=
=CwBZ
-----END PGP SIGNATURE-----

J
J
John Soo wrote on 15 Feb 2021 17:13
(address . 46162@debbugs.gnu.org)
87ft1xwbqh.fsf@asu.edu
Rebased on staging.

There is one more question that I have about search paths. A lot of rust
tools use the $RUST_SRC_PATH to find the source of rust. I add the
source output as "src" but I am not sure how to specify a search path to
the top directory of a separate output. Any pointers?

Thanks!

John
From cd849893e7f9b6ad8c380b3ebc74ea92530f8c48 Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Wed, 25 Nov 2020 06:25:43 -0800
Subject: [PATCH] gnu: Add more tools to rust outputs.

The goal is to provide standard rust tools as outputs of rustc. The tools we
were missing were rls, clippy, src, and rust-analyzer. as a separate output of
rust.

* gnu/packages/rust.scm (rust-1.46): [outputs] add rls, clippy, src, and
rust-analyzer, [arguments] alter phases to build, test, and install each new
output, [arguments] patch RUNPATHS of outputs that require it, [arguments]
delete all uninstall scripts from outputs, [arguments] delete all the install
logs and manifests.
---
gnu/packages/rust.scm | 145 +++++++++++++++++++++++++++++++-----------
1 file changed, 109 insertions(+), 36 deletions(-)

Toggle diff (210 lines)
diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index f72f310169..8a09e1077e 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -11,6 +11,7 @@
 ;;; Copyright © 2020, 2021 Jakub Kądziołka <kuba@kadziolka.net>
 ;;; Copyright © 2020 Pierre Langlois <pierre.langlois@gmx.com>
 ;;; Copyright © 2020 Matthew Kraai <kraai@ftbfs.org>
+;;; Copyright © 2021 John Soo <jsoo1@asu.edu>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -41,6 +42,7 @@
   #:use-module (gnu packages jemalloc)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages llvm)
+  #:use-module (gnu packages node)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
   #:use-module (gnu packages ssh)
@@ -1322,19 +1324,41 @@ move around."
            "0a17jby2pd050s24cy4dfc0gzvgcl585v3vvyfilniyvjrqknsid")))
     (package
       (inherit base-rust)
-      (outputs (cons "rustfmt" (package-outputs base-rust)))
+      (outputs (append '("rustfmt" "rls" "src" "clippy" "rust-analyzer")
+                       (package-outputs base-rust)))
+      (inputs
+       `(("gcc-lib" ,gcc "lib")
+         ,@(package-inputs base-rust)))
+      (native-inputs
+       `(("node" ,node)
+         ("patchelf" ,patchelf)
+         ,@(package-native-inputs base-rust)))
       (arguments
        (substitute-keyword-arguments (package-arguments base-rust)
          ((#:phases phases)
           `(modify-phases ,phases
+             (replace 'patch-cargo-checksums
+               ;; Generate checksums after patching generated files (in
+               ;; particular, vendor/jemalloc/rep/Makefile).
+               (lambda* _
+                 (use-modules (guix build cargo-utils))
+                 (substitute* '("Cargo.lock"
+                                "src/tools/rust-analyzer/Cargo.lock")
+                   (("(checksum = )\".*\"" all name)
+                    (string-append name "\"" ,%cargo-reference-hash "\"")))
+                 (generate-all-checksums "vendor")
+                 #t))
              (replace 'build
                (lambda* _
                  (invoke "./x.py" "build")
                  (invoke "./x.py" "build" "src/tools/cargo")
-                 (invoke "./x.py" "build" "src/tools/rustfmt")))
+                 (invoke "./x.py" "build" "src/tools/rustfmt")
+                 (invoke "./x.py" "build" "src/tools/clippy")
+                 (invoke "./x.py" "build" "src/tools/rls")
+                 (invoke "./x.py" "build"
+                         "src/tools/rust-analyzer/crates/rust-analyzer")))
              (replace 'check
                (lambda* _
-                 ;; Test rustfmt.
                  (let ((parallel-job-spec
                         (string-append "-j" (number->string
                                              (min 4
@@ -1343,44 +1367,84 @@ move around."
                    (invoke "./x.py" parallel-job-spec "test"
                            "src/tools/cargo")
                    (invoke "./x.py" parallel-job-spec "test"
-                           "src/tools/rustfmt"))))
+                           "src/tools/rustfmt")
+                   ;; Clippy tests do not work. See
+                   ;; https://github.com/rust-lang/rust/issues/78717
+                   ;; Even with --stage 1, they fail to compile
+                   ;; (invoke "./x.py" parallel-job-spec "test" "--stage" "1"
+                   ;;         "src/tools/clippy")
+                   (substitute* "src/tools/rls/tests/client.rs"
+                     (("fn client_dependency_typo_and_fix" all)
+                      (string-append "#[ignore]\n" all)))
+                   (invoke "./x.py" parallel-job-spec "test"
+                           "src/tools/rls"))))
              (replace 'install
                (lambda* (#:key outputs #:allow-other-keys)
                  (invoke "./x.py" "install")
+                 (for-each delete-file-recursively
+                           (find-files (assoc-ref outputs "out")
+                                       "^uninstall\\.sh$"))
                  (substitute* "config.toml"
                    ;; replace prefix to specific output
-                   (("prefix = \"[^\"]*\"")
-                    (string-append "prefix = \"" (assoc-ref outputs "cargo") "\"")))
-                 (invoke "./x.py" "install" "cargo")
-                 (substitute* "config.toml"
-                   ;; replace prefix to specific output
-                   (("prefix = \"[^\"]*\"")
-                    (string-append "prefix = \"" (assoc-ref outputs "rustfmt") "\"")))
-                 (invoke "./x.py" "install" "rustfmt")))
+                   (("\\[build\\]" all)
+                    (string-append all "
+extended = true
+tools =
+")))
+                 (define (install-component component)
+                   (substitute* "config.toml"
+                     ;; replace prefix to specific output
+                     (("(tools =).*" all tools)
+                      (string-append tools " [\"" component "\"]\n"))
+                     (("prefix = \"[^\"]*\"")
+                      (string-append
+                       "prefix = \"" (assoc-ref outputs component) "\"")))
+                   (mkdir-p (assoc-ref outputs component))
+                   (invoke "./x.py" "install" component)
+                   (for-each delete-file-recursively
+                             (find-files (assoc-ref outputs component)
+                                         "uninstall\\.sh")))
+                 (for-each install-component
+                           '("cargo"
+                             "rustfmt"
+                             "clippy"
+                             "rls"
+                             "src"
+                             "rust-analyzer"))
+                 #t))
+             (add-after 'install 'patch-tools-runpaths
+               (lambda* (#:key outputs inputs #:allow-other-keys)
+                 (use-modules (ice-9 popen)
+                              (ice-9 textual-ports))
+                 (define (patch-path path)
+                   (let* ((read-rpath
+                           (string-append
+                            "patchelf --print-rpath " path))
+                          (pipe (open-input-pipe read-rpath))
+                          (current-rpath (get-string-all pipe))
+                          (out (assoc-ref outputs "out"))
+                          (libc (assoc-ref inputs "libc"))
+                          (gcc-lib (assoc-ref inputs "gcc-lib")))
+                     (close-pipe pipe)
+                     (invoke "patchelf" "--set-rpath"
+                             (string-append current-rpath
+                                            ":" out "/lib"
+                                            ":" libc "/lib"
+                                            ":" gcc-lib "/lib")
+                             path)))
+                 (define (patch-component component)
+                   (for-each patch-path
+                             (find-files (assoc-ref outputs component)
+                                         (lambda (p s) (executable-file? p)))))
+                 (for-each patch-component '("clippy" "rls"))))
              (replace 'delete-install-logs
                (lambda* (#:key outputs #:allow-other-keys)
-                 (define (delete-manifest-file out-path file)
-                   (delete-file (string-append out-path "/lib/rustlib/" file)))
-
-                 (let ((out (assoc-ref outputs "out"))
-                       (cargo-out (assoc-ref outputs "cargo"))
-                       (rustfmt-out (assoc-ref outputs "rustfmt")))
-                   (for-each
-                     (lambda (file) (delete-manifest-file out file))
-                     '("install.log"
-                       "manifest-rust-docs"
-                       ,(string-append "manifest-rust-std-"
-                                       (nix-system->gnu-triplet-for-rust))
-                       "manifest-rustc"))
-                   (for-each
-                     (lambda (file) (delete-manifest-file cargo-out file))
-                     '("install.log"
-                       "manifest-cargo"))
-                   (for-each
-                     (lambda (file) (delete-manifest-file rustfmt-out file))
-                     '("install.log"
-                       "manifest-rustfmt-preview"))
-                   #t))))))))))
+                 (define log-manifest-re
+                   "^install\\.log$|^manifest-([a-z]|[0-9]|_|-)+(-preview)?$")
+                 (define (delete-install-log output)
+                   (for-each delete-file-recursively
+                             (find-files output log-manifest-re)))
+                 (for-each delete-install-log (map cdr outputs)))))))))))
 
 (define-public rust-1.47
   (let ((base-rust
@@ -1413,7 +1477,8 @@ move around."
                ;; particular, vendor/jemalloc/rep/Makefile).
                (lambda* _
                  (use-modules (guix build cargo-utils))
-                 (substitute* "Cargo.lock"
+                 (substitute* '("Cargo.lock"
+                                "src/tools/rust-analyzer/Cargo.lock")
                    (("(checksum = )\".*\"" all name)
                     (string-append name "\"" ,%cargo-reference-hash "\"")))
                  (generate-all-checksums "vendor")
@@ -1445,7 +1510,15 @@ move around."
                      (("fn test_process_mask") "#[allow(unused_attributes)]
     #[ignore]
     fn test_process_mask"))
-                   #t))))))))))
+                   #t)))
+             ;; FIXME: This fixes the "src" output which is not critical.  We
+             ;; should probably copy the source of the de-vendored libunwind
+             ;; for completeness' sake.
+             (add-before 'install 'remove-vendored-llvm-reference-in-src
+               (lambda _
+                 (substitute* "src/bootstrap/dist.rs"
+                   ((", \"src/llvm-project/libunwind\"") ""))
+                 #t)))))))))
 
 (define-public rust-1.49
   (rust-bootstrapped-package rust-1.48 "1.49.0"
-- 
2.30.0
J
J
John Soo wrote on 15 Feb 2021 19:09
(address . 46162@debbugs.gnu.org)(address . kuba@kadziolka.net)
877dn9w6cq.fsf_-_@asu.edu
Hi Jakub,

Thanks again for your work on rust.

I cc'd you for feedback on adding the "extended" tools to rust's
outputs.

- John
J
J
Jakub Kądziołka wrote on 16 Feb 2021 00:41
C9AI2TKPWQ4B.2SAN12EWNA247@gravity
On Mon Feb 15, 2021 at 7:09 PM CET, John Soo wrote:
Toggle quote (7 lines)
> Hi Jakub,
>
> Thanks again for your work on rust.
>
> I cc'd you for feedback on adding the "extended" tools to rust's
> outputs.

I don't think tools beyond rustc and cargo should be included in the
main rust package, as this causes them to be built in each step of the
bootstrap. I believe a better approach would be to define separate
packages for them.

We would have something like

;; TODO(staging): Bump this variable to the latest packaged rust.
(define-public rust rust-1.45)

+(define-public rust-for-tools rust-1.50)

I'm not sure if rustbuild can be convinced to not build the compiler
itself when the version used for the build is the same as the sources'.
If so, defining packages for each tool shouldn't need any guix-side
tricks.

Otherwise, I would define a single rust-tools package with
(outputs '("rustfmt" "clippy" ...)). Perhaps it would help with UX if
rust-tools itself was hidden, and instead the tools would be exposed
with simple packages that expose each tool separately, with a symlink or
similar.

I'll see if I can find some time to try this out this week.

Regards,
Jakub Kądziołka
J
J
John Soo wrote on 16 Feb 2021 17:40
(name . Jakub Kądziołka)(address . kuba@kadziolka.net)(address . 46162@debbugs.gnu.org)
87wnv80xvv.fsf@asu.edu
Hi Jakub,

Jakub Kądziołka <kuba@kadziolka.net> writes:

Toggle quote (5 lines)
> I don't think tools beyond rustc and cargo should be included in the
> main rust package, as this causes them to be built in each step of the
> bootstrap. I believe a better approach would be to define separate
> packages for them.

Yeah that would make sense. Have we explored any of the incremental or
--keep-stage options to speedup the bootstrap?

Toggle quote (12 lines)
> We would have something like
>
> ;; TODO(staging): Bump this variable to the latest packaged rust.
> (define-public rust rust-1.45)
>
> +(define-public rust-for-tools rust-1.50)
>
> I'm not sure if rustbuild can be convinced to not build the compiler
> itself when the version used for the build is the same as the sources'.
> If so, defining packages for each tool shouldn't need any guix-side
> tricks.

Even if it did build the compiler for each tool it may not be a problem
if only the last 3 versions had the tools available (for instance).

Toggle quote (6 lines)
> Otherwise, I would define a single rust-tools package with
> (outputs '("rustfmt" "clippy" ...)). Perhaps it would help with UX if
> rust-tools itself was hidden, and instead the tools would be exposed
> with simple packages that expose each tool separately, with a symlink or
> similar.

I could see this working nicely. I think this is just more evidence for
language-environment related documentation.

Toggle quote (2 lines)
> I'll see if I can find some time to try this out this week.

Thanks! That would be helpful. It is really painful to have all these
unmerged patches to rust.

Kindly,

John
J
J
Jakub Kądziołka wrote on 16 Feb 2021 17:41
(name . John Soo)(address . jsoo1@asu.edu)
C9B3S080FBJ3.1CPQDB4088F2L@gravity
On Tue Feb 16, 2021 at 5:40 PM CET, John Soo wrote:
Toggle quote (12 lines)
> Hi Jakub,
>
> Jakub Kądziołka <kuba@kadziolka.net> writes:
>
> > I don't think tools beyond rustc and cargo should be included in the
> > main rust package, as this causes them to be built in each step of the
> > bootstrap. I believe a better approach would be to define separate
> > packages for them.
>
> Yeah that would make sense. Have we explored any of the incremental or
> --keep-stage options to speedup the bootstrap?

apteryx mentioned some experiments with those on #mrustc.

Regards,
Jakub Kądziołka
M
M
Maxim Cournoyer wrote on 18 Feb 2021 19:01
(name . Jakub Kądziołka)(address . kuba@kadziolka.net)
87wnv5uufu.fsf@gmail.com
Hi,

Jakub Kądziołka <kuba@kadziolka.net> writes:

Toggle quote (18 lines)
> On Tue Feb 16, 2021 at 5:40 PM CET, John Soo wrote:
>> Hi Jakub,
>>
>> Jakub Kądziołka <kuba@kadziolka.net> writes:
>>
>> > I don't think tools beyond rustc and cargo should be included in the
>> > main rust package, as this causes them to be built in each step of the
>> > bootstrap. I believe a better approach would be to define separate
>> > packages for them.
>>
>> Yeah that would make sense. Have we explored any of the incremental or
>> --keep-stage options to speedup the bootstrap?
>
> apteryx mentioned some experiments with those on #mrustc.
>
> Regards,
> Jakub Kądziołka

Those weren't fruitful, but I didn't push it; I think it caused cargo to
fail compiling itself, or its test suite.

Maxim
?