[PATCH 0/4] build-system/cargo: Propagations across the crate closure.

OpenSubmitted by Jakub Kądziołka.
Details
3 participants
  • Efraim Flashner
  • Jakub Kądziołka
  • Leo Famulari
Owner
unassigned
Severity
normal
J
J
Jakub Kądziołka wrote on 25 Jun 2020 23:25
(address . guix-patches@gnu.org)(address . efraim@flashner.co.il)
20200625212523.25016-1-kuba@kadziolka.net
Due to the unusual (for Guix) compilation model used by
cargo-build-system, any phases or inputs added by a given library crate
need to be duplicated in all its dependents. This patchstack attempts to
solve this by allowing to propagate inputs, native-inputs and phases
across cargo-inputs edges in the dependency graph.

Apart from the build system work itself, I have included samples of the
cleanup they allow. Apart from being a good example, these are the
changes I have used to test the feature.

Jakub Kądziołka (4):
build-system/cargo: Allow propagating inputs across CARGO-INPUTS edges
gnu: crates-io: Use propagated-inputs and propagated-native-inputs.
build-system/cargo: Add a propagated-phases argument.
gnu: crates-io: Use propagated-phases.

gnu/packages/crates-io.scm | 157 +++++++++++-------------------------
gnu/packages/rust-apps.scm | 9 +--
gnu/packages/sequoia.scm | 3 +-
guix/build-system/cargo.scm | 92 +++++++++++++++------
4 files changed, 120 insertions(+), 141 deletions(-)

--
2.26.2
J
J
Jakub Kądziołka wrote on 25 Jun 2020 23:26
[PATCH 1/4] build-system/cargo: Allow propagating inputs across CARGO-INPUTS edges
(address . 42049@debbugs.gnu.org)
20200625212627.25622-1-kuba@kadziolka.net
* guix/build-system/cargo.scm (package-propagated-native-inputs):
New procedure.
(expand-crate-sources): Take crate closure instead of input lists.
(lower): Expand host-inputs and build-inputs to include inputs
from the crate closure.
---
guix/build-system/cargo.scm | 59 +++++++++++++++++++++++--------------
1 file changed, 37 insertions(+), 22 deletions(-)

Toggle diff (101 lines)
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 6c8edf6bac..3f518343ec 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -143,6 +143,12 @@ to NAME and VERSION."
       cargo-development-inputs)
     (package-arguments p)))
 
+(define (package-propagated-native-inputs p)
+  (apply
+    (lambda* (#:key (propagated-native-inputs '()) #:allow-other-keys)
+      propagated-native-inputs)
+    (package-arguments p)))
+
 (define (crate-closure inputs)
   "Return the closure of INPUTS when considering the 'cargo-inputs' and
 'cargod-dev-deps' edges.  Omit duplicate inputs, except for those
@@ -182,9 +188,8 @@ do not extract the conventional inputs)."
       ((input rest ...)
        (loop rest (cons input result) propagated first? seen)))))
 
-(define (expand-crate-sources cargo-inputs cargo-development-inputs)
-  "Extract all transitive sources for CARGO-INPUTS and CARGO-DEVELOPMENT-INPUTS
-along their 'cargo-inputs' edges.
+(define (expand-crate-sources crate-closure)
+  "Extract all sources for the transitive cargo inputs provided in CRATE-CLOSURE.
 
 Cargo requires all transitive crate dependencies' sources to be available
 in its index, even if they are optional (this is so it can generate
@@ -236,40 +241,50 @@ any dependent crates. This can be a benefits:
        (list label (package-source p)))
       ((label input)
        (list label input)))
-    (crate-closure (append cargo-inputs cargo-development-inputs))))
+    crate-closure))
 
 (define* (lower name
                 #:key source inputs native-inputs outputs system target
                 (rust (default-rust))
                 (cargo-inputs '())
                 (cargo-development-inputs '())
+                (propagated-native-inputs '())
                 #:allow-other-keys
                 #:rest arguments)
   "Return a bag for NAME."
 
   (define private-keywords
     '(#:source #:target #:rust #:inputs #:native-inputs #:outputs
-      #:cargo-inputs #:cargo-development-inputs))
+      #:cargo-inputs #:cargo-development-inputs #:propagated-native-inputs))
 
   (and (not target) ;; TODO: support cross-compilation
-       (bag
-         (name name)
-         (system system)
-         (target target)
-         (host-inputs `(,@(if source
-                              `(("source" ,source))
-                              '())
-                        ,@inputs
+       (let ((closure (crate-closure
+                        (append cargo-inputs cargo-development-inputs))))
+         (bag
+           (name name)
+           (system system)
+           (target target)
+           (host-inputs `(,@(if source
+                                `(("source" ,source))
+                                '())
+                          ,@inputs
+                          ,@(append-map
+                              (compose package-propagated-inputs second)
+                              closure)
 
-                        ;; Keep the standard inputs of 'gnu-build-system'
-                        ,@(standard-packages)))
-         (build-inputs `(("cargo" ,rust "cargo")
-                         ("rustc" ,rust)
-                         ,@(expand-crate-sources cargo-inputs cargo-development-inputs)
-                         ,@native-inputs))
-         (outputs outputs)
-         (build cargo-build)
-         (arguments (strip-keyword-arguments private-keywords arguments)))))
+                          ;; Keep the standard inputs of 'gnu-build-system'
+                          ,@(standard-packages)))
+           (build-inputs `(("cargo" ,rust "cargo")
+                           ("rustc" ,rust)
+                           ,@(expand-crate-sources closure)
+                           ,@(append-map
+                               (compose package-propagated-native-inputs second)
+                               closure)
+                           ,@propagated-native-inputs
+                           ,@native-inputs))
+           (outputs outputs)
+           (build cargo-build)
+           (arguments (strip-keyword-arguments private-keywords arguments))))))
 
 (define cargo-build-system
   (build-system
-- 
2.26.2
J
J
Jakub Kądziołka wrote on 25 Jun 2020 23:26
[PATCH 2/4] gnu: crates-io: Use propagated-inputs and propagated-native-inputs.
(address . 42049@debbugs.gnu.org)
20200625212632.25840-1-kuba@kadziolka.net
* gnu/packages/crates-io.scm
(rust-curl-sys-0.4, rust-freetype-rs-0.23, rust-freetype-sys-0.9,
rust-git2-0.11, rust-grep-pcre2-0.1, rust-libgit2-sys-0.10,
rust-libz-sys-1.0, rust-libssh2-sys-0.2, rust-pcre2-0.2,
rust-pcre2-sys-0.2, rust-servo-fontconfig-0.4,
rust-servo-fontconfig-sys-4): Don't include transitive dependencies in inputs.
(rust-capnp-rpc-0.10, rust-expat-sys-2.1, rust-libz-sys-1.0,
rust-pcre2-sys-0.2, rust-pkg-config-0.3, rust-servo-fontconfig-sys-4):
Make inputs propagated.
(rust-flate2-1.0, rust-grep-0.2): Don't skip build.
* gnu/packages/rust-apps.scm (exa, ripgrep): Don't include some
transitive dependencies in inputs.
* gnu/packages/sequoia.scm (sequoia): Likewise.
---
gnu/packages/crates-io.scm | 83 ++++++++++++--------------------------
gnu/packages/rust-apps.scm | 9 +----
gnu/packages/sequoia.scm | 3 +-
3 files changed, 29 insertions(+), 66 deletions(-)

Toggle diff (265 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index bc29df5871..5a4b513f1a 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -2482,15 +2482,15 @@ messages.")
        (sha256
         (base32 "1j6xg7yays1hlm1045wviyn1642yvvi2p4kba26yk07a0kafr3jn"))))
     (build-system cargo-build-system)
-    (native-inputs
-     `(("capnproto" ,capnproto)))
     (arguments
      `(#:cargo-inputs
        (("rust-capnp" ,rust-capnp-0.10)
         ("rust-capnp-futures" ,rust-capnp-futures-0.10)
         ("rust-futures" ,rust-futures-0.1))
        #:cargo-development-inputs
-       (("rust-capnpc" ,rust-capnpc-0.10))))
+       (("rust-capnpc" ,rust-capnpc-0.10))
+       #:propagated-native-inputs
+       (("capnproto" ,capnproto))))
     (home-page "https://github.com/capnproto/capnproto-rust")
     (synopsis "Cap'n Proto remote procedure call protocol implementation")
     (description "This package provides an implementation of the Cap'n Proto
@@ -4659,13 +4659,10 @@ Transparency logs for use with sct crate.")
             (let ((openssl (assoc-ref inputs "openssl")))
               (setenv "OPENSSL_DIR" openssl))
             #t)))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
     (inputs
      `(("curl" ,curl)
        ("nghttp2" ,nghttp2)
-       ("openssl" ,openssl)
-       ("zlib" ,zlib)))
+       ("openssl" ,openssl)))
     (home-page "https://github.com/alexcrichton/curl-rust")
     (synopsis "Native bindings to the libcurl library")
     (description
@@ -6341,10 +6338,10 @@ variables.")
     (arguments
      `(#:cargo-inputs
        (("rust-cmake" ,rust-cmake-0.1)
-        ("rust-pkg-config" ,rust-pkg-config-0.3))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
-    (inputs
+        ("rust-pkg-config" ,rust-pkg-config-0.3))
+       #:propagated-native-inputs
+       (("pkg-config" ,pkg-config))))
+    (propagated-inputs
      `(("expat" ,expat)))
     (home-page "http://www.libexpat.org/")
     (synopsis "XML parser library written in C")
@@ -6663,8 +6660,7 @@ cross platform API.")
          "0hlb2zmn5ixrgr0i1qvrd3a7j4fpp002d0kddn2hm7hjj49z9zrc"))))
     (build-system cargo-build-system)
     (arguments
-     `(#:skip-build? #t
-       #:cargo-inputs
+     `(#:cargo-inputs
        (("rust-cfg-if" ,rust-cfg-if-0.1)
         ("rust-cloudflare-zlib-sys"
          ,rust-cloudflare-zlib-sys-0.2)
@@ -7014,9 +7010,6 @@ values to other threads.")
         ("rust-libc" ,rust-libc-0.2))
        #:cargo-development-inputs
        (("rust-unicode-normalization" ,rust-unicode-normalization-0.1))))
-    (inputs
-     `(("freetype" ,freetype)
-       ("zlib" ,zlib)))
     (home-page "https://github.com/PistonDevelopers/freetype-rs")
     (synopsis "Bindings for FreeType font library")
     (description "This package provides bindings for FreeType font library.")
@@ -7041,9 +7034,8 @@ values to other threads.")
        (("rust-libc" ,rust-libc-0.2)
         ("rust-libz-sys" ,rust-libz-sys-1.0)
         ("rust-pkg-config" ,rust-pkg-config-0.3))))
-    (inputs
-     `(("freetype" ,freetype)
-       ("zlib" ,zlib)))
+    (propagated-inputs
+     `(("freetype" ,freetype)))
     (home-page "https://github.com/PistonDevelopers/freetype-sys")
     (synopsis "Low level binding for FreeType font library")
     (description
@@ -8484,9 +8476,7 @@ DWARF debugging format.")
     (native-inputs
      `(("libgit2" ,libgit2)
        ("libssh2" ,libssh2)
-       ("openssl" ,openssl)
-       ("pkg-config" ,pkg-config)
-       ("zlib" ,zlib)))
+       ("openssl" ,openssl)))
     (home-page "https://github.com/rust-lang/git2-rs")
     (synopsis "Rust bindings to libgit2")
     (description
@@ -9164,8 +9154,7 @@ loading crate.")
          "0s3y1rx94swqnciz2zzifm8pmy2iyck270skgxhgkq7ab6x96bjq"))))
     (build-system cargo-build-system)
     (arguments
-     `(#:skip-build? #t
-       #:cargo-inputs
+     `(#:cargo-inputs
        (("rust-grep-cli" ,rust-grep-cli-0.1)
         ("rust-grep-matcher" ,rust-grep-matcher-0.1)
         ("rust-grep-pcre2" ,rust-grep-pcre2-0.1)
@@ -9259,9 +9248,6 @@ the regex engine it uses pluggable.")
      `(#:cargo-inputs
        (("rust-grep-matcher" ,rust-grep-matcher-0.1)
         ("rust-pcre2" ,rust-pcre2-0.2))))
-    (native-inputs
-     `(("pcre2" ,pcre2)
-       ("pkg-config" ,pkg-config)))
     (home-page
      "https://github.com/BurntSushi/ripgrep")
     (synopsis "Use PCRE2 with the grep crate")
@@ -11419,9 +11405,7 @@ macros on libc without stdlib.")
              #t)))))
     (native-inputs
      `(("libgit2" ,libgit2)
-       ("openssl" ,openssl)
-       ("pkg-config" ,pkg-config)
-       ("zlib" ,zlib)))
+       ("openssl" ,openssl)))
     (home-page "https://github.com/rust-lang/git2-rs")
     (synopsis "Native bindings to the libgit2 library")
     (description
@@ -11639,10 +11623,9 @@ functions and static variables these libraries contain.")
         ;; Build dependencies:
         ("rust-cc" ,rust-cc-1.0)
         ("rust-pkg-config" ,rust-pkg-config-0.3)
-        ("rust-vcpkg" ,rust-vcpkg-0.2))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)
-       ("zlib" ,zlib)))
+        ("rust-vcpkg" ,rust-vcpkg-0.2))
+       #:propagated-native-inputs
+       (("zlib" ,zlib))))
     (home-page "https://github.com/rust-lang/libz-sys")
     (synopsis "Bindings to the system libz library")
     (description
@@ -11810,9 +11793,7 @@ pairs in insertion order.")
              #t)))))
     (native-inputs
      `(("libssh2" ,libssh2)
-       ("openssl" ,openssl)
-       ("pkg-config" ,pkg-config)
-       ("zlib" ,zlib)))
+       ("openssl" ,openssl)))
     (home-page "https://github.com/alexcrichton/ssh2-rs")
     (synopsis "Native bindings to the libssh2 library")
     (description
@@ -15558,9 +15539,6 @@ synchronization primitives.")
         ("rust-log" ,rust-log-0.4)
         ("rust-pcre2-sys" ,rust-pcre2-sys-0.2)
         ("rust-thread-local" ,rust-thread-local-1.0))))
-    (native-inputs
-     `(("pcre2" ,pcre2)
-       ("pkg-config" ,pkg-config)))
     (home-page "https://github.com/BurntSushi/rust-pcre2")
     (synopsis "High level wrapper library for PCRE2")
     (description
@@ -15588,10 +15566,9 @@ synchronization primitives.")
      `(#:cargo-inputs
        (("rust-libc" ,rust-libc-0.2)
         ("rust-pkg-config" ,rust-pkg-config-0.3)
-        ("rust-cc" ,rust-cc-1.0))))
-    (native-inputs
-     `(("pcre2" ,pcre2)
-       ("pkg-config" ,pkg-config)))
+        ("rust-cc" ,rust-cc-1.0))
+       #:propagated-native-inputs
+       (("pcre2" ,pcre2))))
     (home-page
      "https://github.com/BurntSushi/rust-pcre2")
     (synopsis "Low level bindings to PCRE2")
@@ -16509,9 +16486,9 @@ written with declarative macros.")
     (build-system cargo-build-system)
     (arguments
      `(#:cargo-development-inputs
-       (("rust-lazy-static" ,rust-lazy-static-1))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
+       (("rust-lazy-static" ,rust-lazy-static-1))
+       #:propagated-native-inputs
+       (("pkg-config" ,pkg-config))))
     (home-page "https://github.com/rust-lang/pkg-config-rs")
     (synopsis "Library to run the pkg-config system tool")
     (description
@@ -20969,10 +20946,6 @@ for the serde framework.")
      `(#:cargo-inputs
        (("rust-libc" ,rust-libc-0.2)
         ("rust-servo-fontconfig-sys" ,rust-servo-fontconfig-sys-4))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
-    (inputs
-     `(("fontconfig" ,fontconfig)))
     (home-page "https://github.com/servo/rust-fontconfig/")
     (synopsis "Rust bindings for fontconfig")
     (description "This package provides Rust bindings for fontconfig.")
@@ -21003,9 +20976,7 @@ for the serde framework.")
        (("rust-expat-sys" ,rust-expat-sys-2.1)
         ("rust-servo-freetype-sys" ,rust-servo-freetype-sys-4)
         ("rust-pkg-config" ,rust-pkg-config-0.3))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
-    (inputs
+    (propagated-inputs
      `(("fontconfig" ,fontconfig)))
     (home-page "https://crates.io/crates/servo-fontconfig-sys")
     (synopsis "Rust wrapper around Fontconfig")
@@ -21034,9 +21005,7 @@ for the serde framework.")
      `(#:cargo-inputs
        (("rust-cmake" ,rust-cmake-0.1)
         ("rust-pkg-config" ,rust-pkg-config-0.3))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
-    (inputs
+    (propagated-inputs
      `(("freetype" ,freetype)))
     (home-page "http://www.freetype.org/")
     (synopsis "Rust wrapper around freetype")
diff --git a/gnu/packages/rust-apps.scm b/gnu/packages/rust-apps.scm
index 1cf22f4a79..9c03b90f9c 100644
--- a/gnu/packages/rust-apps.scm
+++ b/gnu/packages/rust-apps.scm
@@ -120,10 +120,7 @@
                           (string-append share "/zsh/site-functions/_exa"))
                #t))))))
     (inputs
-     `(("libgit2" ,libgit2)
-       ("zlib" ,zlib)))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
+     `(("libgit2" ,libgit2)))
     (home-page "https://the.exa.website/")
     (synopsis "Modern replacement for ls")
     (description "@code{exa} is a modern replacement for the command-line
@@ -251,9 +248,7 @@ provides defaults for 80% of the use cases.")
              #t)))
        #:features '("pcre2")))
     (native-inputs
-     `(("asciidoc" ,asciidoc)
-       ("pcre2" ,pcre2)
-       ("pkg-config" ,pkg-config)))
+     `(("asciidoc" ,asciidoc)))
     (home-page "https://github.com/BurntSushi/ripgrep")
     (synopsis "Line-oriented search tool")
     (description
diff --git a/gnu/packages/sequoia.scm b/gnu/packages/sequoia.scm
index 554b1d65ea..06ccd6aa78 100644
--- a/gnu/packages/sequoia.scm
+++ b/gnu/packages/sequoia.scm
@@ -57,8 +57,7 @@
        ("python-pytest" ,python-pytest)
        ("python-pytest-runner" ,python-pytest-runner)))
     (inputs
-     `(("capnproto" ,capnproto)
-       ("gmp" ,gmp)
+     `(("gmp" ,gmp)
        ("nettle" ,nettle)
        ("openssl" ,openssl)
        ("python" ,python)
-- 
2.26.2
J
J
Jakub Kądziołka wrote on 25 Jun 2020 23:26
[PATCH 3/4] build-system/cargo: Add a propagated-phases argument.
(address . 42049@debbugs.gnu.org)
20200625212637.26099-1-kuba@kadziolka.net
* guix/build-system/cargo.scm (package-propagated-phases, add-phases):
New procedures.
(lower): Collect phases from the crate closure.
---
guix/build-system/cargo.scm | 39 +++++++++++++++++++++++++++++++++----
1 file changed, 35 insertions(+), 4 deletions(-)

Toggle diff (80 lines)
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 3f518343ec..d3ec97f7fd 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -149,6 +149,12 @@ to NAME and VERSION."
       propagated-native-inputs)
     (package-arguments p)))
 
+(define (package-propagated-phases p)
+  (apply
+    (lambda* (#:key (propagated-phases '()) #:allow-other-keys)
+      propagated-phases)
+    (package-arguments p)))
+
 (define (crate-closure inputs)
   "Return the closure of INPUTS when considering the 'cargo-inputs' and
 'cargod-dev-deps' edges.  Omit duplicate inputs, except for those
@@ -243,23 +249,43 @@ any dependent crates. This can be a benefits:
        (list label input)))
     crate-closure))
 
+(define (add-phases propagated-phases base-phases)
+  ;; TODO(rebuild-rust): This could be simpler if avoiding rebuilds wasn't a goal.
+  (if (null? propagated-phases)
+    base-phases
+    (let ((phase-list `(modify-phases %standard-phases ,@propagated-phases)))
+      (if base-phases
+        `(let ((%standard-phases ,phase-list)) ,base-phases)
+        phase-list))))
+
 (define* (lower name
-                #:key source inputs native-inputs outputs system target
+                #:key source inputs native-inputs outputs system target phases
                 (rust (default-rust))
                 (cargo-inputs '())
                 (cargo-development-inputs '())
                 (propagated-native-inputs '())
+                (propagated-phases '())
                 #:allow-other-keys
                 #:rest arguments)
   "Return a bag for NAME."
 
   (define private-keywords
     '(#:source #:target #:rust #:inputs #:native-inputs #:outputs
+      #:phases #:propagated-phases
       #:cargo-inputs #:cargo-development-inputs #:propagated-native-inputs))
 
   (and (not target) ;; TODO: support cross-compilation
-       (let ((closure (crate-closure
-                        (append cargo-inputs cargo-development-inputs))))
+       (let* ((closure (crate-closure
+                         (append cargo-inputs cargo-development-inputs)))
+              (closure-phases
+                (append-map
+                  (compose package-propagated-phases second)
+                  ;; Make sure the leaves of the dependency graph come first,
+                  ;; such that their dependents may refer to the names of
+                  ;; the phases
+                  (reverse closure)))
+              (closure-phases (append closure-phases propagated-phases))
+              (phases (add-phases closure-phases phases)))
          (bag
            (name name)
            (system system)
@@ -284,7 +310,12 @@ any dependent crates. This can be a benefits:
                            ,@native-inputs))
            (outputs outputs)
            (build cargo-build)
-           (arguments (strip-keyword-arguments private-keywords arguments))))))
+           (arguments
+             (let ((provided-arguments
+                     (strip-keyword-arguments private-keywords arguments)))
+               (if phases
+                 (cons* #:phases phases provided-arguments)
+                 provided-arguments)))))))
 
 (define cargo-build-system
   (build-system
-- 
2.26.2
J
J
Jakub Kądziołka wrote on 25 Jun 2020 23:26
[PATCH 4/4] gnu: crates-io: Use propagated-phases.
(address . 42049@debbugs.gnu.org)
20200625212643.26344-1-kuba@kadziolka.net
* gnu/packages/crates-io.scm (rust-metadeps-1.1)[arguments]:
Don't skip build, skip tests instead. Add rust-lazy-static-0.2 as
development input.
(rust-clang-sys-0.28)[arguments]: Propagate phase, give it a
descriptive name.
[inputs]: Move libclang...
[propagated-inputs]: ...here.
(rust-clang-sys-0.26)[arguments]: Use substitute-keyword-arguments
to avoid duplicating the code of the phase.
(rust-bindgen-0.52, rust-aom-sys-0.1): Don't duplicate inputs and
phases of dependencies.
---
gnu/packages/crates-io.scm | 74 ++++++++++++--------------------------
1 file changed, 22 insertions(+), 52 deletions(-)

Toggle diff (127 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 5a4b513f1a..8d57f4f8d0 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -535,24 +535,11 @@ text or blue underlined text, on ANSI terminals.")
           "0ix3djcf84kk53h6fac73n7jc614745n7kbmikxwi3s73b6vzgsr"))))
     (build-system cargo-build-system)
     (arguments
-     `(;#:skip-build? #t
-       #:cargo-inputs
+     `(#:cargo-inputs
        (("rust-bindgen" ,rust-bindgen-0.51)
-        ("rust-metadeps" ,rust-metadeps-1.1))
-       #:phases
-       (modify-phases %standard-phases
-         (add-after 'unpack 'set-environmental-variable
-           (lambda* (#:key inputs #:allow-other-keys)
-             (let ((clang (assoc-ref inputs "libclang")))
-               (setenv "LIBCLANG_PATH"
-                       (string-append clang "/lib")))
-             #t)))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
+        ("rust-metadeps" ,rust-metadeps-1.1))))
     (inputs
-     `(("libaom" ,libaom)
-       ("libclang" ,clang)
-       ("llvm" ,llvm)))
+     `(("libaom" ,libaom)))
     (home-page "https://github.com/rust-av/aom-rs")
     (synopsis "FFI bindings to aom")
     (description "This package provides FFI bindings to aom.")
@@ -1338,17 +1325,7 @@ that uses Serde for transforming structs into bytes and vice versa!")
        #:cargo-development-inputs
        (("rust-clap" ,rust-clap-2)
         ("rust-diff" ,rust-diff-0.1)
-        ("rust-shlex" ,rust-shlex-0.1))
-       #:phases
-       (modify-phases %standard-phases
-         (add-after 'unpack 'set-environmental-variable
-           (lambda* (#:key inputs #:allow-other-keys)
-             (let ((clang (assoc-ref inputs "libclang")))
-               (setenv "LIBCLANG_PATH"
-                       (string-append clang "/lib")))
-             #t)))))
-    (inputs
-     `(("libclang" ,clang)))
+        ("rust-shlex" ,rust-shlex-0.1))))
     (home-page "https://rust-lang.github.io/rust-bindgen/")
     (synopsis
      "Automatically generates Rust FFI bindings to C and C++ libraries")
@@ -1391,8 +1368,7 @@ that uses Serde for transforming structs into bytes and vice versa!")
        #:cargo-development-inputs
        (("rust-clap" ,rust-clap-2)
         ("rust-diff" ,rust-diff-0.1)
-        ("rust-shlex" ,rust-shlex-0.1))))
-    (inputs `())))
+        ("rust-shlex" ,rust-shlex-0.1))))))
 
 (define-public rust-bindgen-0.50
   (package
@@ -2968,15 +2944,14 @@ for computer graphics.")
        (("rust-glob" ,rust-glob-0.3)
         ("rust-libc" ,rust-libc-0.2)
         ("rust-libloading" ,rust-libloading-0.5))
-       #:phases
-       (modify-phases %standard-phases
-         (add-after 'unpack 'set-environmental-variable
-           (lambda* (#:key inputs #:allow-other-keys)
-             (let ((clang (assoc-ref inputs "libclang")))
-               (setenv "LIBCLANG_PATH"
-                       (string-append clang "/lib")))
-             #t)))))
-    (inputs
+       #:propagated-phases
+       ((add-after 'unpack 'bindgen:set-libclang-path
+          (lambda* (#:key inputs #:allow-other-keys)
+            (let ((clang (assoc-ref inputs "libclang")))
+              (setenv "LIBCLANG_PATH"
+                      (string-append clang "/lib")))
+            #t)))))
+    (propagated-inputs
      `(("libclang" ,clang)))
     (home-page "https://github.com/KyleMayes/clang-sys")
     (synopsis "Rust bindings for libclang")
@@ -2998,18 +2973,11 @@ for computer graphics.")
          (base32
           "1r50dwy5hj5gq07dn0qf8222d07qv0970ymx0j8n9779yayc3w3f"))))
     (arguments
-     `(#:cargo-inputs
-       (("rust-glob" ,rust-glob-0.2)
-        ("rust-libc" ,rust-libc-0.2)
-        ("rust-libloading" ,rust-libloading-0.5))
-       #:phases
-       (modify-phases %standard-phases
-         (add-after 'unpack 'set-environmental-variable
-           (lambda* (#:key inputs #:allow-other-keys)
-             (let ((clang (assoc-ref inputs "libclang")))
-               (setenv "LIBCLANG_PATH"
-                       (string-append clang "/lib")))
-             #t)))))))
+      (substitute-keyword-arguments (package-arguments rust-clang-sys-0.28)
+        (#:cargo-inputs
+         `(("rust-glob" ,rust-glob-0.2)
+           ("rust-libc" ,rust-libc-0.2)
+           ("rust-libloading" ,rust-libloading-0.5)))))))
 
 (define-public rust-clang-sys-0.23
   (package
@@ -12712,11 +12680,13 @@ for Rust structs.")
           "1hjla9ypycqw1snd2qf87cckcc0d5z5qvxpcijn5yrrs3f825cbk"))))
     (build-system cargo-build-system)
     (arguments
-     `(#:skip-build? #t
+     `(#:tests? #f ;; Test files aren't shipped to crates.io
        #:cargo-inputs
        (("rust-error-chain" ,rust-error-chain-0.10)
         ("rust-toml" ,rust-toml-0.2)
-        ("rust-pkg-config" ,rust-pkg-config-0.3))))
+        ("rust-pkg-config" ,rust-pkg-config-0.3))
+       #:cargo-development-inputs
+       (("rust-lazy-static" ,rust-lazy-static-0.2))))
     (home-page "https://github.com/joshtriplett/metadeps")
     (synopsis "Run pkg-config from declarative dependencies in Cargo.toml")
     (description "Run pkg-config from declarative dependencies in Cargo.toml.")
-- 
2.26.2
E
E
Efraim Flashner wrote on 13 Aug 2020 11:48
Re: [PATCH 0/4] build-system/cargo: Propagations across the crate closure.
(name . Jakub Kądziołka)(address . kuba@kadziolka.net)(address . guix-patches@gnu.org)
20200813094843.GC1228@E5400
On Thu, Jun 25, 2020 at 11:25:23PM +0200, Jakub Kądziołka wrote:
Toggle quote (26 lines)
> Due to the unusual (for Guix) compilation model used by
> cargo-build-system, any phases or inputs added by a given library crate
> need to be duplicated in all its dependents. This patchstack attempts to
> solve this by allowing to propagate inputs, native-inputs and phases
> across cargo-inputs edges in the dependency graph.
>
> Apart from the build system work itself, I have included samples of the
> cleanup they allow. Apart from being a good example, these are the
> changes I have used to test the feature.
>
> Jakub Kądziołka (4):
> build-system/cargo: Allow propagating inputs across CARGO-INPUTS edges
> gnu: crates-io: Use propagated-inputs and propagated-native-inputs.
> build-system/cargo: Add a propagated-phases argument.
> gnu: crates-io: Use propagated-phases.
>
> gnu/packages/crates-io.scm | 157 +++++++++++-------------------------
> gnu/packages/rust-apps.scm | 9 +--
> gnu/packages/sequoia.scm | 3 +-
> guix/build-system/cargo.scm | 92 +++++++++++++++------
> 4 files changed, 120 insertions(+), 141 deletions(-)
>
> --
> 2.26.2
>

I'm going to respond here so the thoughts don't get lost. While it would
take care of some of the issues we have regarding adding regular inputs
or propagated-/native- inputs, I don't think this is the way we want to
go. If we can't figure out how to re-use build artifacts then I'd rather
copy the go-build-system and install the sources into the output and use
that as the input for the next package. That would give us the
build-graph which we really want.

--
Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAl81DHgACgkQQarn3Mo9
g1GcNRAAtcLi609PF7MtuCfbmrubnmZ4+uP8Z1idngrVOxgN/xAdtqHmCjTjukcg
dH31/AKm1PvTteFKps5X95b2OzOtlvsOhzn2AU8kmCrnP1qUnLLRbrIymy0I0GLe
UIRJmbc/QPhdia670reP6OJ/l1HC9xWnnBofccdNSj0gvdKWsyrB6sImNfhrZ6aH
Z+b+54B8QiN8syMoB8dQd2QKG0rREj7Nnr40+4uRyHFa3SzYhkETA297V177wZKl
ADueC2qP48Cc2vJSxoOvA4/22LIsFfeWcI7tdWJskvGJByK4uAcfsuO6TxxZ71Pa
leQ8fXjLDZi5/rf3OvIa5oqZYGpvnwTRY/FZiKE1YlmZ9c0J2EW/CXDKHC1AUttq
2jS3obNivfkkBE/6yuI8uXkVfKiYByLYBhcl/vYZBVN1zYNQ8HCDN1NlQ+2x4LPU
g/QbZ9SfTaO1GVBK9HJXdBwoHTKV47ASoC4W2XLzbbuczR0mTX2Gd50uNJ2V6QMK
3AHUn7/l2Iegg6v8rxLCp6ALQkG97BKojahj7AP2NsA0DdZHz1YgTk/BTB39QQX7
krUs+fWBm+qLkRu00BK5EJu16BTDCFdiARad23YdD/8kD+fwVIN5/hQyztO+xOnK
0Vz0tgxum6l+KnPERJqljXRI4VkRa7HM/o1xJfE1utyEAVhK9U4=
=hh+1
-----END PGP SIGNATURE-----


J
J
Jakub Kądziołka wrote on 13 Aug 2020 18:16
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . guix-patches@gnu.org)
20200813161638.ffgfusjplz5zcfgx@gravity
On Thu, Aug 13, 2020 at 12:48:43PM +0300, Efraim Flashner wrote:
Toggle quote (8 lines)
> I'm going to respond here so the thoughts don't get lost. While it would
> take care of some of the issues we have regarding adding regular inputs
> or propagated-/native- inputs, I don't think this is the way we want to
> go. If we can't figure out how to re-use build artifacts then I'd rather
> copy the go-build-system and install the sources into the output and use
> that as the input for the next package. That would give us the
> build-graph which we really want.

Note that this wouldn't solve all the issues, we would still need
an equivalent for propagated phases, to set any environment variables
necessary.

Moreover, note that the reason the current system was introduced in the
first place was to avoid the quadratic builds. I suppose this is less of
an issue in go-build-system due to the order-of-magnitude difference in
compiler speed on typical source code.

As for re-using build artifacts, once we figure out how to do it, we can
always revert this patch, together with the one that originally added
cargo-inputs. I don't think it's going to be any time soon, though, as
upstream doesn't support this style of building.

Regards,
Jakub Kądziołka
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEE5Xa/ss9usT31cTO54xWnWEYTFWQFAl81Z2YACgkQ4xWnWEYT
FWRJKw/5ARe5CUp+YkggXptVWzZWZjQ+n5AEipSEBeD6iTVCIciUxFa6k3Y+bylo
s3D1o1krAwclV2rQREC+gRdN274JRAcFiv9BilHzgVffefMZNP2mPZK9fFD4thlJ
/xYBQInGP4KePn9f4hFcapgSYQWQ3N3k2RDDOg27P1tFqSNlT5NihV215xbPhdAi
Xl9TNKOJTCdks8SxL27841lw9KhMOADahjlGO3kc9WOYeQ7OzBipTrCoUEdekQl8
+IYiZ63AqXZFx7aK93S2Kc6aD134Jd8mjKqnywrJwCCDrGh0zmOnSGVbT4W4vOBe
zDqVRZYvGvYwfAe6XiVaZaF9BJBZE9RnftfHSnFioVHAUy1qeQ3NZyIHsJEvAV83
vCoApdfccmKbxkbzchvr4fkECCsbsjKYWGI8f7w9fc99cH7kuWe9X7E5tTmjjJMv
7R83cG8IxGBJzIw/plfdLmXswy2RyyO2iZqW92Q4jc44W6WDi/7jAKl/ipAAOmiI
QbhQVdQlAN5NxzXeDNXlZFZ7dp8mxCVdApluV39cAaIgqrNTRMFUB3Lbeo87LlyW
u9/mUzryu1gdoAhSV7IHandQWJXv+1apzd7bbpKu+9oXV+/fQIhDUdT7ViqKgIrc
VwmN7izB2j4QBDo2hj6fbbp149uf9MKgTJCgTaczp7H3JzuEOFc=
=r91e
-----END PGP SIGNATURE-----


L
L
Leo Famulari wrote on 14 Aug 2020 23:26
Re: [bug#42049] [PATCH 0/4] build-system/cargo: Propagations across the crate closure.
(name . Jakub Kądziołka)(address . kuba@kadziolka.net)
20200814212618.GB30339@jasmine.lan
On Thu, Aug 13, 2020 at 06:16:38PM +0200, Jakub Kądziołka wrote:
Toggle quote (5 lines)
> Moreover, note that the reason the current system was introduced in the
> first place was to avoid the quadratic builds. I suppose this is less of
> an issue in go-build-system due to the order-of-magnitude difference in
> compiler speed on typical source code.

Right. The go-build-system is far from optimally efficient but, in
practice, it's not too slow. I haven't counted but I figure that Rust
dependency graphs are an order of magnitude larger.
?