[PATCH 0/4] build-system/cargo: Propagations across the crate closure.

Open

Details

3 participants

Efraim Flashner
Jakub K?dzio?ka
Leo Famulari

Owner: unassigned

Submitted by: Jakub K?dzio?ka

Severity: normal

Jakub K?dzio?ka wrote on 25 Jun 2020 23:25

Recipients:(address . guix-patches@gnu.org)(address . efraim@flashner.co.il)

Message-ID:20200625212523.25016-1-kuba@kadziolka.net

Due to the unusual (for Guix) compilation model used by
cargo-build-system, any phases or inputs added by a given library crate
need to be duplicated in all its dependents. This patchstack attempts to
solve this by allowing to propagate inputs, native-inputs and phases
across cargo-inputs edges in the dependency graph.

Apart from the build system work itself, I have included samples of the
cleanup they allow. Apart from being a good example, these are the
changes I have used to test the feature.

Jakub K?dzio?ka (4):
  build-system/cargo: Allow propagating inputs across CARGO-INPUTS edges
  gnu: crates-io: Use propagated-inputs and propagated-native-inputs.
  build-system/cargo: Add a propagated-phases argument.
  gnu: crates-io: Use propagated-phases.

 gnu/packages/crates-io.scm  | 157 +++++++++++-------------------------
 gnu/packages/rust-apps.scm  |   9 +--
 gnu/packages/sequoia.scm    |   3 +-
 guix/build-system/cargo.scm |  92 +++++++++++++++------
 4 files changed, 120 insertions(+), 141 deletions(-)

-- 
2.26.2

Jakub K?dzio?ka wrote on 25 Jun 2020 23:26

[PATCH 1/4] build-system/cargo: Allow propagating inputs across CARGO-INPUTS edges

Recipients:(address . 42049@debbugs.gnu.org)

Message-ID:20200625212627.25622-1-kuba@kadziolka.net

* guix/build-system/cargo.scm (package-propagated-native-inputs):
  New procedure.
  (expand-crate-sources): Take crate closure instead of input lists.
  (lower): Expand host-inputs and build-inputs to include inputs
  from the crate closure.
---
 guix/build-system/cargo.scm | 59 +++++++++++++++++++++++--------------
 1 file changed, 37 insertions(+), 22 deletions(-)

Toggle diff (101 lines)diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 6c8edf6bac..3f518343ec 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -143,6 +143,12 @@ to NAME and VERSION."
       cargo-development-inputs)
     (package-arguments p)))
 
+(define (package-propagated-native-inputs p)
+  (apply
+    (lambda* (#:key (propagated-native-inputs '()) #:allow-other-keys)
+      propagated-native-inputs)
+    (package-arguments p)))
+
 (define (crate-closure inputs)
   "Return the closure of INPUTS when considering the 'cargo-inputs' and
 'cargod-dev-deps' edges.  Omit duplicate inputs, except for those
@@ -182,9 +188,8 @@ do not extract the conventional inputs)."
       ((input rest ...)
        (loop rest (cons input result) propagated first? seen)))))
 
-(define (expand-crate-sources cargo-inputs cargo-development-inputs)
-  "Extract all transitive sources for CARGO-INPUTS and CARGO-DEVELOPMENT-INPUTS
-along their 'cargo-inputs' edges.
+(define (expand-crate-sources crate-closure)
+  "Extract all sources for the transitive cargo inputs provided in CRATE-CLOSURE.
 
 Cargo requires all transitive crate dependencies' sources to be available
 in its index, even if they are optional (this is so it can generate
@@ -236,40 +241,50 @@ any dependent crates. This can be a benefits:
        (list label (package-source p)))
       ((label input)
        (list label input)))
-    (crate-closure (append cargo-inputs cargo-development-inputs))))
+    crate-closure))
 
 (define* (lower name
                 #:key source inputs native-inputs outputs system target
                 (rust (default-rust))
                 (cargo-inputs '())
                 (cargo-development-inputs '())
+                (propagated-native-inputs '())
                 #:allow-other-keys
                 #:rest arguments)
   "Return a bag for NAME."
 
   (define private-keywords
     '(#:source #:target #:rust #:inputs #:native-inputs #:outputs
-      #:cargo-inputs #:cargo-development-inputs))
+      #:cargo-inputs #:cargo-development-inputs #:propagated-native-inputs))
 
   (and (not target) ;; TODO: support cross-compilation
-       (bag
-         (name name)
-         (system system)
-         (target target)
-         (host-inputs `(,@(if source
-                              `(("source" ,source))
-                              '())
-                        ,@inputs
+       (let ((closure (crate-closure
+                        (append cargo-inputs cargo-development-inputs))))
+         (bag
+           (name name)
+           (system system)
+           (target target)
+           (host-inputs `(,@(if source
+                                `(("source" ,source))
+                                '())
+                          ,@inputs
+                          ,@(append-map
+                              (compose package-propagated-inputs second)
+                              closure)
 
-                        ;; Keep the standard inputs of 'gnu-build-system'
-                        ,@(standard-packages)))
-         (build-inputs `(("cargo" ,rust "cargo")
-                         ("rustc" ,rust)
-                         ,@(expand-crate-sources cargo-inputs cargo-development-inputs)
-                         ,@native-inputs))
-         (outputs outputs)
-         (build cargo-build)
-         (arguments (strip-keyword-arguments private-keywords arguments)))))
+                          ;; Keep the standard inputs of 'gnu-build-system'
+                          ,@(standard-packages)))
+           (build-inputs `(("cargo" ,rust "cargo")
+                           ("rustc" ,rust)
+                           ,@(expand-crate-sources closure)
+                           ,@(append-map
+                               (compose package-propagated-native-inputs second)
+                               closure)
+                           ,@propagated-native-inputs
+                           ,@native-inputs))
+           (outputs outputs)
+           (build cargo-build)
+           (arguments (strip-keyword-arguments private-keywords arguments))))))
 
 (define cargo-build-system
   (build-system
-- 
2.26.2

Jakub K?dzio?ka wrote on 25 Jun 2020 23:26

[PATCH 2/4] gnu: crates-io: Use propagated-inputs and propagated-native-inputs.

Recipients:(address . 42049@debbugs.gnu.org)

Message-ID:20200625212632.25840-1-kuba@kadziolka.net

* gnu/packages/crates-io.scm
  (rust-curl-sys-0.4, rust-freetype-rs-0.23, rust-freetype-sys-0.9,
  rust-git2-0.11, rust-grep-pcre2-0.1, rust-libgit2-sys-0.10,
  rust-libz-sys-1.0, rust-libssh2-sys-0.2, rust-pcre2-0.2,
  rust-pcre2-sys-0.2, rust-servo-fontconfig-0.4,
  rust-servo-fontconfig-sys-4): Don't include transitive dependencies in inputs.
  (rust-capnp-rpc-0.10, rust-expat-sys-2.1, rust-libz-sys-1.0,
  rust-pcre2-sys-0.2, rust-pkg-config-0.3, rust-servo-fontconfig-sys-4):
  Make inputs propagated.
  (rust-flate2-1.0, rust-grep-0.2): Don't skip build.
* gnu/packages/rust-apps.scm (exa, ripgrep): Don't include some
  transitive dependencies in inputs.
* gnu/packages/sequoia.scm (sequoia): Likewise.
---
 gnu/packages/crates-io.scm | 83 ++++++++++++--------------------------
 gnu/packages/rust-apps.scm |  9 +----
 gnu/packages/sequoia.scm   |  3 +-
 3 files changed, 29 insertions(+), 66 deletions(-)

Toggle diff (265 lines)diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index bc29df5871..5a4b513f1a 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -2482,15 +2482,15 @@ messages.")
        (sha256
         (base32 "1j6xg7yays1hlm1045wviyn1642yvvi2p4kba26yk07a0kafr3jn"))))
     (build-system cargo-build-system)
-    (native-inputs
-     `(("capnproto" ,capnproto)))
     (arguments
      `(#:cargo-inputs
        (("rust-capnp" ,rust-capnp-0.10)
         ("rust-capnp-futures" ,rust-capnp-futures-0.10)
         ("rust-futures" ,rust-futures-0.1))
        #:cargo-development-inputs
-       (("rust-capnpc" ,rust-capnpc-0.10))))
+       (("rust-capnpc" ,rust-capnpc-0.10))
+       #:propagated-native-inputs
+       (("capnproto" ,capnproto))))
     (home-page "https://github.com/capnproto/capnproto-rust")
     (synopsis "Cap'n Proto remote procedure call protocol implementation")
     (description "This package provides an implementation of the Cap'n Proto
@@ -4659,13 +4659,10 @@ Transparency logs for use with sct crate.")
             (let ((openssl (assoc-ref inputs "openssl")))
               (setenv "OPENSSL_DIR" openssl))
             #t)))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
     (inputs
      `(("curl" ,curl)
        ("nghttp2" ,nghttp2)
-       ("openssl" ,openssl)
-       ("zlib" ,zlib)))
+       ("openssl" ,openssl)))
     (home-page "https://github.com/alexcrichton/curl-rust")
     (synopsis "Native bindings to the libcurl library")
     (description
@@ -6341,10 +6338,10 @@ variables.")
     (arguments
      `(#:cargo-inputs
        (("rust-cmake" ,rust-cmake-0.1)
-        ("rust-pkg-config" ,rust-pkg-config-0.3))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
-    (inputs
+        ("rust-pkg-config" ,rust-pkg-config-0.3))
+       #:propagated-native-inputs
+       (("pkg-config" ,pkg-config))))
+    (propagated-inputs
      `(("expat" ,expat)))
     (home-page "http://www.libexpat.org/")
     (synopsis "XML parser library written in C")
@@ -6663,8 +6660,7 @@ cross platform API.")
          "0hlb2zmn5ixrgr0i1qvrd3a7j4fpp002d0kddn2hm7hjj49z9zrc"))))
     (build-system cargo-build-system)
     (arguments
-     `(#:skip-build? #t
-       #:cargo-inputs
+     `(#:cargo-inputs
        (("rust-cfg-if" ,rust-cfg-if-0.1)
         ("rust-cloudflare-zlib-sys"
          ,rust-cloudflare-zlib-sys-0.2)
@@ -7014,9 +7010,6 @@ values to other threads.")
         ("rust-libc" ,rust-libc-0.2))
        #:cargo-development-inputs
        (("rust-unicode-normalization" ,rust-unicode-normalization-0.1))))
-    (inputs
-     `(("freetype" ,freetype)
-       ("zlib" ,zlib)))
     (home-page "https://github.com/PistonDevelopers/freetype-rs")
     (synopsis "Bindings for FreeType font library")
     (description "This package provides bindings for FreeType font library.")
@@ -7041,9 +7034,8 @@ values to other threads.")
        (("rust-libc" ,rust-libc-0.2)
         ("rust-libz-sys" ,rust-libz-sys-1.0)
         ("rust-pkg-config" ,rust-pkg-config-0.3))))
-    (inputs
-     `(("freetype" ,freetype)
-       ("zlib" ,zlib)))
+    (propagated-inputs
+     `(("freetype" ,freetype)))
     (home-page "https://github.com/PistonDevelopers/freetype-sys")
     (synopsis "Low level binding for FreeType font library")
     (description
@@ -8484,9 +8476,7 @@ DWARF debugging format.")
     (native-inputs
      `(("libgit2" ,libgit2)
        ("libssh2" ,libssh2)
-       ("openssl" ,openssl)
-       ("pkg-config" ,pkg-config)
-       ("zlib" ,zlib)))
+       ("openssl" ,openssl)))
     (home-page "https://github.com/rust-lang/git2-rs")
     (synopsis "Rust bindings to libgit2")
     (description
@@ -9164,8 +9154,7 @@ loading crate.")
          "0s3y1rx94swqnciz2zzifm8pmy2iyck270skgxhgkq7ab6x96bjq"))))
     (build-system cargo-build-system)
     (arguments
-     `(#:skip-build? #t
-       #:cargo-inputs
+     `(#:cargo-inputs
        (("rust-grep-cli" ,rust-grep-cli-0.1)
         ("rust-grep-matcher" ,rust-grep-matcher-0.1)
         ("rust-grep-pcre2" ,rust-grep-pcre2-0.1)
@@ -9259,9 +9248,6 @@ the regex engine it uses pluggable.")
      `(#:cargo-inputs
        (("rust-grep-matcher" ,rust-grep-matcher-0.1)
         ("rust-pcre2" ,rust-pcre2-0.2))))
-    (native-inputs
-     `(("pcre2" ,pcre2)
-       ("pkg-config" ,pkg-config)))
     (home-page
      "https://github.com/BurntSushi/ripgrep")
     (synopsis "Use PCRE2 with the grep crate")
@@ -11419,9 +11405,7 @@ macros on libc without stdlib.")
              #t)))))
     (native-inputs
      `(("libgit2" ,libgit2)
-       ("openssl" ,openssl)
-       ("pkg-config" ,pkg-config)
-       ("zlib" ,zlib)))
+       ("openssl" ,openssl)))
     (home-page "https://github.com/rust-lang/git2-rs")
     (synopsis "Native bindings to the libgit2 library")
     (description
@@ -11639,10 +11623,9 @@ functions and static variables these libraries contain.")
         ;; Build dependencies:
         ("rust-cc" ,rust-cc-1.0)
         ("rust-pkg-config" ,rust-pkg-config-0.3)
-        ("rust-vcpkg" ,rust-vcpkg-0.2))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)
-       ("zlib" ,zlib)))
+        ("rust-vcpkg" ,rust-vcpkg-0.2))
+       #:propagated-native-inputs
+       (("zlib" ,zlib))))
     (home-page "https://github.com/rust-lang/libz-sys")
     (synopsis "Bindings to the system libz library")
     (description
@@ -11810,9 +11793,7 @@ pairs in insertion order.")
              #t)))))
     (native-inputs
      `(("libssh2" ,libssh2)
-       ("openssl" ,openssl)
-       ("pkg-config" ,pkg-config)
-       ("zlib" ,zlib)))
+       ("openssl" ,openssl)))
     (home-page "https://github.com/alexcrichton/ssh2-rs")
     (synopsis "Native bindings to the libssh2 library")
     (description
@@ -15558,9 +15539,6 @@ synchronization primitives.")
         ("rust-log" ,rust-log-0.4)
         ("rust-pcre2-sys" ,rust-pcre2-sys-0.2)
         ("rust-thread-local" ,rust-thread-local-1.0))))
-    (native-inputs
-     `(("pcre2" ,pcre2)
-       ("pkg-config" ,pkg-config)))
     (home-page "https://github.com/BurntSushi/rust-pcre2")
     (synopsis "High level wrapper library for PCRE2")
     (description
@@ -15588,10 +15566,9 @@ synchronization primitives.")
      `(#:cargo-inputs
        (("rust-libc" ,rust-libc-0.2)
         ("rust-pkg-config" ,rust-pkg-config-0.3)
-        ("rust-cc" ,rust-cc-1.0))))
-    (native-inputs
-     `(("pcre2" ,pcre2)
-       ("pkg-config" ,pkg-config)))
+        ("rust-cc" ,rust-cc-1.0))
+       #:propagated-native-inputs
+       (("pcre2" ,pcre2))))
     (home-page
      "https://github.com/BurntSushi/rust-pcre2")
     (synopsis "Low level bindings to PCRE2")
@@ -16509,9 +16486,9 @@ written with declarative macros.")
     (build-system cargo-build-system)
     (arguments
      `(#:cargo-development-inputs
-       (("rust-lazy-static" ,rust-lazy-static-1))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
+       (("rust-lazy-static" ,rust-lazy-static-1))
+       #:propagated-native-inputs
+       (("pkg-config" ,pkg-config))))
     (home-page "https://github.com/rust-lang/pkg-config-rs")
     (synopsis "Library to run the pkg-config system tool")
     (description
@@ -20969,10 +20946,6 @@ for the serde framework.")
      `(#:cargo-inputs
        (("rust-libc" ,rust-libc-0.2)
         ("rust-servo-fontconfig-sys" ,rust-servo-fontconfig-sys-4))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
-    (inputs
-     `(("fontconfig" ,fontconfig)))
     (home-page "https://github.com/servo/rust-fontconfig/")
     (synopsis "Rust bindings for fontconfig")
     (description "This package provides Rust bindings for fontconfig.")
@@ -21003,9 +20976,7 @@ for the serde framework.")
        (("rust-expat-sys" ,rust-expat-sys-2.1)
         ("rust-servo-freetype-sys" ,rust-servo-freetype-sys-4)
         ("rust-pkg-config" ,rust-pkg-config-0.3))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
-    (inputs
+    (propagated-inputs
      `(("fontconfig" ,fontconfig)))
     (home-page "https://crates.io/crates/servo-fontconfig-sys")
     (synopsis "Rust wrapper around Fontconfig")
@@ -21034,9 +21005,7 @@ for the serde framework.")
      `(#:cargo-inputs
        (("rust-cmake" ,rust-cmake-0.1)
         ("rust-pkg-config" ,rust-pkg-config-0.3))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
-    (inputs
+    (propagated-inputs
      `(("freetype" ,freetype)))
     (home-page "http://www.freetype.org/")
     (synopsis "Rust wrapper around freetype")
diff --git a/gnu/packages/rust-apps.scm b/gnu/packages/rust-apps.scm
index 1cf22f4a79..9c03b90f9c 100644
--- a/gnu/packages/rust-apps.scm
+++ b/gnu/packages/rust-apps.scm
@@ -120,10 +120,7 @@
                           (string-append share "/zsh/site-functions/_exa"))
                #t))))))
     (inputs
-     `(("libgit2" ,libgit2)
-       ("zlib" ,zlib)))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
+     `(("libgit2" ,libgit2)))
     (home-page "https://the.exa.website/")
     (synopsis "Modern replacement for ls")
     (description "@code{exa} is a modern replacement for the command-line
@@ -251,9 +248,7 @@ provides defaults for 80% of the use cases.")
              #t)))
        #:features '("pcre2")))
     (native-inputs
-     `(("asciidoc" ,asciidoc)
-       ("pcre2" ,pcre2)
-       ("pkg-config" ,pkg-config)))
+     `(("asciidoc" ,asciidoc)))
     (home-page "https://github.com/BurntSushi/ripgrep")
     (synopsis "Line-oriented search tool")
     (description
diff --git a/gnu/packages/sequoia.scm b/gnu/packages/sequoia.scm
index 554b1d65ea..06ccd6aa78 100644
--- a/gnu/packages/sequoia.scm
+++ b/gnu/packages/sequoia.scm
@@ -57,8 +57,7 @@
        ("python-pytest" ,python-pytest)
        ("python-pytest-runner" ,python-pytest-runner)))
     (inputs
-     `(("capnproto" ,capnproto)
-       ("gmp" ,gmp)
+     `(("gmp" ,gmp)
        ("nettle" ,nettle)
        ("openssl" ,openssl)
        ("python" ,python)
-- 
2.26.2

Jakub K?dzio?ka wrote on 25 Jun 2020 23:26

[PATCH 3/4] build-system/cargo: Add a propagated-phases argument.

Recipients:(address . 42049@debbugs.gnu.org)

Message-ID:20200625212637.26099-1-kuba@kadziolka.net

* guix/build-system/cargo.scm (package-propagated-phases, add-phases):
  New procedures.
  (lower): Collect phases from the crate closure.
---
 guix/build-system/cargo.scm | 39 +++++++++++++++++++++++++++++++++----
 1 file changed, 35 insertions(+), 4 deletions(-)

Toggle diff (80 lines)diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 3f518343ec..d3ec97f7fd 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -149,6 +149,12 @@ to NAME and VERSION."
       propagated-native-inputs)
     (package-arguments p)))
 
+(define (package-propagated-phases p)
+  (apply
+    (lambda* (#:key (propagated-phases '()) #:allow-other-keys)
+      propagated-phases)
+    (package-arguments p)))
+
 (define (crate-closure inputs)
   "Return the closure of INPUTS when considering the 'cargo-inputs' and
 'cargod-dev-deps' edges.  Omit duplicate inputs, except for those
@@ -243,23 +249,43 @@ any dependent crates. This can be a benefits:
        (list label input)))
     crate-closure))
 
+(define (add-phases propagated-phases base-phases)
+  ;; TODO(rebuild-rust): This could be simpler if avoiding rebuilds wasn't a goal.
+  (if (null? propagated-phases)
+    base-phases
+    (let ((phase-list `(modify-phases %standard-phases ,@propagated-phases)))
+      (if base-phases
+        `(let ((%standard-phases ,phase-list)) ,base-phases)
+        phase-list))))
+
 (define* (lower name
-                #:key source inputs native-inputs outputs system target
+                #:key source inputs native-inputs outputs system target phases
                 (rust (default-rust))
                 (cargo-inputs '())
                 (cargo-development-inputs '())
                 (propagated-native-inputs '())
+                (propagated-phases '())
                 #:allow-other-keys
                 #:rest arguments)
   "Return a bag for NAME."
 
   (define private-keywords
     '(#:source #:target #:rust #:inputs #:native-inputs #:outputs
+      #:phases #:propagated-phases
       #:cargo-inputs #:cargo-development-inputs #:propagated-native-inputs))
 
   (and (not target) ;; TODO: support cross-compilation
-       (let ((closure (crate-closure
-                        (append cargo-inputs cargo-development-inputs))))
+       (let* ((closure (crate-closure
+                         (append cargo-inputs cargo-development-inputs)))
+              (closure-phases
+                (append-map
+                  (compose package-propagated-phases second)
+                  ;; Make sure the leaves of the dependency graph come first,
+                  ;; such that their dependents may refer to the names of
+                  ;; the phases
+                  (reverse closure)))
+              (closure-phases (append closure-phases propagated-phases))
+              (phases (add-phases closure-phases phases)))
          (bag
            (name name)
            (system system)
@@ -284,7 +310,12 @@ any dependent crates. This can be a benefits:
                            ,@native-inputs))
            (outputs outputs)
            (build cargo-build)
-           (arguments (strip-keyword-arguments private-keywords arguments))))))
+           (arguments
+             (let ((provided-arguments
+                     (strip-keyword-arguments private-keywords arguments)))
+               (if phases
+                 (cons* #:phases phases provided-arguments)
+                 provided-arguments)))))))
 
 (define cargo-build-system
   (build-system
-- 
2.26.2

Jakub K?dzio?ka wrote on 25 Jun 2020 23:26

[PATCH 4/4] gnu: crates-io: Use propagated-phases.

Recipients:(address . 42049@debbugs.gnu.org)

Message-ID:20200625212643.26344-1-kuba@kadziolka.net

* gnu/packages/crates-io.scm (rust-metadeps-1.1)[arguments]:
  Don't skip build, skip tests instead. Add rust-lazy-static-0.2 as
  development input.
  (rust-clang-sys-0.28)[arguments]: Propagate phase, give it a
  descriptive name.
  [inputs]: Move libclang...
  [propagated-inputs]: ...here.
  (rust-clang-sys-0.26)[arguments]: Use substitute-keyword-arguments
  to avoid duplicating the code of the phase.
  (rust-bindgen-0.52, rust-aom-sys-0.1): Don't duplicate inputs and
  phases of dependencies.
---
 gnu/packages/crates-io.scm | 74 ++++++++++++--------------------------
 1 file changed, 22 insertions(+), 52 deletions(-)

Toggle diff (127 lines)diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 5a4b513f1a..8d57f4f8d0 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -535,24 +535,11 @@ text or blue underlined text, on ANSI terminals.")
           "0ix3djcf84kk53h6fac73n7jc614745n7kbmikxwi3s73b6vzgsr"))))
     (build-system cargo-build-system)
     (arguments
-     `(;#:skip-build? #t
-       #:cargo-inputs
+     `(#:cargo-inputs
        (("rust-bindgen" ,rust-bindgen-0.51)
-        ("rust-metadeps" ,rust-metadeps-1.1))
-       #:phases
-       (modify-phases %standard-phases
-         (add-after 'unpack 'set-environmental-variable
-           (lambda* (#:key inputs #:allow-other-keys)
-             (let ((clang (assoc-ref inputs "libclang")))
-               (setenv "LIBCLANG_PATH"
-                       (string-append clang "/lib")))
-             #t)))))
-    (native-inputs
-     `(("pkg-config" ,pkg-config)))
+        ("rust-metadeps" ,rust-metadeps-1.1))))
     (inputs
-     `(("libaom" ,libaom)
-       ("libclang" ,clang)
-       ("llvm" ,llvm)))
+     `(("libaom" ,libaom)))
     (home-page "https://github.com/rust-av/aom-rs")
     (synopsis "FFI bindings to aom")
     (description "This package provides FFI bindings to aom.")
@@ -1338,17 +1325,7 @@ that uses Serde for transforming structs into bytes and vice versa!")
        #:cargo-development-inputs
        (("rust-clap" ,rust-clap-2)
         ("rust-diff" ,rust-diff-0.1)
-        ("rust-shlex" ,rust-shlex-0.1))
-       #:phases
-       (modify-phases %standard-phases
-         (add-after 'unpack 'set-environmental-variable
-           (lambda* (#:key inputs #:allow-other-keys)
-             (let ((clang (assoc-ref inputs "libclang")))
-               (setenv "LIBCLANG_PATH"
-                       (string-append clang "/lib")))
-             #t)))))
-    (inputs
-     `(("libclang" ,clang)))
+        ("rust-shlex" ,rust-shlex-0.1))))
     (home-page "https://rust-lang.github.io/rust-bindgen/")
     (synopsis
      "Automatically generates Rust FFI bindings to C and C++ libraries")
@@ -1391,8 +1368,7 @@ that uses Serde for transforming structs into bytes and vice versa!")
        #:cargo-development-inputs
        (("rust-clap" ,rust-clap-2)
         ("rust-diff" ,rust-diff-0.1)
-        ("rust-shlex" ,rust-shlex-0.1))))
-    (inputs `())))
+        ("rust-shlex" ,rust-shlex-0.1))))))
 
 (define-public rust-bindgen-0.50
   (package
@@ -2968,15 +2944,14 @@ for computer graphics.")
        (("rust-glob" ,rust-glob-0.3)
         ("rust-libc" ,rust-libc-0.2)
         ("rust-libloading" ,rust-libloading-0.5))
-       #:phases
-       (modify-phases %standard-phases
-         (add-after 'unpack 'set-environmental-variable
-           (lambda* (#:key inputs #:allow-other-keys)
-             (let ((clang (assoc-ref inputs "libclang")))
-               (setenv "LIBCLANG_PATH"
-                       (string-append clang "/lib")))
-             #t)))))
-    (inputs
+       #:propagated-phases
+       ((add-after 'unpack 'bindgen:set-libclang-path
+          (lambda* (#:key inputs #:allow-other-keys)
+            (let ((clang (assoc-ref inputs "libclang")))
+              (setenv "LIBCLANG_PATH"
+                      (string-append clang "/lib")))
+            #t)))))
+    (propagated-inputs
      `(("libclang" ,clang)))
     (home-page "https://github.com/KyleMayes/clang-sys")
     (synopsis "Rust bindings for libclang")
@@ -2998,18 +2973,11 @@ for computer graphics.")
          (base32
           "1r50dwy5hj5gq07dn0qf8222d07qv0970ymx0j8n9779yayc3w3f"))))
     (arguments
-     `(#:cargo-inputs
-       (("rust-glob" ,rust-glob-0.2)
-        ("rust-libc" ,rust-libc-0.2)
-        ("rust-libloading" ,rust-libloading-0.5))
-       #:phases
-       (modify-phases %standard-phases
-         (add-after 'unpack 'set-environmental-variable
-           (lambda* (#:key inputs #:allow-other-keys)
-             (let ((clang (assoc-ref inputs "libclang")))
-               (setenv "LIBCLANG_PATH"
-                       (string-append clang "/lib")))
-             #t)))))))
+      (substitute-keyword-arguments (package-arguments rust-clang-sys-0.28)
+        (#:cargo-inputs
+         `(("rust-glob" ,rust-glob-0.2)
+           ("rust-libc" ,rust-libc-0.2)
+           ("rust-libloading" ,rust-libloading-0.5)))))))
 
 (define-public rust-clang-sys-0.23
   (package
@@ -12712,11 +12680,13 @@ for Rust structs.")
           "1hjla9ypycqw1snd2qf87cckcc0d5z5qvxpcijn5yrrs3f825cbk"))))
     (build-system cargo-build-system)
     (arguments
-     `(#:skip-build? #t
+     `(#:tests? #f ;; Test files aren't shipped to crates.io
        #:cargo-inputs
        (("rust-error-chain" ,rust-error-chain-0.10)
         ("rust-toml" ,rust-toml-0.2)
-        ("rust-pkg-config" ,rust-pkg-config-0.3))))
+        ("rust-pkg-config" ,rust-pkg-config-0.3))
+       #:cargo-development-inputs
+       (("rust-lazy-static" ,rust-lazy-static-0.2))))
     (home-page "https://github.com/joshtriplett/metadeps")
     (synopsis "Run pkg-config from declarative dependencies in Cargo.toml")
     (description "Run pkg-config from declarative dependencies in Cargo.toml.")
-- 
2.26.2

Efraim Flashner wrote on 13 Aug 2020 11:48

Re: [PATCH 0/4] build-system/cargo: Propagations across the crate closure.

Recipients:(name . Jakub K?dzio?ka)(address . kuba@kadziolka.net)(address . guix-patches@gnu.org)

Message-ID:20200813094843.GC1228@E5400

On Thu, Jun 25, 2020 at 11:25:23PM +0200, Jakub K?dzio?ka wrote:

Toggle quote (26 lines)> Due to the unusual (for Guix) compilation model used by
> cargo-build-system, any phases or inputs added by a given library crate
> need to be duplicated in all its dependents. This patchstack attempts to
> solve this by allowing to propagate inputs, native-inputs and phases
> across cargo-inputs edges in the dependency graph.
> 
> Apart from the build system work itself, I have included samples of the
> cleanup they allow. Apart from being a good example, these are the
> changes I have used to test the feature.
> 
> Jakub K?dzio?ka (4):
>   build-system/cargo: Allow propagating inputs across CARGO-INPUTS edges
>   gnu: crates-io: Use propagated-inputs and propagated-native-inputs.
>   build-system/cargo: Add a propagated-phases argument.
>   gnu: crates-io: Use propagated-phases.
> 
>  gnu/packages/crates-io.scm  | 157 +++++++++++-------------------------
>  gnu/packages/rust-apps.scm  |   9 +--
>  gnu/packages/sequoia.scm    |   3 +-
>  guix/build-system/cargo.scm |  92 +++++++++++++++------
>  4 files changed, 120 insertions(+), 141 deletions(-)
> 
> -- 
> 2.26.2
> 

I'm going to respond here so the thoughts don't get lost. While it would
take care of some of the issues we have regarding adding regular inputs
or propagated-/native- inputs, I don't think this is the way we want to
go. If we can't figure out how to re-use build artifacts then I'd rather
copy the go-build-system and install the sources into the output and use
that as the input for the next package. That would give us the
build-graph which we really want.

-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Jakub K?dzio?ka wrote on 13 Aug 2020 18:16

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . guix-patches@gnu.org)

Message-ID:20200813161638.ffgfusjplz5zcfgx@gravity

On Thu, Aug 13, 2020 at 12:48:43PM +0300, Efraim Flashner wrote:

Toggle quote (8 lines)

> I'm going to respond here so the thoughts don't get lost. While it would

> take care of some of the issues we have regarding adding regular inputs

> or propagated-/native- inputs, I don't think this is the way we want to

> go. If we can't figure out how to re-use build artifacts then I'd rather

> copy the go-build-system and install the sources into the output and use

> that as the input for the next package. That would give us the

> build-graph which we really want.

Note that this wouldn't solve all the issues, we would still need
an equivalent for propagated phases, to set any environment variables
necessary.

Moreover, note that the reason the current system was introduced in the
first place was to avoid the quadratic builds. I suppose this is less of
an issue in go-build-system due to the order-of-magnitude difference in
compiler speed on typical source code.

As for re-using build artifacts, once we figure out how to do it, we can
always revert this patch, together with the one that originally added
cargo-inputs. I don't think it's going to be any time soon, though, as
upstream doesn't support this style of building.

Regards,
Jakub K?dzio?ka

Leo Famulari wrote on 14 Aug 2020 23:26

Re: [bug#42049] [PATCH 0/4] build-system/cargo: Propagations across the crate closure.

Recipients:(name . Jakub K?dzio?ka)(address . kuba@kadziolka.net)

Message-ID:20200814212618.GB30339@jasmine.lan

On Thu, Aug 13, 2020 at 06:16:38PM +0200, Jakub K?dzio?ka wrote:

Toggle quote (5 lines)

> Moreover, note that the reason the current system was introduced in the

> first place was to avoid the quadratic builds. I suppose this is less of

> an issue in go-build-system due to the order-of-magnitude difference in

> compiler speed on typical source code.

Right. The go-build-system is far from optimally efficient but, in
practice, it's not too slow. I haven't counted but I figure that Rust
dependency graphs are an order of magnitude larger.

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 42049@debbugs.gnu.org

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date