[PATCH 0/2] Better interoperability between guix and Julia built-in package manager

OpenSubmitted by Jean-Baptiste Volatier.
Details
2 participants
  • Efraim Flashner
  • Jean-Baptiste Volatier
Owner
unassigned
Severity
normal
J
J
Jean-Baptiste Volatier wrote on 21 Oct 15:06 +0200
(address . guix-patches@gnu.org)(name . Jean-Baptiste Volatier)(address . jbv@pm.me)
20211021130648.14594-1-jbv@pm.me
With this patch, when a user install a package using Julia package manager,the package manager finds packages that were already installed in the guixprofile and use them.
First patch of the serie does the implementation, the second patch is an example usingsundials_jll package.
Sundials_jll is a julia package that wraps the sundials library, it is typically notinstalled on its own but as a dependency of DifferentialEquations.jl for example.
As DifferentialEquations.jl is not currently packaged in julia, a fallback is toinstall it with julia package manager. Unfortunately that will also installsundials_jll and downloads some .so that will fail to load on Guix because oflinker incompatilibities.
Now with the proposed patch, the user can install sundials_jll using guix,and then julia package manager will use this package instead of downloading somebinary files.
As for the patch itself, it works by following more closely the directorystructure that Julia expects in JULIA_DEPOT_PATH. Namely the packagessubfolder should not contain packages but packages/XXXX/ where XXXX isa string encoding package uuid and sha1 of package files.
$JULIA_DEPOT_PATH/packages was also used as JULIA_LOAD_PATH, this will notwork anymore as packages are behind a layer of XXXX sub directories.So instead I added a new subfolder $JULIA_DEPOT_PATH/loadpath where the packageis installed and $JULIA_DEPOT_PATH/packages/XXXX is a symlink to it.
Jean-Baptiste Volatier (2): build-system/julia: Enable Julia Pkg to find installed packages gnu: Add julia-sundials-jull
gnu/packages/julia-jll.scm | 38 +++++++++++++++++++++++++ gnu/packages/julia-xyz.scm | 2 +- gnu/packages/julia.scm | 2 +- gnu/packages/maths.scm | 24 ++++++++++++++++ guix/build/julia-build-system.scm | 46 ++++++++++++++++++++++++++++--- 5 files changed, 106 insertions(+), 6 deletions(-)
--2.33.1
J
J
Jean-Baptiste Volatier wrote on 21 Oct 15:09 +0200
(address . 51319@debbugs.gnu.org)(name . Jean-Baptiste Volatier)(address . jbv@pm.me)
20211021130952.14696-1-jbv@pm.me
With this patch, when a user install a package using Julia package manager,the package manager finds packages that were already installed in the guixprofile and use them.
First patch of the serie does the implementation, the second patch is an example usingsundials_jll package.
Sundials_jll is a julia package that wraps the sundials library, it is typically notinstalled on its own but as a dependency of DifferentialEquations.jl for example.
As DifferentialEquations.jl is not currently packaged in julia, a fallback is toinstall it with julia package manager. Unfortunately that will also installsundials_jll and downloads some .so that will fail to load on Guix because oflinker incompatilibities.
Now with the proposed patch, the user can install sundials_jll using guix,and then julia package manager will use this package instead of downloading somebinary files.
As for the patch itself, it works by following more closely the directorystructure that Julia expects in JULIA_DEPOT_PATH. Namely the packagessubfolder should not contain packages but packages/XXXX/ where XXXX isa string encoding package uuid and sha1 of package files.
$JULIA_DEPOT_PATH/packages was also used as JULIA_LOAD_PATH, this will notwork anymore as packages are behind a layer of XXXX sub directories.So instead I added a new subfolder $JULIA_DEPOT_PATH/loadpath where the packageis installed and $JULIA_DEPOT_PATH/packages/XXXX is a symlink to it.
Jean-Baptiste Volatier (2): build-system/julia: Enable Julia Pkg to find installed packages gnu: Add julia-sundials-jull
gnu/packages/julia-jll.scm | 38 +++++++++++++++++++++++++ gnu/packages/julia-xyz.scm | 2 +- gnu/packages/julia.scm | 2 +- gnu/packages/maths.scm | 24 ++++++++++++++++ guix/build/julia-build-system.scm | 46 ++++++++++++++++++++++++++++--- 5 files changed, 106 insertions(+), 6 deletions(-)
--2.33.1
J
J
Jean-Baptiste Volatier wrote on 21 Oct 15:10 +0200
[PATCH 2/2] gnu: Add julia-sundials-jull
(address . 51319@debbugs.gnu.org)(name . Jean-Baptiste Volatier)(address . jbv@pm.me)
20211021130952.14696-3-jbv@pm.me
* gnu/packages/julia-jll.scm (julia-sundials-jll): New variable.* gnu/packages/maths.scm (sundials-julia): New variable.--- gnu/packages/julia-jll.scm | 38 ++++++++++++++++++++++++++++++++++++++ gnu/packages/maths.scm | 24 ++++++++++++++++++++++++ 2 files changed, 62 insertions(+)
Toggle diff (81 lines)diff --git a/gnu/packages/julia-jll.scm b/gnu/packages/julia-jll.scmindex d3677d0a33..e1c33903dd 100644--- a/gnu/packages/julia-jll.scm+++ b/gnu/packages/julia-jll.scm@@ -2709,3 +2709,41 @@ (define-public julia-zstd-jll (synopsis "Zstd library wrappers") (description "This package provides a wrapper for the zstd library.") (license license:expat)))++(define-public julia-sundials-jll+ (package+ (name "julia-sundials-jll")+ (version "5.2.1+0")+ (source+ (origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/JuliaBinaryWrappers/Sundials_jll.jl")+ (commit (string-append "Sundials-v" version))))+ (file-name (git-file-name name version))+ (sha256+ (base32 "0cijb9frq8gj8bjpqf2lr5d0jxlj262y6h6xi4z3536dingrvffc"))))+ (build-system julia-build-system)+ (arguments+ '(#:tests? #f ; no runtests+ #:phases+ (modify-phases %standard-phases+ (add-after 'link-depot 'override-binary-path+ (lambda* (#:key inputs #:allow-other-keys)+ (map+ (lambda (wrapper)+ (substitute* wrapper+ (("global artifact_dir.*")+ (string-append+ "global artifact_dir = \""+ (assoc-ref inputs "sundials") "\"\n"))))+ ;; There's a Julia file for each platform, override them all+ (find-files "src/wrappers/" "\\.jl$")))))))+ (inputs+ `(("sundials" ,sundials-julia)))+ (propagated-inputs+ `(("julia-jllwrappers" ,julia-jllwrappers)))+ (home-page "https://github.com/JuliaBinaryWrappers/Sundials_jll.jl")+ (synopsis "SUndials library wrappers")+ (description "This package provides a wrapper for the sundials library.")+ (license license:expat)))diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scmindex 038ca5518c..54f44a3b55 100644--- a/gnu/packages/maths.scm+++ b/gnu/packages/maths.scm@@ -5943,6 +5943,30 @@ (define-public sundials-openmpi ,%openmpi-setup))))) (synopsis "SUNDIALS with OpenMPI support")))
+(define-public sundials-julia+ (package+ (inherit sundials)+ (name "sundials-julia")+ (version "5.2.0")+ (source+ (origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/LLNL/sundials.git")+ (commit (string-append "v" version))))+ (sha256+ (base32+ "08q9svjdl6fmqrwyd9p12m98n0wd2lws38yh223wyp6590zi3i49"))))+ (inputs+ `(("lapack" ,lapack)+ ,@(package-inputs sundials)))+ (arguments+ (substitute-keyword-arguments (package-arguments sundials)+ ((#:configure-flags flags '())+ `(cons* "-DLAPACK_ENABLE:BOOL=ON"+ ,flags))))+ (synopsis "SUNDIALS with lapack support as required by julia-sundials-jll")))+ (define-public combinatorial-blas (package (name "combinatorial-blas")
--2.33.1
J
J
Jean-Baptiste Volatier wrote on 21 Oct 15:09 +0200
[PATCH 1/2] build-system/julia: Enable Julia Pkg to find installed packages
(address . 51319@debbugs.gnu.org)(name . Jean-Baptiste Volatier)(address . jbv@pm.me)
20211021130952.14696-2-jbv@pm.me
* guix/build/julia-build-system.scm (link-depot): New phase.Julia built-in package manager (Pkg) looks for packages inJULIA_DEPOT_PATH/packages/PACKAGENAME/XXXX where XXXX isa string encoding package UUID and SHA1 of files.The link-depot phase creates a link at the correct locationto allow Pkg to find packages that were already installed byGuix.(%package-path): modified package path from packages/ toloadpath/(project.toml->uuid): retrive package uuid from TOML file
* gnu/packages/julia-xyz.scm: correct paths to reflectchange of %package-path
* gnu/packages/julia.scm: correct paths to reflect change of%package-path--- gnu/packages/julia-xyz.scm | 2 +- gnu/packages/julia.scm | 2 +- guix/build/julia-build-system.scm | 46 ++++++++++++++++++++++++++++--- 3 files changed, 44 insertions(+), 6 deletions(-)
Toggle diff (135 lines)diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scmindex c1ebc0fba1..1e407a2277 100644--- a/gnu/packages/julia-xyz.scm+++ b/gnu/packages/julia-xyz.scm@@ -4610,7 +4610,7 @@ (define-public julia-uris (lambda* (#:key source outputs #:allow-other-keys) (let ((out (assoc-ref outputs "out"))) (chdir- (string-append out "/share/julia/packages/URIs/test")))+ (string-append out "/share/julia/loadpath/URIs/test"))) #t))))) ;; required for tests (inputs `(("julia-json" ,julia-json)))diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scmindex 411f2e2e10..5b9bf91ed6 100644--- a/gnu/packages/julia.scm+++ b/gnu/packages/julia.scm@@ -673,7 +673,7 @@ (define-public julia (native-search-paths (list (search-path-specification (variable "JULIA_LOAD_PATH")- (files (list "share/julia/packages/")))+ (files (list "share/julia/loadpath/"))) (search-path-specification (variable "JULIA_DEPOT_PATH") (files (list "share/julia/")))))diff --git a/guix/build/julia-build-system.scm b/guix/build/julia-build-system.scmindex d74acf2a05..c7f2119974 100644--- a/guix/build/julia-build-system.scm+++ b/guix/build/julia-build-system.scm@@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2019, 2020 Nicolò Balzarotti <nicolo@nixo.xyz>+;;; Copyright © 2021 Jean-Baptiste Volatier <jbv@pm.me> ;;; ;;; This file is part of GNU Guix. ;;;@@ -20,10 +21,13 @@ (define-module (guix build julia-build-system) #:use-module ((guix build gnu-build-system) #:prefix gnu:) #:use-module (guix build utils)+ #:use-module (rnrs io ports) #:use-module (ice-9 match) #:use-module (ice-9 regex) #:use-module (ice-9 rdelim)+ #:use-module (ice-9 popen) #:export (%standard-phases+ %package-path julia-create-package-toml julia-build))
@@ -37,7 +41,7 @@ (define (invoke-julia code) (invoke "julia" "-e" code))
;; subpath where we store the package content-(define %package-path "/share/julia/packages/")+(define %package-path "/share/julia/loadpath/")
(define (project.toml->name file) "Look for Julia package name in the TOML file FILE (usually named@@ -51,6 +55,18 @@ (define (project.toml->name file) (if m (match:substring m 1) (loop (read-line in 'concat)))))))))
+(define (project.toml->uuid file)+ "Look for Julia package uuid in the TOML file FILE (usually named+Project.toml)."+ (call-with-input-file file+ (lambda (in)+ (let loop ((line (read-line in 'concat)))+ (if (eof-object? line)+ #f+ (let ((m (string-match "uuid\\s*=\\s*\"(.*)\"" line)))+ (if m (match:substring m 1)+ (loop (read-line in 'concat)))))))))+ (define* (install #:key source inputs outputs julia-package-name #:allow-other-keys) (let* ((out (assoc-ref outputs "out"))@@ -73,7 +89,7 @@ (define* (precompile #:key source inputs outputs julia-package-name (setenv "JULIA_DEPOT_PATH" builddir) ;; Add new package dir to the load path. (setenv "JULIA_LOAD_PATH"- (string-append builddir "packages/" ":"+ (string-append builddir "loadpath/" ":" (or (getenv "JULIA_LOAD_PATH") ""))) ;; Actual precompilation:@@ -97,15 +113,36 @@ (define* (check #:key tests? source inputs outputs julia-package-name (setenv "SOURCE_DATE_EPOCH" "1") (setenv "JULIA_DEPOT_PATH" builddir) (setenv "JULIA_LOAD_PATH"- (string-append builddir "packages/" ":"+ (string-append builddir "loadpath/" ":" (or (getenv "JULIA_LOAD_PATH") ""))) (setenv "HOME" "/tmp") (invoke "julia" "--depwarn=yes"- (string-append builddir "packages/"+ (string-append builddir "loadpath/" package "/test/runtests.jl")))) #t)
+(define* (link-depot #:key source inputs outputs julia-package-name+ #:allow-other-keys)+ (let* ((out (assoc-ref outputs "out"))+ (package-name (or+ julia-package-name+ (project.toml->name "Project.toml")))+ (package-dir (string-append out %package-path package-name))+ (uuid (project.toml->uuid "Project.toml"))+ (pipe (open-pipe* OPEN_READ "julia" "-e"+ (format #f "using Pkg;+println(Base.version_slug(Base.UUID(\"~a\"),+ Base.SHA1(Pkg.GitTools.tree_hash(\".\"))))" uuid package-dir)))+ (slug (string-trim-right (get-string-all pipe))))+ ;; When installing a package, julia looks first at in the JULIA_DEPOT_PATH+ ;; for a path like packages/PACKAGE/XXXX+ ;; Where XXXX is a slug encoding the package UUID and SHA1 of the files+ ;; Here we create a link with the correct path to enable julia to find the package+ (mkdir-p (string-append out "/share/julia/packages/" package-name))+ (symlink package-dir (string-append out "/share/julia/packages/" package-name "/" slug)))+ #t)+ (define (julia-create-package-toml outputs source name uuid version deps)@@ -138,6 +175,7 @@ (define %standard-phases (delete 'check) ; tests must be run after installation (replace 'install install) (add-after 'install 'precompile precompile)+ (add-after 'unpack 'link-depot link-depot) (add-after 'install 'check check) ;; TODO: In the future we could add a "system-image-generation" phase ;; where we use PackageCompiler.jl to speed up package loading times
--2.33.1
E
E
Efraim Flashner wrote on 8 Nov 08:30 +0100
(name . Jean-Baptiste Volatier)(address . jbv@pm.me)(address . 51319@debbugs.gnu.org)
YYjSF2CewB3kkAhg@3900XT
Thanks for the patch series! I'm really looking forward to this workingout in Guix, being able to use Julia to download as-yet unpackaged juliapackages will make using Julia much easier.
A couple of comments inline in the patch:
On Thu, Oct 21, 2021 at 01:09:58PM +0000, Jean-Baptiste Volatier via Guix-patches via wrote:
Toggle quote (128 lines)> * guix/build/julia-build-system.scm (link-depot): New phase.> Julia built-in package manager (Pkg) looks for packages in> JULIA_DEPOT_PATH/packages/PACKAGENAME/XXXX where XXXX is> a string encoding package UUID and SHA1 of files.> The link-depot phase creates a link at the correct location> to allow Pkg to find packages that were already installed by> Guix.> (%package-path): modified package path from packages/ to> loadpath/> (project.toml->uuid): retrive package uuid from TOML file> > * gnu/packages/julia-xyz.scm: correct paths to reflect> change of %package-path> > * gnu/packages/julia.scm: correct paths to reflect change of> %package-path> ---> gnu/packages/julia-xyz.scm | 2 +-> gnu/packages/julia.scm | 2 +-> guix/build/julia-build-system.scm | 46 ++++++++++++++++++++++++++++---> 3 files changed, 44 insertions(+), 6 deletions(-)> > diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm> index c1ebc0fba1..1e407a2277 100644> --- a/gnu/packages/julia-xyz.scm> +++ b/gnu/packages/julia-xyz.scm> @@ -4610,7 +4610,7 @@ (define-public julia-uris> (lambda* (#:key source outputs #:allow-other-keys)> (let ((out (assoc-ref outputs "out")))> (chdir> - (string-append out "/share/julia/packages/URIs/test")))> + (string-append out "/share/julia/loadpath/URIs/test")))> #t)))))> ;; required for tests> (inputs `(("julia-json" ,julia-json)))> diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm> index 411f2e2e10..5b9bf91ed6 100644> --- a/gnu/packages/julia.scm> +++ b/gnu/packages/julia.scm> @@ -673,7 +673,7 @@ (define-public julia> (native-search-paths> (list (search-path-specification> (variable "JULIA_LOAD_PATH")> - (files (list "share/julia/packages/")))> + (files (list "share/julia/loadpath/")))> (search-path-specification> (variable "JULIA_DEPOT_PATH")> (files (list "share/julia/")))))> diff --git a/guix/build/julia-build-system.scm b/guix/build/julia-build-system.scm> index d74acf2a05..c7f2119974 100644> --- a/guix/build/julia-build-system.scm> +++ b/guix/build/julia-build-system.scm> @@ -1,5 +1,6 @@> ;;; GNU Guix --- Functional package management for GNU> ;;; Copyright © 2019, 2020 Nicolò Balzarotti <nicolo@nixo.xyz>> +;;; Copyright © 2021 Jean-Baptiste Volatier <jbv@pm.me>> ;;;> ;;; This file is part of GNU Guix.> ;;;> @@ -20,10 +21,13 @@> (define-module (guix build julia-build-system)> #:use-module ((guix build gnu-build-system) #:prefix gnu:)> #:use-module (guix build utils)> + #:use-module (rnrs io ports)> #:use-module (ice-9 match)> #:use-module (ice-9 regex)> #:use-module (ice-9 rdelim)> + #:use-module (ice-9 popen)> #:export (%standard-phases> + %package-path> julia-create-package-toml> julia-build))> > @@ -37,7 +41,7 @@ (define (invoke-julia code)> (invoke "julia" "-e" code))> > ;; subpath where we store the package content> -(define %package-path "/share/julia/packages/")> +(define %package-path "/share/julia/loadpath/")> > (define (project.toml->name file)> "Look for Julia package name in the TOML file FILE (usually named> @@ -51,6 +55,18 @@ (define (project.toml->name file)> (if m (match:substring m 1)> (loop (read-line in 'concat)))))))))> > +(define (project.toml->uuid file)> + "Look for Julia package uuid in the TOML file FILE (usually named> +Project.toml)."> + (call-with-input-file file> + (lambda (in)> + (let loop ((line (read-line in 'concat)))> + (if (eof-object? line)> + #f> + (let ((m (string-match "uuid\\s*=\\s*\"(.*)\"" line)))> + (if m (match:substring m 1)> + (loop (read-line in 'concat)))))))))> +> (define* (install #:key source inputs outputs julia-package-name> #:allow-other-keys)> (let* ((out (assoc-ref outputs "out"))> @@ -73,7 +89,7 @@ (define* (precompile #:key source inputs outputs julia-package-name> (setenv "JULIA_DEPOT_PATH" builddir)> ;; Add new package dir to the load path.> (setenv "JULIA_LOAD_PATH"> - (string-append builddir "packages/" ":"> + (string-append builddir "loadpath/" ":"> (or (getenv "JULIA_LOAD_PATH")> "")))> ;; Actual precompilation:> @@ -97,15 +113,36 @@ (define* (check #:key tests? source inputs outputs julia-package-name> (setenv "SOURCE_DATE_EPOCH" "1")> (setenv "JULIA_DEPOT_PATH" builddir)> (setenv "JULIA_LOAD_PATH"> - (string-append builddir "packages/" ":"> + (string-append builddir "loadpath/" ":"> (or (getenv "JULIA_LOAD_PATH")> "")))> (setenv "HOME" "/tmp")> (invoke "julia" "--depwarn=yes"> - (string-append builddir "packages/"> + (string-append builddir "loadpath/"> package "/test/runtests.jl"))))> #t)> > +(define* (link-depot #:key source inputs outputs julia-package-name> + #:allow-other-keys)
I had to wrap this with '(when (file-exists? "Project.toml")' since wedo have some packages that pre-date the use of Project.toml. Buildingjulia-bufferedstreams will test both a package with and without aProject.toml.
Toggle quote (11 lines)> + (let* ((out (assoc-ref outputs "out"))> + (package-name (or> + julia-package-name> + (project.toml->name "Project.toml")))> + (package-dir (string-append out %package-path package-name))> + (uuid (project.toml->uuid "Project.toml"))> + (pipe (open-pipe* OPEN_READ "julia" "-e"> + (format #f "using Pkg;> +println(Base.version_slug(Base.UUID(\"~a\"),> + Base.SHA1(Pkg.GitTools.tree_hash(\".\"))))" uuid package-dir)))
Here you have a '~a' to use uuid but package-dir isn't used anywhere. Ispackage-dir not needed or is there a missing substitution?
Toggle quote (24 lines)> + (slug (string-trim-right (get-string-all pipe))))> + ;; When installing a package, julia looks first at in the JULIA_DEPOT_PATH> + ;; for a path like packages/PACKAGE/XXXX> + ;; Where XXXX is a slug encoding the package UUID and SHA1 of the files> + ;; Here we create a link with the correct path to enable julia to find the package> + (mkdir-p (string-append out "/share/julia/packages/" package-name))> + (symlink package-dir (string-append out "/share/julia/packages/" package-name "/" slug)))> + #t)> +> (define (julia-create-package-toml outputs source> name uuid version> deps)> @@ -138,6 +175,7 @@ (define %standard-phases> (delete 'check) ; tests must be run after installation> (replace 'install install)> (add-after 'install 'precompile precompile)> + (add-after 'unpack 'link-depot link-depot)> (add-after 'install 'check check)> ;; TODO: In the future we could add a "system-image-generation" phase> ;; where we use PackageCompiler.jl to speed up package loading times> --> 2.33.1>
-- Efraim Flashner <efraim@flashner.co.il> רנשלפ םירפאGPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmGI0hcACgkQQarn3Mo9g1FvIxAAh4UeF65a0A6KTwS+RNL6qlEfYbOAfd55vzqYQo+wW2hjIGK7+H6+N9SBmlsrw2nwMnjJmwnEcLrISYk5GaL2JT4oIA4ajzVOhcw3bOgbZcaTLry1loLUPawNjbHQZf6V3U6yd/nbNcMj/vkP1nwPLV42vqBuZMxhRh2ri5r1iJ/PB2xcZRO/U6NWm3h36TLqVKoa6F9xp41rYntrmqZc3Sp+MF2tUKg8/5rwQmcoCArLw/egYm2wI/BhuQmRq9TOY09doTYCeK6LEw6L3ApE3IcIQmDE7jtvLtYkl8DornJrkDcnkt8pfZ7uCJWtYLKiwT+Z915sWhkJLQwpOo2MUQvpZhZB9gU8Ix7isNhlZz0aWc7EHnbYgiybqRQsUzF1SO5vR0+i1O1DlGHvy2fv3bmwjFgB/iLg/jTDE1BvjIuTlFLl1XHTXoBqJxu0Nck+VKTXzQ0c559rKzZ1URXHDJWlWhdKMq87qOsxpm75npnOgq5mdm++bqLHCO1/3FySy1U9Cp9rrxR5AWZDUKU6G9c2WMYzyJU7YrtU21C9MKlQ9wilPP5LpGk9GjPWk/OqNabtHNnXxetJoXW1OYAKdqPyonVf/0Ztp+/lMbZdwbBeFAvMpdXLaT81fb2BryQIdqpXDiGOxG6vlwepotByDWm1Kh8oGf7c4k/e+8qTFo8==KRz2-----END PGP SIGNATURE-----

J
J
Jean-Baptiste Volatier wrote on 12 Nov 15:23 +0100
[PATCH v2 0/2] Better interoperability between guix and Julia built-in package manager
(name . Jean-Baptiste Volatier)(address . jbv@pm.me)
20211112142328.28400-1-jbv@pm.me
On Monday, November 8th, 2021 at 8:30 AM, Efraim Flashner <efraim@flashner.co.il> wrote:
Toggle quote (4 lines)> Thanks for the patch series! I'm really looking forward to this working> out in Guix, being able to use Julia to download as-yet unpackaged julia> packages will make using Julia much easier.
Me too! Then we can concentrate first on the JLL packages sincesource only packages are likely to work as is.
I forgot to mention that for this patch to work, the link-depotphase needs to be called before the source is modified.So in this new version, I have modified all packages that wheremodifiying the source after unpack to do it after link-depot instead.
Toggle quote (5 lines)> I had to wrap this with '(when (file-exists? "Project.toml")' since we> do have some packages that pre-date the use of Project.toml. Building> julia-bufferedstreams will test both a package with and without a> Project.toml.
I added a julia-package-uuid keyword argument like julia-package-namefor packages without a Project.toml. I also modified the packagesthat needed it.
Toggle quote (3 lines)> Here you have a '~a' to use uuid but package-dir isn't used anywhere. Is> package-dir not needed or is there a missing substitution?
This was some left over from a previous iteration.

Jean-Baptiste Volatier (2): build-system/julia: Enable Julia Pkg to find installed packages gnu: Add julia-sundials-jull
gnu/packages/julia-jll.scm | 176 ++++++++++++++++++------------ gnu/packages/julia-xyz.scm | 69 ++++++------ gnu/packages/julia.scm | 2 +- gnu/packages/maths.scm | 24 ++++ guix/build-system/julia.scm | 5 +- guix/build/julia-build-system.scm | 49 ++++++++- 6 files changed, 217 insertions(+), 108 deletions(-)

base-commit: 6e67c701a88064babd1aef93b58890aed2dcd0aa--2.33.1
J
J
Jean-Baptiste Volatier wrote on 12 Nov 15:23 +0100
[PATCH v2 1/2] build-system/julia: Enable Julia Pkg to find installed packages
(name . Jean-Baptiste Volatier)(address . jbv@pm.me)
20211112142328.28400-2-jbv@pm.me
* guix/build/julia-build-system.scm (link-depot): New phase.Julia built-in package manager (Pkg) looks for packages inJULIA_DEPOT_PATH/packages/PACKAGENAME/XXXX where XXXX isa string encoding package UUID and SHA1 of files.The link-depot phase creates a link at the correct locationto allow Pkg to find packages that were already installed byGuix.(%package-path): modified package path from packages/ toloadpath/(project.toml->uuid): retrive package uuid from TOML file
* gnu/packages/julia-xyz.scm: correct paths to reflectchange of %package-pathmodify order of phases as link-depot needs to run on anunmodified src directoryadd package-name-uuid keyword argument to packages withoutProject.toml file.
* gnu/packages/julia.scm: correct paths to reflect change of%package-pathmodify order of phases as link-depot needs to run on anunmodified src directory--- gnu/packages/julia-jll.scm | 138 +++++++++++++++--------------- gnu/packages/julia-xyz.scm | 69 ++++++++------- gnu/packages/julia.scm | 2 +- guix/build-system/julia.scm | 5 +- guix/build/julia-build-system.scm | 49 +++++++++-- 5 files changed, 155 insertions(+), 108 deletions(-)
Toggle diff (1088 lines)diff --git a/gnu/packages/julia-jll.scm b/gnu/packages/julia-jll.scmindex 51643819ae..d8cf5b9eaf 100644--- a/gnu/packages/julia-jll.scm+++ b/gnu/packages/julia-jll.scm@@ -77,7 +77,7 @@ (define-public julia-bzip2-jll '(#:tests? #f ; No runtests.jl #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -117,7 +117,7 @@ (define-public julia-cairo-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -163,7 +163,7 @@ (define-public julia-compilersupportlibraries-jll `(#:tests? #f ; no runtests.jl #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -205,7 +205,7 @@ (define-public julia-expat-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -243,7 +243,7 @@ (define-public julia-ffmpeg-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -292,7 +292,7 @@ (define-public julia-fontconfig-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -335,7 +335,7 @@ (define-public julia-freetype2-jll '(#:tests? #f ; No runtests.jl #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -377,7 +377,7 @@ (define-public julia-fribidi-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -413,7 +413,7 @@ (define-public julia-gettext-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -454,7 +454,7 @@ (define-public julia-glfw-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -497,7 +497,7 @@ (define-public julia-glib-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -541,7 +541,7 @@ (define-public julia-gr-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -590,7 +590,7 @@ (define-public julia-gumbo-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (let ((gumbo (string-append (assoc-ref inputs "gumbo-parser")))) (for-each@@ -630,7 +630,7 @@ (define-public julia-imagemagick-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -672,7 +672,7 @@ (define-public julia-jllwrappers '(#:tests? #f #:phases (modify-phases %standard-phases- (add-after 'unpack 'custom-override-path+ (add-after 'link-depot 'custom-override-path (lambda* (#:key inputs #:allow-other-keys) ;; Make @generate_wrapper_header take an optional argument that ;; guix packagers can pass to override the default "override"@@ -718,7 +718,7 @@ (define-public julia-jpegturbo-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -754,7 +754,7 @@ (define-public julia-lame-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -790,7 +790,7 @@ (define-public julia-libass-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -830,7 +830,7 @@ (define-public julia-libfdk-aac-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -866,7 +866,7 @@ (define-public julia-libffi-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -903,7 +903,7 @@ (define-public julia-libgcrypt-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -942,7 +942,7 @@ (define-public julia-libglvnd-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -980,7 +980,7 @@ (define-public julia-libgpg-error-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1018,7 +1018,7 @@ (define-public julia-libiconv-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1056,7 +1056,7 @@ (define-public julia-libmount-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1095,7 +1095,7 @@ (define-public julia-libpng-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1133,7 +1133,7 @@ (define-public julia-libsass-jll `(#:tests? #f ; no runtests.jl #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1175,7 +1175,7 @@ (define-public julia-libtiff-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1214,7 +1214,7 @@ (define-public julia-libuuid-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1252,7 +1252,7 @@ (define-public julia-libvorbis-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1290,7 +1290,7 @@ (define-public julia-lzo-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1328,7 +1328,7 @@ (define-public julia-mbedtls-jll '(#:tests? #f ; No runtests.jl #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1366,7 +1366,7 @@ (define-public julia-ogg-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1403,7 +1403,7 @@ (define-public julia-openspecfun-jll `(#:tests? #f ; no runtests.jl #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1446,7 +1446,7 @@ (define-public julia-openssl-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1482,7 +1482,7 @@ (define-public julia-opus-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1518,7 +1518,7 @@ (define-public julia-pcre-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1556,7 +1556,7 @@ (define-public julia-pixman-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1594,7 +1594,7 @@ (define-public julia-qt5base-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1645,7 +1645,7 @@ (define-public julia-wayland-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1684,7 +1684,7 @@ (define-public julia-wayland-protocols-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1721,7 +1721,7 @@ (define-public julia-x264-jll `(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (let ((libx264 (assoc-ref inputs "libx264"))) (map@@ -1762,7 +1762,7 @@ (define-public julia-x265-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1798,7 +1798,7 @@ (define-public julia-xkbcommon-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1838,7 +1838,7 @@ (define-public julia-xml2-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1878,7 +1878,7 @@ (define-public julia-xorg-libpthread-stubs-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1915,7 +1915,7 @@ (define-public julia-xorg-libx11-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1953,7 +1953,7 @@ (define-public julia-xorg-libxau-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -1989,7 +1989,7 @@ (define-public julia-xorg-libxcb-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2029,7 +2029,7 @@ (define-public julia-xorg-libxcursor-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2067,7 +2067,7 @@ (define-public julia-xorg-libxdmcp-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2103,7 +2103,7 @@ (define-public julia-xorg-libxext-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2140,7 +2140,7 @@ (define-public julia-xorg-libxfixes-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2177,7 +2177,7 @@ (define-public julia-xorg-libxi-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2215,7 +2215,7 @@ (define-public julia-xorg-libxinerama-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2252,7 +2252,7 @@ (define-public julia-xorg-libxkbfile-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2289,7 +2289,7 @@ (define-public julia-xorg-libxrandr-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2327,7 +2327,7 @@ (define-public julia-xorg-libxrender-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2364,7 +2364,7 @@ (define-public julia-xorg-xcb-util-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2401,7 +2401,7 @@ (define-public julia-xorg-xcb-util-image-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2438,7 +2438,7 @@ (define-public julia-xorg-xcb-util-keysyms-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2475,7 +2475,7 @@ (define-public julia-xorg-xcb-util-renderutil-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2512,7 +2512,7 @@ (define-public julia-xorg-xcb-util-wm-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2549,7 +2549,7 @@ (define-public julia-xorg-xkbcomp-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2586,7 +2586,7 @@ (define-public julia-xorg-xkeyboard-config-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2623,7 +2623,7 @@ (define-public julia-xorg-xtrans-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2659,7 +2659,7 @@ (define-public julia-xslt-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2702,7 +2702,7 @@ (define-public julia-zlib-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)@@ -2740,7 +2740,7 @@ (define-public julia-zstd-jll '(#:tests? #f ; no runtests #:phases (modify-phases %standard-phases- (add-after 'unpack 'override-binary-path+ (add-after 'link-depot 'override-binary-path (lambda* (#:key inputs #:allow-other-keys) (map (lambda (wrapper)diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scmindex 93fadf318c..73768317b8 100644--- a/gnu/packages/julia-xyz.scm+++ b/gnu/packages/julia-xyz.scm@@ -410,7 +410,8 @@ (define-public julia-bufferedstreams ;; https://travis-ci.org/BioJulia/BufferedStreams.jl/jobs/491050182 (arguments '(#:tests? #f- #:julia-package-name "BufferedStreams"))+ #:julia-package-name "BufferedStreams"+ #:julia-package-uuid "e1450e63-4bb3-523b-b2a4-4ffa8c0fd77d")) (propagated-inputs `(("julia-compat" ,julia-compat))) (home-page "https://github.com/BioJulia/BufferedStreams.jl") (synopsis "Fast composable IO streams")@@ -778,12 +779,12 @@ (define-public julia-configurations (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'fix-tests+ (add-after 'link-depot 'fix-tests (lambda _ (substitute* "test/runtests.jl" (("option.toml") "test/option.toml")) #t))- (add-after 'unpack 'dont-use-exproniconlite+ (add-after 'link-depot 'dont-use-exproniconlite (lambda _ (substitute* '("Project.toml" "src/Configurations.jl"@@ -919,7 +920,7 @@ (define-public julia-dataframes (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'skip-failing-test+ (add-after 'link-depot 'skip-failing-test (lambda _ ;; Tests with non-standard colors. (substitute* "test/show.jl"@@ -1022,7 +1023,7 @@ (define-public julia-datavalues `(#:tests? #f ; Tests need upgrading with newer Julia version. #:phases (modify-phases %standard-phases- (add-after 'unpack 'skip-known-failing-tests+ (add-after 'link-depot 'skip-known-failing-tests (lambda _ ;; See upstream report: ;; https://github.com/queryverse/DataValues.jl/issues/83@@ -1100,7 +1101,7 @@ (define-public julia-distances (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'skip-flakey-tests+ (add-after 'link-depot 'skip-flakey-tests (lambda _ ;; Some combination of these tests fail nondeterministically ;; each of the times this package is built.@@ -1166,14 +1167,14 @@ (define-public julia-documenter (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'patch-source+ (add-after 'link-depot 'patch-source (lambda* (#:key inputs #:allow-other-keys) (substitute* "src/Deps.jl" (("pip install") (string-append (assoc-ref inputs "python") "/bin/pip install"))) #t))- (add-after 'unpack 'remove-javascript-downloads+ (add-after 'link-depot 'remove-javascript-downloads (lambda _ (substitute* "src/Writers/HTMLWriter.jl" (("cdnjs.cloudflare.com") "example.com"))@@ -1352,7 +1353,7 @@ (define-public julia-dualnumbers (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'adjust-test-suite+ (add-after 'link-depot 'adjust-test-suite (lambda _ (substitute* "test/runtests.jl" ;; Seems to not play nicely with SpecialFunctions@@ -1386,7 +1387,7 @@ (define-public julia-ellipsisnotation (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'adjust-test-suite+ (add-after 'link-depot 'adjust-test-suite (lambda _ (substitute* "test/runtests.jl" ;; Seems to not play nicely with Julia-1.6.@@ -1438,7 +1439,7 @@ (define-public julia-expronicon (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'skip-network-tests+ (add-after 'link-depot 'skip-network-tests (lambda _ (substitute* "test/runtests.jl" ;; This test tries to access the Julia package registry.@@ -1517,7 +1518,7 @@ (define-public julia-fileio `(#:phases (modify-phases %standard-phases (delete 'reset-gzip-timestamps)- (add-after 'unpack 'skip-network-tests+ (add-after 'link-depot 'skip-network-tests (lambda _ ;; These tests try to download audio/video files. (substitute* "test/query.jl"@@ -1677,7 +1678,7 @@ (define-public julia-fixedpointnumbers (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'disable-failing-test+ (add-after 'link-depot 'disable-failing-test (lambda* (#:key outputs #:allow-other-keys) (substitute* "test/fixed.jl" ;; A deprecation warning is not thrown@@ -1764,7 +1765,7 @@ (define-public julia-functionwrappers (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'adjust-tests+ (add-after 'link-depot 'adjust-tests (lambda _ (substitute* "test/runtests.jl" (("testset \\\"Abstract.*" all)@@ -1817,7 +1818,7 @@ (define-public julia-fuzzycompletions (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'skip-failing-test+ (add-after 'link-depot 'skip-failing-test (lambda _ (substitute* "test/runtests.jl" ((".*RPLE.*") "")))))))@@ -1844,7 +1845,7 @@ (define-public julia-genericlinearalgebra (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'adjust-test-suite+ (add-after 'link-depot 'adjust-test-suite (lambda _ (substitute* "test/runtests.jl" ((".*lapack.*") "")))))))@@ -1875,7 +1876,7 @@ (define-public julia-genericschur (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'adjust-test-suite+ (add-after 'link-depot 'adjust-test-suite (lambda _ (substitute* "test/complex.jl" ;; expected Array{Int32,1}, got a value of type Array{Int64,1}@@ -1912,13 +1913,13 @@ (define-public julia-geometrybasics (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'remove-earcut+ (add-after 'link-depot 'remove-earcut (lambda _ (substitute* '("Project.toml" "src/GeometryBasics.jl") ((".*EarCut.*") "")) #t))- (add-after 'unpack 'skip-incompatible-test+ (add-after 'link-depot 'skip-incompatible-test (lambda _ (substitute* "test/runtests.jl" (("@testset.*MetaT and heterogeneous data.*" all)@@ -2268,7 +2269,7 @@ (define-public julia-imagemagick (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'skip-failing-test+ (add-after 'link-depot 'skip-failing-test (lambda _ ;; These tests try to download from the imagemagick.org (substitute* "test/runtests.jl"@@ -2465,7 +2466,7 @@ (define-public julia-infinity (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'remove-timezones.jl+ (add-after 'link-depot 'remove-timezones.jl (lambda _ (substitute* "test/runtests.jl" (("using TimeZones.*") "")@@ -2864,7 +2865,7 @@ (define-public julia-matrixfactorizations (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'skip-failing-test+ (add-after 'link-depot 'skip-failing-test (lambda _ ;; Tests with math functions are hard. (substitute* "test/test_ul.jl"@@ -2981,7 +2982,8 @@ (define-public julia-media (build-system julia-build-system) ;; Package without Project.toml (arguments- '(#:julia-package-name "Media"))+ '(#:julia-package-name "Media"+ #:julia-package-uuid "e89f7d12-3494-54d1-8411-f7d8b9ae1f27")) (propagated-inputs `(("julia-macrotools" ,julia-macrotools))) (home-page "https://github.com/JunoLab/Media.jl")@@ -3202,7 +3204,7 @@ (define-public julia-nnlib (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'skip-cuda-tests+ (add-after 'link-depot 'skip-cuda-tests (lambda _ (substitute* "test/runtests.jl" (("using CUDA") "")@@ -3238,7 +3240,8 @@ (define-public julia-optimtestproblems (base32 "10h47x5ws42pkqjccimaz0yxfvz41w0yazq6inamfk4lg5g2g3d9")))) (build-system julia-build-system) (arguments- `(#:julia-package-name "OptimTestProblems"))+ `(#:julia-package-name "OptimTestProblems"+ #:julia-package-uuid "cec144fc-5a64-5bc6-99fb-dde8f63e154c")) (home-page "https://github.com/JuliaNLSolvers/OptimTestProblems.jl") (synopsis "Collection of optimization test problems") (description "The purpose of this package is to provide test problems for@@ -3528,7 +3531,7 @@ (define-public julia-prettytables (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'skip-color-tests+ (add-after 'link-depot 'skip-color-tests (lambda _ (substitute* "test/text_backend.jl" ((".*colors\\.jl.*") ""))@@ -3567,7 +3570,7 @@ (define-public julia-pycall ((guix build python-build-system) #:prefix python:)) #:phases (modify-phases %standard-phases- (add-after 'unpack 'remove-conda+ (add-after 'link-depot 'remove-conda (lambda _ (substitute* "Project.toml" ((".*Conda.*") ""))@@ -3577,7 +3580,7 @@ (define-public julia-pycall (substitute* "deps/depsutils.jl" (("Conda.PYTHONDIR") "\"/\"")) #t))- (add-after 'unpack 'set-python+ (add-after 'link-depot 'set-python (lambda* (#:key inputs outputs #:allow-other-keys) (let ((python (assoc-ref inputs "python"))) (setenv "PYCALL_JL_RUNTIME_PYTHON"@@ -3690,7 +3693,7 @@ (define-public julia-quadmath (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'hardcode-libmath-location+ (add-after 'link-depot 'hardcode-libmath-location (lambda* (#:key inputs #:allow-other-keys) (let ((gcclib (assoc-ref inputs "gcc:lib"))) (substitute* "src/Quadmath.jl"@@ -4082,7 +4085,8 @@ (define-public julia-safetestsets (base32 "1fb1dfdmiw2ggx60hf70954xlps0r48fcb3k3dvxynlz7ylphp96")))) (build-system julia-build-system) (arguments- `(#:julia-package-name "SafeTestsets"))+ `(#:julia-package-name "SafeTestsets"+ #:julia-package-uuid "1bc83da4-3b8d-516f-aca4-4fe02f6d838f")) (native-inputs `(("julia-staticarrays" ,julia-staticarrays))) (home-page "https://github.com/YingboMa/SafeTestsets.jl")@@ -4308,7 +4312,7 @@ (define-public julia-stackviews (arguments `(#:phases (modify-phases %standard-phases- (add-after 'unpack 'skip-doctest+ (add-after 'link-depot 'skip-doctest (lambda _ (substitute* "test/runtests.jl" ((".*doctest.*") ""))@@ -4811,6 +4815,7 @@ (define-public julia-uris (build-system julia-build-system) (arguments '(#:julia-package-name "URIs" ;required to run tests+ #:julia-package-uuid "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4" #:phases (modify-phases %standard-phases (add-before 'check 'change-dir@@ -4818,7 +4823,7 @@ (define-public julia-uris (lambda* (#:key source outputs #:allow-other-keys) (let ((out (assoc-ref outputs "out"))) (chdir- (string-append out "/share/julia/packages/URIs/test")))+ (string-append out "/share/julia/loadpath/URIs/test"))) #t))))) ;; required for tests (inputs `(("julia-json" ,julia-json)))diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scmindex fe31e545c9..8bf2c8e25e 100644--- a/gnu/packages/julia.scm+++ b/gnu/packages/julia.scm@@ -670,7 +670,7 @@ (define-public julia (native-search-paths (list (search-path-specification (variable "JULIA_LOAD_PATH")- (files (list "share/julia/packages/")))+ (files (list "share/julia/loadpath/"))) (search-path-specification (variable "JULIA_DEPOT_PATH") (files (list "share/julia/")))))diff --git a/guix/build-system/julia.scm b/guix/build-system/julia.scmindex 63cb7cd864..8ceac090c7 100644--- a/guix/build-system/julia.scm+++ b/guix/build-system/julia.scm@@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2019 Nicolò Balzarotti <nicolo@nixo.xyz>+;;; Copyright © 2021 Jean-Baptiste Volatier <jbv@pm.me> ;;; ;;; This file is part of GNU Guix. ;;;@@ -83,6 +84,7 @@ (define* (julia-build store name inputs (system (%current-system)) (guile #f) (julia-package-name #f)+ (julia-package-uuid #f) (imported-modules %julia-build-system-modules) (modules '((guix build julia-build-system) (guix build utils))))@@ -105,7 +107,8 @@ (define builder #:search-paths ',(map search-path-specification->sexp search-paths) #:inputs %build-inputs- #:julia-package-name ,julia-package-name)))+ #:julia-package-name ,julia-package-name+ #:julia-package-uuid ,julia-package-uuid)))
(define guile-for-build (match guilediff --git a/guix/build/julia-build-system.scm b/guix/build/julia-build-system.scmindex d74acf2a05..a23f241ba6 100644--- a/guix/build/julia-build-system.scm+++ b/guix/build/julia-build-system.scm@@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2019, 2020 Nicolò Balzarotti <nicolo@nixo.xyz>+;;; Copyright © 2021 Jean-Baptiste Volatier <jbv@pm.me> ;;; ;;; This file is part of GNU Guix. ;;;@@ -20,10 +21,13 @@ (define-module (guix build julia-build-system) #:use-module ((guix build gnu-build-system) #:prefix gnu:) #:use-module (guix build utils)+ #:use-module (rnrs io ports) #:use-module (ice-9 match) #:use-module (ice-9 regex) #:use-module (ice-9 rdelim)+ #:use-module (ice-9 popen) #:export (%standard-phases+ %package-path julia-create-package-toml julia-build))
@@ -37,7 +41,7 @@ (define (invoke-julia code) (invoke "julia" "-e" code))
;; subpath where we store the package content-(define %package-path "/share/julia/packages/")+(define %package-path "/share/julia/loadpath/")
(define (project.toml->name file) "Look for Julia package name in the TOML file FILE (usually named@@ -51,6 +55,18 @@ (define (project.toml->name file) (if m (match:substring m 1) (loop (read-line in 'concat)))))))))
+(define (project.toml->uuid file)+ "Look for Julia package uuid in the TOML file FILE (usually named+Project.toml)."+ (call-with-input-file file+ (lambda (in)+ (let loop ((line (read-line in 'concat)))+ (if (eof-object? line)+ #f+ (let ((m (string-match "uuid\\s*=\\s*\"(.*)\"" line)))+ (if m (match:substring m 1)+ (loop (read-line in 'concat)))))))))+ (define* (install #:key source inputs outputs julia-package-name #:allow-other-keys) (let* ((out (assoc-ref outputs "out"))@@ -73,7 +89,7 @@ (define* (precompile #:key source inputs outputs julia-package-name (setenv "JULIA_DEPOT_PATH" builddir) ;; Add new package dir to the load path. (setenv "JULIA_LOAD_PATH"- (string-append builddir "packages/" ":"+ (string-append builddir "loadpath/" ":" (or (getenv "JULIA_LOAD_PATH") ""))) ;; Actual precompilation:@@ -97,15 +113,36 @@ (define* (check #:key tests? source inputs outputs julia-package-name (setenv "SOURCE_DATE_EPOCH" "1") (setenv "JULIA_DEPOT_PATH" builddir) (setenv "JULIA_LOAD_PATH"- (string-append builddir "packages/" ":"+ (string-append builddir "loadpath/" ":" (or (getenv "JULIA_LOAD_PATH") ""))) (setenv "HOME" "/tmp") (invoke "julia" "--depwarn=yes"- (string-append builddir "packages/"+ (string-append builddir "loadpath/" package "/test/runtests.jl")))) #t)
+(define* (link-depot #:key source inputs outputs julia-package-name julia-package-uuid+ #:allow-other-keys)+ (let* ((out (assoc-ref outputs "out"))+ (package-name (or+ julia-package-name+ (project.toml->name "Project.toml")))+ (package-dir (string-append out %package-path package-name))+ (uuid (or julia-package-uuid (project.toml->uuid "Project.toml")))+ (pipe (open-pipe* OPEN_READ "julia" "-e"+ (format #f "using Pkg;+println(Base.version_slug(Base.UUID(\"~a\"),+ Base.SHA1(Pkg.GitTools.tree_hash(\".\"))))" uuid)))+ (slug (string-trim-right (get-string-all pipe))))+ ;; When installing a package, julia looks first at in the JULIA_DEPOT_PATH+ ;; for a path like packages/PACKAGE/XXXX+ ;; Where XXXX is a slug encoding the package UUID and SHA1 of the files+ ;; Here we create a link with the correct path to enable julia to find the package+ (mkdir-p (string-append out "/share/julia/packages/" package-name))+ (symlink package-dir (string-append out "/share/julia/packages/" package-name "/" slug)))+ #t)+ (define (julia-create-package-toml outputs source name uuid version deps)@@ -138,6 +175,7 @@ (define %standard-phases (delete 'check) ; tests must be run after installation (replace 'install install) (add-after 'install 'precompile precompile)+ (add-after 'unpack 'link-depot link-depot) (add-after 'install 'check check) ;; TODO: In the future we could add a "system-image-generation" phase ;; where we use PackageCompiler.jl to speed up package loading times@@ -146,11 +184,12 @@ (define %standard-phases (delete 'patch-usr-bin-file) (delete 'build)))
-(define* (julia-build #:key inputs julia-package-name+(define* (julia-build #:key inputs julia-package-name julia-package-uuid (phases %standard-phases) #:allow-other-keys #:rest args) "Build the given Julia package, applying all of PHASES in order." (apply gnu:gnu-build #:inputs inputs #:phases phases #:julia-package-name julia-package-name+ #:julia-package-uuid julia-package-uuid args))
--2.33.1
J
J
Jean-Baptiste Volatier wrote on 12 Nov 15:24 +0100
[PATCH v2 2/2] gnu: Add julia-sundials-jull
(name . Jean-Baptiste Volatier)(address . jbv@pm.me)
20211112142328.28400-3-jbv@pm.me
* gnu/packages/julia-jll.scm (julia-sundials-jll): New variable.* gnu/packages/maths.scm (sundials-julia): New variable.--- gnu/packages/julia-jll.scm | 38 ++++++++++++++++++++++++++++++++++++++ gnu/packages/maths.scm | 24 ++++++++++++++++++++++++ 2 files changed, 62 insertions(+)
Toggle diff (81 lines)diff --git a/gnu/packages/julia-jll.scm b/gnu/packages/julia-jll.scmindex d8cf5b9eaf..097d2f1123 100644--- a/gnu/packages/julia-jll.scm+++ b/gnu/packages/julia-jll.scm@@ -2759,3 +2759,41 @@ (define-public julia-zstd-jll (synopsis "Zstd library wrappers") (description "This package provides a wrapper for the zstd library.") (license license:expat)))++(define-public julia-sundials-jll+ (package+ (name "julia-sundials-jll")+ (version "5.2.1+0")+ (source+ (origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/JuliaBinaryWrappers/Sundials_jll.jl")+ (commit (string-append "Sundials-v" version))))+ (file-name (git-file-name name version))+ (sha256+ (base32 "0cijb9frq8gj8bjpqf2lr5d0jxlj262y6h6xi4z3536dingrvffc"))))+ (build-system julia-build-system)+ (arguments+ '(#:tests? #f ; no runtests+ #:phases+ (modify-phases %standard-phases+ (add-after 'link-depot 'override-binary-path+ (lambda* (#:key inputs #:allow-other-keys)+ (map+ (lambda (wrapper)+ (substitute* wrapper+ (("global artifact_dir.*")+ (string-append+ "global artifact_dir = \""+ (assoc-ref inputs "sundials") "\"\n"))))+ ;; There's a Julia file for each platform, override them all+ (find-files "src/wrappers/" "\\.jl$")))))))+ (inputs+ `(("sundials" ,sundials-julia)))+ (propagated-inputs+ `(("julia-jllwrappers" ,julia-jllwrappers)))+ (home-page "https://github.com/JuliaBinaryWrappers/Sundials_jll.jl")+ (synopsis "SUndials library wrappers")+ (description "This package provides a wrapper for the sundials library.")+ (license license:expat)))diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scmindex 5152fe6942..eba45cd6b6 100644--- a/gnu/packages/maths.scm+++ b/gnu/packages/maths.scm@@ -6159,6 +6159,30 @@ (define-public sundials-openmpi ,%openmpi-setup))))) (synopsis "SUNDIALS with OpenMPI support")))
+(define-public sundials-julia+ (package+ (inherit sundials)+ (name "sundials-julia")+ (version "5.2.0")+ (source+ (origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/LLNL/sundials.git")+ (commit (string-append "v" version))))+ (sha256+ (base32+ "08q9svjdl6fmqrwyd9p12m98n0wd2lws38yh223wyp6590zi3i49"))))+ (inputs+ `(("lapack" ,lapack)+ ,@(package-inputs sundials)))+ (arguments+ (substitute-keyword-arguments (package-arguments sundials)+ ((#:configure-flags flags '())+ `(cons* "-DLAPACK_ENABLE:BOOL=ON"+ ,flags))))+ (synopsis "SUNDIALS with lapack support as required by julia-sundials-jll")))+ (define-public combinatorial-blas (package (name "combinatorial-blas")
--2.33.1
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send email to 51319@debbugs.gnu.org