Julia keeps build directory in the output
(name . Bug Guix)(address . bug-guix@gnu.org)(name . Nicolò Balzarotti)(address . anothersms@gmail.com)
864kfh48nh.fsf@gmail.com
Hi,
Packaging Julia stuff, I note something as a bug, I guess. Julia seems
to keep references to the build directory in the output.
Toggle snippet (15 lines)
$ find $(guix build julia --no-grafts) -type f \
-exec grep '/tmp/guix-build-julia-1.5.3.drv-0' {} \;
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/lib/libjulia.so.1.5 matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/lib/julia/sys.so matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/lib/julia/libccalltest.so.debug matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/lib/julia/libllvmcalltest.so matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/bin/.julia-real matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/share/julia/base.cache matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/share/julia/test/depot/compiled/v1.5/Bar/HXSAn_w3IH9.ji matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/share/julia/test/depot/compiled/v1.5/Foo/MYb1d_w3IH9.ji matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/share/julia/test/depot/compiled/v1.5/Foo/TeeT6_w3IH9.ji matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/share/julia/test/depot/compiled/v1.5/Qux/YFfiR_w3IH9.ji matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/share/julia/test/depot/compiled/v1.5/Baz/rONVA_w3IH9.ji matches
The consequence is ’dlopen’ is broken and some Julia packages cannot be
pre-compiled. Here, an example using the Julia package manager:
Toggle snippet (32 lines)
$ rm -fr ~/.julia # for the sake of the illustration ;-)
$ cat /tmp/foo.jl
using Pkg
Pkg.add("GZip")
println("# => Now let pre-compile")
using GZip
exit()
$ guix environment --ad-hoc julia -- julia --load /tmp/foo.jl
Installing known registries into `~/.julia`
######################################################################### 100.0%
Added registry `General` to `~/.julia/registries/General`
Resolving package versions...
Installed GZip ? v0.5.1
Updating `~/.julia/environments/v1.5/Project.toml`
[92fee26a] + GZip v0.5.1
Updating `~/.julia/environments/v1.5/Manifest.toml`
[92fee26a] + GZip v0.5.1
[8f399da3] + Libdl
# => Now let pre-compile
ERROR: LoadError: LoadError: could not load library "libz"
libz.so: cannot open shared object file: No such file or directory
Stacktrace:
[...]
in expression starting at /home/simon/.julia/packages/GZip/JNmGn/src/zlib_h.jl:13
in expression starting at /home/simon/.julia/packages/GZip/JNmGn/src/GZip.jl:73
ERROR: LoadError: Failed to precompile GZip [92fee26a-97fe-5a0c-ad85-20a5f3185b63] to /home/simon/.julia/compiled/v1.5/GZip/s2LKY_jQTtL.ji.
…
Let tweak the GZip ’ccall’ and pre-compile again to show that the
build-directory is contained in the output, and the Julia internal trace
raises it.
Toggle snippet (30 lines)
$ diff ~/.julia/packages/GZip/JNmGn/src/GZip.jl{,.orig}
75c75
< const GZLIB_VERSION = unsafe_string(ccall(:zlibVersion, Ptr{UInt8}, ()))
---
> const GZLIB_VERSION = unsafe_string(ccall((:zlibVersion, GZip._zlib), Ptr{UInt8}, ()))
$ diff ~/.julia/packages/GZip/JNmGn/src/zlib_h.jl{,.orig}
13c13
< zlib_version = unsafe_string(ccall(:zlibVersion, Ptr{UInt8}, ()))
---
> zlib_version = unsafe_string(ccall((:zlibVersion, _zlib), Ptr{UInt8}, ()))
84c84
< const zlib_compile_flags = ccall(:zlibCompileFlags, UInt, ())
---
> const zlib_compile_flags = ccall((:zlibCompileFlags, _zlib), UInt, ())
$ guix environment --ad-hoc julia -- julia -e 'using GZip'
ERROR: LoadError: could not load library "libz"
libz.so: cannot open shared object file: No such file or directory
Stacktrace:
[1] dlopen(::String, ::UInt32; throw_error::Bool) at /tmp/guix-build-julia-1.5.3.drv-0/julia-1.5.3/usr/share/julia/stdlib/v1.5/Libdl/src/Libdl.jl:109
[2] dlopen at /tmp/guix-build-julia-1.5.3.drv-0/julia-1.5.3/usr/share/julia/stdlib/v1.5/Libdl/src/Libdl.jl:109 [inlined] (repeats 2 times)
[...]
in expression starting at /home/simon/.julia/packages/GZip/JNmGn/src/GZip.jl:206
…
I have tried to tweak LD_LIBRARY_PATH in the julia definition and
rebuild it. Still the same error. Note building the julia package
takes ~2h on my machine, so the trial-error takes ages. ;-)
Another illustration:
Toggle snippet (82 lines)
$ cat /tmp/bar/pkg.scm
(define-module (pkg)
#:use-module (guix packages)
#:use-module (guix git-download)
#:use-module (guix build-system julia)
#:use-module ((guix licenses) #:prefix license:))
(define-public julia-gzip
(package
(name "julia-gzip")
(version "0.5.1")
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/JuliaIO/GZip.jl")
(commit (string-append "v" version))))
(file-name (git-file-name name version))
(sha256
(base32 "06bx3l1nsvdswgl344347cjh6ffd2r1hw3lz4h0aicayc566717f"))))
(build-system julia-build-system)
(home-page "https://github.com/JuliaIO/GZip.jl")
(synopsis "TODO")
(description "TODO")
(license license:expat)))
$ guix build -L /tmp/bar/ julia-gzip
The following derivation will be built:
/gnu/store/2wiyhvqrhq8bbiw34gyxclv6zzf98vy3-julia-gzip-0.5.1.drv
building /gnu/store/2wiyhvqrhq8bbiw34gyxclv6zzf98vy3-julia-gzip-0.5.1.drv...
starting phase `set-SOURCE-DATE-EPOCH'
phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds
starting phase `set-paths'
environment variable `PATH' set to `/gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/bin:/gnu/store/v6f44zccwh9z5zk3pjlywjybbi8n2hjh-tar-1.32/bin:/gnu/store/ncydgq2znms5n1d2k5yqshhf58nsixwv-gzip-1.10/bin:/gnu/store/i8h2pcxqdq07ijm3ibkka8f4smn1w48v-bzip2-1.0.8/bin:/gnu/store/9860f1abqj8wjjnwl8a9v54pdcc3bhgf-xz-5.2.4/bin:/gnu/store/60g7r3l01fd7c58yjbm6krgcwj1jkpwg-file-5.38/bin:/gnu/store/n4n560pfvvw50a9369axw5vj5rrqfj1n-diffutils-3.7/bin:/gnu/store/cd5qf3kcnlq35p9k392pjdpdzpsnds70-patch-2.7.6/bin:/gnu/store/hic7snhayfl7m6cpfqqr73nmm19bpqkg-findutils-4.7.0/bin:/gnu/store/swqdvwri9dbv6zssg6v0by7l05hd6wxp-gawk-5.0.1/bin:/gnu/store/ishk7fswcs4gkwcp8mh788z4mvvl9bxh-sed-4.8/bin:/gnu/store/bhs4rj58v8j1narb2454raan2ps38xd8-grep-3.4/bin:/gnu/store/57xj5gcy1jbl9ai2lnrqnpr0dald9i65-coreutils-8.32/bin:/gnu/store/hm40bxnv8jxmbc1lpb7zfimii4xm9m81-make-4.3/bin:/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin:/gnu/store/mpa04aq8lblbcviyxywxcsb1zbi0mf39-ld-wrapper-0/bin:/gnu/store/m1z7cdbqsqyp9xnjw5cvlb4a7gkcg3m4-binutils-2.34/bin:/gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1y17lk-gcc-7.5.0/bin:/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/bin:/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/sbin'
environment variable `JULIA_LOAD_PATH' unset
environment variable `JULIA_DEPOT_PATH' set to `/gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/share/julia/'
environment variable `BASH_LOADABLES_PATH' unset
environment variable `C_INCLUDE_PATH' set to `/gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/include:/gnu/store/i8h2pcxqdq07ijm3ibkka8f4smn1w48v-bzip2-1.0.8/include:/gnu/store/9860f1abqj8wjjnwl8a9v54pdcc3bhgf-xz-5.2.4/include:/gnu/store/60g7r3l01fd7c58yjbm6krgcwj1jkpwg-file-5.38/include:/gnu/store/swqdvwri9dbv6zssg6v0by7l05hd6wxp-gawk-5.0.1/include:/gnu/store/hm40bxnv8jxmbc1lpb7zfimii4xm9m81-make-4.3/include:/gnu/store/m1z7cdbqsqyp9xnjw5cvlb4a7gkcg3m4-binutils-2.34/include:/gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1y17lk-gcc-7.5.0/include:/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/include:/gnu/store/gfapkk5c6hvl1d94m4sqnhn7f9l5gqyh-linux-libre-headers-5.4.20/include'
environment variable `CPLUS_INCLUDE_PATH' set to `/gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/include:/gnu/store/i8h2pcxqdq07ijm3ibkka8f4smn1w48v-bzip2-1.0.8/include:/gnu/store/9860f1abqj8wjjnwl8a9v54pdcc3bhgf-xz-5.2.4/include:/gnu/store/60g7r3l01fd7c58yjbm6krgcwj1jkpwg-file-5.38/include:/gnu/store/swqdvwri9dbv6zssg6v0by7l05hd6wxp-gawk-5.0.1/include:/gnu/store/hm40bxnv8jxmbc1lpb7zfimii4xm9m81-make-4.3/include:/gnu/store/m1z7cdbqsqyp9xnjw5cvlb4a7gkcg3m4-binutils-2.34/include:/gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1y17lk-gcc-7.5.0/include/c++:/gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1y17lk-gcc-7.5.0/include:/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/include:/gnu/store/gfapkk5c6hvl1d94m4sqnhn7f9l5gqyh-linux-libre-headers-5.4.20/include'
environment variable `LIBRARY_PATH' set to `/gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/lib:/gnu/store/i8h2pcxqdq07ijm3ibkka8f4smn1w48v-bzip2-1.0.8/lib:/gnu/store/9860f1abqj8wjjnwl8a9v54pdcc3bhgf-xz-5.2.4/lib:/gnu/store/60g7r3l01fd7c58yjbm6krgcwj1jkpwg-file-5.38/lib:/gnu/store/swqdvwri9dbv6zssg6v0by7l05hd6wxp-gawk-5.0.1/lib:/gnu/store/m1z7cdbqsqyp9xnjw5cvlb4a7gkcg3m4-binutils-2.34/lib:/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib:/gnu/store/s3dcqzwqaakv1yx37by9chksdbkgih17-glibc-2.31-static/lib:/gnu/store/hwcky7446s952w0mwchhmm211ll07zrq-glibc-utf8-locales-2.31/lib'
environment variable `GUIX_LOCPATH' set to `/gnu/store/hwcky7446s952w0mwchhmm211ll07zrq-glibc-utf8-locales-2.31/lib/locale'
phase `set-paths' succeeded after 0.0 seconds
starting phase `install-locale'
using 'en_US.utf8' locale for category "LC_ALL"
phase `install-locale' succeeded after 0.0 seconds
starting phase `unpack'
[...]
phase `unpack' succeeded after 0.0 seconds
starting phase `patch-source-shebangs'
phase `patch-source-shebangs' succeeded after 0.0 seconds
starting phase `patch-generated-file-shebangs'
phase `patch-generated-file-shebangs' succeeded after 0.0 seconds
starting phase `install'
[...]
phase `install' succeeded after 0.0 seconds
starting phase `check'
ERROR: LoadError: LoadError: could not load library "libz"
libz.so: cannot open shared object file: No such file or directory
Stacktrace:
[...]
in expression starting at /gnu/store/dak0gnj5kqdma55xy279345qzsal9zdq-julia-gzip-0.5.1/share/julia/packages/GZip/src/zlib_h.jl:13
in expression starting at /gnu/store/dak0gnj5kqdma55xy279345qzsal9zdq-julia-gzip-0.5.1/share/julia/packages/GZip/src/GZip.jl:73
ERROR: LoadError: Failed to precompile GZip [92fee26a-97fe-5a0c-ad85-20a5f3185b63] to /tmp/.julia/compiled/v1.5/GZip/s2LKY_y94HW.ji.
Stacktrace:
[...]
in expression starting at /gnu/store/dak0gnj5kqdma55xy279345qzsal9zdq-julia-gzip-0.5.1/share/julia/packages/GZip/test/runtests.jl:1
command "julia" "--depwarn=yes" "/gnu/store/dak0gnj5kqdma55xy279345qzsal9zdq-julia-gzip-0.5.1/share/julia/packages/GZip/test/runtests.jl" failed with status 1
builder for `/gnu/store/2wiyhvqrhq8bbiw34gyxclv6zzf98vy3-julia-gzip-0.5.1.drv' failed with exit code 1
build of /gnu/store/2wiyhvqrhq8bbiw34gyxclv6zzf98vy3-julia-gzip-0.5.1.drv failed
View build log at '/var/log/guix/drvs/2w/iyhvqrhq8bbiw34gyxclv6zzf98vy3-julia-gzip-0.5.1.drv.bz2'.
guix build: error: build of `/gnu/store/2wiyhvqrhq8bbiw34gyxclv6zzf98vy3-julia-gzip-0.5.1.drv' failed
Cheers,
simon