Julia is very slow

Done

Details

6 participants

Cayetano Santos
Efraim Flashner
Ludovic Courtès
Ludovic Courtès
Tobias Geerinckx-Rice
Simon Tournier

Owner: unassigned

Submitted by: Cayetano Santos

Severity: normal

Cayetano Santos wrote on 9 Jun 2023 23:42

Recipients:(address . bug-guix@gnu.org)

Message-ID:87legsl3cm.fsf@inventati.org

Hi guix,

I just noticed that the following line

julia -e 'using Pkg; Pkg.add("BenchmarkTools"); using BenchmarkTools; N = 1000; A = rand(N, N); B = rand(N, N); @btime $A*$B'

gives around 530 ms in my laptop when using guix provided julia. Same

behavior when running within a guix container.

This very same code, using the binary one may download from the julia

site gives 15 ms.

I’m using here an up to date guix. As a reference, julia binary

provided by archlinux takes also 15 ms.

Thanks,

Cayetano Santos

Ludovic Courtès wrote on 14 Jun 2023 18:00

Recipients:(name . Cayetano Santos)(address . csantosb@inventati.org)

Message-ID:87jzw6rq48.fsf@gnu.org

Hi Cayetano,

Cc’in Efraim, Simon, and Nicolas who’ve looked into Julia packaging in

the past. Hopefully we can get inspiration from Arch’s build recipe!

Thanks,

Ludo’.

Cayetano Santos <csantosb@inventati.org> skribis:

Toggle quote (18 lines)> Hi guix,
>
>   I just noticed that the following line
>
>     julia -e 'using Pkg; Pkg.add("BenchmarkTools"); using BenchmarkTools; N = 1000; A = rand(N, N); B = rand(N, N); @btime $A*$B'
>
>   gives around 530 ms in my laptop when using guix provided julia. Same
>   behavior when running within a guix container.
>
>   This very same code, using the binary one may download from the julia
>   site gives 15 ms.
>
>   I’m using here an up to date guix. As a reference, julia binary
>   provided by archlinux takes also 15 ms.
>
> Thanks,
>
> Cayetano Santos

Simon Tournier wrote on 19 Jun 2023 18:13

Recipients:

Message-ID:871qi7a0qr.fsf@gmail.com

Hi,

On Wed, 14 Jun 2023 at 18:00, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (3 lines)

> Cc’in Efraim, Simon, and Nicolas who’ve looked into Julia packaging in

> the past. Hopefully we can get inspiration from Arch’s build recipe!

Hum, the difference seems:

USE_SYSTEM_DSFMT=0

USE_SYSTEM_LIBUV=0

which are set to 1 in our Guix recipe; I guess it does not come from

that. And we link against OpenBLAS and they link against some NETLIB; I

guess it does not come from that. Well, the other difference could be

“make release” that we do not run. Maybe?

Somehow, it seems from the ability to exploit the multicore, IIUC.

Using the binary generated by upstream:

Toggle snippet (17 lines)$ ldd julia-1.9.1/bin/julia 
	linux-vdso.so.1 (0x00007fffd83f1000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f08fb274000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f08fb251000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f08fb05f000)
	libjulia.so.1 => /tmp/julia-1.9.1/bin/../lib/libjulia.so.1 (0x00007f08fb03c000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f08fb28e000)

$ ldd julia-1.6.7/bin/julia 
	linux-vdso.so.1 (0x00007fffcdbd7000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f822423f000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f822421c000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f822402a000)
	libjulia.so.1 => /tmp/julia-1.6.7/bin/../lib/libjulia.so.1 (0x00007f8223e04000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f8224259000)

And please note it also contain all these binaries:

Toggle snippet (30 lines)libamd.so -> libamd.so.2.4.6
libamd.so.2 -> libamd.so.2.4.6
libamd.so.2.4.6
libatomic.so -> libatomic.so.1.2.0
libatomic.so.1 -> libatomic.so.1.2.0
libatomic.so.1.2.0
libblastrampoline.so -> libblastrampoline.so.5
libblastrampoline.so.5
libblastrampoline.so.5.4.0 -> libblastrampoline.so.5

[...]

libsuitesparseconfig.so -> libsuitesparseconfig.so.5.10.1
libsuitesparseconfig.so.5 -> libsuitesparseconfig.so.5.10.1
libsuitesparseconfig.so.5.10.1
libumfpack.so -> libumfpack.so.5.7.9
libumfpack.so.5 -> libumfpack.so.5.7.9
libumfpack.so.5.7.9
libunwind.so -> libunwind.so.8.0.1
libunwind.so.8 -> libunwind.so.8.0.1
libunwind.so.8.0.1
libuv.so -> libuv.so.2.0.0
libuv.so.2 -> libuv.so.2.0.0
libuv.so.2.0.0
libz.so -> libz.so.1
libz.so.1 -> libz.so.1.2.13
libz.so.1.2.13
sys.so

I get these time:

+ ~7ms v1.9.1

+ ~18ms v1.6.7

compared to ~500ms of v1.8.3 provided by Guix.

I guess the issue is about “threading”. Most of the time is about

’futex’ in the Guix version.

Toggle snippet (37 lines)$ head v1.9.1.txt 
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 83.48    5.682198           6    926675           sched_yield
 14.87    1.011882         987      1025           futex
  0.42    0.028269           2     12308           rt_sigprocmask
  0.35    0.023628           9      2592           madvise
  0.18    0.012532        6266         2           wait4
  0.18    0.012353          10      1227           epoll_wait
  0.17    0.011707           2      5015        13 read
  0.09    0.006235          13       448           brk

$ head v1.6.7.txt
245193 ????( <detached ...>
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 86.41    5.870043           5   1008236           sched_yield
 12.85    0.872865         467      1869           futex
  0.26    0.017538           3      5165           madvise
  0.09    0.006171           2      2173        12 read
  0.07    0.004486          13       321           brk
  0.06    0.004242           2      1772           rt_sigprocmask
  0.05    0.003197           2      1554       456 statx

$ head vguix.txt
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 88.96    7.843293        4621      1697         2 futex
  6.88    0.606245          31     19080           sched_yield
  1.74    0.153092      153092         1         1 rt_sigtimedwait
  0.48    0.041975           1     26317         1 read
  0.36    0.032148           1     27602           rt_sigprocmask
  0.34    0.030236       10078         3         1 wait4
  0.33    0.028833           1     20780           mincore
  0.33    0.028801           1     22424           write

Hum, I do not know… It needs some investigations.

Thanks for the report.

Cheers,

simon

Ludovic Courtès wrote on 21 Jun 2023 16:36

Recipients:(name . Simon Tournier)(address . zimon.toutoune@gmail.com)

Message-ID:87pm5o7uh9.fsf@gnu.org

Hey!

The benchmark you posted, Cayetano, is:

julia -e 'using Pkg; Pkg.add("BenchmarkTools"); using BenchmarkTools; N = 1000; A = rand(N, N); B = rand(N, N); @btime $A*$B'

This is a matrix multiplication that gets delegated to the underlying

BLAS right. Running it under ‘perf record’ confirms it:

Toggle snippet (8 lines)

Samples: 139K of event 'cycles:u', Event count (approx.): 99624880590

Overhead Command Shared Object Symbol

35.27% .julia-real libblas.so.3.9.0 [.] dgemm_

3.99% .julia-real libjulia-internal.so.1.8 [.] gc_mark_loop

2.60% .julia-real libjulia-internal.so.1.8 [.] apply_cl

1.06% .julia-real libjulia-internal.so.1.8 [.] jl_get_binding_

We’re using libblas.so (presumably from the ‘lapack’ package) and not

OpenBLAS, so no wonder it’s slow.

Could it be that:

"LIBBLAS=-lopenblas"

"LIBBLASNAME=libopenblas"

is ineffective? I think we have a lead!

Ludo’.

Cayetano Santos wrote on 21 Jun 2023 22:39

Recipients:(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)

Message-ID:87ttv0mth3.fsf@inventati.org

Toggle quote (28 lines)>mer. 21 juin 2023 at 16:36, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

> Hey!
>
> The benchmark you posted, Cayetano, is:
>
>   julia -e 'using Pkg; Pkg.add("BenchmarkTools"); using BenchmarkTools; N = 1000; A = rand(N, N); B = rand(N, N); @btime $A*$B'
>
> This is a matrix multiplication that gets delegated to the underlying
> BLAS right.  Running it under ‘perf record’ confirms it:
>
> Samples: 139K of event 'cycles:u', Event count (approx.): 99624880590
> Overhead  Command      Shared Object               Symbol
>   35.27%  .julia-real  libblas.so.3.9.0            [.] dgemm_
>    3.99%  .julia-real  libjulia-internal.so.1.8    [.] gc_mark_loop
>    2.60%  .julia-real  libjulia-internal.so.1.8    [.] apply_cl
>    1.06%  .julia-real  libjulia-internal.so.1.8    [.] jl_get_binding_
>
> We’re using libblas.so (presumably from the ‘lapack’ package) and not
> OpenBLAS, so no wonder it’s slow.
>
> Could it be that:
>
>          "LIBBLAS=-lopenblas"
>          "LIBBLASNAME=libopenblas"
>
> is ineffective?  I think we have a lead!

Are we following all instructions here ?

https://docs.julialang.org/en/v1.8/devdocs/build/distributing/#Notes-on-BLAS-and-LAPACK

I’m thinking about the variables LIBLAPACK and LIBLAPACKNAME.

Cayetano Santos wrote on 22 Jun 2023 15:26

Recipients:

Message-ID:87wmzvmxt8.fsf@inventati.org

Toggle quote (36 lines)>mer. 21 juin 2023 at 22:39, Cayetano Santos <csantosb@inventati.org> wrote:

>>mer. 21 juin 2023 at 16:36, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:
>
>> Hey!
>>
>> The benchmark you posted, Cayetano, is:
>>
>>   julia -e 'using Pkg; Pkg.add("BenchmarkTools"); using BenchmarkTools; N = 1000; A = rand(N, N); B = rand(N, N); @btime $A*$B'
>>
>> This is a matrix multiplication that gets delegated to the underlying
>> BLAS right.  Running it under ‘perf record’ confirms it:
>>
>> Samples: 139K of event 'cycles:u', Event count (approx.): 99624880590
>> Overhead  Command      Shared Object               Symbol
>>   35.27%  .julia-real  libblas.so.3.9.0            [.] dgemm_
>>    3.99%  .julia-real  libjulia-internal.so.1.8    [.] gc_mark_loop
>>    2.60%  .julia-real  libjulia-internal.so.1.8    [.] apply_cl
>>    1.06%  .julia-real  libjulia-internal.so.1.8    [.] jl_get_binding_
>>
>> We’re using libblas.so (presumably from the ‘lapack’ package) and not
>> OpenBLAS, so no wonder it’s slow.
>>
>> Could it be that:
>>
>>          "LIBBLAS=-lopenblas"
>>          "LIBBLASNAME=libopenblas"
>>
>> is ineffective?  I think we have a lead!
>
> Are we following all instructions here ?
>
>   https://docs.julialang.org/en/v1.8/devdocs/build/distributing/#Notes-on-BLAS-and-LAPACK
>
> I’m thinking about the variables LIBLAPACK and LIBLAPACKNAME.

To complete my previous comment, I just realised that

Base.USE_BLAS64

gives "true" when running fast. Guix julia gives "false".

Efraim Flashner wrote on 22 Jun 2023 16:37

Recipients:(name . Cayetano Santos)(address . csantosb@inventati.org)

Message-ID:ZJRcq8BieSPFPIEk@3900XT

On Thu, Jun 22, 2023 at 03:26:30PM +0200, Cayetano Santos wrote:

Toggle quote (12 lines)> 
> 
> >mer. 21 juin 2023 at 22:39, Cayetano Santos <csantosb@inventati.org> wrote:
> 
> >>mer. 21 juin 2023 at 16:36, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:
> >
> >> Hey!
> >>
> >> The benchmark you posted, Cayetano, is:
> >>
> >>   julia -e 'using Pkg; Pkg.add("BenchmarkTools"); using BenchmarkTools; N = 1000; A = rand(N, N); B = rand(N, N); @btime $A*$B'

I've been having a hard time with that command, so I've been running

julia -e 'using Pkg; Pkg.add("BenchmarkTools"); using BenchmarkTools; N = 1000; A = rand(N, N); B = rand(N, N); @time A*B'

I'm not sure if that's useful or not but I didn't see a different with

LIBBLAS* set.

Toggle quote (35 lines)> >>
> >> This is a matrix multiplication that gets delegated to the underlying
> >> BLAS right.  Running it under ‘perf record’ confirms it:
> >>
> >> Samples: 139K of event 'cycles:u', Event count (approx.): 99624880590
> >> Overhead  Command      Shared Object               Symbol
> >>   35.27%  .julia-real  libblas.so.3.9.0            [.] dgemm_
> >>    3.99%  .julia-real  libjulia-internal.so.1.8    [.] gc_mark_loop
> >>    2.60%  .julia-real  libjulia-internal.so.1.8    [.] apply_cl
> >>    1.06%  .julia-real  libjulia-internal.so.1.8    [.] jl_get_binding_
> >>
> >> We’re using libblas.so (presumably from the ‘lapack’ package) and not
> >> OpenBLAS, so no wonder it’s slow.
> >>
> >> Could it be that:
> >>
> >>          "LIBBLAS=-lopenblas"
> >>          "LIBBLASNAME=libopenblas"
> >>
> >> is ineffective?  I think we have a lead!
> >
> > Are we following all instructions here ?
> >
> >   https://docs.julialang.org/en/v1.8/devdocs/build/distributing/#Notes-on-BLAS-and-LAPACK
> >
> > I’m thinking about the variables LIBLAPACK and LIBLAPACKNAME.
> 
> To complete my previous comment, I just realised that
> 
>   Base.USE_BLAS64
> 
> gives "true" when running fast. Guix julia gives "false".
> 
> C.

I'll see what comes with switching it to blas64

-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Simon Tournier wrote on 22 Jun 2023 17:52

Recipients:

Message-ID:87h6qz8pg0.fsf@gmail.com

Hi,

On Thu, 22 Jun 2023 at 15:26, Cayetano Santos via Bug reports for GNU Guix <bug-guix@gnu.org> wrote:

Toggle quote (4 lines)

>> Are we following all instructions here ?

>> https://docs.julialang.org/en/v1.8/devdocs/build/distributing/#Notes-on-BLAS-and-LAPACK

[...]

Toggle quote (4 lines)

> Base.USE_BLAS64

> gives "true" when running fast. Guix julia gives "false".

When I try USE_BLAS64=1, then I get:

Toggle snippet (5 lines)

? Error: No loaded BLAS libraries were built with ILP64 support

? @ LinearAlgebra.BLAS /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/blas.jl:155

Quitting.

And from the documentation above, it reads:

[...] while on 64-bit architectures, Julia builds OpenBLAS to

use 64-bit integers (ILP64). It is essential that all Julia

functions that call BLAS and LAPACK API routines use integers of

the correct width.

Well using the patch attached, I get:

6.884 ms (2 allocations: 7.63 MiB)

compared to the previous

494.345 ms (2 allocations: 7.63 MiB)

WDYT about this patch?

From 024c92fac091f59dcdbd3a78eb6ea77bb15b2170 Mon Sep 17 00:00:00 2001
Message-Id: <024c92fac091f59dcdbd3a78eb6ea77bb15b2170.1687449033.git.zimon.toutoune@gmail.com>
From: Simon Tournier <zimon.toutoune@gmail.com>
Date: Thu, 22 Jun 2023 17:45:50 +0200
Subject: [PATCH] gnu: julia: Use openblas with ILP64 support.

Fixes https://bugs.gnu.org/63986.
Reported by Cayetano Santos <csantosb@inventati.org>.

* gnu/packages/julia.scm (julia)[arguments]<#:make-flags>: Use OpenBLAS with
ILP64 support.
[inputs]: Replace openblas by openblas-ilp64.
---
 gnu/packages/julia.scm | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

Toggle diff (31 lines)diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
index ba54175822..a034cbf543 100644
--- a/gnu/packages/julia.scm
+++ b/gnu/packages/julia.scm
@@ -481,10 +481,10 @@ (define-public julia
 
          ,@(if (target-aarch64?)
              `("USE_BLAS64=0")
-             '())
+             `("USE_BLAS64=1"))
 
-         "LIBBLAS=-lopenblas"
-         "LIBBLASNAME=libopenblas"
+         "LIBBLAS=-lopenblas_ilp64"
+         "LIBBLASNAME=libopenblas_ilp64"
 
          (string-append "UTF8PROC_INC="
                         (assoc-ref %build-inputs "utf8proc")
@@ -513,7 +513,7 @@ (define-public julia
        ("llvm" ,llvm-julia)
        ("mbedtls-apache" ,mbedtls-apache)
        ("mpfr" ,mpfr)
-       ("openblas" ,openblas)
+       ("openblas" ,openblas-ilp64)
        ("openlibm" ,openlibm)
        ("p7zip" ,p7zip)
        ("pcre2" ,pcre2)

base-commit: 37c2e94cec6cb8b5e0e93e7b6c712c3b187ca5db
-- 
2.38.1

Well, I need to do more tests but I guess that’s the good direction. :-)

Cheers,

simon

Efraim Flashner wrote on 22 Jun 2023 18:25

Recipients:(name . Simon Tournier)(address . zimon.toutoune@gmail.com)

Message-ID:ZJR14QoB4SoqhsrP@3900XT

On Thu, Jun 22, 2023 at 05:52:15PM +0200, Simon Tournier wrote:

Toggle quote (38 lines)> Hi,
> 
> On Thu, 22 Jun 2023 at 15:26, Cayetano Santos via Bug reports for GNU Guix <bug-guix@gnu.org> wrote:
> >> Are we following all instructions here ?
> >>
> >>   https://docs.julialang.org/en/v1.8/devdocs/build/distributing/#Notes-on-BLAS-and-LAPACK
> 
> [...]
> 
> >   Base.USE_BLAS64
> >
> > gives "true" when running fast. Guix julia gives "false".
> 
> When I try USE_BLAS64=1, then I get:
> 
> --8<---------------cut here---------------start------------->8---
> ? Error: No loaded BLAS libraries were built with ILP64 support
> ? @ LinearAlgebra.BLAS /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/blas.jl:155
> Quitting.
> --8<---------------cut here---------------end--------------->8---
> 
> And from the documentation above, it reads:
> 
>         [...] while on 64-bit architectures, Julia builds OpenBLAS to
>         use 64-bit integers (ILP64). It is essential that all Julia
>         functions that call BLAS and LAPACK API routines use integers of
>         the correct width.
> 
> Well using the patch attached, I get:
> 
>     6.884 ms (2 allocations: 7.63 MiB)
> 
> compared to the previous
> 
>     494.345 ms (2 allocations: 7.63 MiB)
> 
> WDYT about this patch?

(ins)efraim@3900XT ~/workspace/guix$ cat /gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/lib/pkgconfig/openblas.pc

libdir=/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/lib

includedir=/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/include

openblas_config= USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER=1 NO_CBLAS= NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= generic MAX_THREADS=128

version=0.3.20

extralib=-lm -lpthread -lgfortran -lm -lpthread -lgfortran

Name: openblas

Description: OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version

Version: ${version}

URL: https://github.com/xianyi/OpenBLAS

Libs: -L${libdir} -lopenblas

Libs.private: ${extralib}

Cflags: -I${includedir}

Looks like it should be "LIBBLAS=-lopenblas"

It might need some tuning anyway, currently we have julia for i686 and

switching to solely openblas-ilp64 we'd lose 32-bit support.

I also noticed the julia expects the 64-bit openblas to be libopenblas64

(which happens to be what Debian¹ has). Would we need to adapt anything

in stdlib/OpenBLAS_jll/src/OpenBLAS_jll.jl?

Also, are we supposed to build lapack with our openblas as an input?

¹ https://sources.debian.org/src/openblas/0.3.21%2Bds-4/debian/rules/#L71

Toggle quote (56 lines)> From 024c92fac091f59dcdbd3a78eb6ea77bb15b2170 Mon Sep 17 00:00:00 2001
> Message-Id: <024c92fac091f59dcdbd3a78eb6ea77bb15b2170.1687449033.git.zimon.toutoune@gmail.com>
> From: Simon Tournier <zimon.toutoune@gmail.com>
> Date: Thu, 22 Jun 2023 17:45:50 +0200
> Subject: [PATCH] gnu: julia: Use openblas with ILP64 support.
> 
> Fixes <https://bugs.gnu.org/63986>.
> Reported by Cayetano Santos <csantosb@inventati.org>.
> 
> * gnu/packages/julia.scm (julia)[arguments]<#:make-flags>: Use OpenBLAS with
> ILP64 support.
> [inputs]: Replace openblas by openblas-ilp64.
> ---
>  gnu/packages/julia.scm | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
> index ba54175822..a034cbf543 100644
> --- a/gnu/packages/julia.scm
> +++ b/gnu/packages/julia.scm
> @@ -481,10 +481,10 @@ (define-public julia
>  
>           ,@(if (target-aarch64?)
>               `("USE_BLAS64=0")
> -             '())
> +             `("USE_BLAS64=1"))
>  
> -         "LIBBLAS=-lopenblas"
> -         "LIBBLASNAME=libopenblas"
> +         "LIBBLAS=-lopenblas_ilp64"
> +         "LIBBLASNAME=libopenblas_ilp64"
>  
>           (string-append "UTF8PROC_INC="
>                          (assoc-ref %build-inputs "utf8proc")
> @@ -513,7 +513,7 @@ (define-public julia
>         ("llvm" ,llvm-julia)
>         ("mbedtls-apache" ,mbedtls-apache)
>         ("mpfr" ,mpfr)
> -       ("openblas" ,openblas)
> +       ("openblas" ,openblas-ilp64)
>         ("openlibm" ,openlibm)
>         ("p7zip" ,p7zip)
>         ("pcre2" ,pcre2)
> 
> base-commit: 37c2e94cec6cb8b5e0e93e7b6c712c3b187ca5db
> -- 
> 2.38.1
> 

> 
> Well, I need to do more tests but I guess that’s the good direction. :-)
> 
> Cheers,
> simon

-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Cayetano Santos wrote on 22 Jun 2023 18:56

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)

Message-ID:874jmz8mc0.fsf@inventati.org

Toggle quote (11 lines)> Looks like it should be "LIBBLAS=-lopenblas"
>
> It might need some tuning anyway, currently we have julia for i686 and
> switching to solely openblas-ilp64 we'd lose 32-bit support.
>
> I also noticed the julia expects the 64-bit openblas to be libopenblas64
> (which happens to be what Debian¹ has). Would we need to adapt anything
> in stdlib/OpenBLAS_jll/src/OpenBLAS_jll.jl?
>
> Also, are we supposed to build lapack with our openblas as an input?

Being used to Arch, it seems to me the way they do things is the way to

go, or at least, a good reference (other than the support for 32-bit)

https://archlinux.org/packages/?sort=&q=blas&maintainer=&flagged=

https://gitlab.archlinux.org/archlinux/packaging/packages/julia/-/blob/main/PKGBUILD

Simon Tournier wrote on 22 Jun 2023 19:12

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)

Message-ID:87cz1n8lq3.fsf@gmail.com

Hi,

On Thu, 22 Jun 2023 at 19:25, Efraim Flashner <efraim@flashner.co.il> wrote:

Toggle quote (3 lines)

> It might need some tuning anyway, currently we have julia for i686 and

> switching to solely openblas-ilp64 we'd lose 32-bit support.

Yes, indeed. This attached patch conditionally switches.

From 9e1912148d4a691a5484fc7f87597b579ba61785 Mon Sep 17 00:00:00 2001
Message-Id: <9e1912148d4a691a5484fc7f87597b579ba61785.1687453627.git.zimon.toutoune@gmail.com>
From: Simon Tournier <zimon.toutoune@gmail.com>
Date: Thu, 22 Jun 2023 17:45:50 +0200
Subject: [PATCH v2] gnu: julia: Conditionally use openblas with ILP64 support.

Fixes https://bugs.gnu.org/63986.
Reported by Cayetano Santos <csantosb@inventati.org>.

* gnu/packages/julia.scm (julia)[arguments]<#:make-flags>: Conditionally use
OpenBLAS with ILP64 support for x86-64 target.
[inputs]: Conditionally replace openblas by openblas-ilp64 for x86-64 target.
---
 gnu/packages/julia.scm | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

Toggle diff (34 lines)diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
index ba54175822..3ffd1fa4cb 100644
--- a/gnu/packages/julia.scm
+++ b/gnu/packages/julia.scm
@@ -483,8 +483,12 @@ (define-public julia
              `("USE_BLAS64=0")
              '())
 
-         "LIBBLAS=-lopenblas"
-         "LIBBLASNAME=libopenblas"
+         ,@(if (target-x86-64?)
+             `("USE_BLAS64=1"
+               "LIBBLAS=-lopenblas_ilp64"
+               "LIBBLASNAME=libopenblas_ilp64")
+             `("LIBBLAS=-lopenblas"
+               "LIBBLASNAME=libopenblas"))
 
          (string-append "UTF8PROC_INC="
                         (assoc-ref %build-inputs "utf8proc")
@@ -513,7 +517,9 @@ (define-public julia
        ("llvm" ,llvm-julia)
        ("mbedtls-apache" ,mbedtls-apache)
        ("mpfr" ,mpfr)
-       ("openblas" ,openblas)
+       ,@(if (target-x86-64?)
+             `(("openblas" ,openblas-ilp64))
+             `(("openblas" ,openblas)))
        ("openlibm" ,openlibm)
        ("p7zip" ,p7zip)
        ("pcre2" ,pcre2)

base-commit: 37c2e94cec6cb8b5e0e93e7b6c712c3b187ca5db
-- 
2.38.1

Well, the current recipe could be improved but let focus on the current
bug report.  And I guess the slowness come from an incorrect use of BLAS
for x86-64.  That seems fixed by the patch, no?

Cheers,
simon

Simon Tournier wrote on 22 Jun 2023 20:47

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)

Message-ID:875y7f8hbf.fsf@gmail.com

Hi,

On Thu, 22 Jun 2023 at 19:25, Efraim Flashner <efraim@flashner.co.il> wrote:

Toggle quote (16 lines)> (ins)efraim@3900XT ~/workspace/guix$ cat /gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/lib/pkgconfig/openblas.pc
> libdir=/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/lib
> includedir=/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/include
> openblas_config= USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER=1 NO_CBLAS= NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= generic MAX_THREADS=128
> version=0.3.20
> extralib=-lm -lpthread -lgfortran -lm -lpthread -lgfortran
> Name: openblas
> Description: OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version
> Version: ${version}
> URL: https://github.com/xianyi/OpenBLAS
> Libs: -L${libdir} -lopenblas
> Libs.private: ${extralib}
> Cflags: -I${includedir}
>
> Looks like it should be "LIBBLAS=-lopenblas"

I propose to tweak openblas-ilp64. Currently it looks like:

Toggle snippet (13 lines)$ tree $(guix build openblas-ilp64)/lib
/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/lib
??? cmake
?�� ??? openblas
?��     ??? OpenBLASConfig.cmake
?��     ??? OpenBLASConfigVersion.cmake
??? libopenblas_ilp64p-r0.3.20.so
??? libopenblas_ilp64.so -> libopenblas_ilp64p-r0.3.20.so
??? libopenblas_ilp64.so.0 -> libopenblas_ilp64p-r0.3.20.so
??? pkgconfig
    ??? openblas.pc

which is inconsistent with pkgconfig as you noticed above.  Therefore, I
am proposing the addition of a symlink of libopenblas_ilp64p.so to
libopenblas.so.  For instance this attached patch.

From 0c8c7e9371d11972f4a6012ef503ef3057c91364 Mon Sep 17 00:00:00 2001
Message-Id: <0c8c7e9371d11972f4a6012ef503ef3057c91364.1687459454.git.zimon.toutoune@gmail.com>
From: Simon Tournier <zimon.toutoune@gmail.com>
Date: Thu, 22 Jun 2023 20:31:26 +0200
Subject: [PATCH v3 1/2] gnu: openblas-ilp64: Install symlink to libopenblas.

* gnu/packages/maths.scm (openblas-ilp64)[arguments]: Add phases for
installing symlink to libopenblas.
---
 gnu/packages/maths.scm | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

Toggle diff (23 lines)diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index f5a2181905..5c39ab8b94 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -4646,7 +4646,13 @@ (define-public openblas-ilp64
      (substitute-keyword-arguments (package-arguments openblas)
        ((#:make-flags flags #~'())
         #~(append (list "INTERFACE64=1" "LIBNAMESUFFIX=ilp64")
-                 #$flags))))
+                 #$flags))
+       ((#:phases phases)
+        #~(modify-phases #$phases
+            (add-after 'install 'install-symlink
+              (lambda _
+                (symlink "libopenblas_ilp64.so"
+                         (string-append #$output "/lib/libopenblas.so"))))))))
     (synopsis "Optimized BLAS library based on GotoBLAS (ILP64 version)")
     (license license:bsd-3)))
 

base-commit: 37c2e94cec6cb8b5e0e93e7b6c712c3b187ca5db
-- 
2.38.1

Then, the patch for Julia looks like the other attached patch.

From 8563a738703d133b44ae05b91b7448ca56d280b5 Mon Sep 17 00:00:00 2001
Message-Id: <8563a738703d133b44ae05b91b7448ca56d280b5.1687459454.git.zimon.toutoune@gmail.com>
In-Reply-To: <0c8c7e9371d11972f4a6012ef503ef3057c91364.1687459454.git.zimon.toutoune@gmail.com>
References: <0c8c7e9371d11972f4a6012ef503ef3057c91364.1687459454.git.zimon.toutoune@gmail.com>
From: Simon Tournier <zimon.toutoune@gmail.com>
Date: Thu, 22 Jun 2023 17:45:50 +0200
Subject: [PATCH v3 2/2] gnu: julia: Conditionally use openblas with ILP64
 support.

Fixes https://bugs.gnu.org/63986.
Reported by Cayetano Santos <csantosb@inventati.org>.

* gnu/packages/julia.scm (julia)[arguments]<#:make-flags>: Conditionally use
64-bit BLAS for x86-64 target.
[inputs]: Conditionally replace openblas by openblas-ilp64 for x86-64 target.
---
 gnu/packages/julia.scm | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

Toggle diff (30 lines)diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
index ba54175822..17a27f4928 100644
--- a/gnu/packages/julia.scm
+++ b/gnu/packages/julia.scm
@@ -479,9 +479,9 @@ (define-public julia
          "NO_GIT=1"             ; build from release tarball.
          "USE_GPL_LIBS=1"       ; proudly
 
-         ,@(if (target-aarch64?)
-             `("USE_BLAS64=0")
-             '())
+         ,@(if (target-x86-64?)
+               `("USE_BLAS64=1")
+               '())
 
          "LIBBLAS=-lopenblas"
          "LIBBLASNAME=libopenblas"
@@ -513,7 +513,9 @@ (define-public julia
        ("llvm" ,llvm-julia)
        ("mbedtls-apache" ,mbedtls-apache)
        ("mpfr" ,mpfr)
-       ("openblas" ,openblas)
+       ,@(if (target-x86-64?)
+             `(("openblas" ,openblas-ilp64))
+             `(("openblas" ,openblas)))
        ("openlibm" ,openlibm)
        ("p7zip" ,p7zip)
        ("pcre2" ,pcre2)
-- 
2.38.1

WDYT?

Well, I am running all the testsuite for checking if all is correct.

Somehow, I think this is direction. Otherwise, what would you suggest?

Cheers,

simon

Efraim Flashner wrote on 22 Jun 2023 20:55

Recipients:(name . Simon Tournier)(address . zimon.toutoune@gmail.com)

Message-ID:ZJSZExs94_1yjjZh@3900XT

Attachment: file

Efraim Flashner wrote on 22 Jun 2023 22:13

Recipients:(name . Simon Tournier)(address . zimon.toutoune@gmail.com)

Message-ID:ZJSrU5ZqofMV3t4T@3900XT

On Thu, Jun 22, 2023 at 08:47:48PM +0200, Simon Tournier wrote:

Toggle quote (40 lines)> Hi,
> 
> On Thu, 22 Jun 2023 at 19:25, Efraim Flashner <efraim@flashner.co.il> wrote:
> 
> > (ins)efraim@3900XT ~/workspace/guix$ cat /gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/lib/pkgconfig/openblas.pc
> > libdir=/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/lib
> > includedir=/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/include
> > openblas_config= USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER=1 NO_CBLAS= NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= generic MAX_THREADS=128
> > version=0.3.20
> > extralib=-lm -lpthread -lgfortran -lm -lpthread -lgfortran
> > Name: openblas
> > Description: OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version
> > Version: ${version}
> > URL: https://github.com/xianyi/OpenBLAS
> > Libs: -L${libdir} -lopenblas
> > Libs.private: ${extralib}
> > Cflags: -I${includedir}
> >
> > Looks like it should be "LIBBLAS=-lopenblas"
> 
> I propose to tweak openblas-ilp64.  Currently it looks like:
> 
> --8<---------------cut here---------------start------------->8---
> $ tree $(guix build openblas-ilp64)/lib
> /gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/lib
> ??? cmake
> ?�� ??? openblas
> ?��     ??? OpenBLASConfig.cmake
> ?��     ??? OpenBLASConfigVersion.cmake
> ??? libopenblas_ilp64p-r0.3.20.so
> ??? libopenblas_ilp64.so -> libopenblas_ilp64p-r0.3.20.so
> ??? libopenblas_ilp64.so.0 -> libopenblas_ilp64p-r0.3.20.so
> ??? pkgconfig
>     ??? openblas.pc
> --8<---------------cut here---------------end--------------->8---
> 
> which is inconsistent with pkgconfig as you noticed above.  Therefore, I
> am proposing the addition of a symlink of libopenblas_ilp64p.so to
> libopenblas.so.  For instance this attached patch.

I've attached the patch that I've made it to. julia is built with
openblas-ilp64 on 64-bit architectures, openblas-ilp64 has its
configure-flags adjusted to match what other programs are expecting, and
I've also patched python-numpy to use openblas-ilp64 as another test
target.

-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Attachment: julia-openblas-ilp64.patch

Ludovic Courtès wrote on 23 Jun 2023 11:09

Recipients:(name . Simon Tournier)(address . zimon.toutoune@gmail.com)

Message-ID:871qi24k9z.fsf@inria.fr

Hi!

Simon Tournier <zimon.toutoune@gmail.com> skribis:

Toggle quote (18 lines)> +         ,@(if (target-x86-64?)
> +             `("USE_BLAS64=1"
> +               "LIBBLAS=-lopenblas_ilp64"
> +               "LIBBLASNAME=libopenblas_ilp64")
> +             `("LIBBLAS=-lopenblas"
> +               "LIBBLASNAME=libopenblas"))
>  
>           (string-append "UTF8PROC_INC="
>                          (assoc-ref %build-inputs "utf8proc")
> @@ -513,7 +517,9 @@ (define-public julia
>         ("llvm" ,llvm-julia)
>         ("mbedtls-apache" ,mbedtls-apache)
>         ("mpfr" ,mpfr)
> -       ("openblas" ,openblas)
> +       ,@(if (target-x86-64?)
> +             `(("openblas" ,openblas-ilp64))
> +             `(("openblas" ,openblas)))

Should it be ‘target-64bit?’ instead?

Thumbs up for finding the solution!

Ludo’.

Efraim Flashner wrote on 23 Jun 2023 11:16

Recipients:(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)

Message-ID:ZJVi0Wwg5aRq_XxJ@3900XT

On Fri, Jun 23, 2023 at 11:09:44AM +0200, Ludovic Courtès wrote:

Toggle quote (24 lines)> Hi!
> 
> Simon Tournier <zimon.toutoune@gmail.com> skribis:
> 
> > +         ,@(if (target-x86-64?)
> > +             `("USE_BLAS64=1"
> > +               "LIBBLAS=-lopenblas_ilp64"
> > +               "LIBBLASNAME=libopenblas_ilp64")
> > +             `("LIBBLAS=-lopenblas"
> > +               "LIBBLASNAME=libopenblas"))
> >  
> >           (string-append "UTF8PROC_INC="
> >                          (assoc-ref %build-inputs "utf8proc")
> > @@ -513,7 +517,9 @@ (define-public julia
> >         ("llvm" ,llvm-julia)
> >         ("mbedtls-apache" ,mbedtls-apache)
> >         ("mpfr" ,mpfr)
> > -       ("openblas" ,openblas)
> > +       ,@(if (target-x86-64?)
> > +             `(("openblas" ,openblas-ilp64))
> > +             `(("openblas" ,openblas)))
> 
> Should it be ‘target-64bit?’ instead?

It should be, but julia fails the precompile stage with openblas-ilp64

Precompilation complete. Summary:
Total ??????? 1006.632176 seconds
Generation ?? 690.274180 seconds 68.5726%
Execution ??? 316.357997 seconds 31.4274%

signal (15): Terminated
in expression starting at none:0
unknown function (ip: 0xfffff4d81b24)
unknown function (ip: 0xfffff4ed10bb)
unknown function (ip: 0xfffff4eda16f)
_ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE at /gnu/store/wa28l476c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line)
_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /gnu/store/wa28l476c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line)
_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /gnu/store/wa28l476c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /gnu/store/wa28l476c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line)
operator() at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/aotcompile.cpp:580 [inlined]
jl_dump_native_impl at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/aotcompile.cpp:592
jl_write_compiler_output at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/precompile.c:94
ijl_atexit_hook at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/init.c:207
jl_repl_entrypoint at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/jlapi.c:720
main at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/cli/loader_exe.c:59
__libc_start_call_main at /gnu/store/a19xbynxc3sg25xpkwmx7g0mdl7g31hx-glibc-2.35/lib/libc.so.6 (unknown line)
__libc_start_main at /gnu/store/a19xbynxc3sg25xpkwmx7g0mdl7g31hx-glibc-2.35/lib/libc.so.6 (unknown line)
_start at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/bin/julia (unknown line)
unknown function (ip: (nil))
Allocations: 145655328 (Pool: 145582643; Big: 72685); GC: 120
*** This error is usually fixed by running `make clean`. If the error persists, try `make cleanall`. ***
make[1]: *** [sysimage.mk:89: /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/lib/julia/sys-o.a] Error 1
make[1]: Leaving directory '/tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3'
make: *** [Makefile:88: julia-sysimg-release] Error 2

Toggle quote (4 lines)

> Thumbs up for finding the solution!

> Ludo’.

-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Simon Tournier wrote on 23 Jun 2023 14:11

Recipients:

Message-ID:87lega74zn.fsf@gmail.com

Hi,

On Fri, 23 Jun 2023 at 12:16, Efraim Flashner <efraim@flashner.co.il> wrote:

Toggle quote (4 lines)

>> Should it be ‘target-64bit?’ instead?

> It should be, but julia fails the precompile stage with openblas-ilp64

Well, I picked target-x86-64? because the recipe contains,

,@(if (target-aarch64?)

`("USE_BLAS64=0")

'())

so using target-64bit? would apply USE_BLAS64=1 to aarch64 when it was

not. Well, since I know few about this architecture, I have tried to

keep the way it was. Somehow, I have no opinion about what is best. :-)

Cheers,

simon

Simon Tournier wrote on 23 Jun 2023 14:06

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)

Message-ID:87v8fe758r.fsf@gmail.com

Hi Efraim,

On Thu, 22 Jun 2023 at 23:13, Efraim Flashner <efraim@flashner.co.il> wrote:

Toggle quote (4 lines)

> - ,@(if (target-aarch64?)

> - `("USE_BLAS64=0")

> - '())

[...]

Toggle quote (3 lines)

> + ,@(if (target-64bit?)

> + `("USE_BLAS64=1"

Well, I do not know but using this change USE_BLAS64=1 will be applied

to aarch64 when it was specifically not previously. Is it expected?

Toggle quote (6 lines)

> diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm

> index b650b71f3b..deeffb67ae 100644

> --- a/gnu/packages/python-xyz.scm

> +++ b/gnu/packages/python-xyz.scm

> @@ -6953,19 +6953,36 @@ (define-public python-numpy

This change leads to a large rebuild. Therefore, it should go first to

a dedicated branch, no?

Cheers,

simon

Simon Tournier wrote on 23 Jun 2023 15:23

Recipients:

Message-ID:87h6qy71oa.fsf@gmail.com

Hi Efraim,

On Fri, 23 Jun 2023 at 12:16, Efraim Flashner <efraim@flashner.co.il> wrote:

Toggle quote (32 lines)> It should be, but julia fails the precompile stage with openblas-ilp64
>
> Precompilation complete. Summary:
> Total ??????? 1006.632176 seconds
> Generation ?? 690.274180 seconds 68.5726%
> Execution ??? 316.357997 seconds 31.4274%
>
> signal (15): Terminated
> in expression starting at none:0
> unknown function (ip: 0xfffff4d81b24)
> unknown function (ip: 0xfffff4ed10bb)
> unknown function (ip: 0xfffff4eda16f)
> _ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE at /gnu/store/wa28l476c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line)
> _ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /gnu/store/wa28l476c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line)
> _ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /gnu/store/wa28l476c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line)
> _ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /gnu/store/wa28l476c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line)
> operator() at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/aotcompile.cpp:580 [inlined]
> jl_dump_native_impl at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/aotcompile.cpp:592
> jl_write_compiler_output at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/precompile.c:94
> ijl_atexit_hook at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/init.c:207
> jl_repl_entrypoint at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/jlapi.c:720
> main at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/cli/loader_exe.c:59
> __libc_start_call_main at /gnu/store/a19xbynxc3sg25xpkwmx7g0mdl7g31hx-glibc-2.35/lib/libc.so.6 (unknown line)
> __libc_start_main at /gnu/store/a19xbynxc3sg25xpkwmx7g0mdl7g31hx-glibc-2.35/lib/libc.so.6 (unknown line)
> _start at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/bin/julia (unknown line)
> unknown function (ip: (nil))
> Allocations: 145655328 (Pool: 145582643; Big: 72685); GC: 120
> *** This error is usually fixed by running `make clean`. If the error persists, try `make cleanall`. ***
> make[1]: *** [sysimage.mk:89: /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/lib/julia/sys-o.a] Error 1
> make[1]: Leaving directory '/tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3'
> make: *** [Makefile:88: julia-sysimg-release] Error 2

Using my patch, it fails with:

Toggle snippet (8 lines)

Error in testset OpenBLAS_jll:

Test Failed at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/share/julia/stdlib/v1.8/OpenBLAS_jll/test/runtests.jl:16

Expression: dlsym(OpenBLAS_jll.libopenblas_handle, #= /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/share/julia/stdlib/v1.8/OpenBLAS_jll/test/runtests.jl:16 =# @blasfunc(openblas_set_num_threads); throw_error = false) != nothing

Evaluated: nothing != nothing

ERROR: LoadError: Test run finished with errors

in expression starting at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/test/runtests.jl:93

Well, I am still missing why the expression

dlsym(OpenBLAS_jll.libopenblas_handle,

#= /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/share/julia/stdlib/v1.8/OpenBLAS_jll/test/runtests.jl:16 =#

@blasfunc(openblas_set_num_threads); throw_error = false)

is evaluated to nothing. Any idea?

Cheers,

simon

Efraim Flashner wrote on 23 Jun 2023 16:16

Recipients:(name . Simon Tournier)(address . zimon.toutoune@gmail.com)

Message-ID:ZJWpKhkhHiMMzHST@3900XT

On Fri, Jun 23, 2023 at 03:23:17PM +0200, Simon Tournier wrote:

Toggle quote (47 lines)> Hi Efraim,
> 
> On Fri, 23 Jun 2023 at 12:16, Efraim Flashner <efraim@flashner.co.il> wrote:
> 
> > It should be, but julia fails the precompile stage with openblas-ilp64
> >
> > Precompilation complete. Summary:
> > Total ??????? 1006.632176 seconds
> > Generation ?? 690.274180 seconds 68.5726%
> > Execution ??? 316.357997 seconds 31.4274%
> >
> > signal (15): Terminated
> > in expression starting at none:0
> > unknown function (ip: 0xfffff4d81b24)
> > unknown function (ip: 0xfffff4ed10bb)
> > unknown function (ip: 0xfffff4eda16f)
> > _ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE at /gnu/store/wa28l476c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line)
> > _ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /gnu/store/wa28l476c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line)
> > _ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /gnu/store/wa28l476c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line)
> > _ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /gnu/store/wa28l476c1bfshfiqcqbmk75zr8ml152-llvm-13.0.1/lib/libLLVM-13jl.so (unknown line)
> > operator() at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/aotcompile.cpp:580 [inlined]
> > jl_dump_native_impl at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/aotcompile.cpp:592
> > jl_write_compiler_output at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/precompile.c:94
> > ijl_atexit_hook at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/init.c:207
> > jl_repl_entrypoint at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/src/jlapi.c:720
> > main at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/cli/loader_exe.c:59
> > __libc_start_call_main at /gnu/store/a19xbynxc3sg25xpkwmx7g0mdl7g31hx-glibc-2.35/lib/libc.so.6 (unknown line)
> > __libc_start_main at /gnu/store/a19xbynxc3sg25xpkwmx7g0mdl7g31hx-glibc-2.35/lib/libc.so.6 (unknown line)
> > _start at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/bin/julia (unknown line)
> > unknown function (ip: (nil))
> > Allocations: 145655328 (Pool: 145582643; Big: 72685); GC: 120
> > *** This error is usually fixed by running `make clean`. If the error persists, try `make cleanall`. ***
> > make[1]: *** [sysimage.mk:89: /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/lib/julia/sys-o.a] Error 1
> > make[1]: Leaving directory '/tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3'
> > make: *** [Makefile:88: julia-sysimg-release] Error 2
> 
> Using my patch, it fails with:
> 
> --8<---------------cut here---------------start------------->8---
> Error in testset OpenBLAS_jll:
> Test Failed at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/share/julia/stdlib/v1.8/OpenBLAS_jll/test/runtests.jl:16
>   Expression: dlsym(OpenBLAS_jll.libopenblas_handle, #= /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/share/julia/stdlib/v1.8/OpenBLAS_jll/test/runtests.jl:16 =# @blasfunc(openblas_set_num_threads); throw_error = false) != nothing
>    Evaluated: nothing != nothing
> ERROR: LoadError: Test run finished with errors
> in expression starting at /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/test/runtests.jl:93
> --8<---------------cut here---------------end--------------->8---

Mine was on aarch64

Toggle quote (8 lines)

> Well, I am still missing why the expression

> dlsym(OpenBLAS_jll.libopenblas_handle,

> #= /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/share/julia/stdlib/v1.8/OpenBLAS_jll/test/runtests.jl:16 =#

> @blasfunc(openblas_set_num_threads); throw_error = false)

> is evaluated to nothing. Any idea?

It is something about the wrong internal interface in openblas, I don't

remember exactly what that error came from.

Try with the attached diff.

Efraim Flashner <efraim@flashner.co.il> ????? ?????

GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351

Confidentiality cannot be guaranteed on emails sent or received unencrypted

Toggle diff (62 lines)diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
index ba54175822..8ba2b480ce 100644
--- a/gnu/packages/julia.scm
+++ b/gnu/packages/julia.scm
@@ -269,7 +269,10 @@ (define-public julia
                (substitute* (jlpath "nghttp2")
                  (((from "libnghttp2")) (to "libnghttp2" "libnghttp2")))
                (substitute* (jlpath "OpenBLAS")
-                 (((from "libopenblas")) (to "openblas" "libopenblas")))
+                 (((from "libopenblas"))
+                  ,@(if (target-x86-64?)
+                      `((to "openblas" "libopenblas64_" "libopenblas"))
+                      `((to "openblas" "libopenblas")))))
                (substitute* (jlpath "OpenLibm")
                  (((from "libopenlibm")) (to "openlibm" "libopenlibm")))
                (substitute* (jlpath "PCRE2")
@@ -479,12 +482,13 @@ (define-public julia
          "NO_GIT=1"             ; build from release tarball.
          "USE_GPL_LIBS=1"       ; proudly
 
-         ,@(if (target-aarch64?)
-             `("USE_BLAS64=0")
-             '())
-
-         "LIBBLAS=-lopenblas"
-         "LIBBLASNAME=libopenblas"
+         ,@(if (target-x86-64?)
+             `("USE_BLAS64=1"
+               "LIBBLAS=-lopenblas64_"
+               "LIBBLASNAME=libopenblas64_")
+             `("USE_BLAS64=0"
+               "LIBBLAS=-lopenblas"
+               "LIBBLASNAME=libopenblas"))
 
          (string-append "UTF8PROC_INC="
                         (assoc-ref %build-inputs "utf8proc")
@@ -513,7 +517,9 @@ (define-public julia
        ("llvm" ,llvm-julia)
        ("mbedtls-apache" ,mbedtls-apache)
        ("mpfr" ,mpfr)
-       ("openblas" ,openblas)
+       ,@(if (target-x86-64?)
+             `(("openblas" ,openblas-ilp64))
+             `(("openblas" ,openblas)))
        ("openlibm" ,openlibm)
        ("p7zip" ,p7zip)
        ("pcre2" ,pcre2)
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index f5a2181905..2d3ce41cb7 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -4645,7 +4645,9 @@ (define-public openblas-ilp64
     (arguments
      (substitute-keyword-arguments (package-arguments openblas)
        ((#:make-flags flags #~'())
-        #~(append (list "INTERFACE64=1" "LIBNAMESUFFIX=ilp64")
+        #~(append (list "INTERFACE64=1"
+                        "SYMBOLSUFFIX=64_"
+                        "LIBPREFIX=libopenblas64_")
                  #$flags))))
     (synopsis "Optimized BLAS library based on GotoBLAS (ILP64 version)")
     (license license:bsd-3)))

Ludovic Courtès wrote on 6 Jul 2023 16:13

Recipients:

Message-ID:87o7kpku0u.fsf@gnu.org

Hi there!

What’s the status? Sounds like we have a couple of fixes already.

Maybe you can submit one of them to guix-patches@gnu.org so qa.guix can

pick it up. And if one of them is more intrusive (more rebuilds), then

submit it separately so it gets merged later? How does that sound?

Ludo’.

Ludovic Courtès wrote on 20 Aug 2023 22:53

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)

Message-ID:87jztpqvtz.fsf@gnu.org

Hi!

Friendly ping. :-)

https://issues.guix.gnu.org/63986

Ludo’.

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (9 lines)> Hi there!
>
> What’s the status?  Sounds like we have a couple of fixes already.
>
> Maybe you can submit one of them to guix-patches@gnu.org so qa.guix can
> pick it up.  And if one of them is more intrusive (more rebuilds), then
> submit it separately so it gets merged later?  How does that sound?
>
> Ludo’.

Efraim Flashner wrote on 14 Sep 2023 12:33

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:ZQLhcr_Y21H1XiYL@3900XT

On Sun, Aug 20, 2023 at 10:53:44PM +0200, Ludovic Courtès wrote:

Toggle quote (20 lines)> Hi!
> 
> Friendly ping.  :-)
> 
>   https://issues.guix.gnu.org/63986
> 
> Ludo’.
> 
> Ludovic Courtès <ludo@gnu.org> skribis:
> 
> > Hi there!
> >
> > What’s the status?  Sounds like we have a couple of fixes already.
> >
> > Maybe you can submit one of them to guix-patches@gnu.org so qa.guix can
> > pick it up.  And if one of them is more intrusive (more rebuilds), then
> > submit it separately so it gets merged later?  How does that sound?
> >
> > Ludo’.

I've attached a diff to adjust openblas64 and to use it for x86_64 in
julia. I don't know if it's faster than the current openblas.

-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Toggle diff (62 lines)diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
index ba54175822..8ba2b480ce 100644
--- a/gnu/packages/julia.scm
+++ b/gnu/packages/julia.scm
@@ -269,7 +269,10 @@ (define-public julia
                (substitute* (jlpath "nghttp2")
                  (((from "libnghttp2")) (to "libnghttp2" "libnghttp2")))
                (substitute* (jlpath "OpenBLAS")
-                 (((from "libopenblas")) (to "openblas" "libopenblas")))
+                 (((from "libopenblas"))
+                  ,@(if (target-x86-64?)
+                      `((to "openblas" "libopenblas64_" "libopenblas"))
+                      `((to "openblas" "libopenblas")))))
                (substitute* (jlpath "OpenLibm")
                  (((from "libopenlibm")) (to "openlibm" "libopenlibm")))
                (substitute* (jlpath "PCRE2")
@@ -479,12 +482,13 @@ (define-public julia
          "NO_GIT=1"             ; build from release tarball.
          "USE_GPL_LIBS=1"       ; proudly
 
-         ,@(if (target-aarch64?)
-             `("USE_BLAS64=0")
-             '())
-
-         "LIBBLAS=-lopenblas"
-         "LIBBLASNAME=libopenblas"
+         ,@(if (target-x86-64?)
+             `("USE_BLAS64=1"
+               "LIBBLAS=-lopenblas64_"
+               "LIBBLASNAME=libopenblas64_")
+             `("USE_BLAS64=0"
+               "LIBBLAS=-lopenblas"
+               "LIBBLASNAME=libopenblas"))
 
          (string-append "UTF8PROC_INC="
                         (assoc-ref %build-inputs "utf8proc")
@@ -513,7 +517,9 @@ (define-public julia
        ("llvm" ,llvm-julia)
        ("mbedtls-apache" ,mbedtls-apache)
        ("mpfr" ,mpfr)
-       ("openblas" ,openblas)
+       ,@(if (target-x86-64?)
+             `(("openblas" ,openblas-ilp64))
+             `(("openblas" ,openblas)))
        ("openlibm" ,openlibm)
        ("p7zip" ,p7zip)
        ("pcre2" ,pcre2)
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index f5a2181905..2d3ce41cb7 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -4645,7 +4645,9 @@ (define-public openblas-ilp64
     (arguments
      (substitute-keyword-arguments (package-arguments openblas)
        ((#:make-flags flags #~'())
-        #~(append (list "INTERFACE64=1" "LIBNAMESUFFIX=ilp64")
+        #~(append (list "INTERFACE64=1"
+                        "SYMBOLSUFFIX=64_"
+                        "LIBPREFIX=libopenblas64_")
                  #$flags))))
     (synopsis "Optimized BLAS library based on GotoBLAS (ILP64 version)")
     (license license:bsd-3)))

Cayetano Santos wrote on 15 Sep 2023 21:45

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)

Message-ID:87jzsr2p69.fsf@inventati.org

Toggle quote (27 lines)>jeu. 14 sept. 2023 at 13:33, Efraim Flashner <efraim@flashner.co.il> wrote:

> [[PGP Signed Part:Undecided]]
> On Sun, Aug 20, 2023 at 10:53:44PM +0200, Ludovic Courtès wrote:
>> Hi!
>>
>> Friendly ping.  :-)
>>
>>   https://issues.guix.gnu.org/63986
>>
>> Ludo’.
>>
>> Ludovic Courtès <ludo@gnu.org> skribis:
>>
>> > Hi there!
>> >
>> > What’s the status?  Sounds like we have a couple of fixes already.
>> >
>> > Maybe you can submit one of them to guix-patches@gnu.org so qa.guix can
>> > pick it up.  And if one of them is more intrusive (more rebuilds), then
>> > submit it separately so it gets merged later?  How does that sound?
>> >
>> > Ludo’.
>
> I've attached a diff to adjust openblas64 and to use it for x86_64 in
> julia. I don't know if it's faster than the current openblas.

I have applied the patch in a freshly cloned guix repo, and build julia

within a shell as for the instructions under ’Contributing’.

I get the 13 ms when running the original test, so I guess the issue is

solved (other than thinking about the feasibility of performance tests

to avoid this kind of situations).

Thanks a lot !

Cayetano Santos wrote on 15 Sep 2023 21:49

Fixed.

Recipients:(address . 63986-done@debbugs.gnu.org)

Message-ID:87fs3f2p3y.fsf@inventati.org

Closed

Simon Tournier wrote on 16 Sep 2023 12:23

[PATCH 0/2] Fix bug#63986: Julia is very slow

Recipients:(address . guix-patches@gnu.org)(name . Simon Tournier)(address . zimon.toutoune@gmail.com)

Message-ID:cover.1694859477.git.zimon.toutoune@gmail.com

Hi,

This series is for triggering the rebuild of the Julia world. Some Julia

packages are long to build so let minimize the annoyance and make some

substitutes available before pushing to master. :-)

bug#63986: Julia is very slow

Cayetano Santos via Bug reports for GNU Guix <bug-guix@gnu.org>

Fri, 09 Jun 2023 23:42:00 +0200

id:87legsl3cm.fsf@inventati.org

https://yhetil.org/guix/87legsl3cm.fsf@inventati.org

https://issues.guix.gnu.org/msgid/87legsl3cm.fsf@inventati.org

Cheers,

simon

Efraim Flashner (2):

gnu: openblas-ilp64: Add symbol suffix.

gnu: julia: Conditionally use openblas with ILP64 support.

gnu/packages/julia.scm | 22 ++++++++++++++--------

gnu/packages/maths.scm | 4 +++-

2 files changed, 17 insertions(+), 9 deletions(-)

base-commit: eeec8371890fc95c60112da299dde78b21c948c2

2.38.1

Simon Tournier wrote on 16 Sep 2023 12:47

Re: bug#63986: Julia is very slow

Recipients:

Message-ID:87v8ca8ke3.fsf@gmail.com

Hi Efraim,

On Thu, 14 Sep 2023 at 13:33, Efraim Flashner <efraim@flashner.co.il> wrote:

Toggle quote (5 lines)

> On Sun, Aug 20, 2023 at 10:53:44PM +0200, Ludovic Courtès wrote:

> I've attached a diff to adjust openblas64 and to use it for x86_64 in

> julia. I don't know if it's faster than the current openblas.

I have applied the diff and turned into two patches. Attached below.

Then, I have sent the cover-letter to guix-patches where you are CC.

The idea was to trigger the Julia world rebuild. However, I do not get

back any Debbugs ID for sending the two patches.

I think that before pushing the change to master, we should wait that QA

builds some substitutes. It will reduce the annoyance for user since

some Julia paackages are long to build.

I have no time today for waiting after some Debbugs troubles.

Cheers,

simon

From 7f2347d8e64562e1de563cf1e4cdfd192e81a9df Mon Sep 17 00:00:00 2001

Message-Id: <cover.1694859477.git.zimon.toutoune@gmail.com>

X-Debbugs-CC: Efraim Flashner <efraim@flashner.co.il>, Cayetano Santos <csantosb@inventati.org>,

Hi,

This series is for triggering the rebuild of the Julia world. Some Julia

packages are long to build so let minimize the annoyance and make some

substitutes available before pushing to master. :-)

bug#63986: Julia is very slow

Cayetano Santos via Bug reports for GNU Guix <bug-guix@gnu.org>

Fri, 09 Jun 2023 23:42:00 +0200

id:87legsl3cm.fsf@inventati.org

https://yhetil.org/guix/87legsl3cm.fsf@inventati.org

https://issues.guix.gnu.org/msgid/87legsl3cm.fsf@inventati.org

Cheers,

simon

Efraim Flashner (2):

gnu: openblas-ilp64: Add symbol suffix.

gnu: julia: Conditionally use openblas with ILP64 support.

gnu/packages/julia.scm | 22 ++++++++++++++--------

gnu/packages/maths.scm | 4 +++-

2 files changed, 17 insertions(+), 9 deletions(-)

base-commit: eeec8371890fc95c60112da299dde78b21c948c2

2.38.1

From 2f044415efb5dc5703bf2b09439c6f5e2fd1fd72 Mon Sep 17 00:00:00 2001
Message-Id: <2f044415efb5dc5703bf2b09439c6f5e2fd1fd72.1694859477.git.zimon.toutoune@gmail.com>
In-Reply-To: <cover.1694859477.git.zimon.toutoune@gmail.com>
References: <cover.1694859477.git.zimon.toutoune@gmail.com>
From: Efraim Flashner <efraim@flashner.co.il>
Date: Sat, 16 Sep 2023 11:59:28 +0200
Subject: [PATCH 1/2] gnu: openblas-ilp64: Add symbol suffix.

* gnu/packages/maths.scm (openblas-ilp64)[arguments]<#:make-flags>: Add the
symbol '64_' as suffix.
---
 gnu/packages/maths.scm | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

Toggle diff (17 lines)diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 6141c0988612..c051e60d5e40 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -4638,7 +4638,9 @@ (define-public openblas-ilp64
     (arguments
      (substitute-keyword-arguments (package-arguments openblas)
        ((#:make-flags flags #~'())
-        #~(append (list "INTERFACE64=1" "LIBNAMESUFFIX=ilp64")
+        #~(append (list "INTERFACE64=1"
+                        "SYMBOLSUFFIX=64_"
+                        "LIBPREFIX=libopenblas64_")
                  #$flags))))
     (synopsis "Optimized BLAS library based on GotoBLAS (ILP64 version)")
     (license license:bsd-3)))
-- 
2.38.1

From 7f2347d8e64562e1de563cf1e4cdfd192e81a9df Mon Sep 17 00:00:00 2001
Message-Id: <7f2347d8e64562e1de563cf1e4cdfd192e81a9df.1694859477.git.zimon.toutoune@gmail.com>
In-Reply-To: <cover.1694859477.git.zimon.toutoune@gmail.com>
References: <cover.1694859477.git.zimon.toutoune@gmail.com>
From: Efraim Flashner <efraim@flashner.co.il>
Date: Sat, 16 Sep 2023 12:05:41 +0200
Subject: [PATCH 2/2] gnu: julia: Conditionally use openblas with ILP64
 support.

Fixes https://bugs.gnu.org/63986.
Reported by Cayetano Santos <csantosb@inventati.org>.

* gnu/packages/julia.scm (julia)[arguments]<#:phases>: Conditionally use
OpenBLAS with ILP64 support for x86-64 target.
<#:make-flags>: Likewise.
[inputs]: Conditionally replace openblas by openblas-ilp64 for x86-64 target.

Co-authored-by: Simon Tournier <zimon.toutoune@gmail.com>
---
 gnu/packages/julia.scm | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)

Toggle diff (49 lines)diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
index ba541758223e..8ba2b480ce55 100644
--- a/gnu/packages/julia.scm
+++ b/gnu/packages/julia.scm
@@ -269,7 +269,10 @@ (define-public julia
                (substitute* (jlpath "nghttp2")
                  (((from "libnghttp2")) (to "libnghttp2" "libnghttp2")))
                (substitute* (jlpath "OpenBLAS")
-                 (((from "libopenblas")) (to "openblas" "libopenblas")))
+                 (((from "libopenblas"))
+                  ,@(if (target-x86-64?)
+                      `((to "openblas" "libopenblas64_" "libopenblas"))
+                      `((to "openblas" "libopenblas")))))
                (substitute* (jlpath "OpenLibm")
                  (((from "libopenlibm")) (to "openlibm" "libopenlibm")))
                (substitute* (jlpath "PCRE2")
@@ -479,12 +482,13 @@ (define-public julia
          "NO_GIT=1"             ; build from release tarball.
          "USE_GPL_LIBS=1"       ; proudly
 
-         ,@(if (target-aarch64?)
-             `("USE_BLAS64=0")
-             '())
-
-         "LIBBLAS=-lopenblas"
-         "LIBBLASNAME=libopenblas"
+         ,@(if (target-x86-64?)
+             `("USE_BLAS64=1"
+               "LIBBLAS=-lopenblas64_"
+               "LIBBLASNAME=libopenblas64_")
+             `("USE_BLAS64=0"
+               "LIBBLAS=-lopenblas"
+               "LIBBLASNAME=libopenblas"))
 
          (string-append "UTF8PROC_INC="
                         (assoc-ref %build-inputs "utf8proc")
@@ -513,7 +517,9 @@ (define-public julia
        ("llvm" ,llvm-julia)
        ("mbedtls-apache" ,mbedtls-apache)
        ("mpfr" ,mpfr)
-       ("openblas" ,openblas)
+       ,@(if (target-x86-64?)
+             `(("openblas" ,openblas-ilp64))
+             `(("openblas" ,openblas)))
        ("openlibm" ,openlibm)
        ("p7zip" ,p7zip)
        ("pcre2" ,pcre2)
-- 
2.38.1

Tobias Geerinckx-Rice wrote on 16 Sep 2023 13:02

(no subject)

Recipients:(address . control@debbugs.gnu.org)

Message-ID:4b62da6efd14aee35d433e8840ef04c0@tobias.gr

tags 63986 patch

Simon Tournier wrote on 16 Sep 2023 13:03

Re: bug#63986: Julia is very slow

Recipients:(name . GNU bug Tracking System)(address . help-debbugs@gnu.org)

Message-ID:86fs3e4bxo.fsf_-_@gmail.com

Hi Cayetano,

On Fri, 15 Sep 2023 at 19:51, help-debbugs@gnu.org (GNU bug Tracking System) wrote:

Toggle quote (7 lines)

> #63986: Julia is very slow

> which was filed against the guix package, has been closed.

> The explanation is attached below, along with your original report.

> If you require more details, please reply to 63986@debbugs.gnu.org.

Since nothing had been applied for fixing the bug, could you please keep

it open. Let close it once all is fine in master. WDYT?

Cheers,

simon

Simon Tournier wrote on 16 Sep 2023 13:18

Recipients:

Message-ID:87o7i28iy3.fsf@gmail.com

Hi,

On Sat, 16 Sep 2023 at 12:47, Simon Tournier <zimon.toutoune@gmail.com> wrote:

Toggle quote (5 lines)

> I have applied the diff and turned into two patches. Attached below.

> Then, I have sent the cover-letter to guix-patches where you are CC.

> The idea was to trigger the Julia world rebuild. However, I do not get

> back any Debbugs ID for sending the two patches.

Now tacked by #66030. And progress here:

https://qa.guix.gnu.org/issue/66030

Cheers,

simon

Simon Tournier wrote on 20 Sep 2023 17:57

Recipients:

Message-ID:87y1h096s5.fsf@gmail.com

Hi Efraim,

Applying the patch is done in v3 of #66030.

https://issues.guix.gnu.org/issue/66030

and QA processed all.

https://qa.guix.gnu.org/issue/66030/

It is almost good except one strong annoyance [1]@

Toggle snippet (36 lines)Singular value decomposition |   57     57  5.0s
Hermitian: Error During Test at /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/test/rectfullpacked.jl:12
  Got exception outside of a @test
  could not load symbol "dsfrk_64_":
  /gnu/store/h5mgc7ar7a05f9rwrd1makhzays5wd3s-julia-1.8.3/bin/../lib/julia/liblapack.so: undefined symbol: dsfrk_64_
  Stacktrace:
    [1] sfrk!(transr::Char, uplo::Char, trans::Char, alpha::Float64, A::Matrix{Float64}, beta::Float64, C::Vector{Float64})
      @ GenericLinearAlgebra.LAPACK2 /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/src/lapack.jl:523
    [2] Ac_mul_A_RFP(A::Matrix{Float64}, uplo::Symbol)
      @ GenericLinearAlgebra /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/src/rectfullpacked.jl:77
    [3] macro expansion
      @ /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/test/rectfullpacked.jl:13 [inlined]
    [4] macro expansion
      @ /gnu/store/h5mgc7ar7a05f9rwrd1makhzays5wd3s-julia-1.8.3/share/julia/stdlib/v1.8/Test/src/Test.jl:1360 [inlined]
    [5] macro expansion
      @ /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/test/rectfullpacked.jl:13 [inlined]
    [6] macro expansion
      @ /gnu/store/h5mgc7ar7a05f9rwrd1makhzays5wd3s-julia-1.8.3/share/julia/stdlib/v1.8/Test/src/Test.jl:1436 [inlined]
    [7] macro expansion
      @ /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/test/rectfullpacked.jl:7 [inlined]
    [8] macro expansion
      @ /gnu/store/h5mgc7ar7a05f9rwrd1makhzays5wd3s-julia-1.8.3/share/julia/stdlib/v1.8/Test/src/Test.jl:1360 [inlined]
    [9] top-level scope
      @ /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/test/rectfullpacked.jl:7
   [10] include(fname::String)
      @ Base.MainInclude ./client.jl:476
   [11] top-level scope
      @ /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/test/runtests.jl:10
   [12] include(mod::Module, _path::String)
      @ Base ./Base.jl:419
   [13] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:303
   [14] _start()
      @ Base ./client.jl:522

Any idea? I have no idea and it is blocking the merge because ~20

packages are then broken. Well, if we have no idea, I will push the fix

for “Julia is slow” and we will fix later these ~20 failures. WDYT?

Cheers,

simon

1: https://bordeaux.guix.gnu.org/build/30e3aaec-dbde-423b-9554-67a64881e0cf/log

Cayetano Santos wrote on 24 Sep 2023 10:37

Recipients:(name . Simon Tournier)(address . zimon.toutoune@gmail.com)

Message-ID:87fs34neyw.fsf@inventati.org

Toggle quote (8 lines)

>mer. 20 sept. 2023 at 17:57, Simon Tournier <zimon.toutoune@gmail.com> wrote:

> Hi Efraim,

> Any idea? I have no idea and it is blocking the merge because ~20

> packages are then broken. Well, if we have no idea, I will push the fix

> for “Julia is slow” and we will fix later these ~20 failures. WDYT?

Sounds good to me.

Ludovic Courtès wrote on 25 Sep 2023 16:46

Recipients:

Message-ID:87fs325mzi.fsf@gnu.org

Hi,

Simon Tournier <zimon.toutoune@gmail.com> skribis:

Toggle quote (8 lines)

> It is almost good except one strong annoyance [1]@

> Singular value decomposition | 57 57 5.0s

> Hermitian: Error During Test at /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/test/rectfullpacked.jl:12

> Got exception outside of a @test

> could not load symbol "dsfrk_64_":

> /gnu/store/h5mgc7ar7a05f9rwrd1makhzays5wd3s-julia-1.8.3/bin/../lib/julia/liblapack.so: undefined symbol: dsfrk_64_

The ‘_64’ suffix is added by a patch in this very series, that adds

‘SYMBOLSUFFIX=64_’ to ‘openblas-ilp64’.

I don’t know what the rationale was for that configuration change, but

this is a good area of investigation.

Efraim, WDYT?

Thanks,

Ludo’.

Efraim Flashner wrote on 26 Sep 2023 11:27

Recipients:(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)

Message-ID:ZRKkDNdJg1DaZVKA@3900XT

On Mon, Sep 25, 2023 at 04:46:57PM +0200, Ludovic Courtès wrote:

Toggle quote (20 lines)> Hi,
> 
> Simon Tournier <zimon.toutoune@gmail.com> skribis:
> 
> > It is almost good except one strong annoyance [1]@
> >
> > Singular value decomposition |   57     57  5.0s
> > Hermitian: Error During Test at /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/test/rectfullpacked.jl:12
> >   Got exception outside of a @test
> >   could not load symbol "dsfrk_64_":
> >   /gnu/store/h5mgc7ar7a05f9rwrd1makhzays5wd3s-julia-1.8.3/bin/../lib/julia/liblapack.so: undefined symbol: dsfrk_64_
> 
> The ‘_64’ suffix is added by a patch in this very series, that adds
> ‘SYMBOLSUFFIX=64_’ to ‘openblas-ilp64’.
> 
> I don’t know what the rationale was for that configuration change, but
> this is a good area of investigation.
> 
> Efraim, WDYT?

the SYMBOLSUFFIX change seems to be standard across other distributions,
and in fact there seem to be packages out there which rely on it
directly.

Looking at this error specifically and also the pcre2-8 errors we get
during the 'test phase for julia, it seems the suggested fix is to build
those libraries with julia. So we'd want to replace the pcre2 and the
lapack libraries in the julia sources (for pcre2 for all the
architectures, for lapack for x86_64 specifically) and let julia build
and link to them during the build phase.

-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Efraim Flashner wrote on 28 Sep 2023 13:41

Recipients:(name . Simon Tournier)(address . zimon.toutoune@gmail.com)

Message-ID:ZRVmc2OxC6DCLX9O@3900XT

On Wed, Sep 20, 2023 at 05:57:30PM +0200, Simon Tournier wrote:

Toggle quote (53 lines)> Hi Efraim,
> 
> Applying the patch is done in v3 of #66030.
> 
>     https://issues.guix.gnu.org/issue/66030
> 
> and QA processed all.
> 
>     https://qa.guix.gnu.org/issue/66030/
> 
> It is almost good except one strong annoyance [1]@
> 
> --8<---------------cut here---------------start------------->8---
> Singular value decomposition |   57     57  5.0s
> Hermitian: Error During Test at /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/test/rectfullpacked.jl:12
>   Got exception outside of a @test
>   could not load symbol "dsfrk_64_":
>   /gnu/store/h5mgc7ar7a05f9rwrd1makhzays5wd3s-julia-1.8.3/bin/../lib/julia/liblapack.so: undefined symbol: dsfrk_64_
>   Stacktrace:
>     [1] sfrk!(transr::Char, uplo::Char, trans::Char, alpha::Float64, A::Matrix{Float64}, beta::Float64, C::Vector{Float64})
>       @ GenericLinearAlgebra.LAPACK2 /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/src/lapack.jl:523
>     [2] Ac_mul_A_RFP(A::Matrix{Float64}, uplo::Symbol)
>       @ GenericLinearAlgebra /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/src/rectfullpacked.jl:77
>     [3] macro expansion
>       @ /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/test/rectfullpacked.jl:13 [inlined]
>     [4] macro expansion
>       @ /gnu/store/h5mgc7ar7a05f9rwrd1makhzays5wd3s-julia-1.8.3/share/julia/stdlib/v1.8/Test/src/Test.jl:1360 [inlined]
>     [5] macro expansion
>       @ /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/test/rectfullpacked.jl:13 [inlined]
>     [6] macro expansion
>       @ /gnu/store/h5mgc7ar7a05f9rwrd1makhzays5wd3s-julia-1.8.3/share/julia/stdlib/v1.8/Test/src/Test.jl:1436 [inlined]
>     [7] macro expansion
>       @ /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/test/rectfullpacked.jl:7 [inlined]
>     [8] macro expansion
>       @ /gnu/store/h5mgc7ar7a05f9rwrd1makhzays5wd3s-julia-1.8.3/share/julia/stdlib/v1.8/Test/src/Test.jl:1360 [inlined]
>     [9] top-level scope
>       @ /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/test/rectfullpacked.jl:7
>    [10] include(fname::String)
>       @ Base.MainInclude ./client.jl:476
>    [11] top-level scope
>       @ /gnu/store/zkx6p7kz3m5k5w5iy0l1d09b2n8b0ib3-julia-genericlinearalgebra-0.2.5/share/julia/loadpath/GenericLinearAlgebra/test/runtests.jl:10
>    [12] include(mod::Module, _path::String)
>       @ Base ./Base.jl:419
>    [13] exec_options(opts::Base.JLOptions)
>       @ Base ./client.jl:303
>    [14] _start()
>       @ Base ./client.jl:522
> --8<---------------cut here---------------end--------------->8---
> 
> Any idea?  I have no idea and it is blocking the merge because ~20
> packages are then broken.  Well, if we have no idea, I will push the fix
> for “Julia is slow” and we will fix later these ~20 failures.  WDYT?

This one was easy, I just upgraded it to 0.3.0. Then I had to adjust
julia-bandedmatrices and julia-arraylayouts and everything looks good.
Except for julia-optim.

I spent a few hours trying to get Julia to use accept '64' instead of
'64_' for SYMBOLSUFFIX and even packaged a newer version of lapack but I
wasn't able to force the issue. Ultimately any other  julia packages
need to use libblastrampoline to determine whether to use openblas or
lapack.

Patches pushed, issue closed. I even tested the original reproducer from
the first email.

-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Closed

Ludovic Courtès wrote on 29 Sep 2023 10:46

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)

Message-ID:877co9wenk.fsf@inria.fr

Hi,

Efraim Flashner <efraim@flashner.co.il> skribis:

Toggle quote (2 lines)

> On Mon, Sep 25, 2023 at 04:46:57PM +0200, Ludovic Courtès wrote:

[...]

Toggle quote (21 lines)>> >   /gnu/store/h5mgc7ar7a05f9rwrd1makhzays5wd3s-julia-1.8.3/bin/../lib/julia/liblapack.so: undefined symbol: dsfrk_64_
>> 
>> The ‘_64’ suffix is added by a patch in this very series, that adds
>> ‘SYMBOLSUFFIX=64_’ to ‘openblas-ilp64’.
>> 
>> I don’t know what the rationale was for that configuration change, but
>> this is a good area of investigation.
>> 
>> Efraim, WDYT?
>
> the SYMBOLSUFFIX change seems to be standard across other distributions,
> and in fact there seem to be packages out there which rely on it
> directly.
>
> Looking at this error specifically and also the pcre2-8 errors we get
> during the 'test phase for julia, it seems the suggested fix is to build
> those libraries with julia. So we'd want to replace the pcre2 and the
> lapack libraries in the julia sources (for pcre2 for all the
> architectures, for lapack for x86_64 specifically) and let julia build
> and link to them during the build phase.

That would seem like a step backwards though, no? Usually we prefer to

unbundle things.

Actually, why does liblapack.so end up in Julia itself, as opposed to

just linking to libopenblas.so?

Ludo’.

Efraim Flashner wrote on 29 Sep 2023 12:12

Recipients:(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)

Message-ID:ZRai-2kitgPkvO8A@3900XT

On Fri, Sep 29, 2023 at 10:46:07AM +0200, Ludovic Courtès wrote:

Toggle quote (32 lines)> Hi,
> 
> Efraim Flashner <efraim@flashner.co.il> skribis:
> 
> > On Mon, Sep 25, 2023 at 04:46:57PM +0200, Ludovic Courtès wrote:
> 
> [...]
> 
> >> >   /gnu/store/h5mgc7ar7a05f9rwrd1makhzays5wd3s-julia-1.8.3/bin/../lib/julia/liblapack.so: undefined symbol: dsfrk_64_
> >> 
> >> The ‘_64’ suffix is added by a patch in this very series, that adds
> >> ‘SYMBOLSUFFIX=64_’ to ‘openblas-ilp64’.
> >> 
> >> I don’t know what the rationale was for that configuration change, but
> >> this is a good area of investigation.
> >> 
> >> Efraim, WDYT?
> >
> > the SYMBOLSUFFIX change seems to be standard across other distributions,
> > and in fact there seem to be packages out there which rely on it
> > directly.
> >
> > Looking at this error specifically and also the pcre2-8 errors we get
> > during the 'test phase for julia, it seems the suggested fix is to build
> > those libraries with julia. So we'd want to replace the pcre2 and the
> > lapack libraries in the julia sources (for pcre2 for all the
> > architectures, for lapack for x86_64 specifically) and let julia build
> > and link to them during the build phase.
> 
> That would seem like a step backwards though, no?  Usually we prefer to
> unbundle things.

I tried it with inserting our source for pcre2 and adding a patch to fix

the configure phase of pcre2, we ended up with other failures, so I

think we're best off with what we have now.

Toggle quote (3 lines)

> Actually, why does liblapack.so end up in Julia itself, as opposed to

> just linking to libopenblas.so?

Julia links to both (open)blas and to lapack. If we build openblas so
that it also provides lapack, or use (c)blas from lapack, then we can
use only one or the other. I suppose in theory it should be possible to
tell julia that libopenblas64_.so is really lapack and to use that for
both of them. I don't know how well that would work though.

-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Your comment

This issue is archived.

To comment on this conversation send an email to 63986@debbugs.gnu.org

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date