Cannot import IJulia in Julia

  • Done
  • quality assurance status badge
Details
3 participants
  • Efraim Flashner
  • Theodore Ehrenborg
  • Simon Tournier
Owner
unassigned
Submitted by
Theodore Ehrenborg
Severity
normal
T
T
Theodore Ehrenborg wrote on 28 Jan 2023 14:45
(address . bug-guix@gnu.org)
CA+oeq6n9L3UTPMSpBb8ZWmhCEHMA0L_Jae7zyCvEZb4UTOPxkQ@mail.gmail.com
Hi Guix,

I would like to run a Jupyter notebook using Julia, so I need to install
the IJulia backend:

guix install julia
julia # Enter julia REPL
] # To go into the julia pkg REPL
add IJulia
# Now type backspace to go to julia REPL
using IJulia

This produces the error:

[ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
ERROR: LoadError: InitError: SystemError: opening file
"/gnu/store/npj8z0g9nx14wl22yphqfs2c5w4qk5jk-julia-1.8.3/share/julia/cert.pem":
No such file or directory

The full error message is here: https://pastebin.com/qC8yyHXT

I saw a very similar bug on Gentoo:

Without this file (which can be a symbolic link to
`/etc/ssl/certs/ca-certificates.crt`) many Julia 1.8.3 packages, e.g.
`HTTP`, do not work.
This is what happens:

julia> import HTTP
[ Info: Precompiling HTTP [cd3eb016-35fb-5094-929b-558a96fad6f3]
ERROR: LoadError: InitError: SystemError: opening file
"/usr/share/julia/cert.pem":


Any help would be greatly appreciated.

Best regards,
Theodore Ehrenborg
Attachment: file
S
S
Simon Tournier wrote on 30 Jan 2023 13:27
(name . Efraim Flashner)(address . efraim@flashner.co.il)
87bkmgky0p.fsf@gmail.com
Hi,

I confirm this bug.

On sam., 28 janv. 2023 at 13:45, Theodore Ehrenborg <theodore.ehrenborg@gmail.com> wrote:

Toggle quote (5 lines)
> [ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
> ERROR: LoadError: InitError: SystemError: opening file
> "/gnu/store/npj8z0g9nx14wl22yphqfs2c5w4qk5jk-julia-1.8.3/share/julia/cert.pem":
> No such file or directory

[...]

Toggle quote (2 lines)
> I saw a very similar bug on Gentoo:

[...]

Toggle quote (2 lines)
Well, that’s because Julia upstream does not take care about packagers;
as explicitly mentioned in this comment:


The Guixer Cayetano Santos fixed upstream the issue for one package.
But as you are noticing it is not done for all.

I do not know what is the best solution because the issue is coming from
Julia itself.

Efraim, any suggestion?

Cheers,
simon
T
T
Theodore Ehrenborg wrote on 30 Jan 2023 22:55
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
CA+oeq6n4Vx1yg0yz2oQ2t6DbFnT-vt3fV4qFo4oEkv-5vF-SXQ@mail.gmail.com
Hi,

Thanks for getting back with me.

Gentoo appears to have fixed this bug by linking julia/cert.pem to the
system's ca-certificates.crt.

Is there a way I could rebuild my own slightly modified Julia with a link
like that?

I understand that there's probably a good reason that Guix's Julia doesn't
by default have cert.pem, but I would be pleased with a hacky custom
solution if it made Jupyter notebooks work.

Thanks,
Theodore

Den mån 30 jan. 2023 kl 12:47 skrev Simon Tournier <zimon.toutoune@gmail.com
Toggle quote (40 lines)
>:

> Hi,
>
> I confirm this bug.
>
> On sam., 28 janv. 2023 at 13:45, Theodore Ehrenborg <
> theodore.ehrenborg@gmail.com> wrote:
>
> > [ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
> > ERROR: LoadError: InitError: SystemError: opening file
> >
> "/gnu/store/npj8z0g9nx14wl22yphqfs2c5w4qk5jk-julia-1.8.3/share/julia/cert.pem":
> > No such file or directory
>
> [...]
>
> > I saw a very similar bug on Gentoo:
>
> [...]
>
> > (https://bugs.gentoo.org/888978)
>
> Well, that’s because Julia upstream does not take care about packagers;
> as explicitly mentioned in this comment:
>
>
> https://github.com/JuliaLang/MbedTLS.jl/pull/261#issuecomment-1346886879
>
> The Guixer Cayetano Santos fixed upstream the issue for one package.
> But as you are noticing it is not done for all.
>
> I do not know what is the best solution because the issue is coming from
> Julia itself.
>
> Efraim, any suggestion?
>
> Cheers,
> simon
>
Attachment: file
S
S
Simon Tournier wrote on 31 Jan 2023 12:34
(name . Theodore Ehrenborg)(address . theodore.ehrenborg@gmail.com)
86o7qfuedj.fsf@gmail.com
Hi,

On Mon, 30 Jan 2023 at 21:55, Theodore Ehrenborg <theodore.ehrenborg@gmail.com> wrote:

Toggle quote (4 lines)
> Gentoo appears to have fixed this bug by linking julia/cert.pem to the
> system's ca-certificates.crt.
> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=26b59330b5222996defa4536237e62404bf21168

This trick is not possible, IIUC.

Toggle quote (3 lines)
> Is there a way I could rebuild my own slightly modified Julia with a link
> like that?

Maybe, by adding the package nss-certs as propagated-inputs in the
definition of julia.

Toggle quote (4 lines)
> I understand that there's probably a good reason that Guix's Julia doesn't
> by default have cert.pem, but I would be pleased with a hacky custom
> solution if it made Jupyter notebooks work.

The reason is security. ;-) It’s Julia that does poorly here.

As pointed with the upstream package MbedTLS.jl, the fix should come
from Julia itself; therefore, it could be worth to open an issue, if it
is not already the case. ;-)

From my understanding, the culprit is this [1]:

Toggle snippet (6 lines)
function __init__()
global artifact_dir = dirname(Sys.BINDIR)
global cacert = normpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "cert.pem")
end

And it is not clear for me if NetworkOptions.jl [2] provides the option
of not, and I am missing why Julia itself does not depend on it.



Efraim, do you think it would be possible to patch Julia to point to
some certificates via bundled_ca_roots or ca_roots_path?

Well, somehow turn back these tests:

Toggle snippet (15 lines)
;; julia embeds a certificate, we are not doing that
(substitute* "stdlib/MozillaCACerts_jll/test/runtests.jl"
(("@test isfile\\(MozillaCACerts_jll.cacert\\)")
"@test_broken isfile(MozillaCACerts_jll.cacert)"))
;; since certificate is not present some tests are failing in network option
(substitute* "usr/share/julia/stdlib/v1.8/NetworkOptions/test/runtests.jl"
(("@test isfile\\(bundled_ca_roots\\(\\)\\)")
"@test_broken isfile(bundled_ca_roots())")
(("@test ispath\\(ca_roots_path\\(\\)\\)")
"@test_broken ispath(ca_roots_path())")
(("@test ca_roots_path\\(\\) \\!= bundled_ca_roots\\(\\)")
"@test_broken ca_roots_path() != bundled_ca_roots()"))


Cheers,
simon
E
E
Efraim Flashner wrote on 9 Feb 2023 10:29
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
Y+S9Ed+J3eJNsGGQ@3900XT
On Tue, Jan 31, 2023 at 12:34:16PM +0100, Simon Tournier wrote:
Toggle quote (16 lines)
> Hi,
>
> On Mon, 30 Jan 2023 at 21:55, Theodore Ehrenborg <theodore.ehrenborg@gmail.com> wrote:
>
> > Gentoo appears to have fixed this bug by linking julia/cert.pem to the
> > system's ca-certificates.crt.
> > https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=26b59330b5222996defa4536237e62404bf21168
>
> This trick is not possible, IIUC.
>
> > Is there a way I could rebuild my own slightly modified Julia with a link
> > like that?
>
> Maybe, by adding the package nss-certs as propagated-inputs in the
> definition of julia.

By itself I don't think this would do anything.

Toggle quote (29 lines)
> > I understand that there's probably a good reason that Guix's Julia doesn't
> > by default have cert.pem, but I would be pleased with a hacky custom
> > solution if it made Jupyter notebooks work.
>
> The reason is security. ;-) It’s Julia that does poorly here.
>
> As pointed with the upstream package MbedTLS.jl, the fix should come
> from Julia itself; therefore, it could be worth to open an issue, if it
> is not already the case. ;-)
>
> From my understanding, the culprit is this [1]:
>
> --8<---------------cut here---------------start------------->8---
> function __init__()
> global artifact_dir = dirname(Sys.BINDIR)
> global cacert = normpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "cert.pem")
> end
> --8<---------------cut here---------------end--------------->8---
>
> And it is not clear for me if NetworkOptions.jl [2] provides the option
> of not, and I am missing why Julia itself does not depend on it.
>
> 1: https://github.com/JuliaLang/julia/blob/master/stdlib/MozillaCACerts_jll/src/MozillaCACerts_jll.jl#L20
> 2: https://github.com/JuliaLang/NetworkOptions.jl
>
>
> Efraim, do you think it would be possible to patch Julia to point to
> some certificates via bundled_ca_roots or ca_roots_path?

In the initial patch for julia-1.8.1 I think there was a substitution to
hardcode /etc/ssl/something instead for 'global cacert' but I took that
out since we don't like hardcoding that.

GIT_SSL_CAINFO=/home/efraim/.guix-home/profile/etc/ssl/certs/ca-certificates.crt
SSL_CERT_DIR=/run/current-system/profile/etc/ssl/certs
CURL_CA_BUNDLE=/home/efraim/.guix-home/profile/etc/ssl/certs/ca-certificates.crt
SSL_CERT_FILE=/run/current-system/profile/etc/ssl/certs/ca-certificates.crt

I think it would be fine to tell Julia to look at SSL_CERT_FILE as the
cacert so it can be overridden as desired, and then we can add a
(native-?)search-path to Julia for SSL_CERT_FILE.

Does anyone know offhand how to get the environment variable? If not
I'll grep the sources and then look online.

Toggle quote (17 lines)
> Well, somehow turn back these tests:
>
> --8<---------------cut here---------------start------------->8---
> ;; julia embeds a certificate, we are not doing that
> (substitute* "stdlib/MozillaCACerts_jll/test/runtests.jl"
> (("@test isfile\\(MozillaCACerts_jll.cacert\\)")
> "@test_broken isfile(MozillaCACerts_jll.cacert)"))
> ;; since certificate is not present some tests are failing in network option
> (substitute* "usr/share/julia/stdlib/v1.8/NetworkOptions/test/runtests.jl"
> (("@test isfile\\(bundled_ca_roots\\(\\)\\)")
> "@test_broken isfile(bundled_ca_roots())")
> (("@test ispath\\(ca_roots_path\\(\\)\\)")
> "@test_broken ispath(ca_roots_path())")
> (("@test ca_roots_path\\(\\) \\!= bundled_ca_roots\\(\\)")
> "@test_broken ca_roots_path() != bundled_ca_roots()"))
> --8<---------------cut here---------------end--------------->8---

That one might be a little harder, I'd rather not add nss-certs to the
build just for the test suite, but I'll see how it goes. Or at least
update the comment afterward.

Toggle quote (4 lines)
>
> Cheers,
> simon

--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmPkvRAACgkQQarn3Mo9
g1H42g/+NBbSto/YVvqnqo1gX6zgFglkzfca4ROj4qznOnZ9ho16YHD+Y2r9tQO2
shkJUzx3XfR4S3Tn8gSwHVzLttk25hcDxDwYcjsiq15qPzfdtmlOubhDQ9PcHKxZ
V51m6hyzwONDxH7mvgbBs1s+R/Sj3UQ6weS5zZ7KodB7wo2XMrfGPGckc+0lz8HO
o+72HsQjuqKrFrdiiLkQ9rqAz8OE/4nVYRRSUSmICLOT/1EDHfMY9T1Ka3QZte1S
YSAyLfey1/s5LFPo0jOcJxNVRAtfC8aqt1m4iOXo7Dpp+jbO8lZLVkloZflHxNjx
oLuwfmri2z/WSCz61Crlk+q9mgwSE2/gDKpzonyi7+F5fzH2wNH5uKy6M124/kUE
TXA5MBUcj+zAReMC0sctkzwHgIrAx8aSF6Cw1RgpY7GHCaOn9YqtoGYefntN5eDX
+3nCuCssy5RMyXnNaVM+XZUvP1jTSbc6X+418pxKBZBA80H/mt2quLlALRIFFZ9G
Mz2KHNf0QyW2Ome0Wj65Q7uYeTc3l4ncZVaK5hTFVn29kNWPnJGJXQTqEJ6htNTZ
vrDLOdtYa/JDSQ7DppbbC4+piYBj9NgMolx0qEYPwOwLbxmYjtQaMtRfSYLd6SJ/
SA7o6HUhOOIuCQjGKhbWzc+NZS5Vg2L/FASwc7clheUr/lH99rY=
=9cwu
-----END PGP SIGNATURE-----


E
E
Efraim Flashner wrote on 9 Feb 2023 15:53
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
Y+UIzdetrRVjOdxZ@3900XT
On Tue, Jan 31, 2023 at 12:34:16PM +0100, Simon Tournier wrote:
Toggle quote (8 lines)
>
> --8<---------------cut here---------------start------------->8---
> function __init__()
> global artifact_dir = dirname(Sys.BINDIR)
> global cacert = normpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "cert.pem")
> end
> --8<---------------cut here---------------end--------------->8---

I've changed this line to:

global cacert = get(ENV, \"SSL_CERT_FILE\", "\"/etc/ssl/certs/ca-certificates.crt\")

and then tested it with the example at the beginning of the bug report.

Toggle quote (17 lines)
> Well, somehow turn back these tests:
>
> --8<---------------cut here---------------start------------->8---
> ;; julia embeds a certificate, we are not doing that
> (substitute* "stdlib/MozillaCACerts_jll/test/runtests.jl"
> (("@test isfile\\(MozillaCACerts_jll.cacert\\)")
> "@test_broken isfile(MozillaCACerts_jll.cacert)"))
> ;; since certificate is not present some tests are failing in network option
> (substitute* "usr/share/julia/stdlib/v1.8/NetworkOptions/test/runtests.jl"
> (("@test isfile\\(bundled_ca_roots\\(\\)\\)")
> "@test_broken isfile(bundled_ca_roots())")
> (("@test ispath\\(ca_roots_path\\(\\)\\)")
> "@test_broken ispath(ca_roots_path())")
> (("@test ca_roots_path\\(\\) \\!= bundled_ca_roots\\(\\)")
> "@test_broken ca_roots_path() != bundled_ca_roots()"))
> --8<---------------cut here---------------end--------------->8---

I wasn't able to turn these tests back on though.

--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmPlCMwACgkQQarn3Mo9
g1FCJxAAwYDBBAH2Xzb6qIO9M7AC1o53GsKY5rktlaIsn0v9O39IHgz6fjMz+F7b
YVyDXTtG+qK7GxByg/PODpzhdl/lk+C38uF4J6aXqzInjgPLc8sPAaAJO7Bg+FEu
kyYADr7wbEmjJx5TDiIWC6AnfhPzO4Jw70W1GIEt2Av8CiZSGapUSJcGKKoMYuCk
cJK3EZo3kfHVnVi+nrOFGSXEIPRMjmILAGZDbC9RopqQt54NoL8Kwv2qrVebKJy1
RPNguyWfjaYE6+G2kPrkHUOSkMCQliNEUVoQKakI8+HOWXTAkA+X9K3gUDDs4BL+
0hFQiIccwS7++FAXZf0fOBWzy+ObwZEfdwVkmHnMzC9VCFoOe26LZMM9/ANexAOG
wqdGLX0HOchPjY+GN+loXzxxs50e4dVkzIzswloqDKN+GnJqq+YIbjoR397w0UYg
9m4rWT7AuWAN0Z/+3nA3fRoZOnjHNTUSrtc7RVQeB2SbyZJIqlR0zjKnweWiXBa8
kruTKWFaZkphi+VSJ9GJyBrvsfogeIvD3gU8yBDNjeHezh+nHzEN6h7pYNhwQfB4
6IYGv6eHpJNuQmRQNjz6yjSvV0HPVK8LdTnhjxsxjrL0PbrBdEtGw2IA46i/pK1Z
690DMayk6tZvHjAW70zCjB7FURdvARsTLsmcV6iEp7zas/P1moI=
=STwv
-----END PGP SIGNATURE-----


Closed
?