Cannot import IJulia in Julia

  • Done
  • quality assurance status badge
Details
3 participants
  • Efraim Flashner
  • Theodore Ehrenborg
  • Simon Tournier
Owner
unassigned
Submitted by
Theodore Ehrenborg
Severity
normal
T
T
Theodore Ehrenborg wrote on 28 Jan 2023 14:45
(address . bug-guix@gnu.org)
CA+oeq6n9L3UTPMSpBb8ZWmhCEHMA0L_Jae7zyCvEZb4UTOPxkQ@mail.gmail.com
Hi Guix,

I would like to run a Jupyter notebook using Julia, so I need to install
the IJulia backend:

guix install julia
julia # Enter julia REPL
] # To go into the julia pkg REPL
add IJulia
# Now type backspace to go to julia REPL
using IJulia

This produces the error:

[ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
ERROR: LoadError: InitError: SystemError: opening file
"/gnu/store/npj8z0g9nx14wl22yphqfs2c5w4qk5jk-julia-1.8.3/share/julia/cert.pem":
No such file or directory

The full error message is here: https://pastebin.com/qC8yyHXT

I saw a very similar bug on Gentoo:

Without this file (which can be a symbolic link to
`/etc/ssl/certs/ca-certificates.crt`) many Julia 1.8.3 packages, e.g.
`HTTP`, do not work.
This is what happens:

julia> import HTTP
[ Info: Precompiling HTTP [cd3eb016-35fb-5094-929b-558a96fad6f3]
ERROR: LoadError: InitError: SystemError: opening file
"/usr/share/julia/cert.pem":


Any help would be greatly appreciated.

Best regards,
Theodore Ehrenborg
Attachment: file
S
S
Simon Tournier wrote on 30 Jan 2023 13:27
(name . Efraim Flashner)(address . efraim@flashner.co.il)
87bkmgky0p.fsf@gmail.com
Hi,

I confirm this bug.

On sam., 28 janv. 2023 at 13:45, Theodore Ehrenborg <theodore.ehrenborg@gmail.com> wrote:

Toggle quote (5 lines)
> [ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
> ERROR: LoadError: InitError: SystemError: opening file
> "/gnu/store/npj8z0g9nx14wl22yphqfs2c5w4qk5jk-julia-1.8.3/share/julia/cert.pem":
> No such file or directory

[...]

Toggle quote (2 lines)
> I saw a very similar bug on Gentoo:

[...]

Toggle quote (2 lines)
Well, that’s because Julia upstream does not take care about packagers;
as explicitly mentioned in this comment:


The Guixer Cayetano Santos fixed upstream the issue for one package.
But as you are noticing it is not done for all.

I do not know what is the best solution because the issue is coming from
Julia itself.

Efraim, any suggestion?

Cheers,
simon
T
T
Theodore Ehrenborg wrote on 30 Jan 2023 22:55
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
CA+oeq6n4Vx1yg0yz2oQ2t6DbFnT-vt3fV4qFo4oEkv-5vF-SXQ@mail.gmail.com
Hi,

Thanks for getting back with me.

Gentoo appears to have fixed this bug by linking julia/cert.pem to the
system's ca-certificates.crt.

Is there a way I could rebuild my own slightly modified Julia with a link
like that?

I understand that there's probably a good reason that Guix's Julia doesn't
by default have cert.pem, but I would be pleased with a hacky custom
solution if it made Jupyter notebooks work.

Thanks,
Theodore

Den mån 30 jan. 2023 kl 12:47 skrev Simon Tournier <zimon.toutoune@gmail.com
Toggle quote (40 lines)
>:

> Hi,
>
> I confirm this bug.
>
> On sam., 28 janv. 2023 at 13:45, Theodore Ehrenborg <
> theodore.ehrenborg@gmail.com> wrote:
>
> > [ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
> > ERROR: LoadError: InitError: SystemError: opening file
> >
> "/gnu/store/npj8z0g9nx14wl22yphqfs2c5w4qk5jk-julia-1.8.3/share/julia/cert.pem":
> > No such file or directory
>
> [...]
>
> > I saw a very similar bug on Gentoo:
>
> [...]
>
> > (https://bugs.gentoo.org/888978)
>
> Well, that’s because Julia upstream does not take care about packagers;
> as explicitly mentioned in this comment:
>
>
> https://github.com/JuliaLang/MbedTLS.jl/pull/261#issuecomment-1346886879
>
> The Guixer Cayetano Santos fixed upstream the issue for one package.
> But as you are noticing it is not done for all.
>
> I do not know what is the best solution because the issue is coming from
> Julia itself.
>
> Efraim, any suggestion?
>
> Cheers,
> simon
>
Attachment: file
S
S
Simon Tournier wrote on 31 Jan 2023 12:34
(name . Theodore Ehrenborg)(address . theodore.ehrenborg@gmail.com)
86o7qfuedj.fsf@gmail.com
Hi,

On Mon, 30 Jan 2023 at 21:55, Theodore Ehrenborg <theodore.ehrenborg@gmail.com> wrote:

Toggle quote (4 lines)
> Gentoo appears to have fixed this bug by linking julia/cert.pem to the
> system's ca-certificates.crt.
> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=26b59330b5222996defa4536237e62404bf21168

This trick is not possible, IIUC.

Toggle quote (3 lines)
> Is there a way I could rebuild my own slightly modified Julia with a link
> like that?

Maybe, by adding the package nss-certs as propagated-inputs in the
definition of julia.

Toggle quote (4 lines)
> I understand that there's probably a good reason that Guix's Julia doesn't
> by default have cert.pem, but I would be pleased with a hacky custom
> solution if it made Jupyter notebooks work.

The reason is security. ;-) It’s Julia that does poorly here.

As pointed with the upstream package MbedTLS.jl, the fix should come
from Julia itself; therefore, it could be worth to open an issue, if it
is not already the case. ;-)

From my understanding, the culprit is this [1]:

Toggle snippet (6 lines)
function __init__()
global artifact_dir = dirname(Sys.BINDIR)
global cacert = normpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "cert.pem")
end

And it is not clear for me if NetworkOptions.jl [2] provides the option
of not, and I am missing why Julia itself does not depend on it.



Efraim, do you think it would be possible to patch Julia to point to
some certificates via bundled_ca_roots or ca_roots_path?

Well, somehow turn back these tests:

Toggle snippet (15 lines)
;; julia embeds a certificate, we are not doing that
(substitute* "stdlib/MozillaCACerts_jll/test/runtests.jl"
(("@test isfile\\(MozillaCACerts_jll.cacert\\)")
"@test_broken isfile(MozillaCACerts_jll.cacert)"))
;; since certificate is not present some tests are failing in network option
(substitute* "usr/share/julia/stdlib/v1.8/NetworkOptions/test/runtests.jl"
(("@test isfile\\(bundled_ca_roots\\(\\)\\)")
"@test_broken isfile(bundled_ca_roots())")
(("@test ispath\\(ca_roots_path\\(\\)\\)")
"@test_broken ispath(ca_roots_path())")
(("@test ca_roots_path\\(\\) \\!= bundled_ca_roots\\(\\)")
"@test_broken ca_roots_path() != bundled_ca_roots()"))


Cheers,
simon
E
E
Efraim Flashner wrote on 9 Feb 2023 10:29
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
Y+S9Ed+J3eJNsGGQ@3900XT
On Tue, Jan 31, 2023 at 12:34:16PM +0100, Simon Tournier wrote:
Toggle quote (16 lines)
> Hi,
>
> On Mon, 30 Jan 2023 at 21:55, Theodore Ehrenborg <theodore.ehrenborg@gmail.com> wrote:
>
> > Gentoo appears to have fixed this bug by linking julia/cert.pem to the
> > system's ca-certificates.crt.
> > https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=26b59330b5222996defa4536237e62404bf21168
>
> This trick is not possible, IIUC.
>
> > Is there a way I could rebuild my own slightly modified Julia with a link
> > like that?
>
> Maybe, by adding the package nss-certs as propagated-inputs in the
> definition of julia.

By itself I don't think this would do anything.

Toggle quote (29 lines)
> > I understand that there's probably a good reason that Guix's Julia doesn't
> > by default have cert.pem, but I would be pleased with a hacky custom
> > solution if it made Jupyter notebooks work.
>
> The reason is security. ;-) It’s Julia that does poorly here.
>
> As pointed with the upstream package MbedTLS.jl, the fix should come
> from Julia itself; therefore, it could be worth to open an issue, if it
> is not already the case. ;-)
>
> From my understanding, the culprit is this [1]:
>
> --8<---------------cut here---------------start------------->8---
> function __init__()
> global artifact_dir = dirname(Sys.BINDIR)
> global cacert = normpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "cert.pem")
> end
> --8<---------------cut here---------------end--------------->8---
>
> And it is not clear for me if NetworkOptions.jl [2] provides the option
> of not, and I am missing why Julia itself does not depend on it.
>
> 1: https://github.com/JuliaLang/julia/blob/master/stdlib/MozillaCACerts_jll/src/MozillaCACerts_jll.jl#L20
> 2: https://github.com/JuliaLang/NetworkOptions.jl
>
>
> Efraim, do you think it would be possible to patch Julia to point to
> some certificates via bundled_ca_roots or ca_roots_path?

In the initial patch for julia-1.8.1 I think there was a substitution to
hardcode /etc/ssl/something instead for 'global cacert' but I took that
out since we don't like hardcoding that.

GIT_SSL_CAINFO=/home/efraim/.guix-home/profile/etc/ssl/certs/ca-certificates.crt
SSL_CERT_DIR=/run/current-system/profile/etc/ssl/certs
CURL_CA_BUNDLE=/home/efraim/.guix-home/profile/etc/ssl/certs/ca-certificates.crt
SSL_CERT_FILE=/run/current-system/profile/etc/ssl/certs/ca-certificates.crt

I think it would be fine to tell Julia to look at SSL_CERT_FILE as the
cacert so it can be overridden as desired, and then we can add a
(native-?)search-path to Julia for SSL_CERT_FILE.

Does anyone know offhand how to get the environment variable? If not
I'll grep the sources and then look online.

Toggle quote (17 lines)
> Well, somehow turn back these tests:
>
> --8<---------------cut here---------------start------------->8---
> ;; julia embeds a certificate, we are not doing that
> (substitute* "stdlib/MozillaCACerts_jll/test/runtests.jl"
> (("@test isfile\\(MozillaCACerts_jll.cacert\\)")
> "@test_broken isfile(MozillaCACerts_jll.cacert)"))
> ;; since certificate is not present some tests are failing in network option
> (substitute* "usr/share/julia/stdlib/v1.8/NetworkOptions/test/runtests.jl"
> (("@test isfile\\(bundled_ca_roots\\(\\)\\)")
> "@test_broken isfile(bundled_ca_roots())")
> (("@test ispath\\(ca_roots_path\\(\\)\\)")
> "@test_broken ispath(ca_roots_path())")
> (("@test ca_roots_path\\(\\) \\!= bundled_ca_roots\\(\\)")
> "@test_broken ca_roots_path() != bundled_ca_roots()"))
> --8<---------------cut here---------------end--------------->8---

That one might be a little harder, I'd rather not add nss-certs to the
build just for the test suite, but I'll see how it goes. Or at least
update the comment afterward.

Toggle quote (4 lines)
>
> Cheers,
> simon

--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmPkvRAACgkQQarn3Mo9
g1H42g/+NBbSto/YVvqnqo1gX6zgFglkzfca4ROj4qznOnZ9ho16YHD+Y2r9tQO2
shkJUzx3XfR4S3Tn8gSwHVzLttk25hcDxDwYcjsiq15qPzfdtmlOubhDQ9PcHKxZ
V51m6hyzwONDxH7mvgbBs1s+R/Sj3UQ6weS5zZ7KodB7wo2XMrfGPGckc+0lz8HO
o+72HsQjuqKrFrdiiLkQ9rqAz8OE/4nVYRRSUSmICLOT/1EDHfMY9T1Ka3QZte1S
YSAyLfey1/s5LFPo0jOcJxNVRAtfC8aqt1m4iOXo7Dpp+jbO8lZLVkloZflHxNjx
oLuwfmri2z/WSCz61Crlk+q9mgwSE2/gDKpzonyi7+F5fzH2wNH5uKy6M124/kUE
TXA5MBUcj+zAReMC0sctkzwHgIrAx8aSF6Cw1RgpY7GHCaOn9YqtoGYefntN5eDX
+3nCuCssy5RMyXnNaVM+XZUvP1jTSbc6X+418pxKBZBA80H/mt2quLlALRIFFZ9G
Mz2KHNf0QyW2Ome0Wj65Q7uYeTc3l4ncZVaK5hTFVn29kNWPnJGJXQTqEJ6htNTZ
vrDLOdtYa/JDSQ7DppbbC4+piYBj9NgMolx0qEYPwOwLbxmYjtQaMtRfSYLd6SJ/
SA7o6HUhOOIuCQjGKhbWzc+NZS5Vg2L/FASwc7clheUr/lH99rY=
=9cwu
-----END PGP SIGNATURE-----


E
E
Efraim Flashner wrote on 9 Feb 2023 15:53
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
Y+UIzdetrRVjOdxZ@3900XT
On Tue, Jan 31, 2023 at 12:34:16PM +0100, Simon Tournier wrote:
Toggle quote (8 lines)
>
> --8<---------------cut here---------------start------------->8---
> function __init__()
> global artifact_dir = dirname(Sys.BINDIR)
> global cacert = normpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "cert.pem")
> end
> --8<---------------cut here---------------end--------------->8---

I've changed this line to:

global cacert = get(ENV, \"SSL_CERT_FILE\", "\"/etc/ssl/certs/ca-certificates.crt\")

and then tested it with the example at the beginning of the bug report.

Toggle quote (17 lines)
> Well, somehow turn back these tests:
>
> --8<---------------cut here---------------start------------->8---
> ;; julia embeds a certificate, we are not doing that
> (substitute* "stdlib/MozillaCACerts_jll/test/runtests.jl"
> (("@test isfile\\(MozillaCACerts_jll.cacert\\)")
> "@test_broken isfile(MozillaCACerts_jll.cacert)"))
> ;; since certificate is not present some tests are failing in network option
> (substitute* "usr/share/julia/stdlib/v1.8/NetworkOptions/test/runtests.jl"
> (("@test isfile\\(bundled_ca_roots\\(\\)\\)")
> "@test_broken isfile(bundled_ca_roots())")
> (("@test ispath\\(ca_roots_path\\(\\)\\)")
> "@test_broken ispath(ca_roots_path())")
> (("@test ca_roots_path\\(\\) \\!= bundled_ca_roots\\(\\)")
> "@test_broken ca_roots_path() != bundled_ca_roots()"))
> --8<---------------cut here---------------end--------------->8---

I wasn't able to turn these tests back on though.

--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmPlCMwACgkQQarn3Mo9
g1FCJxAAwYDBBAH2Xzb6qIO9M7AC1o53GsKY5rktlaIsn0v9O39IHgz6fjMz+F7b
YVyDXTtG+qK7GxByg/PODpzhdl/lk+C38uF4J6aXqzInjgPLc8sPAaAJO7Bg+FEu
kyYADr7wbEmjJx5TDiIWC6AnfhPzO4Jw70W1GIEt2Av8CiZSGapUSJcGKKoMYuCk
cJK3EZo3kfHVnVi+nrOFGSXEIPRMjmILAGZDbC9RopqQt54NoL8Kwv2qrVebKJy1
RPNguyWfjaYE6+G2kPrkHUOSkMCQliNEUVoQKakI8+HOWXTAkA+X9K3gUDDs4BL+
0hFQiIccwS7++FAXZf0fOBWzy+ObwZEfdwVkmHnMzC9VCFoOe26LZMM9/ANexAOG
wqdGLX0HOchPjY+GN+loXzxxs50e4dVkzIzswloqDKN+GnJqq+YIbjoR397w0UYg
9m4rWT7AuWAN0Z/+3nA3fRoZOnjHNTUSrtc7RVQeB2SbyZJIqlR0zjKnweWiXBa8
kruTKWFaZkphi+VSJ9GJyBrvsfogeIvD3gU8yBDNjeHezh+nHzEN6h7pYNhwQfB4
6IYGv6eHpJNuQmRQNjz6yjSvV0HPVK8LdTnhjxsxjrL0PbrBdEtGw2IA46i/pK1Z
690DMayk6tZvHjAW70zCjB7FURdvARsTLsmcV6iEp7zas/P1moI=
=STwv
-----END PGP SIGNATURE-----


Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 61121@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 61121
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch