[PATCH] gnu: emacs-julia-snail: Vendor julia libraries.

  • Open
  • quality assurance status badge
Details
3 participants
  • Danny Milosavljevic
  • Liliana Marie Prikler
  • Ludovic Courtès
Owner
unassigned
Submitted by
Danny Milosavljevic
Severity
normal
D
D
Danny Milosavljevic wrote on 3 Sep 02:02 +0200
(address . guix-patches@gnu.org)(name . Danny Milosavljevic)(address . dannym@friendly-machines.com)
58ce361528f55e5ad76ed7da4123525a9e9375e5.1725319687.git.dannym@friendly-machines.com
* gnu/packages/patches/emacs-julia-snail-1.3.1-vendor.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.
* gnu/packages/emacs-xyz.scm (emacs-julia-snail)[arguments]<#:phases>[vendor]:
New phase.
[inputs]: Add julia-cstparser, julia-tokenize.
[propagated-inputs]: Remove julia-cstparser, julia-tokenize.
[source]: Add patch.

Change-Id: I61c0a4493fcfd219809bcc35f8765be0928a7e81
---
gnu/local.mk | 1 +
gnu/packages/emacs-xyz.scm | 25 ++++++--
.../emacs-julia-snail-1.3.1-vendor.patch | 57 +++++++++++++++++++
3 files changed, 78 insertions(+), 5 deletions(-)
create mode 100644 gnu/packages/patches/emacs-julia-snail-1.3.1-vendor.patch

Toggle diff (126 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 0c4ab96bf3..8647956f2f 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1179,6 +1179,7 @@ dist_patch_DATA = \
%D%/packages/patches/emacs-git-email-missing-parens.patch \
%D%/packages/patches/emacs-helpful-fix-tests.patch \
%D%/packages/patches/emacs-highlight-stages-add-gexp.patch \
+ %D%/packages/patches/emacs-julia-snail-1.3.1-vendor.patch \
%D%/packages/patches/emacs-json-reformat-fix-tests.patch \
%D%/packages/patches/emacs-kv-fix-tests.patch \
%D%/packages/patches/emacs-lispy-fix-thread-last-test.patch \
diff --git a/gnu/packages/emacs-xyz.scm b/gnu/packages/emacs-xyz.scm
index 370daaf50e..1837c759d2 100644
--- a/gnu/packages/emacs-xyz.scm
+++ b/gnu/packages/emacs-xyz.scm
@@ -14656,22 +14656,37 @@ (define-public emacs-julia-snail
(file-name (git-file-name name version))
(sha256
(base32
- "0h5lwc2hsk4rc82idhf9qj9553v7x76wmy9x2z1h60pdd93ilcyr"))))
+ "0h5lwc2hsk4rc82idhf9qj9553v7x76wmy9x2z1h60pdd93ilcyr"))
+ (patches
+ (search-patches "emacs-julia-snail-1.3.1-vendor.patch"))))
(build-system emacs-build-system)
(arguments
(list
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'validate-compiled-autoloads 'vendor
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (base (dirname (car (find-files out "JuliaSnail.jl"))))
+ (vendor (string-append base "/vendor")))
+ (mkdir-p vendor)
+ (copy-recursively (string-append (assoc-ref inputs "julia-cstparser")
+ "/share/julia/loadpath/CSTParser")
+ (string-append vendor "/CSTParser"))
+ (copy-recursively (string-append (assoc-ref inputs "julia-tokenize")
+ "/share/julia/loadpath/Tokenize")
+ (string-append vendor "/Tokenize"))))))
#:include
#~(cons* "^JuliaSnail\\.jl" "extensions" %default-include)))
(inputs
- (list emacs-dash emacs-s emacs-spinner emacs-xref))
+ (list emacs-dash emacs-s emacs-spinner emacs-xref julia-cstparser
+ julia-tokenize))
(propagated-inputs
(list libvterm
emacs-julia-mode ;required by parser
emacs-parsec ;required by parser
emacs-popup
- emacs-vterm
- julia-tokenize
- julia-cstparser))
+ emacs-vterm))
(home-page "https://github.com/gcv/julia-snail")
(synopsis "Development environment and REPL interaction package for Julia")
(description "This package provides a development environment and REPL
diff --git a/gnu/packages/patches/emacs-julia-snail-1.3.1-vendor.patch b/gnu/packages/patches/emacs-julia-snail-1.3.1-vendor.patch
new file mode 100644
index 0000000000..eaa38fb868
--- /dev/null
+++ b/gnu/packages/patches/emacs-julia-snail-1.3.1-vendor.patch
@@ -0,0 +1,57 @@
+Author: Danny Milosavljevic <dannym@friendly-machines.com>
+Date: 2024-09-03
+License: GPL3+
+
+Previously, we propagated julia-cstparser and julia-tokenize. This would only
+work if the user had installed julia in their profile (because only then
+JULIA_LOAD_PATH of julia-cstparser would be propagated).
+That doesn't seem likely or desireable.
+
+It turns out that it's not difficult to make julia read vendored packages.
+Let's do that (since there's a comment inside JuliaSnail.jl that warns about
+CSTParser APIs being easily incompatible, vendoring is better anyway).
+
+diff -ru orig/4l8xr3ldffb30a44x157lj3asaj3ykls-emacs-julia-snail-1.3.1-checkout/JuliaSnail.jl 4l8xr3ldffb30a44x157lj3asaj3ykls-emacs-julia-snail-1.3.1-checkout/JuliaSnail.jl
+--- orig/4l8xr3ldffb30a44x157lj3asaj3ykls-emacs-julia-snail-1.3.1-checkout/JuliaSnail.jl 2024-09-03 00:02:03.798373662 +0200
++++ 4l8xr3ldffb30a44x157lj3asaj3ykls-emacs-julia-snail-1.3.1-checkout/JuliaSnail.jl 2024-09-03 01:05:26.310766009 +0200
+@@ -30,12 +30,12 @@
+ catch err
+ if isa(err, ArgumentError)
+ if isfile(joinpath($dir, "Project.toml"))
+- # force dependency installation
+- Main.Pkg.activate($dir)
+- Main.Pkg.instantiate()
+- Main.Pkg.precompile()
+- # activate what was the first entry before Snail was pushed to the head of LOAD_PATH
+- Main.Pkg.activate(LOAD_PATH[2])
++ # force dependency installation
++ if isa(err, ArgumentError)
++ error("Vendored dependencies not found. Please make sure the vendored packages are located in the 'vendor' directory.")
++ else
++ rethrow(err)
++ end
+ end
+ end
+ finally
+@@ -51,11 +51,16 @@
+ end
+
+ @with_pkg_env (@__DIR__) begin
+- # list all external dependency imports here (from the appropriate Project.toml, either Snail's or an extension's):
+- import CSTParser
+- # check for dependency API compatibility
+- !isdefined(CSTParser, :iscall) &&
+- throw(ArgumentError("CSTParser API not compatible, must install Snail-specific version"))
++ @with_pkg_env (joinpath(@__DIR__, "vendor", "CSTParser")) begin
++ @with_pkg_env (joinpath(@__DIR__, "vendor", "Tokenize")) begin
++ println(LOAD_PATH)
++ # list all external dependency imports here (from the appropriate Project.toml, either Snail's or an extension's):
++ import CSTParser
++ # check for dependency API compatibility
++ !isdefined(CSTParser, :iscall) &&
++ throw(ArgumentError("CSTParser API not compatible, must install Snail-specific version"))
++ end
++ end
+ end
+
+

base-commit: b833aaaee7c95ec0339428a6b602f26831494798
--
2.45.2
L
L
Ludovic Courtès wrote 4 days ago
(name . Danny Milosavljevic)(address . dannym@friendly-machines.com)
87cyl49fbv.fsf@gnu.org
Hey Danny,

Danny Milosavljevic <dannym@friendly-machines.com> skribis:

Toggle quote (10 lines)
> * gnu/packages/patches/emacs-julia-snail-1.3.1-vendor.patch: New file.
> * gnu/local.mk (dist_patch_DATA): Add it.
> * gnu/packages/emacs-xyz.scm (emacs-julia-snail)[arguments]<#:phases>[vendor]:
> New phase.
> [inputs]: Add julia-cstparser, julia-tokenize.
> [propagated-inputs]: Remove julia-cstparser, julia-tokenize.
> [source]: Add patch.
>
> Change-Id: I61c0a4493fcfd219809bcc35f8765be0928a7e81

[...]

Toggle quote (12 lines)
> + (add-after 'validate-compiled-autoloads 'vendor
> + (lambda* (#:key inputs outputs #:allow-other-keys)
> + (let* ((out (assoc-ref outputs "out"))
> + (base (dirname (car (find-files out "JuliaSnail.jl"))))
> + (vendor (string-append base "/vendor")))
> + (mkdir-p vendor)
> + (copy-recursively (string-append (assoc-ref inputs "julia-cstparser")
> + "/share/julia/loadpath/CSTParser")
> + (string-append vendor "/CSTParser"))
> + (copy-recursively (string-append (assoc-ref inputs "julia-tokenize")
> + "/share/julia/loadpath/Tokenize")

Use ‘search-input-directory’ rather than ‘assoc-ref’.

Toggle quote (15 lines)
> +++ b/gnu/packages/patches/emacs-julia-snail-1.3.1-vendor.patch
> @@ -0,0 +1,57 @@
> +Author: Danny Milosavljevic <dannym@friendly-machines.com>
> +Date: 2024-09-03
> +License: GPL3+
> +
> +Previously, we propagated julia-cstparser and julia-tokenize. This would only
> +work if the user had installed julia in their profile (because only then
> +JULIA_LOAD_PATH of julia-cstparser would be propagated).
> +That doesn't seem likely or desireable.
> +
> +It turns out that it's not difficult to make julia read vendored packages.
> +Let's do that (since there's a comment inside JuliaSnail.jl that warns about
> +CSTParser APIs being easily incompatible, vendoring is better anyway).

I’m not sure I understand. For most programming languages (Python,
etc.), we use propagated inputs like this. How does that differ?

Normally we unbundle (“unvendor”?) libraries and this seems to go in the
opposite direction, right?

Thanks,
Ludo’.
D
D
dannym wrote 3 days ago
(name . Ludovic Courtès)(address . ludo@gnu.org)
c2ee03afc41a15b6fa7624498dd32a50@friendly-machines.com
Hi Ludo,

On 2024-09-16 10:24, Ludovic Courtès wrote:
Toggle quote (7 lines)
>> + (copy-recursively (string-append (assoc-ref inputs
>> "julia-tokenize")
>> +
>> "/share/julia/loadpath/Tokenize")
>
> Use ‘search-input-directory’ rather than ‘assoc-ref’.

Will do!

Toggle quote (7 lines)
> I’m not sure I understand. For most programming languages (Python,
> etc.), we use propagated inputs like this. How does that differ?
>
> Normally we unbundle (“unvendor”?) libraries and this seems to go in
> the
> opposite direction, right?

The idea is to make Guix more user-friendly.

I touched this only after I myself got bitten by the following:

I have guix home (with only a few packages--notably no julia (!) or
gcc-toolchain
or anything like that in it),
and I have emacs set up with emacs-buffer-env and the following
configuration:

(setq buffer-env-script-name '("manifest.scm" ".envrc"))

In a julia project, manifest.scm contains:

(specifications->manifest
(list "julia"))

The idea is that emacs-bufferenv will automatically load that manifest
when I
open a file in that directory within emacs--and then have julia-snail
REPL
(including sending expression on point to the REPL) work.

Since the emacs julia-snail REPL integration needs to find the bounds of
the expression,
it has to have a parser for the Julia language.

However, what happens now is that because I don't have julia-cstparser
and julia-tokenize
in guix home, it won't work.

But from the standpoint of the profile I shouldn't have julia-cstparser
and julia-tokenize
in my home profile since I don't personally directly use it (it's a
requirement of emacs-julia-snail--I didn't even know what
julia-cstparser is a few days ago :) ).

But nevermind that. Even if I added julia-cstparser and julia-tokenize
to my
guix home profile, it would NOT work. The reason is because julia
itself is not part of
the guix home profile, and the "julia" package would have a search-path
specification that
would set some environment variables that julia needs to find
julia-cstparser and
julia-tokenize.
Since the package is not there (in guix home) it won't set the
variables.

The idea of this patch is that emacs will automatically set up whatever
it needs to in order
to make the REPL work (including the finding of the boundaries of the
current expression
that the emacs package needs for "send expression at point to REPL")
without weird
dependencies bleeding explicitly into the user profile or into the local
directory's manifest.

In my opinion, the user should not need to be concerned with what
julia-cstparser is in this case.

Emacs should just magically load whatever it needs for the Julia REPL to
work (whether
that's written in Julia or Python or whatever is an implementation
detail--you COULD
have a Julia parser written in Python. Notably, most parsers for any
language in
IntelliJ IDEA are written in Java--the Python parser, Rust parser etc).

That said, I'm totally open to other ways to do it.

The obvious alternative would be for the user to put julia-cstparser and
julia-tokenize
into every Julia project's manifest.scm . But the error message is
inscrutable if you
don't have those in there[1]--and I have no idea how any normal person
would figure out
that julia-cstparser is needed in the first place.

[1] julia-snail--send-to-server: No Julia REPL buffer *julia* found; run
julia-snail
Starting Julia process and loading Snail...
julia-snail--repl-enable: The REPL terminal buffer is inactive;
double-check julia-snail-executable path

(In buffer "*scratch*" : )
julia-snail-executable
"julia"

(note: that "julia" excutable IS available in the manifest of the buffer
env--and launching "julia" does work when launched manually within
emacs)
L
L
Liliana Marie Prikler wrote 2 days ago
63bc7d0629493cca55ee2d2b8623c7bd6d18995b.camel@gmail.com
Hi Danny,

Am Montag, dem 16.09.2024 um 20:01 +0200 schrieb
dannym@friendly-machines.com:
Toggle quote (6 lines)
> The idea of this patch is that emacs will automatically set up
> whatever it needs to in order to make the REPL work (including the
> finding of the boundaries of the current expression that the emacs
> package needs for "send expression at point to REPL") without weird
> dependencies bleeding explicitly into the user profile or into the
> local directory's manifest.
You could write a meta package à la "julia stuff for emacs-julia-snail"
and mention that in the description, no? I think bundling (or
propagating) these dependencies from the package is likely not going to
do us favours in weird edge cases where you actually want more julia
packages, e.g. in the julia package that you may or may not be hacking
on, which has other dependencies :)

Cheers
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 72994@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 72994
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch