[PATCH emacs-team 0/2] Think ahead when compiling

Hi Guix,

this series (hopefully) makes it so that everything we need to be natively

compiled in Emacs a) is natively compiled, and b) is found in the right

location. Please check that you no longer get gratuitous writes to your

local eln-cache when trying this out.

Cheers

Fixes: emacs-build-system … mismatching hashes https://bugs.gnu.org/66864

Liliana Marie Prikler (2):

gnu: emacs: Build trampolines.

gnu: emacs: Don't hash file names in native compilation.

gnu/local.mk | 1 +

gnu/packages/emacs.scm | 6 +-

.../emacs-native-comp-fix-filenames.patch | 93 +++++++++++++++++++

3 files changed, 99 insertions(+), 1 deletion(-)

create mode 100644 gnu/packages/patches/emacs-native-comp-fix-filenames.patch

base-commit: 60c97924e9519361494aaf0686e28eb831a42315

--

2.41.0

Change-Id: I2b7f6b45742a985760f0097bb53910f068e3d8e5

---

gnu/local.mk | 1 +

gnu/packages/emacs.scm | 1 +

.../emacs-native-comp-fix-filenames.patch | 93 +++++++++++++++++++

3 files changed, 95 insertions(+)

create mode 100644 gnu/packages/patches/emacs-native-comp-fix-filenames.patch

On 2023-11-18 14:42, Liliana Marie Prikler wrote:

--

Best regards,

Andrew Tropin

Am Samstag, dem 18.11.2023 um 19:44 +0400 schrieb Andrew Tropin:

Oof.

2023-11-18 14:42 liliana.prikler@gmail.com:

Hello and thank you for your efforts on this issue.

It seems like this is a series of two commits but I only received one, namely the second one ("2/2"). Is it just me or did the first commit went missing for everyone?

Also, are these commits still on emacs-team branch? Andrew's command (guix time-machine --branch=emacs-team -- shell ...) made me assume that these commits were pushed to emacs-team branch, but I can't find them there.

I'd be glad to try this patch-series out - at some point, when I have more time.

Am Mittwoch, dem 22.11.2023 um 12:09 +0000 schrieb Mekeor Melire:

These are commits submitted to the emacs-branch – they are not

upstreamed yet.

I'll have a v2 up hopefully soon; things sadly aren't as smooth as I'd

assumed.

Cheers

With the previous commit, relative file names are expanded relative to

ELN_DIR -- more or less. To make use of this in emacs-build-system, we must

also pass relative file names.

relative file names of the files to compile.

Change-Id: I8983f80fb0fe1573e46748222403ba8873f1599f

---

guix/build/emacs-utils.scm | 4 +++-

1 file changed, 3 insertions(+), 1 deletion(-)

Change-Id: I33303bcbaf6cbda15867a5546e793c05d1f0e67b

---

gnu/packages/emacs.scm | 5 ++++-

1 file changed, 4 insertions(+), 1 deletion(-)

Change-Id: I2b7f6b45742a985760f0097bb53910f068e3d8e5

---

gnu/local.mk | 1 +

gnu/packages/emacs.scm | 1 +

.../emacs-native-comp-fix-filenames.patch | 197 ++++++++++++++++++

3 files changed, 199 insertions(+)

create mode 100644 gnu/packages/patches/emacs-native-comp-fix-filenames.patch

With the previous commit, relative file names are expanded relative to

ELN_DIR -- more or less. To make use of this in emacs-build-system, we must

also pass relative file names.

relative file names of the files to compile.

Change-Id: I8983f80fb0fe1573e46748222403ba8873f1599f

---

guix/build/emacs-utils.scm | 4 +++-

1 file changed, 3 insertions(+), 1 deletion(-)

directory excursion to the actual lisp directory.

Change-Id: Ifa10f9e91fe21cd4c34da11b68ddb77a03d847ca

---

gnu/packages/emacs-xyz.scm | 4 ++++

1 file changed, 4 insertions(+)

a directory excursion.

Change-Id: I332325989a1bbaa95552c2cbf50f336f0075c1c4

---

gnu/packages/emacs-xyz.scm | 6 +++++-

1 file changed, 5 insertions(+), 1 deletion(-)

On 2023-11-25 12:13, Liliana Marie Prikler wrote:

Applied the patch series v4 to emacs-team branch, removed all eln files

in ~/.cache/emacs and ~/.config/emacs, executed

*Warnings*:

Modified the command

*Warnings*:

*Native-compile-log*:

*Async-native-compile-log*:

Am Mittwoch, dem 27.12.2023 um 14:00 +0300 schrieb Andrew Tropin:

You're missing --with-input=emacs-minimal=emacs. You are not natively

compiling emacs-magit atm.

Cheers

On 2023-12-27 17:40, Liliana Marie Prikler wrote:

Right, thank you!

*Native-compile-log*:

*Async-native-compile-log*:

Am Mittwoch, dem 27.12.2023 um 21:27 +0300 schrieb Andrew Tropin:

Turns out you also need to do this with emacs-no-x due to emacs-libgit.

Friendly reminder that we don't have build system/packaging utilities

to easily replace emacs versions yet, so magit might not have been the

wisest choice to showcase here.

More worryingly, it appears you're recompiling Emacs itself here. This

should not be happening. However, it turns out that your eln-cache is

always first on the native load path and thus already compiled files

won't be found.¹ Try adding 

--eval='(setq native-comp-jit-compilation nil)'

and see whether you can load natively compiled procedures and have them

show as natively compiled.

Cheers

¹ Maybe there's a smarter way to locate them files? Did I go wrong in

one of my patches?

Am Mittwoch, dem 27.12.2023 um 21:03 +0100 schrieb Liliana Marie

Prikler:

… or not, as emacs-libgit is not pulled in by emacs-magit.

Still a potential problem, however.

Cheers

On 2023-12-27 21:03, Liliana Marie Prikler wrote:

magit-status is an interactive native-compiled Lisp function in

‘magit-status.el’.

--

Best regards,

Andrew Tropin

Reporting some observations for the benefit of the community.

I managed to install the following sequence of patches on emacs-team

commit 60c97924e9 :

- v3 1/3

- v3 2/3

- v3 3/3

- v4 4/5

- v4 5/5

I was able to build emacs, emacs-org and a few additional packages using

the options "--no-grafts --with-input=emacs-minimal=emacs". When

building "emacs-org-contrib" the additional option

"--without-tests=emacs-clojure-mode" was needed as well.

Having done so, upon starting emacs via guix-shell, I noticed a few

messages in *Async-native-compile-log* which noted that libraries such

as rx.el.gz and cl-macs.el.gz were being compiled. I observed

native-compiled versions of these show up in ~/.emacs.d/eln-cache. Now

this seems similar to the behaviour I observe in a non-Guix host where

Emacs 29.1 is installed via the OS. There too my ~/.emacs.d/eln-cache

has a number of entries that I was a little surprised to see such as

those corresponding to rx and cl-macs.

One difference, however, is in the names and the relative locations of

these natively-compiled files. Outside of the guix shell (i.e., using

the OS's native Emacs package) the eln files are named like

~/.emacs.d/eln-cache/29.1-115521d4/cl-macs-7ae82f81-e626a10e.eln whereas

within the Guix shell using Emacs built from the installed patches you

get names like

~/.emacs.d/eln-cache/29.1-c187d49d/emacs-lisp/cl-macs.eln. Note the

sub-directory within the 29.1-* directory present within guix-shell

container, as well as the lack of hash fingerprint in the *.eln file

itself.

I do not know if these differences are to be expected, or whether or not

they're material in any way.

Thoughts? Can this patch series be installed into emacs-team as-is? If

not, what are the current blockers, if any? On a somewhat related note,

when is the next merge from emacs-team to master expected to be?

--

Suhail

This email is not an offer capable of acceptance, does not evidence an

intention to enter into an agreement, has no operative effect until a

definitive agreement is signed in writing by both parties, and that no

party should act in reliance on the email or any representations of the

sender until a definitive agreement is signed in writing by both

parties.

Change-Id: I33303bcbaf6cbda15867a5546e793c05d1f0e67b

---

gnu/packages/emacs.scm | 5 ++++-

1 file changed, 4 insertions(+), 1 deletion(-)

Change-Id: I2b7f6b45742a985760f0097bb53910f068e3d8e5

---

gnu/local.mk | 1 +

gnu/packages/emacs.scm | 1 +

.../emacs-native-comp-fix-filenames.patch | 197 ++++++++++++++++++

3 files changed, 199 insertions(+)

create mode 100644 gnu/packages/patches/emacs-native-comp-fix-filenames.patch

a directory excursion.

Change-Id: I332325989a1bbaa95552c2cbf50f336f0075c1c4

---

gnu/packages/emacs-xyz.scm | 6 +++++-

1 file changed, 5 insertions(+), 1 deletion(-)

---

gnu/local.mk | 1 +

gnu/packages/emacs.scm | 3 ++-

.../emacs-disable-jit-compilation.patch | 19 +++++++++++++++++++

3 files changed, 22 insertions(+), 1 deletion(-)

create mode 100644 gnu/packages/patches/emacs-disable-jit-compilation.patch

directory excursion to the actual lisp directory.

Change-Id: Ifa10f9e91fe21cd4c34da11b68ddb77a03d847ca

---

gnu/packages/emacs-xyz.scm | 4 ++++

1 file changed, 4 insertions(+)

With the previous commit, relative file names are expanded relative to

ELN_DIR -- more or less. To make use of this in emacs-build-system, we must

also pass relative file names.

relative file names of the files to compile.

Change-Id: I8983f80fb0fe1573e46748222403ba8873f1599f

---

guix/build/emacs-utils.scm | 4 +++-

1 file changed, 3 insertions(+), 1 deletion(-)

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

I don't believe this patch is sufficient. It seems that the variable

'native-comp-eln-load-path may also need to be tweaked. The path to the

"native-lisp" directory needs to be specified absolutely to make it not

be dependent on the invocation-directory. Details below.

#+begin_src bash :results replace

emacs --batch --eval "(message \"%s\" (car (split-string (describe-function 'describe-function) \"\n\")))"

#+end_src

#+RESULTS:

: Type q in help window to delete it

: describe-function is an autoloaded interactive byte-compiled Lisp

#+begin_src bash :results replace

type emacs

ls ~/.guix-profile/lib/emacs/29.2/native-lisp/29.2-e9db68ab/help-fns.eln

ls ~/.guix-profile/lib/emacs/29.2/native-lisp/29.2-e9db68ab/*.eln | wc -l

ls ~/.guix-profile/lib/emacs/native-site-lisp/29.2-e9db68ab/*.eln | wc -l

emacs --batch --eval "(message \"%s\" native-comp-eln-load-path)"

#+end_src

#+RESULTS:

: emacs is hashed (/home/user/.guix-profile/bin/emacs)

: /home/user/.guix-profile/lib/emacs/29.2/native-lisp/29.2-e9db68ab/help-fns.eln

: 1715

: 236

: (/home/user/.emacs.d/eln-cache/ /home/user/.guix-profile/lib/emacs/native-site-lisp ../native-lisp/)

#+begin_src bash :results replace

emacs --batch --eval "(message \"%s\" (progn (add-to-list 'native-comp-eln-load-path \"/home/user/.guix-profile/lib/emacs/29.2/native-lisp\") (car (split-string (describe-function 'describe-function) \"\n\"))))"

#+end_src

#+RESULTS:

: Type q in help window to delete it

: describe-function is an autoloaded interactive native-compiled Lisp

--

Suhail

This email is not an offer capable of acceptance, does not evidence an

intention to enter into an agreement, has no operative effect until a

definitive agreement is signed in writing by both parties, and that no

party should act in reliance on the email or any representations of the

sender until a definitive agreement is signed in writing by both

parties.

Suhail via Guix-patches via <guix-patches@gnu.org> writes:

Or it's possible that a patch that ensures that

'native-comp-eln-load-path contains the absolute value of the

"native-lisp" directory obviates this one (6/6) in its entirety.

--

Suhail

EMACSNATIVELOADPATH.

---

Am Montag, dem 22.01.2024 um 04:36 +0000 schrieb Suhail:

This was the most cryptic hint towards the actual issue, but I've

managed to resolve it anyhow. We still shouldn't enable jit compilation

however; I've updated our reason as to why.

gnu/packages/emacs.scm | 12 ++++++++++--

1 file changed, 10 insertions(+), 2 deletions(-)

Change-Id: I33303bcbaf6cbda15867a5546e793c05d1f0e67b

---

gnu/packages/emacs.scm | 5 ++++-

1 file changed, 4 insertions(+), 1 deletion(-)

With the previous commit, relative file names are expanded relative to

ELN_DIR -- more or less. To make use of this in emacs-build-system, we must

also pass relative file names.

relative file names of the files to compile.

Change-Id: I8983f80fb0fe1573e46748222403ba8873f1599f

---

guix/build/emacs-utils.scm | 4 +++-

1 file changed, 3 insertions(+), 1 deletion(-)

Change-Id: I2b7f6b45742a985760f0097bb53910f068e3d8e5

---

gnu/local.mk | 1 +

gnu/packages/emacs.scm | 1 +

.../emacs-native-comp-fix-filenames.patch | 197 ++++++++++++++++++

3 files changed, 199 insertions(+)

create mode 100644 gnu/packages/patches/emacs-native-comp-fix-filenames.patch

directory excursion to the actual lisp directory.

Change-Id: Ifa10f9e91fe21cd4c34da11b68ddb77a03d847ca

---

gnu/packages/emacs-xyz.scm | 4 ++++

1 file changed, 4 insertions(+)

a directory excursion.

Change-Id: I332325989a1bbaa95552c2cbf50f336f0075c1c4

---

gnu/packages/emacs-xyz.scm | 6 +++++-

1 file changed, 5 insertions(+), 1 deletion(-)

---

gnu/local.mk | 1 +

gnu/packages/emacs.scm | 3 ++-

.../emacs-disable-jit-compilation.patch | 19 +++++++++++++++++++

3 files changed, 22 insertions(+), 1 deletion(-)

create mode 100644 gnu/packages/patches/emacs-disable-jit-compilation.patch

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

I can confirm that Emacs built by installing v6 patches on emacs-team

branch and built with --no-grafts is now able to locate

natively-compiled versions of features like help-fns etc. (which it

couldn't with v5 patches).

#+begin_src bash :results replace

emacs --batch --eval "(message \"%s\" (car (split-string (describe-function 'describe-function) \"\n\")))"

#+end_src

#+RESULTS:

: Type q in help window to delete it

: describe-function is an autoloaded interactive native-compiled Lisp

However, there may be a spurious entry in native-comp-eln-load-path that

should probably be removed.

#+begin_src bash :results replace

emacs --batch --eval "(message \"%s\" native-comp-eln-load-path)"

#+end_src

#+RESULTS:

: (/home/user/.emacs.d/eln-cache/ /home/user/.guix-profile/lib/emacs/native-site-lisp /gnu/store/4jvap9wxifizm56p6pmxc6rk0zyvm5zc-emacs-29.2/lib/emacs/29.2/native-lisp ../native-lisp/)

The "../native-lisp/" entry above, seems out of place. Based on the

documentation of native-comp-eln-load-path:

Contrasting the above with an Emacs 29.1 installation on a non-Guix

system, I notice that the latter doesn't have any non-absolute paths in

native-comp-eln-load-path. Thoughts?

--

Suhail

Hi,

Am Donnerstag, dem 25.01.2024 um 23:53 +0000 schrieb Suhail:

I have noticed this myself, but I don't think that it's related to this

series. Can you try 29.2 without the patches? It might be related to

that bump.

Cheers

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

It seems to be. It's possible that it may also be related to other

things already in the emacs-team branch (e.g. if the patch series is

only responsible for uncovering a pre-existing defect as opposed to

causing it), but the relative path only shows up during the series.

emacs-team branch at aae61f54ff doesn't have the relative path in

native-comp-eln-load-path. The relative path shows up only after

[PATCH v6 3/7] is applied. Hope this helps and hopefully the patch

series can soon be merged into master for general availability.

[PATCH v6 3/7] gnu: emacs: Don't hash file names in native compilation.

--

Suhail

Suhail via Guix-patches via <guix-patches@gnu.org> writes:

Btw, does every package require something like [PATCH v6 5/7] or

[PATCH v6 6/7] in order for it to be natively compiled?

--

Suhail

Am Freitag, dem 26.01.2024 um 08:20 +0000 schrieb Suhail:

No, these patches are only for natively compiled packages that don't

use emacs-build-system. That being said, this is likely an incomplete

list.

Cheers

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

I.e., any and all packages using the emacs-build-system should be

natively-compiled? If so, I believe I've found a counter-example.

After installing v6 series and building emacs-htmlize my expectation is

that functions provided by it, such as htmlize-buffer, would be

natively-compiled. However, I get:

Note the lack of "native-compiled" in the above.

--

Suhail

Am Freitag, dem 26.01.2024 um 22:45 +0000 schrieb Suhail:

No, but packages using emacs-build-system should not need any changes

to accomodate the currently existing hack of adding the option 

--with-input=emacs-minimal=emacs

to ‘guix build’, ‘guix package’, ‘guix shell’, etc.

If you compile with emacs-minimal, that's the expected results.

Different builds of emacs are still incompatible w.r.t. native code and

produce directories with different hashes. We'll have to see as to

whether this bugs grafts, but I'm hoping it doesn't.

Cheers

EMACSNATIVELOADPATH.

---

gnu/packages/emacs.scm | 12 ++++++++++--

1 file changed, 10 insertions(+), 2 deletions(-)

Change-Id: I33303bcbaf6cbda15867a5546e793c05d1f0e67b

---

gnu/packages/emacs.scm | 5 ++++-

1 file changed, 4 insertions(+), 1 deletion(-)

---

gnu/local.mk | 1 +

gnu/packages/emacs.scm | 3 ++-

.../emacs-disable-jit-compilation.patch | 19 +++++++++++++++++++

3 files changed, 22 insertions(+), 1 deletion(-)

create mode 100644 gnu/packages/patches/emacs-disable-jit-compilation.patch

With the previous commit, relative file names are expanded relative to

ELN_DIR -- more or less. To make use of this in emacs-build-system, we must

also pass relative file names.

relative file names of the files to compile.

Change-Id: I8983f80fb0fe1573e46748222403ba8873f1599f

---

guix/build/emacs-utils.scm | 4 +++-

1 file changed, 3 insertions(+), 1 deletion(-)

Change-Id: I2b7f6b45742a985760f0097bb53910f068e3d8e5

---

gnu/local.mk | 1 +

gnu/packages/emacs.scm | 1 +

.../emacs-native-comp-fix-filenames.patch | 291 ++++++++++++++++++

3 files changed, 293 insertions(+)

create mode 100644 gnu/packages/patches/emacs-native-comp-fix-filenames.patch

directory excursion to the actual lisp directory.

Change-Id: Ifa10f9e91fe21cd4c34da11b68ddb77a03d847ca

---

gnu/packages/emacs-xyz.scm | 4 ++++

1 file changed, 4 insertions(+)

a directory excursion.

Change-Id: I332325989a1bbaa95552c2cbf50f336f0075c1c4

---

gnu/packages/emacs-xyz.scm | 6 +++++-

1 file changed, 5 insertions(+), 1 deletion(-)

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

It was compiled with emacs-minimal=emacs. For reference, I have inlined

the manifest file that was used (with the --no-grafts CLI option to

guix-package) towards the end of this email. I'm sharing the one that

was actually used, but it could be minimized.

I believe there are two issues (not including the spurious entry in

native-comp-eln-load-path which may already have been resolved in v7 of

the series). All my observations below are based on v6 patch series.

For emacs-htmlize, I believe the issue may stem from the fact that some

.eln files are available from entries in native-comp-eln-load-path only

via symlinks and the .eln loader may not be able to load symlinks.

Packages such as ox-html are available from two locations in the

native-comp-eln-load-path. In one location they are symlinked, but

(presumably because org is a builtin package) it is also available from

another location in the native-comp-eln-load-path and in the latter

location it's not symlinked in. I believe this difference is why for

some packages natively-compiled versions are loaded (e.g. org, ox-html

etc) whereas for others it's not the case.

In addition, I believe there's another issue. Some packages' names are

getting truncated. For instance, instead of uniquify.eln, I observe

niquify.eln. In this case, the .eln isn't symlinked (presumably because

it's a builtin), but due to the name being messed up (perhaps too

aggressive a truncation of hashes?) the natively compiled version is not

available:

The manifest file that was used is below.

#+begin_src scheme

(use-modules (guix transformations)

(gnu packages))

(define transform1

(options->transformation

'((with-input . "emacs-minimal=emacs")

(without-tests . "emacs-clojure-mode"))))

(packages->manifest

(list (transform1 (specification->package "bash"))

(transform1 (specification->package "coreutils"))

(transform1 (specification->package "git"))

(transform1 (specification->package "make"))

(transform1 (specification->package "emacs"))

(transform1 (specification->package "emacs-org"))

(transform1

(specification->package "emacs-org-contrib"))

(transform1

(specification->package "emacs-citeproc-el"))

(transform1

(specification->package "emacs-engrave-faces"))

(transform1

(specification->package "emacs-haskell-mode"))

(transform1

(specification->package "emacs-htmlize"))

(transform1

(specification->package "emacs-markdown-mode"))

(transform1

(specification->package

"emacs-org-pandoc-import"))

(transform1 (specification->package "pandoc"))

))

#+end_src

Thank you for your continued work on this!

--

Suhail

This email is not an offer capable of acceptance, does not evidence an

intention to enter into an agreement, has no operative effect until a

definitive agreement is signed in writing by both parties, and that no

party should act in reliance on the email or any representations of the

sender until a definitive agreement is signed in writing by both

parties.

Suhail via Guix-patches via <guix-patches@gnu.org> writes:

I can confirm that the v7 series fixes the spurious entry in

native-comp-eln-load-path.

However, the 2 issues pointed out in the previous email remain:

1. failure to load natively-compiled version of non-builtin libraries

like emacs-htmlize possibly due to symlinks, and

2. failure to load natively-compiled version of some builtin libraries

like uniquify due to incorrect naming of .eln files

--

Suhail

Am Samstag, dem 27.01.2024 um 15:36 +0000 schrieb Suhail:

Would you care to debug this a bit more and report your findings?

Well, as pointed out in the deleted code, dlopen has a "one shared

library per file name" limitation. I don't think we're hit by that

thanks to unique hashes in the store directory, but I might be wrong

about that. Maybe we have to do java-style FQDNs instead.

I am not truncating any hashes, I'm not even computing them in the

first place. The functions I'm modifying are publicly callable, namely

comp-el-to-eln-rel-filename for the relative file names, and

comp-el-to-eln-filename for the absolute ones.

There could be an off-by-one error hidden in the stripping of the

BOGUS_DIRS, however. Let's investigate that.

Note, that you can map transform1, saving some typing overhead, and

since you are transforming all of your packages you could compose that

with specification->package.

Cheers

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

The locations in question are in the store and hence write-protected.

What's a "safe" way to debug this?

Perhaps. I'd wondered, more simply, if symlinked .eln files were ever

being loaded. In the limited testing I did, I didn't find such an

instance. For what it's worth, only one of the entries in

native-comp-eln-load-path seems to have symlinked entries (namely,

~/.guix-profile/lib/emacs/native-site-lisp).

Were you thinking out loud, or are there specific steps you'd like me to

help with?

Appreciate the tips, but I should clarify that the manifest file in

question was generated by way of guix shell --export-manifest. I'd

omitted the header.

--

Suhail

This email is not an offer capable of acceptance, does not evidence an

intention to enter into an agreement, has no operative effect until a

definitive agreement is signed in writing by both parties, and that no

party should act in reliance on the email or any representations of the

sender until a definitive agreement is signed in writing by both

parties.

Am Samstag, dem 27.01.2024 um 17:15 +0000 schrieb Suhail:

You could try stepping through loading a natively-compiled, but

symlinked library, either with the emacs debugger or gdb. I sadly

don't have more hints to give; you probably know more about this bug

than I do.

Alternatively, you could try union-building emacs + your libraries and

resolving those symlinks as you do. This should give us a clear hint

that it's the symlinks and not something else that goes wrong.

In addition, you might want to verify that comp-el-to-*-filename

matches the files you want to load. We could try resolving symlinks in

there with realpath as well.

That last sentence is me thinking loud for a solution that I'll try for

v8.

Cheers

Suhail via Guix-patches via <guix-patches@gnu.org> writes:

Correction, there's only 1 issue. The first issue was a false positive.

Apologies for the noise.

This was a false positive. Turns out, if the function in question is

only autoloaded describe-function won't correctly identify the function

as being natively-compiled. This is consistent with what happens on

Emacs 29.1 on a non-Guix system. Not an issue.

This issue remains as of v7 of the patch series.

--

Suhail

Am Samstag, dem 27.01.2024 um 19:39 +0000 schrieb Suhail:

Interesting, but probably not avoidable. There's only so much

information you can put into autoloads.

I fixed this one locally, but ‘uniquify’ remains byte-compiled for no

explainable reason. Will send v8 soon.

Cheer

Change-Id: I33303bcbaf6cbda15867a5546e793c05d1f0e67b

---

gnu/packages/emacs.scm | 5 ++++-

1 file changed, 4 insertions(+), 1 deletion(-)

EMACSNATIVELOADPATH.

---

gnu/packages/emacs.scm | 12 ++++++++++--

1 file changed, 10 insertions(+), 2 deletions(-)

With the previous commit, relative file names are expanded relative to

ELN_DIR -- more or less. To make use of this in emacs-build-system, we must

also pass relative file names.

relative file names of the files to compile.

Change-Id: I8983f80fb0fe1573e46748222403ba8873f1599f

---

guix/build/emacs-utils.scm | 4 +++-

1 file changed, 3 insertions(+), 1 deletion(-)

---

gnu/local.mk | 1 +

gnu/packages/emacs.scm | 3 ++-

.../emacs-disable-jit-compilation.patch | 19 +++++++++++++++++++

3 files changed, 22 insertions(+), 1 deletion(-)

create mode 100644 gnu/packages/patches/emacs-disable-jit-compilation.patch

Change-Id: I2b7f6b45742a985760f0097bb53910f068e3d8e5

---

gnu/local.mk | 1 +

gnu/packages/emacs.scm | 1 +

.../emacs-native-comp-fix-filenames.patch | 292 ++++++++++++++++++

3 files changed, 294 insertions(+)

create mode 100644 gnu/packages/patches/emacs-native-comp-fix-filenames.patch

directory excursion to the actual lisp directory.

Change-Id: Ifa10f9e91fe21cd4c34da11b68ddb77a03d847ca

---

gnu/packages/emacs-xyz.scm | 4 ++++

1 file changed, 4 insertions(+)

a directory excursion.

Change-Id: I332325989a1bbaa95552c2cbf50f336f0075c1c4

---

gnu/packages/emacs-xyz.scm | 6 +++++-

1 file changed, 5 insertions(+), 1 deletion(-)

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

I can confirm that the .eln filenames are no longer truncated (as far as

I can tell).

And too that as of v8 patch, uniquify still isn't natively-compiled.

However, if you unload it and then reload it, it becomes

natively-compiled.

#+begin_src bash

emacs --batch --eval "(message \"%s\" (progn (unload-feature 'uniquify) (require 'uniquify) (take 1 (split-string (substring-no-properties (describe-function 'uniquify-item-p)) \"\\n\" t))))"

#+end_src

#+RESULTS:

: Type q in help window to delete it

: (uniquify-item-p is a native-compiled Lisp function in `uniquify.el'.)

I think the issue is because uniquify (along with some others such as

prog-mode, backquote etc that I tested) are preloaded in Emacs. I

believe that these preloaded packages need to be treated specially. In

a non-Guix system, these preloaded packages have their .eln files stored

under something like

<native-comp-eln-load-path-entry>/29.x-<hash>/preloaded/ , whereas in

the v8 series there is no "preloaded" directory. Instead .eln files for

packages such as uniquify, prog-mode, backquote etc. are in various

locations:

- uniquify :: <n-c-e-l-p-entry>/29.2-<hash>/uniquify.eln

- prog-mode :: <n-c-e-l-p-entry>/29.2-<hash>/progmodes/prog-mode.eln

- backquote :: <n-c-e-l-p-entry>/29.2-<hash>/emacs-lisp/backquote.eln

I suspect that the code which does the preloading may need to be patched

and/or the preloaded packages may need to have their .eln files

generated under a preloaded sub-directory (similar to what happens in

non-Guix systems).

Below is a list of .eln files that exist in the preloaded sub-directory

in Emacs 29.1 on a non-Guix system. It's possible that the set of

preloaded packages on Emacs 29.2 is different. I suspect that all of

these packages are similarly affected.

#+begin_example

abbrev.eln

backquote.eln

bindings.eln

buff-menu.eln

burmese.eln

button.eln

byte-run.eln

case-table.eln

cconv.eln

cham.eln

characters.eln

chinese.eln

cl-generic.eln

cl-preloaded.eln

common-win.eln

composite.eln

cp51932.eln

cus-face.eln

cus-start.eln

custom.eln

cyrillic.eln

czech.eln

debug-early.eln

disp-table.eln

dnd.eln

dos-fns.eln

dos-vars.eln

dos-w32.eln

dynamic-setting.eln

easymenu.eln

ediff-hook.eln

eldoc.eln

electric.eln

elisp-mode.eln

english.eln

env.eln

epa-hook.eln

ethiopic.eln

eucjp-ms.eln

european.eln

faces.eln

files.eln

fill.eln

float-sup.eln

font-core.eln

font-lock.eln

fontset.eln

format.eln

frame.eln

fringe.eln

georgian.eln

greek.eln

haiku-win.eln

hebrew.eln

help.eln

image.eln

indent.eln

indian.eln

indonesian.eln

internal.eln

isearch.eln

iso-transl.eln

japanese.eln

jit-lock.eln

jka-cmpr-hook.eln

keymap.eln

khmer.eln

korean.eln

lao.eln

lisp.eln

lisp-mode.eln

ls-lisp.eln

macroexp.eln

map-ynp.eln

menu-bar.eln

minibuffer.eln

misc-lang.eln

mouse.eln

mule.eln

mule-cmds.eln

mule-conf.eln

mule-util.eln

mwheel.eln

nadvice.eln

newcomment.eln

ns-win.eln

obarray.eln

oclosure.eln

page.eln

paragraphs.eln

paren.eln

pc-win.eln

pgtk-dnd.eln

pgtk-win.eln

philippine.eln

prog-mode.eln

regexp-opt.eln

register.eln

replace.eln

rfn-eshadow.eln

rmc.eln

romanian.eln

scroll-bar.eln

select.eln

seq.eln

shorthands.eln

simple.eln

sinhala.eln

slovak.eln

startup.eln

subr.eln

syntax.eln

tab-bar.eln

tabulated-list.eln

tai-viet.eln

text-mode.eln

thai.eln

tibetan.eln

timer.eln

tool-bar.eln

tooltip.eln

tty-colors.eln

ucs-normalize.eln

uniquify.eln

utf-8-lang.eln

vc-hooks.eln

version.eln

vietnamese.eln

w32-fns.eln

w32-vars.eln

w32-win.eln

widget.eln

window.eln

x-dnd.eln

x-win.eln

#+end_example

--

Suhail

This email is not an offer capable of acceptance, does not evidence an

intention to enter into an agreement, has no operative effect until a

definitive agreement is signed in writing by both parties, and that no

party should act in reliance on the email or any representations of the

sender until a definitive agreement is signed in writing by both

parties.

Suhail via Guix-patches via <guix-patches@gnu.org> writes:

Minor correction. The list of files I posted was /adapted/ from the

list of files in the preloaded sub-directory on a non-Guix system (with

Emacs 29.1). Specifically, the list I shared had the hash suffix

removed to make comparison with the .eln file on Guix easier.

--

Suhail

Am Sonntag, dem 28.01.2024 um 00:13 +0000 schrieb Suhail:

From my experiments, the installing of these files is a red herring. I

instrumented lread.c to print the relative and absolute file names

tried, and it appears that these aren't loaded at all after install

while EMACSNATIVELOADPATH is ignored when dumping.

Cheers

I'm assuming you're able to reproduce the issue on your end as well with

other features like prog-mode, backquote etc.

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

Interesting. I did a grep of the Emacs source (branch emacs-29) and

line 1770 of src/lread.c (in maybe_swap_for_eln) seemed relevant:

What surprised me was that not only was the "preloaded" directory

special-cased, but so too was the last entry of "comp-eln-load-path"

(which I imagine is referring to native-comp-eln-load-path). Judging by

your message, I'm guessing you're well aware of this. I, unfortunately,

have no additional insights or suggestions.

However, given that ensuring Emacs is able to find the natively-compiled

version of files seems non-trivial, it might help for a future patch

series to be accompanied with tests. Thoughts?

--

Suhail

Am Sonntag, dem 28.01.2024 um 16:17 +0000 schrieb Suhail:

On upstream, this would add ../native-lisp/preloaded to the search

(since everything is dropped in one directory, this probably works as

expected), but with our patch, we do multiple directories *and* drop

../native-lisp from that early. We could try expanding

EMACSNATIVELOADPATH earlier, but we can't yet use Fgetenv_internal.

But yeah, we're looking at the same source file right now.

Cheers

EMACSNATIVELOADPATH.

---

gnu/packages/emacs.scm | 12 ++++++++++--

1 file changed, 10 insertions(+), 2 deletions(-)

Change-Id: I33303bcbaf6cbda15867a5546e793c05d1f0e67b

---

gnu/packages/emacs.scm | 5 ++++-

1 file changed, 4 insertions(+), 1 deletion(-)

---

gnu/local.mk | 1 +

gnu/packages/emacs.scm | 3 ++-

.../emacs-disable-jit-compilation.patch | 19 +++++++++++++++++++

3 files changed, 22 insertions(+), 1 deletion(-)

create mode 100644 gnu/packages/patches/emacs-disable-jit-compilation.patch

With the previous commit, relative file names are expanded relative to

ELN_DIR -- more or less. To make use of this in emacs-build-system, we must

also pass relative file names.

relative file names of the files to compile.

Change-Id: I8983f80fb0fe1573e46748222403ba8873f1599f

---

guix/build/emacs-utils.scm | 4 +++-

1 file changed, 3 insertions(+), 1 deletion(-)

directory excursion to the actual lisp directory.

Change-Id: Ifa10f9e91fe21cd4c34da11b68ddb77a03d847ca

---

gnu/packages/emacs-xyz.scm | 4 ++++

1 file changed, 4 insertions(+)