emacs-build-system builds .eln-files with mismatching path-hashes

BUG EXPLANATION

Emacs's natively-compiled .eln-files have a basename following the pattern "{feature-name}-{path-hash}-{content-hash}.eln". [0]

Guix' emacs-build-system is used to build Emacs-related packages. By

default, it uses the "emacs-minimal" package during build, which

does not support native-compilation. But if you replace the

"emacs-minimal" input with "emacs-no-x", e.g. by using

--with-input=emacs-minimal=emacs-no-x, then emacs-build-system

will make use of emacs-no-x' support of native-compilation [1]:

The build will contain .eln-files.

Hereby I'd like to report the bug that consists of mismatched path-hashes in the .eln-files that builds of Emacs-related packages contain when build with emacs-no-x (or any other Emacs that supports native compilation).

BUG REPRODUCTION

To reproduce this bug follow the following steps. Please note that guix-shell seems to leak .eln-files. (This should be reported as

another bug.) That why the reproduction steps avoid guix-shell.

Instead, we'll work with the current user profile.

Delete Emacs' eln-cache (so that we can later see if new

.eln-files have been generated):

rm -rf ~/.emacs.d/eln-cache

Remove all Emacs- and Emacs-related packages from Guix profile:

guix package -I | cut -f 4 | grep emacs | xargs guix remove

Install Emacs and emacs-unfill, as exemplary package, while

replacing input "emacs-minimal" with "emacs", so that .eln-files

are generated during the build:

guix install emacs emacs-unfill

--with-input=emacs-minimal=emacs

Launch the freshly installed Emacs and load the "unfill" package.

If the .eln-files that the emacs-unfill package provides match

Emacs' expectations (path- and content-hash), it'll use it;

otherwise, Emacs will compile a new .eln-file and save it into

~/.emacs.d/eln-cache/*/unfill-{path-hash}-{content-hash}.eln.

emacs -q --eval "(require 'unfill)"

Close Emacs after some seconds. Now determine the path-hash from

Guix' build:

basename

~/.guix-profile/lib/emacs/native-site-lisp/*/unfill-*.eln \

| cut -d - -f 2

Determine the path-hash from Emacs' native-compilation, which

apparently has happened:

basename ~/.emacs.d/eln-cache/*/unfill*.eln \

| cut -d - -f 2

The path-hashes from the last two steps are not equal.

BUG SOLUTION HINTS

In the #guix:libera.chat IRC channel, jpoiret pointed out: "the .eln file hash problem is due to grafts, grafts change the

final output name, but they can't also update the file hashes...

we'd need to modify emacs' behavior for this to work".

CITATIONS

[0]: Emacs' source code documents the meaning of the two hashes here: https://git.sv.gnu.org/cgit/emacs.git/tree/src/comp.c?h=194a8f5c1406dd7e762376bdfde78d1b7d01b6b1#n4405

[1]: Here you can see that emacs-no-x supports native-compilation unlike emacs-minimal: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/emacs.scm?h=92913703448c8e1a488ab066f60741262cdbf923#n294

Am Dienstag, dem 31.10.2023 um 23:49 +0000 schrieb Mekeor Melire:

What do you mean by "leaks .eln-files"?

Just deleting the eln-cache should be enough for a MWE. When doing an

MWE, make sure that its actually minimal :)

This is already the bug. There should not be a file written to the

eln-cache (save for the trampolines that we still write there, which is

also a known bug among those who care).

As jpoiret points out, this has to do with the file naming choices of

Emacs, not with emacs-build-system per se. We would need to get rid of

a lot of hashes if we wanted interoperable native-compiled Emacs

libraries. I wonder what upstream has to say about this.

Cheers

2023-11-01 12:52 liliana.prikler@gmail.com:

To be honest, I can't reproduce the leakage right now. I'll create another bug report if I can.

I wanted to make sure that an Emacs-related package is installed, and specifically with the --with-input=emacs-minimal=emacs transformation because otherwise .eln-files won't be built. The MRE is minimal in that sense that it ensures what's needed; only one Emacs-related package is installed; and commands are kept simple.

Yes, this is already the bug. The reason for the eln-cache to be created is that the two path-hashes do not equal.

The problem is that the .el-file-path that is passed to the Emacs function comp-el-to-eln-filename during build [1] does not equal to the

.el-file-path when Emacs is invoked. Personally, I do not

understand how grafting causes this. But I can confirm that when

--no-grafts is passed to "guix install emacs emacs-unfill

--with-input=emacs-minimal=emacs", then no eln-cache is created.

[1]: See these lines of code:

Am Mittwoch, dem 01.11.2023 um 13:03 +0000 schrieb Mekeor Melire:

I think Emacs might be calculating its own hash at runtime rather than

baking in the value at build time. I would need to investigate this,

however.

Cheers

2023-11-01 15:16 liliana.prikler@gmail.com:

Exactly. That's what I was trying to express.

Am Donnerstag, dem 02.11.2023 um 08:13 +0000 schrieb Mekeor Melire:

I'm not sure whether this is reproducible. On my system

$ guix build emacs-dash --with-input=emacs-minimal=emacs

/gnu/store/zr16hd25338imljqxxfsf07smbfv3wxd-emacs-dash-2.19.1

$ ls /gnu/store/zr16hd25338imljqxxfsf07smbfv3wxd-emacs-dash-2.19.1/lib/emacs/native-site-lisp

29.1-e9e5c1ce

$ emacs --batch --eval='(message "%s" comp-abi-hash)'

e9e5c1ce

Looks like everything's alright?

Cheers

Hi,

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

It's the .eln file itself that has the hash of the .el's path in its

name. That's computed by `comp-el-to-eln-filename`.

Best,

--

Josselin Poiret

Am Donnerstag, dem 09.11.2023 um 12:21 +0100 schrieb Josselin Poiret:

Does this still occur on the emacs-team branch, where we compile

everything from the build directory?

Cheers