emacs-next/git-master broken due to pdmp file naming changes

  • Done
  • quality assurance status badge
Details
3 participants
  • Morgan.J.Smith
  • Liliana Marie Prikler
  • Paxton Evans
Owner
unassigned
Submitted by
Paxton Evans
Severity
normal
P
P
Paxton Evans wrote on 17 Oct 2021 23:20
(address . bug-guix@gnu.org)
a31d056d98bc5617a40dada3de8aed87@riseup.net
As of Emacs commit e81f1faca4382ed5c8f15fec84fb7c900a5468f9, building
emacs-next
with the git-master branch will cause some problems on Guix. The
"emacs-next"
build itself will work without any errors, but it will complain whenever
you try
run it:

Toggle quote (4 lines)
> emacs: could not load dump file
> "/gnu/store/a5bx5v96snvgiv9r1b7i1im5ccn5mz54-emacs-next-git.master/libexec/emacs/29.0.50/x86_64-pc-linux-gnu/emacs-66e6890a5565f2bed1ee56075d21e0051d891a59200cdd092c0946403fb84ac2.pdmp":
> not a dump file

This will also cause new builds of packages that use
emacs-next/git-master as an
input to fail as well, with a version of the same error. Here's a
(failed) build
log for emacs-pdf-tools:

Toggle quote (32 lines)
> [top part snipped]
>
> make[1]: Leaving directory '/tmp/guix-build-emacs-pdf-tools-0.90-2.5f77dae.drv-0/source/server'
> phase `install' succeeded after 0.3 seconds
> starting phase `patch-shebangs'
> phase `patch-shebangs' succeeded after 0.0 seconds
> starting phase `strip'
> stripping binaries in "/gnu/store/swfjir05k1bg22l50yawhibbv0y5qhr9-emacs-pdf-tools-0.90-2.5f77dae/bin" with "strip" and flags ("--strip-debug" "--enable-deterministic-archives")
> phase `strip' succeeded after 0.0 seconds
> starting phase `validate-runpath'
> validating RUNPATH of 1 binaries in "/gnu/store/swfjir05k1bg22l50yawhibbv0y5qhr9-emacs-pdf-tools-0.90-2.5f77dae/bin"...
> phase `validate-runpath' succeeded after 0.0 seconds
> starting phase `validate-documentation-location'
> phase `validate-documentation-location' succeeded after 0.0 seconds
> starting phase `delete-info-dir-file'
> phase `delete-info-dir-file' succeeded after 0.0 seconds
> starting phase `patch-dot-desktop-files'
> phase `patch-dot-desktop-files' succeeded after 0.0 seconds
> starting phase `install-license-files'
> installing 0 license files from '.'
> phase `install-license-files' succeeded after 0.0 seconds
> starting phase `reset-gzip-timestamps'
> phase `reset-gzip-timestamps' succeeded after 0.0 seconds
> starting phase `compress-documentation'
> phase `compress-documentation' succeeded after 0.0 seconds
> starting phase `enter-lisp-dir'
> phase `enter-lisp-dir' succeeded after 0.0 seconds
> starting phase `emacs-patch-variables'
> emacs: could not load dump file
> "/gnu/store/rv72rvqa3vh2vw7jpkm3d9ww0xb4ibxv-emacs-next-git.master/libexec/emacs/29.0.50/x86_64-pc-linux-gnu/emacs-4588128eef9937d195927b0cccee280697619db43041dd08b620788b8dd59b77.pdmp":
> not a dump file

I think this is because of some recent changes made to the naming of the
Emacs
pdmp file in the master branch. Previously, it just made a file called
"emacs.pdmp" but now it appends the version or commit hash, i.e.
"emacs-4588128eef9937d195927b0cccee280697619db43041dd08b620788b8dd59b77.pdmp".
This new naming scheme started with Emacs commit
e81f1faca4382ed5c8f15fec84fb7c900a5468f9.

This might be confusing Guix's Emacs package recipe, which has a regexp
that
simply looks for "emacs.pdmp" when it does some cleanup after building
Emacs.

guix/gnu/packages/emacs.scm:182 and following:

> (lambda* (#:key outputs target #:allow-other-keys)
> (let* ((libexec (string-append (assoc-ref outputs "out")
> "/libexec"))
> ;; each of these find-files should return one file
> (pdmp (find-files libexec "^emacs\\.pdmp$"))
> (pdmp-real (find-files libexec
> "^\\.emacs\\.pdmp-real$")))

So I think that regexp might need to be changed a little to accommodate
those
recent Emacs changes. That's as far as I'm able to get with it, though,
as I'm
not very good at Guile or regexps.

-paxton
M
M
Morgan.J.Smith wrote on 19 Oct 2021 01:50
[PATCH] gnu: emacs: Handle pdump filenames that contain a fingerprint
(name . Morgan Smith)(address . Morgan.J.Smith@outlook.com)
BYAPR05MB40235F9D0819D249C3E6FBA7C5BC9@BYAPR05MB4023.namprd05.prod.outlook.com
From: Morgan Smith <Morgan.J.Smith@outlook.com>

* gnu/packages/emacs.scm (emacs) [restore-emacs-pdmp]: Add regex to handle
filenames that have a fingerprint in them
---

Fixed your problem. Well it was also my problem. The regex is pretty simple but probably a little intimidating if you've never done it before. I highly recommend reading up on ERE's and BRE's from the POSIX specification if you have time. I still don't exactly know how guile does regex but knowing POSIX regex has served me quite well.

I've tested this patch and it works great for me on the latest emacs commit. It should also be backwards compatible so that's nice.


gnu/packages/emacs.scm | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

Toggle diff (19 lines)
diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 6d9950d068..6ebcfecbc7 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -183,9 +183,10 @@ (define* (emacs-byte-compile-directory dir)
(let* ((libexec (string-append (assoc-ref outputs "out")
"/libexec"))
;; each of these find-files should return one file
- (pdmp (find-files libexec "^emacs\\.pdmp$"))
+ ;; there might be a fingerprint in the name
+ (pdmp (find-files libexec "^emacs(-[[:xdigit:]]+)?\\.pdmp$"))
(pdmp-real (find-files libexec
- "^\\.emacs\\.pdmp-real$")))
+ "^\\.emacs(-[[:xdigit:]]+)?\\.pdmp-real$")))
(for-each (lambda (wrapper real)
(delete-file wrapper)
(rename-file real wrapper))
--
2.33.0
L
L
Liliana Marie Prikler wrote on 19 Oct 2021 20:45
fb033300ef41fb9771513e502d88afbd09777738.camel@gmail.com
Hi,

Am Montag, den 18.10.2021, 19:50 -0400 schrieb
Morgan.J.Smith@outlook.com:
Toggle quote (42 lines)
> From: Morgan Smith <Morgan.J.Smith@outlook.com>
>
> * gnu/packages/emacs.scm (emacs) [restore-emacs-pdmp]: Add regex to
> handle
> filenames that have a fingerprint in them
> ---
>
> Fixed your problem. Well it was also my problem. The regex is
> pretty simple but probably a little intimidating if you've never done
> it before. I highly recommend reading up on ERE's and BRE's from the
> POSIX specification if you have time. I still don't exactly know how
> guile does regex but knowing POSIX regex has served me quite well.
>
> I've tested this patch and it works great for me on the latest emacs
> commit. It should also be backwards compatible so that's nice.
>
>
> gnu/packages/emacs.scm | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
> index 6d9950d068..6ebcfecbc7 100644
> --- a/gnu/packages/emacs.scm
> +++ b/gnu/packages/emacs.scm
> @@ -183,9 +183,10 @@ (define* (emacs-byte-compile-directory dir)
> (let* ((libexec (string-append (assoc-ref outputs
> "out")
> "/libexec"))
> ;; each of these find-files should return one
> file
> - (pdmp (find-files libexec "^emacs\\.pdmp$"))
> + ;; there might be a fingerprint in the name
> + (pdmp (find-files libexec "^emacs(-
> [[:xdigit:]]+)?\\.pdmp$"))
> (pdmp-real (find-files libexec
> - "^\\.emacs\\.pdmp-
> real$")))
> + "^\\.emacs(-
> [[:xdigit:]]+)?\\.pdmp-real$")))
> (for-each (lambda (wrapper real)
> (delete-file wrapper)
> (rename-file real wrapper))
Hi, this might sound somewhat odd, but I think this solution could be a
little over-engineered (also it does not match the case where the
version is an actual version rather than a git hash). Perhaps instead
we ought to simply look for the .pdmp-real suffix, construct the old
file name and move the file back. WDYT?
M
M
Morgan.J.Smith wrote on 19 Oct 2021 21:10
[PATCH v2] gnu: emacs: Handle pdump filenames that contain a fingerprint
(name . Morgan Smith)(address . Morgan.J.Smith@outlook.com)
BYAPR05MB40237D49F7DC95B8A0D77905C5BD9@BYAPR05MB4023.namprd05.prod.outlook.com
From: Morgan Smith <Morgan.J.Smith@outlook.com>

* gnu/packages/emacs.scm (emacs) [restore-emacs-pdmp]: Change regex to handle
filenames that have a fingerprint in them. Also remove #t at the end of the phase.
---

I'm a fan of your idea. I tested this patch and it seems to work as expected.

I do think that the fingerprint will never be a version and my original patch would've been fine. However, I'm not certain of that and there isn't a good reason to have a super precise regex here anyways.


gnu/packages/emacs.scm | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)

Toggle diff (24 lines)
diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 6d9950d068..2aae21be6d 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -183,14 +183,13 @@ (define* (emacs-byte-compile-directory dir)
(let* ((libexec (string-append (assoc-ref outputs "out")
"/libexec"))
;; each of these find-files should return one file
- (pdmp (find-files libexec "^emacs\\.pdmp$"))
+ (pdmp (find-files libexec "\\.pdmp$"))
(pdmp-real (find-files libexec
- "^\\.emacs\\.pdmp-real$")))
+ "\\.pdmp-real$")))
(for-each (lambda (wrapper real)
(delete-file wrapper)
(rename-file real wrapper))
- pdmp pdmp-real))
- #t))
+ pdmp pdmp-real))))
(add-after 'glib-or-gtk-wrap 'strip-double-wrap
(lambda* (#:key outputs #:allow-other-keys)
;; Directly copy emacs-X.Y to emacs, so that it is not wrapped
--
2.33.1
L
L
Liliana Marie Prikler wrote on 19 Oct 2021 21:53
df9fdfdc6ab1c121aac0362a4d36d49eb1a5039c.camel@gmail.com
Am Dienstag, den 19.10.2021, 15:10 -0400 schrieb
Morgan.J.Smith@outlook.com:
Toggle quote (14 lines)
> From: Morgan Smith <Morgan.J.Smith@outlook.com>
>
> * gnu/packages/emacs.scm (emacs) [restore-emacs-pdmp]: Change regex
> to handle filenames that have a fingerprint in them. Also remove #t
> at the end of the phase.
> ---
>
> I'm a fan of your idea. I tested this patch and it seems to work as
> expected.
>
> I do think that the fingerprint will never be a version and my
> original patch would've been fine. However, I'm not certain of that
> and there isn't a good reason to have a super precise regex here
> anyways.
Mostly LGTM, but keep in mind that your new pdmp and pdmp-real could
potentially match more than one file even if realistically there's only
one to be matched. Hence why I've said we ought to construct the old
file name from the pdmp-real, because find-files need not be sorted.

Toggle quote (31 lines)
>
> gnu/packages/emacs.scm | 7 +++----
> 1 file changed, 3 insertions(+), 4 deletions(-)
>
> diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
> index 6d9950d068..2aae21be6d 100644
> --- a/gnu/packages/emacs.scm
> +++ b/gnu/packages/emacs.scm
> @@ -183,14 +183,13 @@ (define* (emacs-byte-compile-directory dir)
> (let* ((libexec (string-append (assoc-ref outputs
> "out")
> "/libexec"))
> ;; each of these find-files should return one
> file
> - (pdmp (find-files libexec "^emacs\\.pdmp$"))
> + (pdmp (find-files libexec "\\.pdmp$"))
> (pdmp-real (find-files libexec
> - "^\\.emacs\\.pdmp-
> real$")))
> + "\\.pdmp-real$")))
> (for-each (lambda (wrapper real)
> (delete-file wrapper)
> (rename-file real wrapper))
> - pdmp pdmp-real))
> - #t))
> + pdmp pdmp-real))))
> (add-after 'glib-or-gtk-wrap 'strip-double-wrap
> (lambda* (#:key outputs #:allow-other-keys)
> ;; Directly copy emacs-X.Y to emacs, so that it is not
> wrapped

Regards,
Liliana
M
M
Morgan.J.Smith wrote on 24 Oct 2021 03:00
[PATCH v3] gnu: emacs: Handle pdump filenames that contain a fingerprint
(name . Morgan Smith)(address . Morgan.J.Smith@outlook.com)
BYAPR05MB40231F6A4062C4D8F4170D60C5829@BYAPR05MB4023.namprd05.prod.outlook.com
From: Morgan Smith <Morgan.J.Smith@outlook.com>

* gnu/packages/emacs.scm (emacs) [restore-emacs-pdmp]: Change regex to handle
filenames that have a fingerprint in them. Remove #t at the end of the
phase. Don't bother deleting the old files because rename will overwrite them
anyways.
---

find-files does sort its return so my previous 2 patches probably work great
regardless of the number of pdmps we encounter (which will only ever be 1
anyways).

Also the fingerprint is a 32 char hexidecimal number used to uniquely[1]
identify the Emacs binary (well more specifically an earlier binary generated
during the build process). If you put the version in there it would
completely invalidate the entire purpose of this fingerprint. Different
distros will have different fingerprints for the same version and that's the
point.

Here is another over engineered solution even though the previous 2 still work
great. I'm not sure if I'm supposed to just throw a '(use-modules ...)' in the
phase but if we want to use the #:modules argument then it will break all of
the other Emacsen that inherit from Emacs since they use different build
systems.


1. Hash collisions are a thing so maybe not actually uniquely


gnu/packages/emacs.scm | 22 +++++++++++-----------
1 file changed, 11 insertions(+), 11 deletions(-)

Toggle diff (36 lines)
diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 6d9950d068..e25b79b832 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -179,18 +179,18 @@ (define* (emacs-byte-compile-directory dir)
(add-after 'glib-or-gtk-wrap 'restore-emacs-pdmp
;; restore the dump file that Emacs installs somewhere in
;; libexec/ to its original state
- (lambda* (#:key outputs target #:allow-other-keys)
- (let* ((libexec (string-append (assoc-ref outputs "out")
+ (lambda* (#:key outputs #:allow-other-keys)
+ (use-modules (ice-9 regex))
+ (let* ((pdmp-regex ".(emacs(-[[:xdigit:]]+)?.pdmp)-real")
+ (libexec (string-append (assoc-ref outputs "out")
"/libexec"))
- ;; each of these find-files should return one file
- (pdmp (find-files libexec "^emacs\\.pdmp$"))
- (pdmp-real (find-files libexec
- "^\\.emacs\\.pdmp-real$")))
- (for-each (lambda (wrapper real)
- (delete-file wrapper)
- (rename-file real wrapper))
- pdmp pdmp-real))
- #t))
+ (pdmp (find-files libexec pdmp-regex)))
+ (map
+ (lambda (pdmp)
+ (rename-file
+ pdmp
+ (regexp-substitute #f (string-match pdmp-regex pdmp) 'pre 1)))
+ pdmp))))
(add-after 'glib-or-gtk-wrap 'strip-double-wrap
(lambda* (#:key outputs #:allow-other-keys)
;; Directly copy emacs-X.Y to emacs, so that it is not wrapped
--
2.33.1
L
L
Liliana Marie Prikler wrote on 24 Oct 2021 10:52
4e87942686af0a94689f8ed22dedd5d1ea5b7af5.camel@gmail.com
Hi,

Am Samstag, den 23.10.2021, 21:00 -0400 schrieb
Morgan.J.Smith@outlook.com:
Toggle quote (14 lines)
> From: Morgan Smith <Morgan.J.Smith@outlook.com>
>
> * gnu/packages/emacs.scm (emacs) [restore-emacs-pdmp]: Change regex
> to handle
> filenames that have a fingerprint in them. Remove #t at the end of
> the
> phase. Don't bother deleting the old files because rename will
> overwrite them
> anyways.
> ---
>
> find-files does sort its return so my previous 2 patches probably
> work great regardless of the number of pdmps we encounter (which will
> only ever be 1 anyways).
You are completely right, thanks for pointing that out. I've pushed an
adjusted v2 with updated comments and everything. I also verified that
emacs-next could be updated with a little hacking around patches that
don't apply.

Thanks!
Closed
?