[PATCH] build-system: emacs: Add generation of -pkg.el files.

Done

Details

2 participants

Andrew Tropin
Liliana Marie Prikler

Owner: unassigned

Submitted by: Andrew Tropin

Severity: normal

Andrew Tropin wrote on 9 Dec 2021 10:01

Recipients:(name . Ivan Sokolov)(address . ivan-p-sokolov@ya.ru)

Message-ID:871r2mrleb.fsf@trop.in

* guix/build/emacs-build-system.scm (%default-exclude): Add generation of

-pkg.el files for packages, which do not provide them.

---

Implemented phase, which generates -pkg.el from comments in library file. The

solution for finding main el file of the package is a little hacky, because

package name isn't available build time.

I took a part of the elisp implementation from melpa source code.

https://github.com/melpa/melpa/blob/master/package-build/package-build.el#L553

guix/build/emacs-build-system.scm | 71 ++++++++++++++++++++++++++++++-

1 file changed, 70 insertions(+), 1 deletion(-)

Toggle diff (91 lines)diff --git a/guix/build/emacs-build-system.scm b/guix/build/emacs-build-system.scm
index f13162d6c4..ee934f4bde 100644
--- a/guix/build/emacs-build-system.scm
+++ b/guix/build/emacs-build-system.scm
@@ -154,6 +154,74 @@ (define (substitute-program-names)
           (substitute-program-names))))
     #t))
 
+(define* (find-root-library-file name)
+  (let lp ((subnames (string-split
+                      (package-name-version->elpa-name-version name) #\-))
+           (possible-name ""))
+    (cond
+     ((file-exists? (string-append possible-name ".el")) possible-name)
+     ((null? subnames) #f)
+     (#t (lp (cdr subnames) (string-append
+                             possible-name
+                             (if (string-null? possible-name) "" "-")
+                             (car subnames)))))))
+
+(define (add-pkg-expr name)
+  `(progn
+    (require 'lisp-mnt)
+    (require 'package)
+
+    (defun build-package-desc-from-library (name)
+      (package-desc-from-define
+       name
+       ;; Workaround for malformed version string (for example "24 (beta)" in
+       ;; paredit.el), try to parse version obtained by lm-version, before
+       ;; trying to create package-desc.  Otherwis the whole process of
+       ;; generation -pkg.el will fail.
+       (condition-case
+        nil
+        (version-to-list (lm-version))
+        (lm-version)
+        (error "0.0.0"))
+       (or (save-excursion
+            (goto-char (point-min))
+            (and (re-search-forward
+                  "^;;; [^ ]*\\.el ---[ \t]*\\(.*?\\)[ \t]*\\(-\\*-.*-\\*-[ \t]*\\)?$"
+                  nil t)
+                 (match-string-no-properties 1)))
+           "No description available.")
+       (when-let ((require-lines (lm-header-multiline "package-requires")))
+                 (package--prepare-dependencies
+                  (package-read-from-string
+                   (mapconcat 'identity require-lines " "))))
+       :kind       'single
+       :url        (lm-homepage)
+       :keywords   (lm-keywords-list)
+       :maintainer (lm-maintainer)
+       :authors    (lm-authors)))
+
+    (defun generate-package-description-file (name)
+      (package-generate-description-file
+       (build-package-desc-from-library name)
+       (concat name "-pkg.el")))
+
+    (condition-case err
+     (generate-package-description-file ,name)
+     (message (concat ,name "-pkg.el file generated."))
+     (error
+      (message "There are some errors during generation of -pkg.el file:")
+      (message "%s" (error-message-string err))))))
+
+(define* (add-pkg-file #:key name outputs #:allow-other-keys)
+  (define source (getcwd))
+  (chdir (elpa-directory (assoc-ref outputs "out")))
+  (let ((el-lib (find-root-library-file name)))
+    (when (and el-lib (not (file-exists? (string-append el-lib "-pkg.el"))))
+      (emacs-batch-edit-file (string-append el-lib ".el")
+        (add-pkg-expr el-lib))))
+  (chdir source)
+  #t)
+
 (define* (check #:key tests? (test-command '("make" "check"))
                 (parallel-tests? #t) #:allow-other-keys)
   "Run the tests by invoking TEST-COMMAND.
@@ -294,7 +362,8 @@ (define %standard-phases
       enable-autoloads-compilation)
     (add-after 'enable-autoloads-compilation 'patch-el-files patch-el-files)
     ;; The .el files are byte compiled directly in the store.
-    (add-after 'patch-el-files 'build build)
+    (add-after 'patch-el-files 'add-pkg-file add-pkg-file)
+    (add-after 'add-pkg-file 'build build)
     (add-after 'build 'validate-compiled-autoloads validate-compiled-autoloads)
     (add-after 'validate-compiled-autoloads 'move-doc move-doc)))
 
-- 
2.34.0

Liliana Marie Prikler wrote on 14 Jan 2022 21:56

Recipients:(name . Ivan Sokolov)(address . ivan-p-sokolov@ya.ru)

Message-ID:80cfa468ef91649dce477779a70d3a8ce64fd0e4.camel@gmail.com

Hi Andrew,

Am Donnerstag, dem 09.12.2021 um 12:01 +0300 schrieb Andrew Tropin:

Toggle quote (11 lines)> 
> * guix/build/emacs-build-system.scm (%default-exclude): Add
> generation of
> -pkg.el files for packages, which do not provide them.
> ---
> Implemented phase, which generates -pkg.el from comments in library
> file.� The solution for finding main el file of the package is a
> little hacky, because package name isn't available build time.
> 
> I took a part of the elisp implementation from melpa source code.
> https://github.com/melpa/melpa/blob/master/package-build/package-build.el#L553

As promised, I took a deeper look at your patch.  As already noted,
there were some clean-up actions I had to perform, such as keeping to
our line limit (it was not easy, I tell you) among other things.  Also,
confusingly, your condition-case code did not handle errors and wrong
handling of the version field blew up everything for me.  Did you test
this code?

In any case, attached is my revised patch.  I so far only checked it
with emacs-olivetti -- a package whose description is missing in
current Guix Emacs.  I'll give everyone some time to test things before
pushing this however; I don't want to break a bunch of Emacs packages
scattered around various files.

Cheers

Attachment: file

Andrew Tropin wrote on 21 Jan 2022 13:11

Recipients:(name . Ivan Sokolov)(address . ivan-p-sokolov@ya.ru)

Message-ID:87zgnp8f0n.fsf@trop.in

On 2022-01-14 21:56, Liliana Marie Prikler wrote:

Toggle quote (21 lines)> Hi Andrew,
>
> Am Donnerstag, dem 09.12.2021 um 12:01 +0300 schrieb Andrew Tropin:
>> 
>> * guix/build/emacs-build-system.scm (%default-exclude): Add
>> generation of
>> -pkg.el files for packages, which do not provide them.
>> ---
>> Implemented phase, which generates -pkg.el from comments in library
>> file.� The solution for finding main el file of the package is a
>> little hacky, because package name isn't available build time.
>> 
>> I took a part of the elisp implementation from melpa source code.
>> https://github.com/melpa/melpa/blob/master/package-build/package-build.el#L553
> As promised, I took a deeper look at your patch.  As already noted,
> there were some clean-up actions I had to perform, such as keeping to
> our line limit (it was not easy, I tell you) among other things.  Also,
> confusingly, your condition-case code did not handle errors and wrong
> handling of the version field blew up everything for me.  Did you test
> this code?

I thought it was the version I used for past a few weeks, however I

could send a little outdated version, not the latest one by accident.

Tested it with guix build emacs-paredit.

See your fix which wraps both forms in one let, thank you, yep, it's

very likely I sent an intermediate version of the patch accidentially.

Toggle quote (66 lines)>
> In any case, attached is my revised patch.  I so far only checked it
> with emacs-olivetti -- a package whose description is missing in
> current Guix Emacs.  I'll give everyone some time to test things before
> pushing this however; I don't want to break a bunch of Emacs packages
> scattered around various files.
>
> Cheers
> From 9fa6a09a131cfe436ca053c960ed9625263bc650 Mon Sep 17 00:00:00 2001
> From: Andrew Tropin <andrew@trop.in>
> Date: Thu, 9 Dec 2021 12:01:46 +0300
> Subject: [PATCH] build-system: emacs: Ensure that package descriptions are
>  generated.
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> This patch addresses the second part of <https://bugs.gnu.org/48331>.
> While existing -pkg.el files were previously installed, no such files
> were generated for packages lacking them, resulting in packages not
> being listed as installed and not being available towards
> “describe-package”.
>
> * guix/build/emacs-build-system.scm (find-root-library-file)
> (ensure-package-description): New variables.
> (%standard-phases): Add ‘ensure-package-description’.
> ---
>  guix/build/emacs-build-system.scm | 77 ++++++++++++++++++++++++++++++-
>  1 file changed, 76 insertions(+), 1 deletion(-)
>
> diff --git a/guix/build/emacs-build-system.scm b/guix/build/emacs-build-system.scm
> index ab77e57f33..58d2a9b9f4 100644
> --- a/guix/build/emacs-build-system.scm
> +++ b/guix/build/emacs-build-system.scm
> @@ -140,6 +140,79 @@ (define (substitute-program-names)
>            (substitute-program-names))))
>      #t))
>  
> +(define (find-root-library-file name)
> +  (let loop ((parts (string-split
> +                     (package-name-version->elpa-name-version name) #\-))
> +             (candidate ""))
> +    (cond
> +     ;; at least one version part is given, so we don't terminate "early"
> +     ((null? parts) #f)
> +     ((string-null? candidate) (loop (cdr parts) (car parts)))
> +     ((file-exists? (string-append candidate ".el")) candidate)
> +     (else
> +      (loop (cdr parts) (string-append candidate "-" (car parts)))))))
> +
> +(define* (ensure-package-description #:key outputs #:allow-other-keys)
> +  (define (write-pkg-file name)
> +    (define summary-regexp
> +      "^;;; [^ ]*\\.el ---[ \t]*\\(.*?\\)[ \t]*\\(-\\*-.*-\\*-[ \t]*\\)?$")
> +    (define %write-pkg-file-form
> +      `(progn
> +        (require 'lisp-mnt)
> +        (require 'package)
> +
> +        (defun build-package-desc-from-library (name)
> +          (package-desc-from-define
> +           name
> +           ;; Workaround for malformed version string (for example "24 (beta)"
> +           ;; in paredit.el), try to parse version obtained by lm-version,
> +           ;; before trying to create package-desc.  Otherwis the whole process

s/Otherwis/Otherwise,

Toggle quote (61 lines)> +           ;; of generation -pkg.el will fail.
> +           (condition-case
> +            nil
> +            (let ((version (lm-version)))
> +              ;; raises an error if version is invalid
> +              (and (version-to-list version) version))
> +            (error "0.0.0"))
> +           (or (save-excursion
> +                (goto-char (point-min))
> +                (and (re-search-forward ,summary-regexp nil t)
> +                     (match-string-no-properties 1)))
> +               package--default-summary)
> +           (let ((require-lines (lm-header-multiline "package-requires")))
> +             (and require-lines
> +                  (package--prepare-dependencies
> +                   (package-read-from-string
> +                    (mapconcat 'identity require-lines " ")))))
> +           :kind       'single
> +           :url        (lm-homepage)
> +           :keywords   (lm-keywords-list)
> +           :maintainer (lm-maintainer)
> +           :authors    (lm-authors)))
> +
> +        (defun generate-package-description-file (name)
> +          (package-generate-description-file
> +           (build-package-desc-from-library name)
> +           (concat name "-pkg.el")))
> +
> +        (condition-case
> +         err
> +         (let ((name (file-name-base (buffer-file-name))))
> +           (generate-package-description-file name)
> +           (message (concat name "-pkg.el file generated.")))
> +         (error
> +          (message "There are some errors during generation of -pkg.el file:")
> +          (message "%s" (error-message-string err))))))
> +
> +    (unless (file-exists? (string-append name "-pkg.el"))
> +      (emacs-batch-edit-file (string-append name ".el")
> +        %write-pkg-file-form)))
> +
> +  (let* ((out (assoc-ref outputs "out"))
> +         (elpa-name-ver (store-directory->elpa-name-version out)))
> +    (with-directory-excursion (elpa-directory out)
> +      (and=> (find-root-library-file elpa-name-ver) write-pkg-file))))
> +
>  (define* (check #:key tests? (test-command '("make" "check"))
>                  (parallel-tests? #t) #:allow-other-keys)
>    "Run the tests by invoking TEST-COMMAND.
> @@ -279,8 +352,10 @@ (define %standard-phases
>      (add-after 'make-autoloads 'enable-autoloads-compilation
>        enable-autoloads-compilation)
>      (add-after 'enable-autoloads-compilation 'patch-el-files patch-el-files)
> +    (add-after 'patch-el-files 'ensure-package-description
> +      ensure-package-description)
>      ;; The .el files are byte compiled directly in the store.
> -    (add-after 'patch-el-files 'build build)
> +    (add-after 'ensure-package-description 'build build)
>      (add-after 'build 'validate-compiled-autoloads validate-compiled-autoloads)
>      (add-after 'validate-compiled-autoloads 'move-doc move-doc)))

Looks cleaner than original implementation, thank you!)

Applied on my local checkout of guix channel, list-packages now shows

117 external packages =) Will keep it for a while and let you know if

any problems will reveal themselves.

Best regards,

Andrew Tropin

Liliana Marie Prikler wrote on 21 Jan 2022 17:57

Recipients:(name . Ivan Sokolov)(address . ivan-p-sokolov@ya.ru)

Message-ID:6dc75712f3ea1cfe12594e45ae977f8f4e8543fb.camel@gmail.com

Hi Andrew,

Am Freitag, dem 21.01.2022 um 15:11 +0300 schrieb Andrew Tropin:

Toggle quote (14 lines)> > As promised, I took a deeper look at your patch.� As already noted,
> > there were some clean-up actions I had to perform, such as keeping
> > to our line limit (it was not easy, I tell you) among other
> > things.� Also, confusingly, your condition-case code did not handle
> > errors and wrong handling of the version field blew up everything
> > for me.� Did you test this code?
> 
> I thought it was the version I used for past a few weeks, however I
> could send a little outdated version, not the latest one by accident.
> Tested it with guix build emacs-paredit.
> 
> See your fix which wraps both forms in one let, thank you, yep, it's
> very likely I sent an intermediate version of the patch
> accidentially.

Git index not up to date, truly a classic :)

Toggle quote (4 lines)

> > +�� ;; before trying to create package-desc.� Otherwis the

> > whole process

> s/Otherwis/Otherwise,

Thanks.

Toggle quote (5 lines)

> Applied on my local checkout of guix channel, list-packages now shows

> 117 external packages =)� Will keep it for a while and let you know

> if any problems will reveal themselves.

Fair enough, I'll reset my "14 days without review" clock then :)

Andrew Tropin wrote on 28 Jan 2022 16:29

Recipients:(name . Ivan Sokolov)(address . ivan-p-sokolov@ya.ru)

Message-ID:8735l73mm5.fsf@trop.in

On 2022-01-21 17:57, Liliana Marie Prikler wrote:

Toggle quote (31 lines)> Hi Andrew,
>
> Am Freitag, dem 21.01.2022 um 15:11 +0300 schrieb Andrew Tropin:
>> > As promised, I took a deeper look at your patch.� As already noted,
>> > there were some clean-up actions I had to perform, such as keeping
>> > to our line limit (it was not easy, I tell you) among other
>> > things.� Also, confusingly, your condition-case code did not handle
>> > errors and wrong handling of the version field blew up everything
>> > for me.� Did you test this code?
>> 
>> I thought it was the version I used for past a few weeks, however I
>> could send a little outdated version, not the latest one by accident.
>> Tested it with guix build emacs-paredit.
>> 
>> See your fix which wraps both forms in one let, thank you, yep, it's
>> very likely I sent an intermediate version of the patch
>> accidentially.
> Git index not up to date, truly a classic :)
>
>> > +���������� ;; before trying to create package-desc.� Otherwis the
>> > whole process
>> 
>> s/Otherwis/Otherwise,
> Thanks.
>> 
>
>> Applied on my local checkout of guix channel, list-packages now shows
>> 117 external packages =)� Will keep it for a while and let you know
>> if any problems will reveal themselves.
> Fair enough, I'll reset my "14 days without review" clock then :)

Have been using your version of the patch for a week now, added a few
more emacs packages to my config, nothing stopped to build or launch, so
it seems to work good.

-- 
Best regards,
Andrew Tropin

Liliana Marie Prikler wrote on 29 Jan 2022 08:53

Recipients:(name . Ivan Sokolov)(address . ivan-p-sokolov@ya.ru)

Message-ID:70ef92949b1b9bd6543add18c53634e005e5260c.camel@gmail.com

Am Freitag, dem 28.01.2022 um 18:29 +0300 schrieb Andrew Tropin:

Toggle quote (3 lines)

> Have been using your version of the patch for a week now, added a few

> more emacs packages to my config, nothing stopped to build or launch,

> so it seems to work good.

"Seems to work good" sounds good enough to me. Let's mark this bug as

done :)

Closed

Andrew Tropin wrote on 31 Jan 2022 18:18

Recipients:(name . Ivan Sokolov)(address . ivan-p-sokolov@ya.ru)

Message-ID:87y22vygb5.fsf@trop.in

On 2022-01-29 08:53, Liliana Marie Prikler wrote:

Toggle quote (7 lines)

> Am Freitag, dem 28.01.2022 um 18:29 +0300 schrieb Andrew Tropin:

>> Have been using your version of the patch for a week now, added a few

>> more emacs packages to my config, nothing stopped to build or launch,

>> so it seems to work good.

> "Seems to work good" sounds good enough to me. Let's mark this bug as

> done :)

Yay! Thank you for helping with it! :)

Best regards,

Andrew Tropin

Closed

Your comment

This issue is archived.

To comment on this conversation send an email to 52388@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 52388

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date