Emacs' describe-package doesn't work for packages managed by guix

  • Done
  • quality assurance status badge
Details
7 participants
  • Andrew Tropin
  • Arun Isaac
  • Ivan Sokolov
  • Leo Prikler
  • Liliana Marie Prikler
  • Ludovic Courtès
  • Maxim Cournoyer
Owner
unassigned
Submitted by
Andrew Tropin
Severity
normal
A
A
Andrew Tropin wrote on 10 May 2021 09:51
(address . bug-guix@gnu.org)
87im3r81so.fsf@trop.in
describe-package and list-packages do not show emacs packages, installed
with guix. Packages themselves work perfectly fine, but not listed in
list-packages and can't be accessed with describe-package.

Way to reproduce:

guix environment --pure --ad-hoc emacs emacs-treemacs

emacs -q

M-x treemacs ;; Works fine
C-h P treemacs ;; Doesn't work
M-x list-packages ;; Doesn't list treemacs

Played around with it a little bit, still not sure how to solve.
L
L
Leo Prikler wrote on 11 May 2021 12:05
78cfb495305fb1137da26e37a7ae5dd9467043bf.camel@student.tugraz.at
Am Montag, den 10.05.2021, 10:51 +0300 schrieb Andrew Tropin:
Toggle quote (16 lines)
> describe-package and list-packages do not show emacs packages,
> installed
> with guix. Packages themselves work perfectly fine, but not listed in
> list-packages and can't be accessed with describe-package.
>
> Way to reproduce:
>
> guix environment --pure --ad-hoc emacs emacs-treemacs
>
> emacs -q
>
> M-x treemacs ;; Works fine
> C-h P treemacs ;; Doesn't work
> M-x list-packages ;; Doesn't list treemacs
>
> Played around with it a little bit, still not sure how to solve.
This mail brought me back to the good old days of 2018, when I was
still using Gentoo and had to solve a similar issue.

The problem here is, that Guix does not include the <package>-pkg.el
file, that would typically be generated by package.el. To deal with
this, you have to provide package specs on your own. There already
exists a utility to locate libraries in a package manager agnostic
fashion [1], all you need to do is to feed back that information to
package.el.

I have now published emacs-dpd [2], which does exactly that. To use it
for your Guix Emacs packages, execute
(dpd (list "$GUIX_PROFILE/share/emacs/site-lisp" ...))
replacing "$GUIX_PROFILE" with the actual profile, after `package-
initialize' has run with `dpd-fuzzy-recognize' in `dpd-recognize-hook'.
I might write a more detailed README later.

Neither packed nor dpd are currently packaged in Guix. packed can
easily be imported from melpa-stable, but for dpd you'd have to write
your own guix.scm. I might do that at some point as well.

We already had modifications in emacs-build-system recently, so if you
want to argue, that all Emacs packages should have that <package>-
pkg.el to work with package.el out of the box, I would ask you to wait,
so as to not cause an "Emacs world" rebuild again after only ten days.
I also don't know whether Guix has the same information as package.el
at build time, but that might change with time as well. Particularly,
there will hopefully be a move towards supplying name and version at
build, which would give us the most important information.

Regards,
Leo

A
A
Andrew Tropin wrote on 11 May 2021 17:57
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)(address . 48331@debbugs.gnu.org)
CABrWRW3bRnP+vMu6j73o=p2GHbOsLQ-2MejDV5bCNFuqLDD9Sw@mail.gmail.com
Toggle quote (7 lines)
> The problem here is, that Guix does not include the <package>-pkg.el
> file, that would typically be generated by package.el. To deal with
> this, you have to provide package specs on your own. There already
> exists a utility to locate libraries in a package manager agnostic
> fashion [1], all you need to do is to feed back that information to
> package.el.

Yep, I figured it out yesterday and even hacked around a little bit.

Patched emacs-build-system to place packages under elpa/NAME-VERSION
subdirectory and removed "-pkg\\.el$" from %default-exclude.

Toggle quote (7 lines)
> I have now published emacs-dpd [2], which does exactly that. To use it
> for your Guix Emacs packages, execute
> (dpd (list "$GUIX_PROFILE/share/emacs/site-lisp" ...))
> replacing "$GUIX_PROFILE" with the actual profile, after `package-
> initialize' has run with `dpd-fuzzy-recognize' in `dpd-recognize-hook'.
> I might write a more detailed README later.

Most of the packages already have -pkg.el in sources, but yep, pretty
cool utility, also thought about implementing something like that
yesterday, but luckily I didn't and now I do not need to do it, because
now I'm aware of already-existing implementations!)

Toggle quote (13 lines)
> Neither packed nor dpd are currently packaged in Guix. packed can
> easily be imported from melpa-stable, but for dpd you'd have to write
> your own guix.scm. I might do that at some point as well.

> We already had modifications in emacs-build-system recently, so if you
> want to argue, that all Emacs packages should have that <package>-
> pkg.el to work with package.el out of the box, I would ask you to wait,
> so as to not cause an "Emacs world" rebuild again after only ten days.
> I also don't know whether Guix has the same information as package.el
> at build time, but that might change with time as well. Particularly,
> there will hopefully be a move towards supplying name and version at
> build, which would give us the most important information.

Very cool, I didn't have the latest changes on my local checkout and didn't
see your commits, but now I see, it is exactly what I needed.

The only side note: it should be site-lisp/elpa/NAME-VERSION (right now
it is site-lisp/NAME-VERSION). Also, on line 137 elpa-directory
function can be used.

When you will be updating the path, please remove -pkg.el from
%default-exclude.

Thank you very much for your work! Really appreciate it!

--
Best regards,
Andrew Tropin
L
L
Leo Prikler wrote on 11 May 2021 18:33
(name . Andrew Tropin)(address . andrew@trop.in)
8164abe40ec6e865bf8385e010702cfe158dabcc.camel@student.tugraz.at
Am Dienstag, den 11.05.2021, 18:57 +0300 schrieb Andrew Tropin:
Toggle quote (2 lines)
> Patched emacs-build-system to place packages under elpa/NAME-VERSION
> subdirectory and removed "-pkg\\.el$" from %default-exclude.
I don't know whether that's a good idea. The elpa/ part I already
dislike, and the "-pkg\\.el$" exclude might have existed for a reason
(I don't know which, put perhaps byte compilation).

Toggle quote (14 lines)
> > I have now published emacs-dpd [2], which does exactly that. To
> > use it
> > for your Guix Emacs packages, execute
> > (dpd (list "$GUIX_PROFILE/share/emacs/site-lisp" ...))
> > replacing "$GUIX_PROFILE" with the actual profile, after `package-
> > initialize' has run with `dpd-fuzzy-recognize' in `dpd-recognize-
> > hook'.
> > I might write a more detailed README later.
>
> Most of the packages already have -pkg.el in sources, but yep, pretty
> cool utility, also thought about implementing something like that
> yesterday, but luckily I didn't and now I do not need to do it,
> because
> now I'm aware of already-existing implementations!)
I know people take package.el for granted nowadays, but alternative
package managers for Emacs have their uses. This is not just a Guix
thing :)

Toggle quote (27 lines)
> > Neither packed nor dpd are currently packaged in Guix. packed can
> > easily be imported from melpa-stable, but for dpd you'd have to
> > write
> > your own guix.scm. I might do that at some point as well.
> > We already had modifications in emacs-build-system recently, so if
> > you
> > want to argue, that all Emacs packages should have that <package>-
> > pkg.el to work with package.el out of the box, I would ask you to
> > wait,
> > so as to not cause an "Emacs world" rebuild again after only ten
> > days.
> > I also don't know whether Guix has the same information as
> > package.el
> > at build time, but that might change with time as
> > well. Particularly,
> > there will hopefully be a move towards supplying name and version
> > at
> > build, which would give us the most important information.
>
> Very cool, I didn't have the latest changes on my local checkout and
> didn't
> see your commits, but now I see, it is exactly what I needed.
>
> The only side note: it should be site-lisp/elpa/NAME-VERSION (right
> now
> it is site-lisp/NAME-VERSION). Also, on line 137 elpa-directory
> function can be used.
I don't think we want to fake elpa that hard. Two iterations ago it
was .guix.d and people didn't really like it. My subdirs.el patch is
also stretching it. So I really don't want to add another subdirectory
layer to it.
If elpa can't deal with that, we'll have to code around it in Elisp.

Toggle quote (2 lines)
> When you will be updating the path, please remove -pkg.el from
> %default-exclude.
I've CC'd Maxim, perhaps they know more about %default-exclude.

Regards,
Leo
A
A
Andrew Tropin wrote on 11 May 2021 20:55
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
CABrWRW1WfF0J=Pe1E-R2fyx7f6+qLPMs9nj5C2_ZS-G2SZ5bGQ@mail.gmail.com
Toggle quote (3 lines)
> the "-pkg\\.el$" exclude might have existed for a reason
> (I don't know which, put perhaps byte compilation).

Perhaps it should be ignored during byte compilation, but still
installing it seems to be a good idea. Ok, let's wait for Maxim answer.

Toggle quote (4 lines)
> I know people take package.el for granted nowadays, but alternative
> package managers for Emacs have their uses. This is not just a Guix
> thing :)

Why not take it for granted?) It's built-in since 24(?), elpa/melpa
archives respect it format and provide package descriptions in -pkg.el
format, AFAIK. Most other package managers seem to respect
"infrastructure" provided by package.el. For example you can view a list
of packages with `list-packages` even for packages installed with other
PMs (Nix for example), BTW they keep "package.el" directory structure.

Don't see too many reasons not to follow this format.

I mean it's easily fixable with current directory structure just by
stripping "/elpa" suffix from package-directory-list, but why we would
do that emacs "customization" instead of just placing packages under /elpa
subdirectory and make everything work out of the box?

Toggle quote (3 lines)
> I don't think we want to fake elpa that hard. Two iterations ago it
> was .guix.d and people didn't really like it.

Do you mean the package installation path was site-lisp/.guix.d/NAME-VERSION?

Toggle quote (2 lines)
> My subdirs.el patch is also stretching it.

Not sure what you mean by this, sorry, I'm not native speaker and
automated translation doesn't make sense to me. Rephrase please.

I do not insist on any particular directory structure, just curious why
not to stick to the widely adopted format. Once again, thank you for placing
packages into subdirectories, now the site-lisp structure seems more
organized and less polluted + problem with describe-package (C-h P) and
list-packages are easily fixable. Appreciate your work!)

--
Best regards,
Andrew Tropin
L
L
Leo Prikler wrote on 11 May 2021 21:57
(name . Andrew Tropin)(address . andrew@trop.in)
6cba9730ace19ecebc397023b243cdb02486cc0c.camel@student.tugraz.at
Am Dienstag, den 11.05.2021, 21:55 +0300 schrieb Andrew Tropin:
Toggle quote (6 lines)
> > the "-pkg\\.el$" exclude might have existed for a reason
> > (I don't know which, put perhaps byte compilation).
>
> Perhaps it should be ignored during byte compilation, but still
> installing it seems to be a good idea. Ok, let's wait for Maxim
> answer.
I think we agree on that.

Toggle quote (7 lines)
> > I know people take package.el for granted nowadays, but alternative
> > package managers for Emacs have their uses. This is not just a
> > Guix thing :)
>
> Why not take it for granted?) It's built-in since 24(?), elpa/melpa
> archives respect it format and provide package descriptions in
> -pkg.el format, AFAIK.
el-get[1] is still active.
straight.el[2] is another package manager for Emacs, its README
comparing 5+1 approaches for package mangers, including el-get and
package.el. Those are very much wild lands, I say.

Not to speak for all the distro-specific ways of handling things.
Gentoo had (and probably still has) an overlay, that magically creates
Gentoo packages from elpa – in which of course they drop the -pkg.el.
Debian has a mix of elpa packages and non-elpa ones, some of which
naturally don't have the -pkg.el either. (Also their packages use
site-lisp/elpa-src instead of site-lisp/elpa). Arch also seems to
install at least some Elisp packages "directly" in site-lisp/$PACKAGE.

Toggle quote (2 lines)
> Most other package managers seem to respect "infrastructure" provided
> by package.el.
I don't think that statement is well-supported by the data we have.

Toggle quote (6 lines)
> Don't see too many reasons not to follow this format.
>
> I mean it's easily fixable with current directory structure just by
> stripping "/elpa" suffix from package-directory-list, but why we
> would do that emacs "customization" instead of just placing packages
> under /elpa subdirectory and make everything work out of the box?
Why should we let ELPA dictate our layout? I have not even once tried
customizing package.el for actual use since I got Guix, because the
elpa importer is trivial.

Toggle quote (5 lines)
> > I don't think we want to fake elpa that hard. Two iterations ago
> > it was .guix.d and people didn't really like it.
>
> Do you mean the package installation path was site-lisp/.guix.d/NAME-
> VERSION?
Yep, that was a kinda ELPA-y convention while also remaining more true
to what we're doing.

Toggle quote (4 lines)
> > My subdirs.el patch is also stretching it.
>
> Not sure what you mean by this, sorry, I'm not native speaker and
> automated translation doesn't make sense to me. Rephrase please.
The patch, which I've made, that adds subdirs.el is not
uncontroversial.

Toggle quote (6 lines)
> I do not insist on any particular directory structure, just curious
> why not to stick to the widely adopted format. Once again, thank you
> for placing packages into subdirectories, now the site-lisp structure
> seems more organized and less polluted + problem with describe-
> package (C-h P) and list-packages are easily fixable. Appreciate
> your work!)
I hope I've shed some light as to how "wide" this adoption actually is
– Emacs users are a contentious people. Just because something is "the
default" and enjoys being shipped with Emacs itself doesn't mean that
everyone is happy with it.

Thus we're not trying to keep in line with any specific package
manager, we just need to make things work "with Emacs" in the sense
that packages installed via Guix should have working autoloads and one
should be able to (require ...) them.

Regards,
Leo

L
L
Ludovic Courtès wrote on 11 May 2021 23:17
Re: bug#48331: Emacs' describe-package doesn't work for packages managed by guix
(name . Andrew Tropin)(address . andrew@trop.in)(address . 48331@debbugs.gnu.org)
87mtt1ug0h.fsf@gnu.org
Hi,

Andrew Tropin <andrew@trop.in> skribis:

Toggle quote (2 lines)
> M-x list-packages ;; Doesn't list treemacs

Let me recommend Emacs-Guix (aka. ‘guix.el’) as a superior alternative.
:-)

I think it’s fine that ‘package.el’ is unaware of Guix-managed software
and vice-versa.

Ludo’.
A
A
Andrew Tropin wrote on 19 May 2021 16:32
Re: Emacs' describe-package doesn't work for packages managed by guix
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
CABrWRW1ZYToRkrwVA5t0J3oz9X-tjF3mEyD_GfusDneV1p_9EQ@mail.gmail.com
Toggle quote (4 lines)
> > Most other package managers seem to respect "infrastructure" provided
> > by package.el.
> I don't think that statement is well-supported by the data we have.

Agree, that was an incorrect statement. I should have said something like:
there are some popular tools like use-package configuration helper, Nix
package manager, Spacemacs configuration framework, some elisp archives
and probably something else, which utilize and follow package.el. Not
all of them use package.el itself, but they follow conventions and
describe-package help command and some other work correctly.

Toggle quote (4 lines)
> Why should we let ELPA dictate our layout? I have not even once tried
> customizing package.el for actual use since I got Guix, because the
> elpa importer is trivial.

We don't have to. Actually, I'm very happy with the new (current) layout we
have right now.

I would say I find the following use case very confusing for newcomers:
- Install emacs package via Guix.
- Use built-in help C-h C-h, find C-h P.
- Get it to work for built-in packages, but not for packages installed by Guix.
- Get frustrated.

I think we could avoid this at least in two ways:
1. Use elpa/ subdirectory.
2. Keep current structure, set package-directory-list to .../site-lisp
instead of .../site-lisp/elpa by default.

Toggle quote (5 lines)
> Thus we're not trying to keep in line with any specific package
> manager, we just need to make things work "with Emacs" in the sense
> that packages installed via Guix should have working autoloads and one
> should be able to (require ...) them.

Yes, but at the same time I don't see reasons why not to implement one
of two options above. We can get both: working autoloads and working
built-in help function (+newcommers won't be so frustrated).

Personally, I'm quite happy that packages got their own subdirectories
and I'm fully satisfied with current state of it, but it would be cool
if inexperienced users will be able to use at least built-in help
commands for packages out of the box without additional configuration.

Hope my original point is a little better worded now.

--
Best regards,
Andrew Tropin
A
A
Andrew Tropin wrote on 19 May 2021 16:41
Re: bug#48331: Emacs' describe-package doesn't work for packages managed by guix
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 48331@debbugs.gnu.org)
CABrWRW22yBqjQMKbGZbNUvjHh+bkRuduHNzXLvgCK0XErQuZnw@mail.gmail.com
Toggle quote (5 lines)
> > M-x list-packages ;; Doesn't list treemacs

> Let me recommend Emacs-Guix (aka. 'guix.el') as a superior alternative.
> :-)

Sure) Aware of it, cool tool.

Toggle quote (3 lines)
> I think it's fine that 'package.el' is unaware of Guix-managed software
> and vice-versa.

Yep, perfectly fine, but why not to make it aware, if it is relatively
easy to achieve?)
L
L
Leo Prikler wrote on 19 May 2021 17:08
Re: Emacs' describe-package doesn't work for packages managed by guix
(name . Andrew Tropin)(address . andrew@trop.in)
437d90a3cc91400fdf0828fea2c989c277a73a7c.camel@student.tugraz.at
Am Mittwoch, den 19.05.2021, 17:32 +0300 schrieb Andrew Tropin:
Toggle quote (11 lines)
> > > Most other package managers seem to respect "infrastructure"
> > > provided by package.el.
> > I don't think that statement is well-supported by the data we have.
>
> Agree, that was an incorrect statement. I should have said something
> like: there are some popular tools like use-package configuration
> helper, Nix package manager, Spacemacs configuration framework, some
> elisp archives and probably something else, which utilize and follow
> package.el. Not all of them use package.el itself, but they follow
> conventions and describe-package help command and some other work
> correctly.
Is package.el really so well supported by all these tools? I might
concede, that some of them don't throw away the package.el blurb like
guix does, but other than that, I think you'd have a hard time stuffing
a random git repo from use-package into package.el. Am I missing
something?

Toggle quote (6 lines)
> > Why should we let ELPA dictate our layout? I have not even once
> > tried customizing package.el for actual use since I got Guix,
> > because the elpa importer is trivial.
>
> We don't have to. Actually, I'm very happy with the new (current)
> layout we have right now.
That's good :)

Toggle quote (7 lines)
> I would say I find the following use case very confusing for
> newcomers:
> - Install emacs package via Guix.
> - Use built-in help C-h C-h, find C-h P.
> - Get it to work for built-in packages, but not for packages
> installed by Guix.
> - Get frustrated.
You mean Emacs newcomers? Tell me something new about the first-time
Emacs experience :P

Toggle quote (4 lines)
> I think we could avoid this at least in two ways:
> 1. Use elpa/ subdirectory.
> 2. Keep current structure, set package-directory-list to .../site-
> lisp instead of .../site-lisp/elpa by default.
Neither sounds very pleasant, but does (2) even work?

Toggle quote (8 lines)
> > Thus we're not trying to keep in line with any specific package
> > manager, we just need to make things work "with Emacs" in the sense
> > that packages installed via Guix should have working autoloads and
> > one should be able to (require ...) them.
>
> Yes, but at the same time I don't see reasons why not to implement
> one of two options above. We can get both: working autoloads and
> working built-in help function (+newcommers won't be so frustrated).
Of course. The glue code for that would go into guix-emacs.el, just
like our autoload glue.

Toggle quote (7 lines)
> Personally, I'm quite happy that packages got their own
> subdirectories and I'm fully satisfied with current state of it, but
> it would be cool if inexperienced users will be able to use at least
> built-in help commands for packages out of the box without additional
> configuration.
>
> Hope my original point is a little better worded now.
Doing something in Emacs without configuration sounds like an oxymoron,
but I get your point. At the same time, I think that this kind of
change is a pretty large request (DPD has more than 100 lines not
counting dependencies, it's not small and neat like guix-emacs.el).

If you find a clever trick to make your troubles go away, do submit a
patch, but don't let it rely on user setup (in particular, don't rely
on "haha, the user always carries about the elpa subdirectory").

Regards,
Leo
A
A
Andrew Tropin wrote on 19 May 2021 19:58
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
CABrWRW2OpFaJU-Wz19-smz3OkMx79+rX2WT-N_ZuHkD_Mu6ghQ@mail.gmail.com
From: Andrew Tropin <andrew@trop.in>
Date: Wed, 19 May 2021 20:44:22 +0300
Subject: [PATCH] guix: build: emacs-build-system: Make package.el aware of
guix packages

After updating the package-directory-list variable, functions like
list-packages,
describe-package become aware of packages installed by guix.

---
This code is getting work done, but I'm not a very experienced elisp
developer, so
please review it thoroughly.

gnu/packages/aux-files/emacs/guix-emacs.el | 10 ++++++++++
guix/build/emacs-build-system.scm | 2 +-
2 files changed, 11 insertions(+), 1 deletion(-)

Toggle diff (38 lines)
diff --git a/gnu/packages/aux-files/emacs/guix-emacs.el
b/gnu/packages/aux-files/emacs/guix-emacs.el
index ca9146c535..4aa4220cda 100644
--- a/gnu/packages/aux-files/emacs/guix-emacs.el
+++ b/gnu/packages/aux-files/emacs/guix-emacs.el
@@ -58,6 +58,16 @@ The files in the list do not have extensions (.el, .elc)."
(load f 'noerror))
autoloads)))

+
+(require 'package)
+
+;; Set `package-directory-list' to the value without elpa/ suffix
+;; to match the structure of site-lisp directory of guix's emacs
+;; build system.
+;;;###autoload
+(setq package-directory-list
+ (list (string-remove-suffix "/elpa" (car package-directory-list))))
+
(provide 'guix-emacs)

;;; guix-emacs.el ends here
diff --git a/guix/build/emacs-build-system.scm
b/guix/build/emacs-build-system.scm
index e41e9a6595..7565b9a422 100644
--- a/guix/build/emacs-build-system.scm
+++ b/guix/build/emacs-build-system.scm
@@ -53,7 +53,7 @@

;; These are the default inclusion/exclusion regexps for the install phase.
(define %default-include '("^[^/]*\\.el$" "^[^/]*\\.info$" "^doc/.*\\.info$"))
-(define %default-exclude '("^\\.dir-locals\\.el$" "-pkg\\.el$"
+(define %default-exclude '("^\\.dir-locals\\.el$"
"^[^/]*tests?\\.el$"))

(define gnu:unpack (assoc-ref gnu:%standard-phases 'unpack))
--
2.31.1
L
L
Leo Prikler wrote on 19 May 2021 20:42
(name . Andrew Tropin)(address . andrew@trop.in)
88a438a74fabe0f4d4bfd8a649c38e960ef82ed2.camel@student.tugraz.at
That looks like it'd mess with people's installed ELPA packages. In
general, hacks based on package-directory-list don't feel very stable.
Consider writing a function similar in nature to `package-load-all-
descriptors' instead.

Also, this seems to rely on us not deleting the -pkg.el, but probably
won't work for packages, that don't ship it, e.g. emacs-howm.

[Adding Arun Isaac to CC. Their commit d8796851 is the first one to
drop -pkg.el, but without explanation.]

Am Mittwoch, den 19.05.2021, 20:58 +0300 schrieb Andrew Tropin:
Toggle quote (59 lines)
> From: Andrew Tropin <andrew@trop.in>
> Date: Wed, 19 May 2021 20:44:22 +0300
> Subject: [PATCH] guix: build: emacs-build-system: Make package.el
> aware of
> guix packages
>
> After updating the package-directory-list variable, functions like
> list-packages,
> describe-package become aware of packages installed by guix.
>
> ---
> This code is getting work done, but I'm not a very experienced elisp
> developer, so
> please review it thoroughly.
>
> gnu/packages/aux-files/emacs/guix-emacs.el | 10 ++++++++++
> guix/build/emacs-build-system.scm | 2 +-
> 2 files changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/aux-files/emacs/guix-emacs.el
> b/gnu/packages/aux-files/emacs/guix-emacs.el
> index ca9146c535..4aa4220cda 100644
> --- a/gnu/packages/aux-files/emacs/guix-emacs.el
> +++ b/gnu/packages/aux-files/emacs/guix-emacs.el
> @@ -58,6 +58,16 @@ The files in the list do not have extensions (.el,
> .elc)."
> (load f 'noerror))
> autoloads)))
>
> +
> +(require 'package)
> +
> +;; Set `package-directory-list' to the value without elpa/ suffix
> +;; to match the structure of site-lisp directory of guix's emacs
> +;; build system.
> +;;;###autoload
> +(setq package-directory-list
> + (list (string-remove-suffix "/elpa" (car package-directory-
> list))))
> +
> (provide 'guix-emacs)
>
> ;;; guix-emacs.el ends here
> diff --git a/guix/build/emacs-build-system.scm
> b/guix/build/emacs-build-system.scm
> index e41e9a6595..7565b9a422 100644
> --- a/guix/build/emacs-build-system.scm
> +++ b/guix/build/emacs-build-system.scm
> @@ -53,7 +53,7 @@
>
> ;; These are the default inclusion/exclusion regexps for the install
> phase.
> (define %default-include '("^[^/]*\\.el$" "^[^/]*\\.info$"
> "^doc/.*\\.info$"))
> -(define %default-exclude '("^\\.dir-locals\\.el$" "-pkg\\.el$"
> +(define %default-exclude '("^\\.dir-locals\\.el$"
> "^[^/]*tests?\\.el$"))
>
> (define gnu:unpack (assoc-ref gnu:%standard-phases 'unpack))
A
A
Andrew Tropin wrote on 20 May 2021 12:01
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
CABrWRW1DiStRz7_OTwixUH9sqcOvjJcx8Big1hmepmQWTMgdSg@mail.gmail.com
Toggle quote (3 lines)
> That looks like it'd mess with people's installed ELPA packages. In
> general, hacks based on package-directory-list don't feel very stable.

If you talk about ~/.emacs.d/elpa, it won't, because there is a separate
package-user-dir variable for that.

The problem can arise if we have emacs installed in a few profiles, I'm
not sure if it is a good idea to do so, but it is possible, in such a case
we will have a few items in the package-directory-list. A fix for that:

(setq package-directory-list
(mapcar (apply-partially 'string-remove-suffix "/elpa")
package-directory-list)))

Toggle quote (3 lines)
> Also, this seems to rely on us not deleting the -pkg.el, but probably
> won't work for packages, that don't ship it, e.g. emacs-howm.

It's true, but it seems relatively easy to implement a build phase,
which will generate -pgk.el in case it is missing.
L
L
Leo Prikler wrote on 20 May 2021 12:20
(name . Andrew Tropin)(address . andrew@trop.in)
43c4f2959e4805aae7414d0acc08f1fd7b35d986.camel@student.tugraz.at
Am Donnerstag, den 20.05.2021, 13:01 +0300 schrieb Andrew Tropin:
Toggle quote (15 lines)
> > That looks like it'd mess with people's installed ELPA
> > packages. In general, hacks based on package-directory-list don't
> > feel very stable.
>
> If you talk about ~/.emacs.d/elpa, it won't, because there is a
> separate package-user-dir variable for that.
>
> The problem can arise if we have emacs installed in a few profiles,
> I'm not sure if it is a good idea to do so, but it is possible, in
> such a case we will have a few items in the package-directory-
> list. A fix for that:
>
> (setq package-directory-list
> (mapcar (apply-partially 'string-remove-suffix "/elpa")
> package-directory-list)))
Multi-profile Emacs should be supported, but this also breaks on
foreign distros with foreign distro ELPA. Again, hacking variables is
not the solution (and even if it was, it'd be better to patch the emacs
default value, not that this is a good idea either).

Toggle quote (6 lines)
> > Also, this seems to rely on us not deleting the -pkg.el, but
> > probably won't work for packages, that don't ship it, e.g. emacs-
> > howm.
>
> It's true, but it seems relatively easy to implement a build phase,
> which will generate -pgk.el in case it is missing.
Generating our own -pkg.el should work, still waiting for Maxim or Arun
on a statement as to why we exclude it.

*Always* generating a -pkg.el (disregarding the upstream one if it
exists) might further be a reasonable thing to do if we decide, that
those -pkg.el files are useful.

Regards,
Leo
A
A
Arun Isaac wrote on 20 May 2021 12:32
87tumxzoey.fsf@systemreboot.net
Toggle quote (3 lines)
> [Adding Arun Isaac to CC. Their commit d8796851 is the first one to
> drop -pkg.el, but without explanation.]

Commit d8796851 was an attempt to not install too many unnecessary files
and be closer to how MELPA packaged emacs packages. See
the detailed discussion.

Cheers!
-----BEGIN PGP SIGNATURE-----

iQFPBAEBCAA5FiEEf3MDQ/Lwnzx3v3nTLiXui2GAK7MFAmCmOqUbHGFydW5pc2Fh
Y0BzeXN0ZW1yZWJvb3QubmV0AAoJEC4l7othgCuzA1wH/0C6jGDkwLFVUib+44r/
bRlzRgC+SGjJmvHZPgvPiPBTSvTvvUae2zeIhju5yQYafTZg96/W8uqyeh/LbmVJ
9L1wBgxze8otsQVHcCsisPk/YcN1fJtsr59rWo6oEmjcoJyo7NwS6lntPyR2KCcN
oNcARvrQDSY+2IuTgzlQq/zzgip4S2Ph3xR3qDJJuu+4KEszX8CMNZpNdHdvqZrh
GeyuZJo3LvkCst6sUjRC5tdn2oK6ETVdonz8Dr4FItMnwEWAhkFe94bV39bx5wBm
DMbQYCEuTJi5A1VbXcBFgrwnF8Zx7vIF1wTttfcd56upa90FeSPDvcqnxjnjCE+X
PaE=
=1WMU
-----END PGP SIGNATURE-----

A
A
Arun Isaac wrote on 20 May 2021 12:39
Re: bug#48331: Emacs' describe-package doesn't work for packages managed by guix
87r1i1zo2w.fsf@systemreboot.net
Toggle quote (8 lines)
>> [Adding Arun Isaac to CC. Their commit d8796851 is the first one to
>> drop -pkg.el, but without explanation.]
>
> Commit d8796851 was an attempt to not install too many unnecessary files
> and be closer to how MELPA packaged emacs packages. See
> https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00274.html for
> the detailed discussion.

In other words, no particular thought was given to -pkg.el. It was
simply dropped along with many other files. So, if consensus is reached
that keeping -pkg.el is a good idea, there is no reason to not do that.

Cheers!
-----BEGIN PGP SIGNATURE-----

iQFPBAEBCAA5FiEEf3MDQ/Lwnzx3v3nTLiXui2GAK7MFAmCmPFcbHGFydW5pc2Fh
Y0BzeXN0ZW1yZWJvb3QubmV0AAoJEC4l7othgCuzZBsIAL332OS6wbQQWXOXnku8
ejyDzp8enbrx0aV3DPjuicsY5EEsPiL2R4+iVTe44RRYT+qlRg7m3iBBWyCTWlh7
VZH0+shMpAr+2oiaZYhcZSv3PlFJMuGNx/6Ny8hOqYhZgMwsFjxI/CuBuqhbgspZ
1TqtP95gV7QquLJnZ3fAlyPznaC53ibeG2R7pPL2LwklNaQ60AUYqffZd8Gatv6K
VN9/cCD3Bfll+B1qCCh+BKQFTzLlN02pnIIYuKIEYNOTXCOtCC8hWu01Tx08aReL
ZwAiHFgHmx68uFu4XblUQhk7KNYdVSCBLhXE4hJ2A6MJcznkWzAAH3gfaDtaazDp
GYI=
=B1+W
-----END PGP SIGNATURE-----

L
L
Leo Prikler wrote on 20 May 2021 13:13
681fe5a3db7bcff56fa20e2124c2cb5276fc5eb0.camel@student.tugraz.at
Am Donnerstag, den 20.05.2021, 16:09 +0530 schrieb Arun Isaac:
Toggle quote (15 lines)
> > > [Adding Arun Isaac to CC. Their commit d8796851 is the first one
> > > to
> > > drop -pkg.el, but without explanation.]
> >
> > Commit d8796851 was an attempt to not install too many unnecessary
> > files
> > and be closer to how MELPA packaged emacs packages. See
> > https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00274.html
> > for
> > the detailed discussion.
>
> In other words, no particular thought was given to -pkg.el. It was
> simply dropped along with many other files. So, if consensus is
> reachedthat keeping -pkg.el is a good idea, there is no reason to not
> do that.
Thanks for clearing that up. In that case, I don't have any qualms
about including them either.

Cheers!
A
A
Andrew Tropin wrote on 20 May 2021 14:24
Re: Emacs' describe-package doesn't work for packages managed by guix
(name . Arun Isaac)(address . arunisaac@systemreboot.net)
CABrWRW30RRCGOo=tDGf0nObFjLPW40amQQNZBioMVBohar2iaA@mail.gmail.com
Toggle quote (7 lines)
> > In other words, no particular thought was given to -pkg.el. It was
> > simply dropped along with many other files. So, if consensus is
> > reachedthat keeping -pkg.el is a good idea, there is no reason to not
> > do that.
> Thanks for clearing that up. In that case, I don't have any qualms
> about including them either.

Cool, seems we can get -pkg.el files back.

Toggle quote (5 lines)
> Multi-profile Emacs should be supported, but this also breaks on
> foreign distros with foreign distro ELPA. Again, hacking variables is
> not the solution (and even if it was, it'd be better to patch the emacs
> default value, not that this is a good idea either).

Yep, the last snippet supports multi-profile Emacs. Installing packages
for Emacs via a few different package manager sounds like a very bad
practice) I agree that current implementation with updating variables
isn't perfect and it doesn't cover the use case, which I expect to be
rare (packages from Guix will be listed in list-packages, files from
foreign distro PM won't). I can dive into package.el implementation and
spend some time next week providing a different workaround.

BTW, can you remind me why we do not place packages under
site-lisp/elpa/NAME-VERSION? It seems almost the same as
site-lisp/NAME-VERSION, but everything related to describe-package and
other functions will work out of the box. This way it will work even
with a foreign distro use case.
L
L
Leo Prikler wrote on 20 May 2021 17:57
5bb7553ac84e241a8ce936033b9ec1e1d6d302dd.camel@student.tugraz.at
Am Donnerstag, den 20.05.2021, 15:24 +0300 schrieb Andrew Tropin:
Toggle quote (10 lines)
> > > In other words, no particular thought was given to -pkg.el. It
> > > was
> > > simply dropped along with many other files. So, if consensus is
> > > reachedthat keeping -pkg.el is a good idea, there is no reason to
> > > not
> > > do that.
> > Thanks for clearing that up. In that case, I don't have any qualms
> > about including them either.
>
> Cool, seems we can get -pkg.el files back.
Yes, we can.

Toggle quote (6 lines)
> > Multi-profile Emacs should be supported, but this also breaks on
> > foreign distros with foreign distro ELPA. Again, hacking variables
> > is not the solution (and even if it was, it'd be better to patch
> > the emacs default value, not that this is a good idea either).
>
> Yep, the last snippet supports multi-profile Emacs.
While that's better, I still don't think it's sufficient.

Toggle quote (7 lines)
> Installing packages for Emacs via a few different package manager
> sounds like a very bad practice) I agree that current implementation
> with updating variables isn't perfect and it doesn't cover the use
> case, which I expect to be rare (packages from Guix will be listed in
> list-packages, files from foreign distro PM won't). I can dive into
> package.el implementation and spend some time next week providing a
> different workaround.
I think this is common practice on other distros. For example your
system provides auctex, but it doesn't provide dash.el. What do you
do? Install it through ELPA.

Now let's say, you have Guix installed. Guix provides packages for
some of ELPA, but not what you want. You could hack together a Guix
package based on the ELPA importer, but perhaps upstream is slow in
accepting it or perhaps you've fetched the package from a shady source,
that Guix won't accept. So you have foreign distro system packages +
Guix + personal ELPA.

As far as getting Guix packages to work without affecting the rest of
package.el or user configuration, I think an advice, that runs after
package-load-all-descriptors might be necessary. As for the candidates
to check for this advice, you can read all subdirs.el files you find
inside load-path. When they're formatted
(normal-top-level-add-to-load-path (list P1 ... PN))
they were likely added by Guix (even if not, one should probably
consider them important) and P1 ... PN should be scanned for
descriptors.

Toggle quote (5 lines)
> BTW, can you remind me why we do not place packages under
> site-lisp/elpa/NAME-VERSION? It seems almost the same as
> site-lisp/NAME-VERSION, but everything related to describe-package
> and other functions will work out of the box. This way it will work
> even with a foreign distro use case.
Again, Guix is not ELPA and calling it ELPA would be misleading. As
for why we don't put stuff in any other site-lisp/ directory, e.g.
site-lisp/guix.d/NAME-VERSION, doing that led to rather tricky issues,
which is why we've decided to use site-lisp "directly". The current
way of handling things is a bit of a compromise. It gives you per-
package directories like ELPA, but unlike ELPA can easily be handled at
Emacs startup.
M
M
Maxim Cournoyer wrote on 22 May 2021 05:09
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
874kevxy5n.fsf@gmail.com
Hello,

Leo Prikler <leo.prikler@student.tugraz.at> writes:

Toggle quote (13 lines)
> Am Donnerstag, den 20.05.2021, 15:24 +0300 schrieb Andrew Tropin:
>> > > In other words, no particular thought was given to -pkg.el. It
>> > > was
>> > > simply dropped along with many other files. So, if consensus is
>> > > reachedthat keeping -pkg.el is a good idea, there is no reason to
>> > > not
>> > > do that.
>> > Thanks for clearing that up. In that case, I don't have any qualms
>> > about including them either.
>>
>> Cool, seems we can get -pkg.el files back.
> Yes, we can.

I'm late, but I think it's OK to have those *-pkg.el files installed, if
they are useful.

[...]

Toggle quote (13 lines)
>> BTW, can you remind me why we do not place packages under
>> site-lisp/elpa/NAME-VERSION? It seems almost the same as
>> site-lisp/NAME-VERSION, but everything related to describe-package
>> and other functions will work out of the box. This way it will work
>> even with a foreign distro use case.
> Again, Guix is not ELPA and calling it ELPA would be misleading. As
> for why we don't put stuff in any other site-lisp/ directory, e.g.
> site-lisp/guix.d/NAME-VERSION, doing that led to rather tricky issues,
> which is why we've decided to use site-lisp "directly". The current
> way of handling things is a bit of a compromise. It gives you per-
> package directories like ELPA, but unlike ELPA can easily be handled at
> Emacs startup.

If you are interested in an alternate view of the world, with the
benefits and drawbacks of integrating with package.el to provide
packages autoloading in Guix, you may be interested in studying the
packages are loaded by the package.el library via (package-initialize).
The main drawback (that was deemed inconvenient enough to not go ahead
with this scheme) is summarized in the introductory message:

Parting with a directly usable EMACSLOADPATH means that site-start.el
*must* run for packages to appear in the load-path; that means for
running a test suite, the -Q or --quick Emacs options cannot be used,
since it implies --no-site-file.

HTH,

Maxim
L
L
Leo Prikler wrote on 22 May 2021 14:54
[PATCH 2/2] gnu: emacs: Load package descriptors from packages referenced by subdirs.el
(address . 48331@debbugs.gnu.org)
20210522125428.12859-2-leo.prikler@student.tugraz.at
* gnu/packages/aux-files/emacs/guix-emacs.el
(guix-emacs--non-core-load-path): New procedure.
(guix-emacs-autoload-packages): Use it here.
(guix-emacs-load-package-descriptors): New procedure.
* gnu/packages/emacs.scm (emacs)[install-site-start]: Install advice to run
‘guix-emacs-load-package-descriptors’.
---
gnu/packages/aux-files/emacs/guix-emacs.el | 34 +++++++++++++++++-----
gnu/packages/emacs.scm | 7 +++--
2 files changed, 31 insertions(+), 10 deletions(-)

Toggle diff (83 lines)
diff --git a/gnu/packages/aux-files/emacs/guix-emacs.el b/gnu/packages/aux-files/emacs/guix-emacs.el
index ca9146c535..eff44bfe90 100644
--- a/gnu/packages/aux-files/emacs/guix-emacs.el
+++ b/gnu/packages/aux-files/emacs/guix-emacs.el
@@ -26,6 +26,7 @@
;;; Code:
(require 'seq)
+(declare-function package-load-descriptor "package" (pkg-dir))
(defvar guix-emacs-autoloads-regexp
(rx (* any) "-autoloads.el" (zero-or-one "c") string-end)
@@ -39,6 +40,12 @@ The files in the list do not have extensions (.el, .elc)."
(directory-files directory 'full-name
guix-emacs-autoloads-regexp))))
+(defun guix-emacs--non-core-load-path ()
+ ;; Filter out core Elisp directories, which are already handled by Emacs.
+ (seq-filter (lambda (dir)
+ (string-match-p "/share/emacs/site-lisp" dir))
+ load-path))
+
;;;###autoload
(defun guix-emacs-autoload-packages ()
"Autoload Emacs packages found in EMACSLOADPATH.
@@ -46,18 +53,29 @@ The files in the list do not have extensions (.el, .elc)."
'Autoload' means to load the 'autoloads' files matching
`guix-emacs-autoloads-regexp'."
(interactive)
- (let* ((emacs-non-core-load-path-directories
- ;; Filter out core Elisp directories, which are already autoloaded
- ;; by Emacs.
- (seq-filter (lambda (dir)
- (string-match-p "/share/emacs/site-lisp" dir))
- load-path))
- (autoloads (mapcan #'guix-emacs-find-autoloads
- emacs-non-core-load-path-directories)))
+ (let ((autoloads (mapcan #'guix-emacs-find-autoloads
+ (guix-emacs--non-core-load-path))))
(mapc (lambda (f)
(load f 'noerror))
autoloads)))
+;;;###autoload
+(defun guix-emacs-load-package-descriptors ()
+ "Load descriptors for packages found in EMACSLOADPATH via subdirs.el."
+ (dolist (dir (guix-emacs--non-core-load-path))
+ (let ((subdirs-file (expand-file-name "subdirs.el" dir)))
+ (when (file-exists-p subdirs-file)
+ (with-temp-buffer
+ (insert-file-contents subdirs-file)
+ (goto-char (point-min))
+ (let ((subdirs (read (current-buffer))))
+ (and (equal (car-safe subdirs) 'normal-top-level-add-to-load-path)
+ (equal (car-safe (cadr subdirs)) 'list)
+ (dolist (subdir (cdadr subdirs))
+ (let ((pkg-dir (expand-file-name subdir dir)))
+ (when (file-directory-p pkg-dir)
+ (package-load-descriptor pkg-dir)))))))))))
+
(provide 'guix-emacs)
;;; guix-emacs.el ends here
diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 5316d25151..e4af6ea6a9 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -166,8 +166,11 @@
(with-output-to-file (string-append lisp-dir "/site-start.el")
(lambda ()
(display
- (string-append "(when (require 'guix-emacs nil t)\n"
- " (guix-emacs-autoload-packages))\n"))))
+ (string-append
+ "(when (require 'guix-emacs nil t)\n"
+ " (guix-emacs-autoload-packages)\n"
+ " (advice-add 'package-load-all-descriptors"
+ " :after #'guix-emacs-load-package-descriptors))"))))
;; Remove the extraneous subdirs.el file, as it causes Emacs to
;; add recursively all the the sub-directories of a profile's
;; share/emacs/site-lisp union when added to EMACSLOADPATH,
--
2.31.1
L
L
Leo Prikler wrote on 22 May 2021 14:54
[PATCH 1/2] build-system: emacs: Keep -pkg.el files.
(address . 48331@debbugs.gnu.org)
20210522125428.12859-1-leo.prikler@student.tugraz.at
Partly fixes https://bugs.gnu.org/48331 -- package descriptions can now be
found in the install directory.

* guix/build/emacs-build-system.scm (%default-exclude): Remove "-pkg\\.el$".
---
guix/build/emacs-build-system.scm | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)

Toggle diff (16 lines)
diff --git a/guix/build/emacs-build-system.scm b/guix/build/emacs-build-system.scm
index e41e9a6595..f13162d6c4 100644
--- a/guix/build/emacs-build-system.scm
+++ b/guix/build/emacs-build-system.scm
@@ -53,8 +53,7 @@
;; These are the default inclusion/exclusion regexps for the install phase.
(define %default-include '("^[^/]*\\.el$" "^[^/]*\\.info$" "^doc/.*\\.info$"))
-(define %default-exclude '("^\\.dir-locals\\.el$" "-pkg\\.el$"
- "^[^/]*tests?\\.el$"))
+(define %default-exclude '("^\\.dir-locals\\.el$" "^[^/]*tests?\\.el$"))
(define gnu:unpack (assoc-ref gnu:%standard-phases 'unpack))
--
2.31.1
A
A
Andrew Tropin wrote on 25 May 2021 15:40
[PATCH draft] build-system: emacs: Generate -pkg.el file in case it is missing
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
CABrWRW0gCFYq5=HnuJj4U98DKpMfV_74zDaFGiQaru3ZzyTSyA@mail.gmail.com
---
Thank you for the patches, tested, works for me! The solution is much more
precise than mutating package-directory-list variable, good job. I don't see
any major problems in the implementation (but I'm not very fluent elisp dev
and maybe missing something).


I drafted a simple build phase, which generates -pkg.el in case it is missing.

There are at least a few problems with this implementation:

1. There is no information about package record available during build, which
makes it hard to get package name and package version. I can't use any
regexp to obtain this information from name or elpa-name-ver, because
package name and version can have arbitrary form: comment-dwim-2-1.0,
cyberpunk-2019-theme-20191008-alpha or something like that.
2. It's also not so easy to extract description of the package from
somewhere, the first option is to pass package record to build phases somehow,
another is to parse PACKAGE-NAME.el file comments section.
3. This one I consider as a minor flaw: there is no generic solution for
packages built with build systems other than emacs-build-system.

So, this patch is very dirty and I publish it only for future reference.

The intuition says that we should split name and version in build phase
arguments, also it seems that it will be useful to provide other information
about package during build time for cases like this one. I'll learn this
area a bit more and probably will make another thread someday.

guix/build/emacs-build-system.scm | 60 ++++++++++++++++++++++++++++++-
1 file changed, 59 insertions(+), 1 deletion(-)

Toggle diff (83 lines)
diff --git a/guix/build/emacs-build-system.scm
b/guix/build/emacs-build-system.scm
index f13162d6c4..2bb102b4be 100644
--- a/guix/build/emacs-build-system.scm
+++ b/guix/build/emacs-build-system.scm
@@ -116,6 +116,63 @@ environment variable\n" source-directory))
(parameterize ((%emacs emacs))
(emacs-byte-compile-directory (elpa-directory out)))))

+
+(define* (add-pkg-file-if-missing #:key name outputs #:allow-other-keys
+ #:rest args)
+ "Generate simple -pkg.el in case package doesn't have it in source code."
+ (define (file-contains-nul-char? file)
+ (call-with-input-file file
+ (lambda (in)
+ (let loop ((line (read-line in 'concat)))
+ (cond
+ ((eof-object? line) #f)
+ ((string-index line #\nul) #t)
+ (else (loop (read-line in 'concat))))))
+ #:binary #t))
+
+ (let* ((out (assoc-ref outputs "out"))
+ (el-dir (elpa-directory out))
+ (elpa-name-ver (store-directory->elpa-name-version out))
+ (el-files (remove file-contains-nul-char?
+ (find-files (getcwd) "\\.el$")))
+ (el-names (map (lambda (x) (basename x ".el")) el-files))
+
+ (possible-names
+ (fold (lambda (x acc)
+ (cons
+ (string-append
+ (if (not (null? acc)) (string-append (first acc) "-") "")
+ x)
+ acc))
+ '()
+ (string-split elpa-name-ver #\-)))
+
+ (package-names (append-map
+ (lambda (name)
+ (let ((m (member name el-names)))
+ (if m (list (car m)) '())))
+ possible-names))
+
+ (package-name (if (null? package-names) "" (car package-names)))
+ (package-version (string-drop elpa-name-ver
+ (1+ (string-length package-name))))
+ (package-description "description should be here")
+ (pkg-file (string-append el-dir "/" package-name "-pkg.el")))
+
+ (when (not (file-exists? pkg-file))
+ (with-output-to-file pkg-file
+ (lambda ()
+ (format
+ #t
+ "\
+(define-package
+ ~s
+ ~s
+ ~s
+ nil)"
+ package-name package-version package-description))))
+ #t))
+
(define* (patch-el-files #:key outputs #:allow-other-keys)
"Substitute the absolute \"/bin/\" directory with the right location in the
store in '.el' files."
@@ -293,8 +350,9 @@ for libraries following the ELPA convention."
(add-after 'make-autoloads 'enable-autoloads-compilation
enable-autoloads-compilation)
(add-after 'enable-autoloads-compilation 'patch-el-files patch-el-files)
+ (add-after 'patch-el-files 'add-pkg-file-if-missing
add-pkg-file-if-missing)
;; The .el files are byte compiled directly in the store.
- (add-after 'patch-el-files 'build build)
+ (add-after 'add-pkg-file-if-missing 'build build)
(add-after 'build 'validate-compiled-autoloads validate-compiled-autoloads)
(add-after 'validate-compiled-autoloads 'move-doc move-doc)))

--
2.31.1
L
L
Leo Prikler wrote on 25 May 2021 17:07
(name . Andrew Tropin)(address . andrew@trop.in)
f743d399de309f8ad005cbf78a3cd70d16b4e0ea.camel@student.tugraz.at
Am Dienstag, den 25.05.2021, 16:40 +0300 schrieb Andrew Tropin:
Toggle quote (5 lines)
> ---
> Thank you for the patches, tested, works for me! The solution is
> much more precise than mutating package-directory-list variable, good
> job. I don't see any major problems in the implementation (but I'm
> not very fluent elisp dev and maybe missing something).
Don't worry, I can wait for a proper review by some of our experts,
since we shouldn't mess with emacs-build-system all too often anyway :P

Toggle quote (16 lines)
> I drafted a simple build phase, which generates -pkg.el in case it is
> missing.
>
> There are at least a few problems with this implementation:
>
> 1. There is no information about package record available during
> build, which makes it hard to get package name and package
> version. I can't use any regexp to obtain this information from name
> or elpa-name-ver, because package name and version can have
> arbitrary form: comment-dwim-2-1.0, cyberpunk-2019-theme-20191008-
> alpha or something like that.
> 2. It's also not so easy to extract description of the package from
> somewhere, the first option is to pass package record to build phases
> somehow, another is to parse PACKAGE-NAME.el file comments section.
> 3. This one I consider as a minor flaw: there is no generic solution
> for packages built with build systems other than emacs-build-system.
3. is easy -- just have those packages call the phase through emacs-
build-system. There already exist packages which do that, and it
should be pretty straightforward to do.

For 1. and 2. I think you're thinking a little too complicated. We can
call Emacs at build time. In particular, we can use all of the lisp-
mnt stuff, that I used in emacs-dpd at build time, we just need to
write the (define-package ...) form to disk.
In order to pick the right file, we can either use package-name (see
discussion below) or the name of the pkg-file without -pkg. Come to
think about it, we might also provide #:pkg-file-name as a keyword and
only generate the package file if it's set to a value other than #f,
i.e. let the packager choose whether they want to generate a -pkg.el
file. Something along this line is done with the build.xml of ant-
build-system.

Toggle quote (8 lines)
> So, this patch is very dirty and I publish it only for future
> reference.
>
> The intuition says that we should split name and version in build
> phase arguments, also it seems that it will be useful to provide
> other information about package during build time for cases like this
> one. I'll learn this area a bit more and probably will make another
> thread someday.
IIRC, there's a demand for having package-name and package-version
available during build (which might get through next core-updates or
might not -- we'll see), but I don't think we can argue for
description, especially when the Guix description might not be the
thing package.el wants here.

Toggle quote (94 lines)
> guix/build/emacs-build-system.scm | 60
> ++++++++++++++++++++++++++++++-
> 1 file changed, 59 insertions(+), 1 deletion(-)
>
> diff --git a/guix/build/emacs-build-system.scm
> b/guix/build/emacs-build-system.scm
> index f13162d6c4..2bb102b4be 100644
> --- a/guix/build/emacs-build-system.scm
> +++ b/guix/build/emacs-build-system.scm
> @@ -116,6 +116,63 @@ environment variable\n" source-directory))
> (parameterize ((%emacs emacs))
> (emacs-byte-compile-directory (elpa-directory out)))))
>
> +
> +(define* (add-pkg-file-if-missing #:key name outputs #:allow-other-
> keys
> + #:rest args)
> + "Generate simple -pkg.el in case package doesn't have it in source
> code."
> + (define (file-contains-nul-char? file)
> + (call-with-input-file file
> + (lambda (in)
> + (let loop ((line (read-line in 'concat)))
> + (cond
> + ((eof-object? line) #f)
> + ((string-index line #\nul) #t)
> + (else (loop (read-line in 'concat))))))
> + #:binary #t))
> +
> + (let* ((out (assoc-ref outputs "out"))
> + (el-dir (elpa-directory out))
> + (elpa-name-ver (store-directory->elpa-name-version out))
> + (el-files (remove file-contains-nul-char?
> + (find-files (getcwd) "\\.el$")))
> + (el-names (map (lambda (x) (basename x ".el")) el-files))
> +
> + (possible-names
> + (fold (lambda (x acc)
> + (cons
> + (string-append
> + (if (not (null? acc)) (string-append (first acc)
> "-") "")
> + x)
> + acc))
> + '()
> + (string-split elpa-name-ver #\-)))
> +
> + (package-names (append-map
> + (lambda (name)
> + (let ((m (member name el-names)))
> + (if m (list (car m)) '())))
> + possible-names))
> +
> + (package-name (if (null? package-names) "" (car package-
> names)))
> + (package-version (string-drop elpa-name-ver
> + (1+ (string-length package-
> name))))
> + (package-description "description should be here")
> + (pkg-file (string-append el-dir "/" package-name "-
> pkg.el")))
> +
> + (when (not (file-exists? pkg-file))
> + (with-output-to-file pkg-file
> + (lambda ()
> + (format
> + #t
> + "\
> +(define-package
> + ~s
> + ~s
> + ~s
> + nil)"
> + package-name package-version package-description))))
> + #t))
> +
> (define* (patch-el-files #:key outputs #:allow-other-keys)
> "Substitute the absolute \"/bin/\" directory with the right
> location in the
> store in '.el' files."
> @@ -293,8 +350,9 @@ for libraries following the ELPA convention."
> (add-after 'make-autoloads 'enable-autoloads-compilation
> enable-autoloads-compilation)
> (add-after 'enable-autoloads-compilation 'patch-el-files patch-
> el-files)
> + (add-after 'patch-el-files 'add-pkg-file-if-missing
> add-pkg-file-if-missing)
> ;; The .el files are byte compiled directly in the store.
> - (add-after 'patch-el-files 'build build)
> + (add-after 'add-pkg-file-if-missing 'build build)
> (add-after 'build 'validate-compiled-autoloads validate-
> compiled-autoloads)
> (add-after 'validate-compiled-autoloads 'move-doc move-doc)))
>
I
I
Ivan Sokolov wrote on 26 May 2021 10:15
RE: [PATCH] guix: build: emacs-build-system: Make package.el aware of guix packages
(address . 48331@debbugs.gnu.org)
871r9t6hdk.fsf@ya.ru
Toggle quote (20 lines)
> After updating the package-directory-list variable, functions like
> list-packages, describe-package become aware of packages installed by
> guix.

> ---
> This code is getting work done, but I'm not a very experienced elisp
> developer, so please review it thoroughly.

> +
> +(require 'package)
> +
> +;; Set `package-directory-list' to the value without elpa/ suffix
> +;; to match the structure of site-lisp directory of guix's emacs
> +;; build system.
> +;;;###autoload
> +(setq package-directory-list
> + (list (string-remove-suffix "/elpa" (car package-directory-list))))
> +
> (provide 'guix-emacs)

For future reference, this approach is absolutely unacceptable, no
package should modify customizable variables unless explicitly requested
by the user.
L
L
Liliana Marie Prikler wrote on 3 Dec 2021 21:46
Re: Emacs' describe-package doesn't work for packages managed by guix
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
e9e488c27aa0ce7edbf7b04f3c1022a6061f97db.camel@gmail.com
At long last, I'm pushing the patch to keep -pkg.el files as well as to
load them from guix-emacs during package-initialize. I'll hereby be
closing this bug. Andrew, if you wish to write a phase that adds such
a file for the packages currently lacking them, I'm pretty sure we can
do with a new bug number for that :)

Thanks everyone involved for your help and patience. Cheers
Closed
A
A
Andrew Tropin wrote on 6 Dec 2021 05:52
877dci49np.fsf@trop.in
On 2021-12-03 21:46, Liliana Marie Prikler wrote:

Toggle quote (9 lines)
> At long last, I'm pushing the patch to keep -pkg.el files as well as to
> load them from guix-emacs during package-initialize. I'll hereby be
> closing this bug. Andrew, if you wish to write a phase that adds such
> a file for the packages currently lacking them, I'm pretty sure we can
> do with a new bug number for that :)
>
> Thanks everyone involved for your help and patience. Cheers
>

Thank you very much for applying this change!)

--
Best regards,
Andrew Tropin
-----BEGIN PGP SIGNATURE-----

iQJDBAEBCgAtFiEEKEGaxlA4dEDH6S/6IgjSCVjB3rAFAmGtlxoPHGFuZHJld0B0
cm9wLmluAAoJECII0glYwd6wOp8P/jb45i1nBfVsnURbr3qqR4ILv3Krj8RyGrn8
d6UO4jUIVjoRK8/XUBCmvrFazdi/KUTzqWsA5dK2GBKSnIkbn1tZ0BVFYPbAPHCU
VxLdQQbcqv0EYxsy6D+vix/Xmm47BYYlGxaJEx1uTnvU+t4C6tEjf3D4giG/JmvM
axoO6/wbrxCM2mUoqA3aNlNIFSzLu3QZILF+QR0L976bWc+k+JyHHa3UHYHVM5SR
vNHReWAKhgL9HhP7SDcJKoX+j4Fi/f2b/pOnd2yMRmUhGYjyQtr43/jXas8PKRZa
AdaLyAfnhUGfu2vb30H/L7dcIKnHQ5RhyI7CnVenNEmXLHq8IvTmqAb3hXvaYcgk
M5sILYRarvnYpxUNZk6+wc2g9H6FQfT5IcubPqFKfmk5dRWERnqcmX7jzWJGxgvK
9jDvrrLHLuHqKQ2pxiFlXtB0LX2ML0v/VskgH7e1DKASbWm1gZRlelfvhNDhhOvf
ia2i9IZ/KgrYHq52VLi0Hebn9B/yHbo2HT9CEcQJIFR7P2H+T+xnvHZ6gASiihbk
6xu4c/tHGnQ8Rq4Dls/9BijZx+SWCCbvp7LWGxJcN6MBJUb2F4XTt0a4S0dk977B
eECMXr6FBlfk60HG1eqc3B+EoLa7/RcWxJIn0PX4abeLC4x4oFBr8HjrzDcCG7gt
3DHh+zsN
=rexi
-----END PGP SIGNATURE-----

Closed
A
A
Andrew Tropin wrote on 30 Dec 2021 09:12
8735mamrz2.fsf@trop.in
On 2021-12-03 21:46, Liliana Marie Prikler wrote:

Toggle quote (6 lines)
> At long last, I'm pushing the patch to keep -pkg.el files as well as to
> load them from guix-emacs during package-initialize. I'll hereby be
> closing this bug. Andrew, if you wish to write a phase that adds such
> a file for the packages currently lacking them, I'm pretty sure we can
> do with a new bug number for that :)

I implemented this phase in issues.guix.gnu.org/52388, I have it applied
to my local guix fork and I didn't notice any issues with it, however,
emacs packages built without emacs-build-system will require tweaking.

Everyone reading this thread can jump to [1] and check it out =)

[1]: The mail thread id is 871r2mrleb.fsf@trop.in
Toggle quote (4 lines)
>
> Thanks everyone involved for your help and patience. Cheers
>

--
Best regards,
Andrew Tropin
-----BEGIN PGP SIGNATURE-----

iQJDBAEBCgAtFiEEKEGaxlA4dEDH6S/6IgjSCVjB3rAFAmHNaeEPHGFuZHJld0B0
cm9wLmluAAoJECII0glYwd6wsgcQAIVaPz2q+DDpQc5aFEW68TURSbkrD/gQTkc0
yNVM72W3AgtczIRoXLuEGkiM6cl8BHjZvCFJ28jTiKdfDsdH73SGyqhLpI7r27xo
2sjdXMQCTiwuQWYH1kt55pZLY9FXjLNTfhAtRqvHtkuJpltXLTLzhN+WR8NrrGhl
n0IL9dHT/0xUdla67jCjo0AX2u2jgMpiES+KN6dsu58L4WJu9rbGtQIJYzlIui07
oA16d1Qa3C0QDGIqWdb3GM1SYZv1wYmsGd6rpD9tTfM4DYIoPBrI0qjgkWF70TCL
cqJpm2cq0kO+zgBSRjtZn41hcV3/epPiXHp2iBKSW2ggA9/vAcZgMv64Jmtn19ri
lPy/7sVfIyVAJ5YmMFlkp+Ftj0/MxZFSU5SLFEWBB+pbpp8PC8qwtf0PTy70rD1/
5Gj4UwsMfWVtbOk5fjzgOxnnPuvknSHWjpc1tV/TUg4Ox8TuqFGbbLEP1zwY2pAO
tUpxq3RFVtdLEy+KqutwBW0lPPjkRdXdEVET4FIcimbc99ymAfVfrtT42FLLixGj
0YvnUu3KHwAckTmg/2YOacj3URNnTTTxejAADKnJfoMCg3dFnhNfigHLrIhRfQZm
MnR30NkHPzq3QLpbMnxK/FG+hJ2MWFzg3T1y0lpwBTL9c0O7E1VJSIXU7C9vZmuk
l3lHzzYq
=Ji8v
-----END PGP SIGNATURE-----

Closed
?