[PATCH] WIP: Add gccemacs

  • Done
  • quality assurance status badge
Details
5 participants
  • Morgan Smith
  • Jelle Licht
  • John Soo
  • Andrew Whatson
  • zimoun
Owner
unassigned
Submitted by
John Soo
Severity
normal
J
J
John Soo wrote on 21 Nov 2020 10:15
(address . guix-patches@gnu.org)
874kljnk8z.fsf@asu.edu
Hi Guix,

I was curious how fast this gccemacs branch might be, so I got it to
compile.

It feels fast but there are bugs and the closure is huge. Also these
patches do not use any of the parameterization machinery.

I just thought I would share my work.

Thanks,

John
From 642c087de1592aeea0e65d04589423a05a96d68e Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Sat, 21 Nov 2020 00:56:30 -0800
Subject: [PATCH 1/3] gnu: Add emacs-next-no-x.

* gnu/packages/emacs.scm (emacs-next-no-x): New variable.
---
gnu/packages/emacs.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (42 lines)
diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 4f5a67093c..2d9f2e7ec1 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -22,6 +22,7 @@
;;; Copyright © 2019 Amin Bandali <bandali@gnu.org>
;;; Copyright © 2020 Jack Hill <jackhill@jackhill.us>
;;; Copyright © 2020 Morgan Smith <Morgan.J.Smith@outlook.com>
+;;; Copyright © 2020 John Soo <jsoo1@asu.edu>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -376,6 +377,27 @@ editor (console only)")
(delete 'restore-emacs-pdmp)
(delete 'strip-double-wrap)))))))
+(define-public emacs-next-no-x
+ (package
+ (inherit emacs-next)
+ (name "emacs-next-no-x")
+ (synopsis (package-synopsis emacs-no-x))
+ (build-system gnu-build-system)
+ (arguments
+ (substitute-keyword-arguments (package-arguments emacs-next)
+ ((#:configure-flags flags)
+ `(append '("--with-x-toolkit=no"
+ "--with-xpm=no"
+ "--with-jpeg=no"
+ "--with-gif=no"
+ "--with-tiff=no")
+ ,flags))
+ ((#:phases phases)
+ `(modify-phases ,phases
+ (delete 'restore-emacs-pdmp)
+ (delete 'strip-double-wrap)))))
+ (inputs (package-inputs emacs-no-x))))
+
(define-public emacs-no-x-toolkit
(package/inherit emacs
(name "emacs-no-x-toolkit")
--
2.29.1
Attachment: 0002-gnu-Add-gccemacs.patch
From f23ebabab5e57b22b45fea3a26f9a1814331f39a Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Sat, 21 Nov 2020 00:59:14 -0800
Subject: [PATCH 3/3] gnu: Add gcceamcs-no-x.

* gnu/packages/emacs.scm (gccemacs-no-x): New variable.
---
gnu/packages/emacs.scm | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)

Toggle diff (33 lines)
diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 6312fde3ad..721a13f429 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -487,6 +487,26 @@ editor (from the native compilation branch)")
((#:configure-flags flags ''())
`(cons "--with-nativecomp" ,flags))))))
+(define-public gccemacs-no-x
+ (package/inherit emacs-next-no-x
+ (name "gccemacs-no-x")
+ (version gccemacs-version)
+ (source gccemacs-source)
+ (synopsis "The extensible, customizable, self-documenting text
+editor (console only, from the native compilation branch)")
+ (build-system gnu-build-system)
+ (inputs
+ `(("libgccjit" ,(canonical-package libgccjit))
+ ,@(package-inputs emacs-next-no-x)))
+ (arguments
+ (substitute-keyword-arguments (package-arguments emacs-next-no-x)
+ ((#:phases p)
+ `(modify-phases ,p
+ (add-after 'bootstrap 'add-libgccjit-gcc-lib-to-library-path
+ ,add-libgccjit-gcc-lib-to-library-path)))
+ ((#:configure-flags flags ''())
+ `(cons "--with-nativecomp" ,flags))))))
+
(define-public emacs-no-x-toolkit
(package/inherit emacs
(name "emacs-no-x-toolkit")
--
2.29.1
Z
Z
zimoun wrote on 23 Nov 2020 23:22
86360zvhkw.fsf@gmail.com
Hi John,

On Sat, 21 Nov 2020 at 01:15, John Soo <jsoo1@asu.edu> wrote:

Toggle quote (3 lines)
> I was curious how fast this gccemacs branch might be, so I got it to
> compile.

Thanks! This motivates me to resume what had been discussed here [1]:
be able to somehow run:

Toggle snippet (3 lines)
guix build emacs-magit emacs-foo emacs-bar --with-input=emacs=gccemacs

At least, have a package transformation that allow to rebuild all the
Emacs packages with the ’gccemacs’ VM instead of the current Emacs 27 one.


Toggle quote (3 lines)
> It feels fast but there are bugs and the closure is huge. Also these
> patches do not use any of the parameterization machinery.

What do you mean by “parameterization machinery”? The new
’with-parameter’ introduced here [2] or the package transformation that
replaces all the dependencies (explicit and implicit).




All the best,
simon
J
J
John Soo wrote on 24 Nov 2020 16:06
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 44775@debbugs.gnu.org)
87tutex08p.fsf@asu.edu
Hi zimoun,

zimoun <zimon.toutoune@gmail.com> writes:

Toggle quote (20 lines)
> Thanks! This motivates me to resume what had been discussed here [1]:
> be able to somehow run:
>
> guix build emacs-magit emacs-foo emacs-bar --with-input=emacs=gccemacs
>
> At least, have a package transformation that allow to rebuild all the
> Emacs packages with the ’gccemacs’ VM instead of the current Emacs 27 one.
>
>
>> It feels fast but there are bugs and the closure is huge. Also these
>> patches do not use any of the parameterization machinery.
>
> What do you mean by “parameterization machinery”? The new
> ’with-parameter’ introduced here [2] or the package transformation that
> replaces all the dependencies (explicit and implicit).
>
>
> 1: <https://yhetil.org/guix-bugs/868scwtt34.fsf@gmail.com/>
> 2: <https://yhetil.org/guix-devel/87eeku8trb.fsf@gnu.org>


--with-parameter=gccemacs or similar seem to be required since the
native-comp branch requires a different source, configure flags, and
probably native-search-paths at least.

There appears to be a separate compiled artifact directory under
$out/lib/emacs/$version/native-lisp/$version-triple which has the
compiled native libraries (.eln files). That directory seems to not be
in the search path. That appears to be causing the first error
I see. I am not sure which env variable would be tweaked to pick those
paths up.

Hope that helps!

John
J
J
John Soo wrote on 24 Nov 2020 16:28
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 44775@debbugs.gnu.org)
87pn42wz8o.fsf@asu.edu
Hi Guix and zimoun,

I fixed a bug in the build process here. Still no big progress on the
other bugs.

- John
Attachment: 0001-gnu-Add-gccemacs.patch
M
M
Morgan Smith wrote on 29 Nov 2020 03:14
(name . John Soo)(address . jsoo1@asu.edu)
SN4PR0801MB3679656DE90AA522CAFC8003C5F60@SN4PR0801MB3679.namprd08.prod.outlook.com
Have you seen this gccemacs package? I haven't looked into it myself but
it might help you.

J
J
John Soo wrote on 11 Dec 2020 21:40
(name . Morgan Smith)(address . Morgan.J.Smith@outlook.com)
87k0tovzy3.fsf@asu.edu
Hi Morgan,

Thank you! I will take a look. If I end up using some of their code,
should I consult them about it and see if they want a copyright line?
How is that supposed to work?

Thank you again,

John
M
M
Morgan Smith wrote on 17 Dec 2020 18:26
SN4PR0801MB3679A3939B2DDB789993B21BC5C40@SN4PR0801MB3679.namprd08.prod.outlook.com
Hi John,

(And hi Andrew. We're discussing including your code into the main Guix
repository. You can view the full conversation here:

The code I linked to is already licensed under GPLv3+ meaning that you
are free to integrate their code into the Guix code-base (Going to put
here that I'm not a lawyer and that this is not legal advice and might
not even be correct). You should indeed put a little copyright line for
Andrew in the code. The exact copyright line you should use seems to be
this (pulled from their git repo):

;;; Copyright © 2020 Andrew Whatson <whatson@gmail.com>

You can do all this without contacting the author, but hopefully Andrew
responds to us with lots of love.

Thanks,

Morgan

On 12/11/20 3:40 PM, John Soo wrote:
Toggle quote (10 lines)
> Hi Morgan,
>
> Thank you! I will take a look. If I end up using some of their code,
> should I consult them about it and see if they want a copyright line?
> How is that supposed to work?
>
> Thank you again,
>
> John
>
A
A
Andrew Whatson wrote on 18 Dec 2020 01:25
(name . Morgan Smith)(address . Morgan.J.Smith@outlook.com)
CAPE069df+8MNuS9oczV8tA2dsW77CULctFPEXshirDRWwjvKtQ@mail.gmail.com
Hi everyone,

Yes, I'm very happy for my work to be included in Guix in whatever
form. I've detailed my work below, and also CC'd Andrea who is
responsible for developing this feature in case he has anything to
add.

1. Enable the `--with-native-comp` configure flag. Self explanatory!

2. Set the `NATIVE_FULL_AOT=1` make flag, maybe. This tells the build
process to native-compile *all* of the Elisp that ships with Emacs.
Otherwise only the minimal set of Elisp packages included in the pdump
will be native-compiled, with the rest to be compiled on-demand at
runtime. This has a significant impact on compilation time, so makes
more sense if the package will be built once and installed many times,
and less sense if everyone is building the package themselves.

3. Extend `LIBRARY_PATH` so libgccjit works at compile time. This is
necessary for a functioning libgccjit and to pass the "smoke test" in
the configure script. I think this should probably be exported by the
libgccjit package as a search path instead of requiring all dependents
to handle it manually.

4. Patch the `comp-native-driver-options` in `comp.el`. This is
necessary to have libgccjit functioning at runtime. Originally I was
setting LIBRARY_PATH in the emacs wrapper to achieve this, but that
had the undesirable effect of leaking to any process launched from
Emacs. Setting the necessary paths via the driver options is a much
more targeted solution.

5. Remove the shell-script wrappers from eln files. The
`glib-or-gtk-build-system` aggressively wraps anything which smells
like an executable, preventing the native-compiled Elisp from being
loaded as shared libraries at runtime. This is based on the
`restore-emacs-pdmp` phase in the base Emacs package which works
around the same problem.

6. Use a newer `libgccjit` based on `gcc-10`. This is not strictly
necessary, but if I recall correctly libgccjit-9 suffers from a bug
related to the inlining of constant strings which was fixed in
libgccjit-10, and this has some impact on the performance of
native-compiled Elisp. The `libgccjit` package in Guix is only
defined for gcc-9, so I created a package factory to define libgccjit
packages based on an arbitrary gcc, and use gcc-10 and libgccjit-10 as
dependencies for the emacs-native-comp package. I haven't tried to
build using gcc-9 and libgccjit-10, so I'm not sure if there's some
interdependency. I think it would be best to upstream libgccjit-10
alongside an emacs-native-comp package.

I hope this all makes sense, happy to answer any questions.

Cheers,
Andrew

On Fri, 18 Dec 2020 at 03:26, Morgan Smith <Morgan.J.Smith@outlook.com> wrote:
Toggle quote (34 lines)
>
> Hi John,
>
> (And hi Andrew. We're discussing including your code into the main Guix
> repository. You can view the full conversation here:
> https://issues.guix.gnu.org/44775)
>
> The code I linked to is already licensed under GPLv3+ meaning that you
> are free to integrate their code into the Guix code-base (Going to put
> here that I'm not a lawyer and that this is not legal advice and might
> not even be correct). You should indeed put a little copyright line for
> Andrew in the code. The exact copyright line you should use seems to be
> this (pulled from their git repo):
>
> ;;; Copyright © 2020 Andrew Whatson <whatson@gmail.com>
>
> You can do all this without contacting the author, but hopefully Andrew
> responds to us with lots of love.
>
> Thanks,
>
> Morgan
>
> On 12/11/20 3:40 PM, John Soo wrote:
> > Hi Morgan,
> >
> > Thank you! I will take a look. If I end up using some of their code,
> > should I consult them about it and see if they want a copyright line?
> > How is that supposed to work?
> >
> > Thank you again,
> >
> > John
> >
Z
Z
zimoun wrote on 18 Dec 2020 03:50
(name . John Soo)(address . jsoo1@asu.edu)(address . 44775@debbugs.gnu.org)
86y2hvn7yt.fsf@gmail.com
Hi John,

On Tue, 24 Nov 2020 at 07:06, John Soo <jsoo1@asu.edu> wrote:

Toggle quote (21 lines)
>> Thanks! This motivates me to resume what had been discussed here [1]:
>> be able to somehow run:
>>
>> guix build emacs-magit emacs-foo emacs-bar --with-input=emacs=gccemacs
>>
>> At least, have a package transformation that allow to rebuild all the
>> Emacs packages with the ’gccemacs’ VM instead of the current Emacs 27 one.
>>
>>> It feels fast but there are bugs and the closure is huge. Also these
>>> patches do not use any of the parameterization machinery.
>>
>> What do you mean by “parameterization machinery”? The new
>> ’with-parameter’ introduced here [2] or the package transformation that
>> replaces all the dependencies (explicit and implicit).

> I am referring to https://yhetil.org/guix-devel/87eeku8trb.fsf@gnu.org
>
> --with-parameter=gccemacs or similar seem to be required since the
> native-comp branch requires a different source, configure flags, and
> probably native-search-paths at least.

I am not convinced that the “parameterization machinery” could help here
(aside it is far to be ready :-)) because in this case, “gccemacs” is
not a «parameter» but an «argument» of the Emacs build system.

Maybe I miss something.


Toggle quote (7 lines)
> There appears to be a separate compiled artifact directory under
> $out/lib/emacs/$version/native-lisp/$version-triple which has the
> compiled native libraries (.eln files). That directory seems to not be
> in the search path. That appears to be causing the first error
> I see. I am not sure which env variable would be tweaked to pick those
> paths up.

Thanks for the explanations.

All the best,
simon
J
J
Jelle Licht wrote on 29 May 2023 14:06
Re: bug#44775: [PATCH] WIP: Add gccemacs
(name . John Soo)(address . jsoo1@asu.edu)(address . 44775-done@debbugs.gnu.org)
87v8gbmjew.fsf@fsfe.org
Most versions of Emacs available on the guix master branch make use of
this work to make everything blazing fast, so thanks. Closing.
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 44775@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 44775
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch