[PATCH] gnu: Add make-single-arch-llvm procedure.

  • Done
  • quality assurance status badge
Details
3 participants
  • Thiago Jung Bauermann
  • Efraim Flashner
  • Ludovic Courtès
Owner
unassigned
Submitted by
Efraim Flashner
Severity
normal
E
E
Efraim Flashner wrote on 15 Jul 2021 12:04
(address . guix-patches@gnu.org)(name . Efraim Flashner)(address . efraim@flashner.co.il)
77ec80a001bb04229289f1e3efff7633c94c71dc.1626343320.git.efraim@flashner.co.il
* gnu/packages/llvm.scm (make-single-arch-llvm): New procedure.
(llvm-native-12, llvm-native): New variables.
---

Using this copy of llvm for packages like mesa will reduce the closure
of those packages. I don't suppose it would work as well for compilers
like rust which use LLVM for compiling and targeting different
architectures.

This drops the size of llvm-12 from 167M to 111M.

gnu/packages/llvm.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (36 lines)
diff --git a/gnu/packages/llvm.scm b/gnu/packages/llvm.scm
index 575c63c713..35cecdfd84 100644
--- a/gnu/packages/llvm.scm
+++ b/gnu/packages/llvm.scm
@@ -1347,3 +1347,27 @@ LLVM."))))
(define-public ocaml-llvm-9 (make-ocaml-llvm llvm-9))
(define-public ocaml-llvm-10 (make-ocaml-llvm llvm-10))
(define-public ocaml-llvm-11 (make-ocaml-llvm llvm-11))
+
+(define* (make-single-arch-llvm llvm #:optional (system (or (%current-target-system)
+ (%current-system))))
+ (package
+ (inherit llvm)
+ (name (string-append "llvm-for-" (string-downcase
+ (system->llvm-target system))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments llvm)
+ ((#:configure-flags cf)
+ `(cons* ,(string-append "-DLLVM_TARGETS_TO_BUILD="
+ (system->llvm-target system))
+ ,cf))))
+ (synopsis "Compiler with support for a single architecture")))
+
+(define-public llvm-native-12
+ (package
+ (inherit (make-single-arch-llvm llvm-12))
+ (name "llvm-native")))
+
+(define-public llvm-native
+ (package
+ (inherit (make-single-arch-llvm llvm))
+ (name "llvm-native")))

base-commit: 17c8fedc256406047c7a1e8317bc52641c621286
--
2.32.0
L
L
Ludovic Courtès wrote on 26 Jul 2021 18:47
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 49569@debbugs.gnu.org)
87a6m981kd.fsf@gnu.org
Hi,

Efraim Flashner <efraim@flashner.co.il> skribis:

Toggle quote (11 lines)
> * gnu/packages/llvm.scm (make-single-arch-llvm): New procedure.
> (llvm-native-12, llvm-native): New variables.
> ---
>
> Using this copy of llvm for packages like mesa will reduce the closure
> of those packages. I don't suppose it would work as well for compilers
> like rust which use LLVM for compiling and targeting different
> architectures.
>
> This drops the size of llvm-12 from 167M to 111M.

That’s less impressive than I would have thought.

For Mesa perhaps we could maybe have a variant that’s even more
stripped?

The downside of this is that we risk having several LLVM copies on
typical installations.

Thoughts?

Thanks,
Ludo’.
E
E
Efraim Flashner wrote on 27 Jul 2021 14:29
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 49569@debbugs.gnu.org)
YP/8KUHlPKBdEutc@3900XT
On Mon, Jul 26, 2021 at 06:47:14PM +0200, Ludovic Courtès wrote:
Toggle quote (28 lines)
> Hi,
>
> Efraim Flashner <efraim@flashner.co.il> skribis:
>
> > * gnu/packages/llvm.scm (make-single-arch-llvm): New procedure.
> > (llvm-native-12, llvm-native): New variables.
> > ---
> >
> > Using this copy of llvm for packages like mesa will reduce the closure
> > of those packages. I don't suppose it would work as well for compilers
> > like rust which use LLVM for compiling and targeting different
> > architectures.
> >
> > This drops the size of llvm-12 from 167M to 111M.
>
> That’s less impressive than I would have thought.
>
> For Mesa perhaps we could maybe have a variant that’s even more
> stripped?
>
> The downside of this is that we risk having several LLVM copies on
> typical installations.
>
> Thoughts?
>
> Thanks,
> Ludo’.

I think you're mostly shocked by the large numbers, it is a drop of 33%.

It turns out that llvm-julia uses something similar to only target one
arch, so the idea's there somewhere. We could probably add it in there
too for dlang. Icecat/icedove/mozjs-78 should be OK to switch. And
xf86-video-vmware already has mesa as an input and could likely use a
special llvm-for-mesa version.

I can look at mesa and see which libraries from llvm it links to and see
about just building those. A first look shows 50 llvm libraries from
mesa and about 150 from llvm@11, with the 4 libraries being
libOSMesa libvulkan_radeon libxatracker libXvMCnouveau


--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmD//CgACgkQQarn3Mo9
g1EW5Q//ayy0RkOmKyvc8SdttL6W7bUVUrfOWsIJyRRX+iGg/Mlt51ioqGPYwE6+
GMfiC+5iO/CSUpGewyZDZywCLPKH1GSKYDW1aZ+GLAcEs9V5zznFurGo4zj4Pry6
fWIRLjo9UOnwIEFaSmURitK7F6PyQyuBEu0lJw1IUdjMqLxzEYhQrHHTCYbkVFQm
pig7l/Bj1unfuIGNlHTuS1f7cCLgYEDJopthvgPdIq+ekUi8+fMaDu78B+/PzMfM
Gjzs7I4noiYfdU0HWAWZxTGFEqwONCHxOPQ3oiA3fSgdSMlyjbDRCcfyCxNLiHfc
ZVEOWBhM/t+lK19uA8+fxg0yhDcHxXLN2pj50ZKWTYsDOl07+PFoEF1UBerDCH0B
acGxmiEiVqgOe5LxfiYDb0iLbwKWdl5lDiI7BtAzDC612/CaF9OMLYInSSPIvRxe
edvjj+FrlBMIzBqfam6RGOJwXowOGoCwRDj3hO8rQWHkuwsat1L5AHbG6JE33XOg
y8zqavrRAU+r/kY38fErdVLoZabUm+OvaJVYlfSiz8UeGrs4GghAfA/14lEIedL2
7OLTmEg6XHnKrw6vvuxMeDum2MrplwgVW7umYaA2ep4tNHwMpu/gkyE5BiI2d7/u
Wc1rgYr8/E4mLecxhEIfT+v2l9WA/v+ghJLmWpiWW1H5MD01hHg=
=6+C0
-----END PGP SIGNATURE-----


T
T
Thiago Jung Bauermann wrote on 27 Jul 2021 15:09
Re: [bug#49569] [PATCH] gnu: Add make-single-arch-llvm procedure.
2615631.XO7ydEMPkx@popigai
Hello,

Em quinta-feira, 15 de julho de 2021, às 07:04:52 -03, Efraim Flashner
escreveu:
Toggle quote (11 lines)
> * gnu/packages/llvm.scm (make-single-arch-llvm): New procedure.
> (llvm-native-12, llvm-native): New variables.
> ---
>
> Using this copy of llvm for packages like mesa will reduce the closure
> of those packages. I don't suppose it would work as well for compilers
> like rust which use LLVM for compiling and targeting different
> architectures.
>
> This drops the size of llvm-12 from 167M to 111M.

I don’t have an opinion on the issue that Ludo’ raised, just one comment
about the code below.

Toggle quote (19 lines)
> gnu/packages/llvm.scm | 24 ++++++++++++++++++++++++
> 1 file changed, 24 insertions(+)
>
> diff --git a/gnu/packages/llvm.scm b/gnu/packages/llvm.scm
> index 575c63c713..35cecdfd84 100644
> --- a/gnu/packages/llvm.scm
> +++ b/gnu/packages/llvm.scm
> @@ -1347,3 +1347,27 @@ LLVM."))))
> (define-public ocaml-llvm-9 (make-ocaml-llvm llvm-9))
> (define-public ocaml-llvm-10 (make-ocaml-llvm llvm-10))
> (define-public ocaml-llvm-11 (make-ocaml-llvm llvm-11))
> +
> +(define* (make-single-arch-llvm llvm #:optional (system (or (%current-target-system)
> + (%current-system))))
> + (package
> + (inherit llvm)
> + (name (string-append "llvm-for-" (string-downcase
> + (system->llvm-target system))))

‘system->llvm-target’ needs a Nix system identifier but
‘%current-target-system’ contains a GNU triplet, so the above won’t work
when cross-compiling for armhf-linux – whose GNU triplet is
“arm-unknown-linux-gnueabihf”.

I suggest changing the default value of ‘system’ to

(or (and=> (%current-target-system) gnu-triplet->nix-system)
(%current-system))

which is the default argument for ‘system->llvm-target’.

If I can add a shameless plug, patch 2 of my “Target check fixes and
cleanups” series¹ adds a helper procedure with the code above. :-)

Toggle quote (6 lines)
> + (arguments
> + (substitute-keyword-arguments (package-arguments llvm)
> + ((#:configure-flags cf)
> + `(cons* ,(string-append "-DLLVM_TARGETS_TO_BUILD="
> + (system->llvm-target system))

Same problem here.

--
Thanks,
Thiago

L
L
Ludovic Courtès wrote on 27 Jul 2021 19:10
Re: bug#49569: [PATCH] gnu: Add make-single-arch-llvm procedure.
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 49569@debbugs.gnu.org)
87o8an3coj.fsf@gnu.org
Efraim Flashner <efraim@flashner.co.il> skribis:

Toggle quote (18 lines)
> On Mon, Jul 26, 2021 at 06:47:14PM +0200, Ludovic Courtès wrote:
>> Hi,
>>
>> Efraim Flashner <efraim@flashner.co.il> skribis:
>>
>> > * gnu/packages/llvm.scm (make-single-arch-llvm): New procedure.
>> > (llvm-native-12, llvm-native): New variables.
>> > ---
>> >
>> > Using this copy of llvm for packages like mesa will reduce the closure
>> > of those packages. I don't suppose it would work as well for compilers
>> > like rust which use LLVM for compiling and targeting different
>> > architectures.
>> >
>> > This drops the size of llvm-12 from 167M to 111M.
>>
>> That’s less impressive than I would have thought.

[...]

Toggle quote (8 lines)
> I think you're mostly shocked by the large numbers, it is a drop of 33%.
>
> It turns out that llvm-julia uses something similar to only target one
> arch, so the idea's there somewhere. We could probably add it in there
> too for dlang. Icecat/icedove/mozjs-78 should be OK to switch. And
> xf86-video-vmware already has mesa as an input and could likely use a
> special llvm-for-mesa version.

OK, maybe that’d be an improvement, and indeed, 33% is not negligible.

Toggle quote (5 lines)
> I can look at mesa and see which libraries from llvm it links to and see
> about just building those. A first look shows 50 llvm libraries from
> mesa and about 150 from llvm@11, with the 4 libraries being
> libOSMesa libvulkan_radeon libxatracker libXvMCnouveau

Makes sense.

Does Mesa require some special GPU compiler backend in addition to the
native backend?

Thanks,
Ludo’.
E
E
Efraim Flashner wrote on 30 Jan 2023 19:12
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 49569-done@debbugs.gnu.org)
Y9gIkV9hbHVkGJKH@3900XT
I think we can drop this with llvm-for-mesa in core-updates.

--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmPYCJEACgkQQarn3Mo9
g1FC8w//YwZBfmF6mVbya0CPlSSIUzjQuBT9cdVAXBw+tcH9ib6+E4Y9oYPZXmXo
06+qOmRRUv4fb/NfFE4bb1sbNb4S4C+xM36roeNllDMg9fzYw8Jglw+GooU+U7gO
DiXnRPwJLDgAXsimCWdoaELWElVHVR/7mWVXlZAt4P+vxTE1qSsU8xjtl1J5Ha3C
cLe+hJ2tUSbB1oG8ZUgZ5NfW75mRik+wUBHmGz2K4DsPnr5x2Q8lwGW3lOtY78V9
KSkucsZ67M2SUSfZ64q5LhHDJ1GvCq3nK2UP/YN9Vz8T9cg9dPQ/T/MUH7/GIOZQ
ZWPJXEzkmhdIThoA0MszhnXQAeBVRN2iYf5bKrG7Vx061X77lAVvUsIreW2ko4u8
j/BYM5j2U80d7RSGOAzeapx+nDXw6pTBp/edWEZaYyshvPUvqbw7R901gv4cK0/K
jn8vhLltk2ExTzHUK+vpalOrPLDKmCGr6UapYLOMwB/L7iHAkMWZgZ2AiWRnz+6J
A3/xa8olflh45TULb4x2SVCZGl/ceZ3IhQdlF/KOUzR5bFaV0SpNMhmY65vuRhm1
Oh733xr2b38hQS7VUI1+4/SQprVfOVu6GVmQG9TVT2AURSTEo+OWViVOH82RAasM
cRpmDWvQofQ8EKRMoIn3N+wqD7cBwC2Z+97QRvT20z+4npKI86o=
=OFAk
-----END PGP SIGNATURE-----


Closed
?