[PATCH 0/2] Support 16K pages sizes with jemalloc on aarch64

  • Open
  • quality assurance status badge
Details
3 participants
  • Efraim Flashner
  • Roman Scherer
  • Roman Scherer
Owner
unassigned
Submitted by
Roman Scherer
Severity
normal
R
R
Roman Scherer wrote on 5 Jan 13:33 +0100
(address . guix-patches@gnu.org)
cover.1704455408.git.roman@burningswell.com
Hello Guix,

I'm running Guix system on an Apple M1 using the Asahi Linux kernel. It works,
but there is a major issue due to the fact that the Asahi Linux kernel uses a
16k page size.

There are some programs that are incompatible with this and are being
fixed. You can find more information under the "Known broken applications"
section in [1].

One of those programs that has issues is jemalloc, used by some of the most
heavy Guix packages, like rust and icecat. Running any program that uses
jemalloc crashes with the following error:

```
roman@localhost guix]$ rustc
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
terminate called without an active exception
Aborted
```

This is because jemalloc is compiled to handle 4K page sizes by default. If
jemalloc is configured to handle larger page sizes it is working. From what I
understand you can use jemalloc configured with page size X on all systems
with a page size <= X, with a small cost in performance. There is an issue
here [2] that has a more detailed discussion.

Note that this only happens when I use substitutes. If I compile jemalloc
myself on a system with 16k page size it works. Compiling the whole rust
toolchain and icecat on a Guix updates however is not really practical.

I already tried to fix this issue a while ago here [3], but unfortunatly this
did not solve it. The Guix substitutes are still not compatible on a system
with 16k page size.

It looks like some distros compile jemalloc with a larger page size. I belive
the Asahi Fedora remix is doing this and Arch Linux ARM [4].

This patch series configures jemalloc to support 16k page sizes on the aarch64
architecture. It uses the --with-lg-page switch which specifies the page size
as log2(16384)=14 for 16k pages.

To make packages using jemalloc via rust-jemalloc-sys compatible, the same is
done by specifying the JEMALLOC_SYS_WITH_LG_PAGE environment variable.

Could you please review the patch series and/or help me to get substitutes
available that are compatible with larger page sizes?

Thanks, Roman.


Roman Scherer (2):
gnu: jemalloc: Build with large page size.
gnu: rust-jemalloc-sys: Build with large page size.

gnu/packages/crates-io.scm | 6 +++++-
gnu/packages/jemalloc.scm | 2 ++
2 files changed, 7 insertions(+), 1 deletion(-)

base-commit: ac69b423865f12310cef5662d9c303aa4b90c869
--
2.41.0
R
R
Roman Scherer wrote on 5 Jan 13:36 +0100
[PATCH 1/2] gnu: jemalloc: Build with large page size.
(address . 68261@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
146b3cf98eda967f04143fa62f5a6631c307d575.1704455408.git.roman@burningswell.com
* gnu/packages/jemalloc.scm (jemalloc): Build with large page size.

Change-Id: Ic813e7b0fe4c7ee79a1e703247abea77ad9d53f0
---
gnu/packages/jemalloc.scm | 2 ++
1 file changed, 2 insertions(+)

Toggle diff (15 lines)
diff --git a/gnu/packages/jemalloc.scm b/gnu/packages/jemalloc.scm
index 5e7facfd5e..b5fdd39921 100644
--- a/gnu/packages/jemalloc.scm
+++ b/gnu/packages/jemalloc.scm
@@ -62,6 +62,8 @@ (define-public jemalloc
;; https://github.com/jemalloc/jemalloc/issues/937
#~'("--disable-initial-exec-tls"
#$@(match (%current-system)
+ ("aarch64-linux"
+ (list "--with-lg-page=14"))
("powerpc-linux"
(list "CPPFLAGS=-maltivec"))
(_
--
2.41.0
R
R
Roman Scherer wrote on 5 Jan 13:36 +0100
[PATCH 2/2] gnu: rust-jemalloc-sys: Build with large page size.
(address . 68261@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
c3d2e3f515fcf29588bf18d32ba50257b309b576.1704455408.git.roman@burningswell.com
* gnu/packages/crates-io.scm (rust-jemalloc-sys): Build with large page size.

Change-Id: I91ed8450952204c1ecba19604521dd8b8ec554ec
---
gnu/packages/crates-io.scm | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)

Toggle diff (19 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 1bfd8fb143..f8eb0c293f 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -39286,7 +39286,11 @@ (define-public rust-jemalloc-sys-0.5
;; https://github.com/tikv/jemallocator/issues/19
(setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1")
(setenv "JEMALLOC_OVERRIDE"
- (string-append jemalloc "/lib/libjemalloc_pic.a"))))))))
+ (string-append jemalloc "/lib/libjemalloc_pic.a")))))
+ (add-after 'configure 'with-lg-page-jemalloc
+ (lambda _
+ (when (target-aarch64?)
+ (setenv "JEMALLOC_SYS_WITH_LG_PAGE" "14")))))))
(native-inputs
(list jemalloc))
(home-page "https://github.com/tikv/jemallocator")
--
2.41.0
E
E
Efraim Flashner wrote on 5 Jan 13:48 +0100
(name . Roman Scherer)(address . roman@burningswell.com)(address . 68261@debbugs.gnu.org)
ZZf6rLdVTiEjeH47@3900XT
Since we don't carry the results of one rust package to the next and
have to rebuild everything, it would be better to add this to
(guix build cargo-build-system), in the configure phase. Then it will
apply to every package which is built using the cargo-build-system.

On Fri, Jan 05, 2024 at 01:36:42PM +0100, Roman Scherer wrote:
Toggle quote (30 lines)
> * gnu/packages/crates-io.scm (rust-jemalloc-sys): Build with large page size.
>
> Change-Id: I91ed8450952204c1ecba19604521dd8b8ec554ec
> ---
> gnu/packages/crates-io.scm | 6 +++++-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
> index 1bfd8fb143..f8eb0c293f 100644
> --- a/gnu/packages/crates-io.scm
> +++ b/gnu/packages/crates-io.scm
> @@ -39286,7 +39286,11 @@ (define-public rust-jemalloc-sys-0.5
> ;; https://github.com/tikv/jemallocator/issues/19
> (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1")
> (setenv "JEMALLOC_OVERRIDE"
> - (string-append jemalloc "/lib/libjemalloc_pic.a"))))))))
> + (string-append jemalloc "/lib/libjemalloc_pic.a")))))
> + (add-after 'configure 'with-lg-page-jemalloc
> + (lambda _
> + (when (target-aarch64?)
> + (setenv "JEMALLOC_SYS_WITH_LG_PAGE" "14")))))))
> (native-inputs
> (list jemalloc))
> (home-page "https://github.com/tikv/jemallocator")
> --
> 2.41.0
>
>
>

--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmWX+qkACgkQQarn3Mo9
g1FYFA//WrDrNgX3IzAqhGXAHF6wfeomZxAnhvy9c7/dVEpFOGc8lqa5LBRzlSEy
hIlbkckKqPm+vb5PuJ7yuHV8RKYn531peZZ4Vkhi8B5YgRAPI7FvOZwKXjrS214f
kVMR41eq+2EBAIu2aukWzROGzhe5i8/itUqV9TkCP25InO/0aBu6fDsRqj4Rirl6
7Fpoj1HaDkpbyXFo+wO8WJL2KR2AgOrgQZvKfl9ibIUkGhGZOmawC+4goQoYjk9U
OUVD6D+ic9Uraa48DqN7PHqmXQzVsm9UMBEX7Y9rlWqqrBpfqIoxYOfe1OckGEuN
PcP+tpee8B9CQquQpjsB/yvr86zwbjyVpkdOa2G32yrawDy7DgbvStp5LHcP+/T2
tgJ/JspWS8E54TeXgp/HcmGHslXL51W1LH8fxi7jeWiuRDQw6E6+ad5WuXuu+l37
aPnq/gm9XRhypQwSbmyWLRAsfVBH1PEEfVOHzkj/O7Kh6iUhRG65cPGEXnaaZqk3
BhXqe7GTJfYBRge1DXgfJ0gPpb7c4L0F+3/UGIW+dHoG8HAdyHQ3vb9O9J9uxOQc
SRXVq5Pf8l43abWi7+bSRqgJHsa/DwymiCANzlM8JYSRGLiicIAk8iKFLuWdBRE8
X/OE5xQxXdNgD0WKTzslohLvFLIWZ2tZRtPWBOQNQYxzeK3V65U=
=U0pl
-----END PGP SIGNATURE-----


R
R
Roman Scherer wrote on 5 Jan 14:30 +0100
[PATCH v2 1/2] gnu: jemalloc: Build with large page size.
(address . 68261@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
146b3cf98eda967f04143fa62f5a6631c307d575.1704461315.git.roman@burningswell.com
* gnu/packages/jemalloc.scm (jemalloc): Build with large page size.

Change-Id: Ic813e7b0fe4c7ee79a1e703247abea77ad9d53f0
---
gnu/packages/jemalloc.scm | 2 ++
1 file changed, 2 insertions(+)

Toggle diff (17 lines)
diff --git a/gnu/packages/jemalloc.scm b/gnu/packages/jemalloc.scm
index 5e7facfd5e..b5fdd39921 100644
--- a/gnu/packages/jemalloc.scm
+++ b/gnu/packages/jemalloc.scm
@@ -62,6 +62,8 @@ (define-public jemalloc
;; https://github.com/jemalloc/jemalloc/issues/937
#~'("--disable-initial-exec-tls"
#$@(match (%current-system)
+ ("aarch64-linux"
+ (list "--with-lg-page=14"))
("powerpc-linux"
(list "CPPFLAGS=-maltivec"))
(_

base-commit: ac69b423865f12310cef5662d9c303aa4b90c869
--
2.41.0
R
R
Roman Scherer wrote on 5 Jan 14:30 +0100
[PATCH v2 2/2] build/cargo-build-system: Support 16k page sizes on aarch64.
(address . 68261@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
d8664f2c5a8119ef69fc0a5f7a42c362a6f3b871.1704461315.git.roman@burningswell.com
* guix/build/cargo-build-system.scm (configure): Support 16k page sizes on aarch64.

Change-Id: I523c192159908483577301da246d75d16b694bc8
---
guix/build/cargo-build-system.scm | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)

Toggle diff (26 lines)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index ffb2ec898e..c5c2299a4c 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -119,7 +119,7 @@ (define* (check-for-pregenerated-files #:rest _)
(error "Possible pre-generated files found:" pregenerated-files))))
(define* (configure #:key inputs
- target
+ target system
(vendor-dir "guix-vendor")
#:allow-other-keys)
"Vendor Cargo.toml dependencies as guix inputs."
@@ -178,6 +178,10 @@ (define* (configure #:key inputs
;; Prevent targeting the build machine.
(setenv "CRATE_CC_NO_DEFAULTS" "1"))
+ ;; Support 16k kernel page sizes on aarch64 with jemalloc.
+ (when (string-prefix? "aarch64" (or target system))
+ (setenv "JEMALLOC_SYS_WITH_LG_PAGE" "14"))
+
;; Configure cargo to actually use this new directory with all the crates.
(setenv "CARGO_HOME" (string-append (getcwd) "/.cargo"))
(mkdir-p ".cargo")
--
2.41.0
R
R
Roman Scherer wrote on 5 Jan 14:31 +0100
Re: [bug#68261] [PATCH 2/2] gnu: rust-jemalloc-sys: Build with large page size.
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 68261@debbugs.gnu.org)
86h6jrx5v6.fsf@burningswell.com
Hi Efraim,

thanks for the review! I moved the setting of the
JEMALLOC_SYS_WITH_LG_PAGE now to the cargo build system, as you
suggested, and submitted a v2 of the patch series.

Can you have another look, please?

Thanks, Roman.

Efraim Flashner <efraim@flashner.co.il> writes:

Toggle quote (36 lines)
> [[PGP Signed Part:Undecided]]
> Since we don't carry the results of one rust package to the next and
> have to rebuild everything, it would be better to add this to
> (guix build cargo-build-system), in the configure phase. Then it will
> apply to every package which is built using the cargo-build-system.
>
> On Fri, Jan 05, 2024 at 01:36:42PM +0100, Roman Scherer wrote:
>> * gnu/packages/crates-io.scm (rust-jemalloc-sys): Build with large page size.
>>
>> Change-Id: I91ed8450952204c1ecba19604521dd8b8ec554ec
>> ---
>> gnu/packages/crates-io.scm | 6 +++++-
>> 1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
>> index 1bfd8fb143..f8eb0c293f 100644
>> --- a/gnu/packages/crates-io.scm
>> +++ b/gnu/packages/crates-io.scm
>> @@ -39286,7 +39286,11 @@ (define-public rust-jemalloc-sys-0.5
>> ;; https://github.com/tikv/jemallocator/issues/19
>> (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1")
>> (setenv "JEMALLOC_OVERRIDE"
>> - (string-append jemalloc "/lib/libjemalloc_pic.a"))))))))
>> + (string-append jemalloc "/lib/libjemalloc_pic.a")))))
>> + (add-after 'configure 'with-lg-page-jemalloc
>> + (lambda _
>> + (when (target-aarch64?)
>> + (setenv "JEMALLOC_SYS_WITH_LG_PAGE" "14")))))))
>> (native-inputs
>> (list jemalloc))
>> (home-page "https://github.com/tikv/jemallocator")
>> --
>> 2.41.0
>>
>>
>>
-----BEGIN PGP SIGNATURE-----

iQFTBAEBCAA9FiEE0iajOdjfRIFd3gygPdpSUn0qwZkFAmWYBREfHHJvbWFuLnNj
aGVyZXJAYnVybmluZ3N3ZWxsLmNvbQAKCRA92lJSfSrBmTdiB/9hjLzx5cMb0cfB
7qWpgI7r9dgL5cnkdVAGKiWQOgQc2rNVNTyuaDR2GO7zeWUKuE8fodWlnc0ySpfJ
d9LjYSDWDH9Yw4b7oYVaexHhqwb6RGflnyu042hUuRLwh+uG1n+o2k1IcFm5J+FG
PolUS8VNs1S/tji/zD/czeivVRyvMKWG3ap6DNo5cPWBMuGXCpDOQe28Ld6JF8Hq
pA0Q4o/7pAuhTuOrGNHMTPpEOhfczZbxlfVkQ/Y1SROh+2Og81Lxmokhv7xobWbX
Wx18QZtYONgST0+22+gH/LUmM1zo9ww/MEsLn1poU9iJFhsKOrlwEMaohUctqUnM
ZVqqdAXA
=0pYV
-----END PGP SIGNATURE-----

?