libglvnd support in mesa

  • Open
  • quality assurance status badge
Details
2 participants
  • aurtzy
  • The Man
Owner
unassigned
Submitted by
aurtzy
Severity
normal
A
A
aurtzy wrote on 6 Oct 22:31 +0200
(address . guix-patches@gnu.org)
726317c4-cc33-42be-af55-69bea3010a68@gmail.com
Hi!

I've been looking further into adding libglvnd support to mesa, following recent
discussions on mesa updates [1]. There is a reportedly working version by The
Man [2] which does so by unionizing libglvnd and mesa files, but with the
suggestion to try a libglvnd-as(-propagated)-input approach first, the rest of
this message will focus on working towards that. Relevant parties from the
previous discussion have been CCed.

Context as I understand it: Adding libglvnd as an input to mesa causes mesa to
build without some/all lib*.so files that it usually has because libglvnd
becomes the package that will have them. This means that if a package with mesa
as a dependency employs any code that assumes libraries in mesa
(e.g. =(string-append (assoc-ref inputs "mesa") "/lib/libGL.so")=), the build
fails. This appears to be the main issue.

I devised a script to find all the packages that probably need fixing. It is
included at the end of this message. I cross-checked its coverage with
consult-grep from emacs-consult (search phrase: =mesa lib=), which doesn't seem
to reveal any other packages that would need changes, but I'm not sure if
there's a better way to validate this without going through what seems like a
/lot/ of packages with mesa as a transitive input.

The script outputs the following packages that may need to be changed:

Toggle snippet (15 lines)
gnu/packages/virtualization.scm:2133:2 "looking-glass-client"
gnu/packages/video.scm:4030:2 "obs"
gnu/packages/tor-browsers.scm:203:2 "torbrowser"
gnu/packages/qt.scm:450:2 "qtbase"
gnu/packages/python-xyz.scm:25513:4 "python-glcontext"
gnu/packages/python-xyz.scm:25441:2 "python-pyopengl"
gnu/packages/perl.scm:8504:2 "perl-opengl"
gnu/packages/music.scm:1122:2 "extempore"
gnu/packages/librewolf.scm:218:2 "librewolf"
gnu/packages/gnuzilla.scm:709:2 "icecat-minimal"
gnu/packages/gl.scm:746:2 "guile-opengl"
gnu/packages/gl.scm:1207:2 "glmark2"
gnu/packages/chromium.scm:484:2 "ungoogled-chromium"

Should we consider adjusting variable names as well? I have noticed GL
libraries are sometimes assigned to e.g. "mesa-lib", but they will no longer be
part of mesa if libglvnd support is enabled (an alternative name could be
"gl-lib"). While this is more "cosmetic" and applies to a larger number of
packages, I wonder if it might be a point of confusion for readers. See
libepoxy and its use of "mesa-lib" for an example.

If no one's already started work or wants to take point on this, I should be
able to make some time these next few weeks to write patches :)

The mentioned script:

Toggle snippet (92 lines)
#!/usr/bin/env -S guix repl --
!#

(use-modules (gnu)
(guix)
(guix diagnostics)
(guix records)
(ice-9 match)
(srfi srfi-1)
(srfi srfi-26))

(define (sexp-contains sexp predicate)
"Call (PREDICATE S-EXPRESSION) on every node of SEXP (recursively), and return
the predicate value on the first instance that it is non-false. If none of the
nodes satisfy the predicate, return false."
(define (%sexp-contains sexp rest-sexps)
(match sexp
((= predicate (and (not #f) return-value))
return-value)
((child rest-children ...)
(%sexp-contains child (append rest-children rest-sexps)))
(else
(match rest-sexps
((next-sexp rest-sexps ...)
(%sexp-contains next-sexp rest-sexps))
(else
#f)))))
(%sexp-contains sexp '()))

(define (sexp-at-location filename line column)
"Return the next s-expression after LINE and COLUMN (both one-indexed) in
FILENAME."
(call-with-input-file filename
(lambda (port)
(go-to-location port line column)
(read port))))

(define (relevant-package? package)
"Return whether PACKAGE is a potentially relevant package."
(->bool
(and (any (match-lambda
(("mesa" _ ...) #t)
((? package? (= package-name "mesa")) #t)
(else #f))
(package-transitive-inputs package))
(and=> (sexp-contains (match (package-location package)
(($ <location> file line column)
(sexp-at-location file line (1+ column))))
(lambda (sexp)
(and (list? sexp)
(match (memq #:phases sexp)
((#:phases phases _ ...) phases)
(else #f)))))
(lambda (phases-sexp)
;; Use the following conditions to determine if the
;; package does stuff with mesa libraries.
(and (sexp-contains phases-sexp
(lambda (sexp)
(and (string? sexp)
(string-contains sexp "lib"))))
(or (sexp-contains phases-sexp
(lambda (sexp)
(and (symbol? sexp)
(eq? sexp 'mesa))))
(sexp-contains phases-sexp
(lambda (sexp)
(and (string? sexp)
(string-contains sexp
"mesa")))))))))))

(define (main . _)
"Run this in the top level of the guix repository with \"./pre-inst-env\"."
(format #t "Relevant packages:\n\n")
(for-each
(lambda (package)
(match (package-location package)
(($ <location> file line column)
(format #t "~a:~a:~a ~s\n"
file
line
column
(package-name package)))))
(fold-packages
(lambda (package result)
(if (relevant-package? package)
(cons package result)
result))
'())))

(main)

T
T
The Man wrote on 16 Oct 05:29 +0200
(name . aurtzy)(address . aurtzy@gmail.com)
CAPfwWajVcqgUf8BfQrWOTwiTR7qNaZxu2aMfryED6orM6jO_Rg@mail.gmail.com
Hey,

looking into this,
has the following:
"libGL is loaded impurely in Nix. That is, NixOS sets LD_LIBRARY_PATH..."
This is backed up by https://nixos.wiki/wiki/OpenGLsuggesting the use
of it as well.
There is a little discussion as well in
possibly being usable. it's currently under active development, but
previous attempts to use it by other nix people seem inconclusive.

In my own small amount of effort on going down this path, doing the
same as nix and setting LD_LIBRARY_PATH within mesa's search paths
yielded promising results, otherwise I was seeing test failures in
relation to opengl.

what I have is a little ugly but i think it's a half decent start;

Toggle diff (390 lines)
diff --git a/gnu/packages/chromium.scm b/gnu/packages/chromium.scm
index cf1703cecb..9e58ea5e56 100644
--- a/gnu/packages/chromium.scm
+++ b/gnu/packages/chromium.scm
@@ -796,7 +796,7 @@ (define-public ungoogled-chromium
"chromium/master-preferences.json")))
(gtk (dirname (dirname
(search-input-file inputs
"lib/libgtk-3.so"))))
- (mesa (dirname (search-input-file inputs "lib/libGL.so")))
+ (libglvnd (dirname (search-input-file inputs
"lib/libGL.so")))
(vulkan (dirname (search-input-file inputs
"lib/libvulkan.so")))
(xdg-utils (dirname (search-input-file inputs
"bin/xdg-open"))))

@@ -838,7 +838,7 @@ (define-public ungoogled-chromium
;; Provide libGL and libvulkan without patching
all references.
;; XXX: How to add on RUNPATH instead of this hack?
`("LD_LIBRARY_PATH" ":" prefix
- (,(string-append mesa ":" vulkan)))
+ (,(string-append libglvnd ":" vulkan)))
;; Ensure xdg-open et al. is found.
`("PATH" ":" prefix (,xdg-utils))))

diff --git a/gnu/packages/gl.scm b/gnu/packages/gl.scm
index 21be697d3b..7b65c77543 100644
--- a/gnu/packages/gl.scm
+++ b/gnu/packages/gl.scm
@@ -314,6 +314,7 @@ (define-public mesa
(propagated-inputs
;; The following are in the Requires.private field of gl.pc.
(list libdrm
+ libglvnd
libvdpau
libx11
libxdamage
@@ -382,6 +383,8 @@ (define-public mesa
;; "-Domx=true"
"-Dosmesa=true"
"-Dgallium-xa=enabled"
+ ;; libglvnd
+ "-Dglvnd=true"

;; features required by wayland
"-Dgles2=enabled"
@@ -572,13 +575,41 @@ (define-public mesa
(((string-append "\"lib" layer-name ".so\""))
(string-append "\"" out "/lib/lib"
layer-name ".so\"")))))))
(for-each fix-layer-path '("VkLayer_MESA_device_select"
- "VkLayer_MESA_overlay"))))))))
+ "VkLayer_MESA_overlay")))))
+ (add-after 'install 'fix-paths
+ (lambda _
+ (substitute*
+ (string-append #$output
+ "/share/glvnd/egl_vendor.d/50_mesa.json")
+ (("libEGL_mesa")
+ (string-append #$output "/lib/libEGL_mesa")))))
+ (add-after 'fix-paths 'add-external-egl
+ (lambda _
+ (mkdir-p (string-append #$output
+ "/share/egl/egl_external_platform.d")))))))
(native-search-paths
(list (search-path-specification
;; Ensure the Mesa VDPAU drivers can be found.
(variable "VDPAU_DRIVER_PATH")
(separator #f)
- (files '("lib/vdpau")))))
+ (files '("lib/vdpau")))
+ (search-path-specification
+ (variable "__EGL_VENDOR_LIBRARY_DIRS")
+ (files '("share/glvnd/egl_vendor.d")))
+ (search-path-specification
+ (variable "GBM_BACKENDS_PATH")
+ (files '("lib")))
+ (search-path-specification
+ (variable "XDG_DATA_DIRS")
+ (files '("share")))
+ (search-path-specification
+ (variable "__EGL_EXTERNAL_PLATFORM_CONFIG_DIRS")
+ (files '("share/egl/egl_external_platform.d")))
+ ;; FIXME: find a way to get opengl to work properly
+ ;; without needing this when running tests or from guix-shell
+ (search-path-specification
+ (variable "LD_LIBRARY_PATH")
+ (files '("lib")))))
(home-page "https://mesa3d.org/")
(synopsis "OpenGL and Vulkan implementations")
(description "Mesa is a free implementation of the OpenGL and Vulkan
@@ -763,7 +794,7 @@ (define-public guile-opengl
(substitute* "gl/runtime.scm"
(("\\(dynamic-link\\)")
(string-append "(dynamic-link \""
- (assoc-ref inputs "mesa")
+ (assoc-ref inputs "libglvnd")
"/lib/libGL.so" "\")")))
(define (dynamic-link-substitute file lib input)
(substitute* file
@@ -773,7 +804,7 @@ (define (dynamic-link-substitute file lib input)
"/lib/lib" lib "\""))))
;; Replace dynamic-link calls for libGL, libGLU, and
;; libglut with absolute paths to the store.
- (dynamic-link-substitute "glx/runtime.scm" "GL" "mesa")
+ (dynamic-link-substitute "glx/runtime.scm" "GL"
"libglvnd")
(dynamic-link-substitute "glu/runtime.scm" "GLU" "glu")
(dynamic-link-substitute "glut/runtime.scm" "glut"
"freeglut"))))))
@@ -809,14 +840,14 @@ (define-public libepoxy
#~(modify-phases %standard-phases
(add-before 'configure 'patch-paths
(lambda* (#:key inputs #:allow-other-keys)
- (let ((mesa-lib
+ (let ((gl-lib
(lambda (file)
(search-input-file inputs (string-append
"lib/" file)))))
(substitute* (find-files "." "\\.[ch]$")
- (("libGL.so.1") (mesa-lib "libGL.so.1"))
- (("libEGL.so.1") (mesa-lib "libEGL.so.1"))
- (("libGLESv1_CM.so.1") (mesa-lib "libGLESv1_CM.so.1"))
- (("libGLESv2.so.2") (mesa-lib "libGLESv2.so.2")))))))))
+ (("libGL.so.1") (gl-lib "libGL.so.1"))
+ (("libEGL.so.1") (gl-lib "libEGL.so.1"))
+ (("libGLESv1_CM.so.1") (gl-lib "libGLESv1_CM.so.1"))
+ (("libGLESv2.so.2") (gl-lib "libGLESv2.so.2")))))))))
(build-system meson-build-system)
(native-inputs
(list pkg-config python))
@@ -1229,11 +1260,11 @@ (define-public glmark2
(modify-phases %standard-phases
(add-after 'unpack 'patch-paths
(lambda* (#:key inputs #:allow-other-keys)
- (let ((mesa (assoc-ref inputs "mesa")))
+ (let ((gl-lib (assoc-ref inputs "libglvnd")))
(substitute* (find-files "src" "gl-state-.*\\.cpp$")
- (("libGL.so") (string-append mesa "/lib/libGL.so"))
- (("libEGL.so") (string-append mesa "/lib/libEGL.so"))
- (("libGLESv2.so") (string-append mesa "/lib/libGLESv2.so")))
+ (("libGL.so") (string-append gl-lib "/lib/libGL.so"))
+ (("libEGL.so") (string-append gl-lib "/lib/libEGL.so"))
+ (("libGLESv2.so") (string-append gl-lib "/lib/libGLESv2.so")))
#t))))))
(native-inputs
(list pkg-config))
diff --git a/gnu/packages/gnome.scm b/gnu/packages/gnome.scm
index cf3d1a1496..ce8629f86a 100644
--- a/gnu/packages/gnome.scm
+++ b/gnu/packages/gnome.scm
@@ -6316,7 +6316,7 @@ (define-public cogl
;; Arrange to pass an absolute file name to
;; dlopen for libGL.so.
(string-append "--with-gl-libname="
- (assoc-ref %build-inputs "mesa")
+ (assoc-ref
%build-inputs "libglvnd")
"/lib/libGL.so"))
#:phases
(modify-phases %standard-phases
diff --git a/gnu/packages/gnuzilla.scm b/gnu/packages/gnuzilla.scm
index 6ba27fac1f..297b12b224 100644
--- a/gnu/packages/gnuzilla.scm
+++ b/gnu/packages/gnuzilla.scm
@@ -748,6 +748,7 @@ (define-public icecat-minimal
pixman
pulseaudio
mesa
+ libglvnd ;; error without
pciutils
mit-krb5
hunspell
@@ -1090,7 +1091,7 @@ (define (runpaths-of-input label)
(file-append
(this-package-input label) "/lib"))
'("libpng-apng"
"libxscrnsaver"
- "mesa"
+ "libglvnd"
"pciutils"
"mit-krb5"
"eudev"
diff --git a/gnu/packages/librewolf.scm b/gnu/packages/librewolf.scm
index d696a3058f..9b8f49de33 100644
--- a/gnu/packages/librewolf.scm
+++ b/gnu/packages/librewolf.scm
@@ -585,7 +585,7 @@ (define (runpaths-of-input label)
(string-append (assoc-ref inputs
lib-name)
"/lib"))
- '("mesa" "libpng-apng" "libnotify" "libva"
+ '("libglvnd" "libpng-apng"
"libnotify" "libva"
"pulseaudio" "gtk+" "pipewire"
;; For U2F and WebAuthn
"eudev")))
@@ -602,7 +602,7 @@ (define (runpaths-of-input label)
(rdd-whitelist (map (cut string-append <> "/")
(delete-duplicates
(append-map

runpaths-of-input
- '("mesa"
+
'("libglvnd"

"ffmpeg")))))
(gtk-share (string-append (assoc-ref inputs
"gtk+")
diff --git a/gnu/packages/music.scm b/gnu/packages/music.scm
index d649c0df9e..c2b11a20d3 100644
--- a/gnu/packages/music.scm
+++ b/gnu/packages/music.scm
@@ -1206,10 +1206,10 @@ (define-public extempore
("fft" "libkiss_fft.so" "kiss-fft")
("stb_image" "libstb_image.so" "stb-image")
("nanovg" "libnanovg.so" "nanovg")
- ("glext" "libGL.so" "mesa")
+ ("glext" "libGL.so" "libglvnd")
("glfw3" "libglfw.so" "glfw")
- ("gl/glcore-directbind" "libGL.so" "mesa")
- ("gl/glcompat-directbind" "libGL.so" "mesa")))
+ ("gl/glcore-directbind" "libGL.so" "libglvnd")
+ ("gl/glcompat-directbind" "libGL.so" "libglvnd")))
#t))
(add-after 'unpack 'use-own-llvm
(lambda* (#:key inputs #:allow-other-keys)
diff --git a/gnu/packages/perl.scm b/gnu/packages/perl.scm
index 31576dedf9..3f47cba2af 100644
--- a/gnu/packages/perl.scm
+++ b/gnu/packages/perl.scm
@@ -8547,7 +8547,7 @@ (define-public perl-opengl
(("-L/usr/local/freeglut/lib")
(string-append "-L" (assoc-ref inputs "freeglut") "/lib\n"
"-L" (assoc-ref inputs "glu") "/lib\n"
- "-L" (assoc-ref inputs "mesa") "/lib\n")))
+ "-L" (assoc-ref inputs "libglvnd") "/lib\n")))
#t)))))
(home-page "https://metacpan.org/release/OpenGL")
(synopsis
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 8c7e1596e6..2a2f0084af 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -25524,15 +25524,15 @@ (define-public python-pyopengl
(("filenames_to_try = \\[\\]") "filenames_to_try = [name]"))
(substitute* '("OpenGL/platform/glx.py"
"tests/check_glut_load.py")
(("'GL'")
- (string-append "'" (assoc-ref inputs "mesa") "/lib/libGL.so'"))
+ (string-append "'" (assoc-ref inputs "libglvnd")
"/lib/libGL.so'"))
(("'GLU'")
(string-append "'" (assoc-ref inputs "glu") "/lib/libGLU.so'"))
(("'glut',")
(string-append "'" (assoc-ref inputs "freeglut")
"/lib/libglut.so',"))
(("'GLESv1_CM'")
- (string-append "'" (assoc-ref inputs "mesa")
"/lib/libGLESv1_CM.so'"))
+ (string-append "'" (assoc-ref inputs "libglvnd")
"/lib/libGLESv1_CM.so'"))
(("'GLESv2'")
- (string-append "'" (assoc-ref inputs "mesa")
"/lib/libGLESv2.so'")))
+ (string-append "'" (assoc-ref inputs "libglvnd")
"/lib/libGLESv2.so'")))
;; Not providing libgle. It seems to be very old.
#t)))))
(home-page "https://pyopengl.sourceforge.net")
@@ -25589,21 +25589,21 @@ (define-public python-glcontext
(list #:phases #~(modify-phases %standard-phases
(add-before 'build 'fix-lib-paths
(lambda* (#:key inputs outputs #:allow-other-keys)
- (let ((mesa (assoc-ref inputs "mesa"))
+ (let ((gl-lib (assoc-ref inputs "libglvnd"))
(libx11 (assoc-ref inputs "libx11")))
(substitute* '("glcontext/x11.cpp"
"glcontext/egl.cpp")
(("\"libGL.so\"")
- (string-append "\"" mesa "/lib/libGL.so\""))
+ (string-append "\"" gl-lib
"/lib/libGL.so\""))
(("\"libEGL.so\"")
- (string-append "\"" mesa
"/lib/libEGL.so\""))
+ (string-append "\"" gl-lib
"/lib/libEGL.so\""))
(("\"libX11.so\"")
(string-append "\"" libx11
"/lib/libX11.so\"")))
(substitute* '("glcontext/__init__.py")
(("find_library\\('GL'\\)")
- (string-append "'" mesa "/lib/libGL.so'"))
+ (string-append "'" gl-lib "/lib/libGL.so'"))
(("find_library\\('EGL'\\)")
- (string-append "'" mesa "/lib/libEGL.so'"))
+ (string-append "'" gl-lib
"/lib/libEGL.so'"))
(("find_library\\(\"X11\"\\)")
(string-append "'" libx11
"/lib/libX11.so'"))))))
(replace 'check
diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
index 687c20cd90..994383edfd 100644
--- a/gnu/packages/qt.scm
+++ b/gnu/packages/qt.scm
@@ -674,7 +674,7 @@ (define-public qtbase-5
;; libGL
(substitute*
"src/plugins/platforms/xcb/gl_integrations/xcb_glx/qglxintegration.cpp"
(("^\\s*(QLibrary lib\\(QLatin1String\\(\")(GL\"\\)\\);)" _ a b)
- (string-append a (assoc-ref inputs "mesa") "/lib/lib" b)))
+ (string-append a (assoc-ref inputs "libglvnd") "/lib/lib" b)))
;; libXcursor
(substitute* "src/plugins/platforms/xcb/qxcbcursor.cpp"
(("^\\s*(QLibrary
xcursorLib\\(QLatin1String\\(\")(Xcursor\"\\), 1\\);)" _ a b)
diff --git a/gnu/packages/tor-browsers.scm b/gnu/packages/tor-browsers.scm
index 6bc1ef5328..d065becb69 100644
--- a/gnu/packages/tor-browsers.scm
+++ b/gnu/packages/tor-browsers.scm
@@ -661,7 +661,7 @@ (define (runpaths-of-input label)
(file-append
(this-package-input label) "/lib"))
'("libpng-apng"
"libxscrnsaver"
- "mesa"
+ "libglvnd"
"pciutils"
"mit-krb5"
"eudev"
diff --git a/gnu/packages/video.scm b/gnu/packages/video.scm
index 1d2ab5fbeb..321ec16a5c 100644
--- a/gnu/packages/video.scm
+++ b/gnu/packages/video.scm
@@ -4065,8 +4065,9 @@ (define-public obs
(,(string-append #$(this-package-input "vlc")
"/lib")
;; TODO: Remove this once our mesa has glvnd support.
- ,(string-append #$(this-package-input "mesa")
- "/lib"))))))))))
+;; ,(string-append #$(this-package-input "mesa")
+;; "/lib")
+ )))))))))
(native-search-paths
(list (search-path-specification
(variable "OBS_PLUGINS_DIRECTORY")
diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.scm
index bfca5885c6..01c731352a 100644
--- a/gnu/packages/virtualization.scm
+++ b/gnu/packages/virtualization.scm
@@ -2206,7 +2206,7 @@ (define-public looking-glass-client
(string-append input "/lib")))
'("gmp" "libxi"
"nettle"
- "mesa"
+ "libglvnd"
"wayland"
"fontconfig-minimal"
"freetype"

On Sun, Oct 6, 2024 at 3:31?PM aurtzy <aurtzy@gmail.com> wrote:
>
> Hi!
>
> I've been looking further into adding libglvnd support to mesa, following recent
> discussions on mesa updates [1]. There is a reportedly working version by The
> Man [2] which does so by unionizing libglvnd and mesa files, but with the
> suggestion to try a libglvnd-as(-propagated)-input approach first, the rest of
> this message will focus on working towards that. Relevant parties from the
> previous discussion have been CCed.
>
> Context as I understand it: Adding libglvnd as an input to mesa causes mesa to
> build without some/all lib*.so files that it usually has because libglvnd
> becomes the package that will have them. This means that if a package with mesa
> as a dependency employs any code that assumes libraries in mesa
> (e.g. =(string-append (assoc-ref inputs "mesa") "/lib/libGL.so")=), the build
> fails. This appears to be the main issue.
>
> I devised a script to find all the packages that probably need fixing. It is
> included at the end of this message. I cross-checked its coverage with
> consult-grep from emacs-consult (search phrase: =mesa lib=), which doesn't seem
> to reveal any other packages that would need changes, but I'm not sure if
> there's a better way to validate this without going through what seems like a
> /lot/ of packages with mesa as a transitive input.
>
This message was truncated. Download the full message here.
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 73668@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 73668
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch