[PATCH 00/11] AMD ROCm OpenCL runtime

DoneSubmitted by Lars-Dominik Braun.
Details
One participant
  • Lars-Dominik Braun
Owner
unassigned
Severity
normal
L
L
Lars-Dominik Braun wrote on 6 Aug 09:06 +0200
(address . guix-patches@gnu.org)
YQzfkzqmdBHHrs+u@noor.fritz.box
Hi,
this patchset adds AMD’s ROCm OpenCL runtime for their GPU’s. It worksfine with Linux’ standard kernel module (no need to package ROCk), butonly if firmware has been loaded. The userspace software itself iscompletely free though.
The final patch adjusts one consumer of ROCm OpenCL, darktable, so bothactually work together. Try it yourself with (obviously needs an AMDGPU):
guix environment --ad-hoc darktable ocl-icd rocm-opencl-runtime -- darktable-cltest
Lars-Dominik Braun (11): gnu: Add rocm-cmake. gnu: Add llvm-for-rocm. gnu: Add rocm-device-libs. gnu: Add rocm-comgr. gnu: Add roct-thunk-interface. gnu: Add rocr-runtime. gnu: Add rocclr. gnu: Add rocm-opencl-runtime. gnu: Add rocminfo. gnu: Add rocm-bandwidth-test. gnu: darktable: Add compatibility for ROCm.
gnu/local.mk | 8 + .../llvm-roc-3.0.0-add_libraries.patch | 22 ++ ...roc-4.0.0-remove-isystem-usr-include.patch | 29 ++ .../patches/llvm-roc-4.2.0-add_Object.patch | 13 + .../rocm-comgr-3.1.0-dependencies.patch | 52 +++ ....10.0-add-rocclr-include-directories.patch | 14 + .../rocm-opencl-runtime-4.3-noclinfo.patch | 28 ++ .../rocm-opencl-runtime-4.3-nocltrace.patch | 25 ++ .../rocm-opencl-runtime-4.3-noopencl.patch | 63 ++++ gnu/packages/photo.scm | 24 +- gnu/packages/rocm.scm | 355 ++++++++++++++++++ 11 files changed, 628 insertions(+), 5 deletions(-) create mode 100644 gnu/packages/patches/llvm-roc-3.0.0-add_libraries.patch create mode 100644 gnu/packages/patches/llvm-roc-4.0.0-remove-isystem-usr-include.patch create mode 100644 gnu/packages/patches/llvm-roc-4.2.0-add_Object.patch create mode 100644 gnu/packages/patches/rocm-comgr-3.1.0-dependencies.patch create mode 100644 gnu/packages/patches/rocm-opencl-runtime-3.10.0-add-rocclr-include-directories.patch create mode 100644 gnu/packages/patches/rocm-opencl-runtime-4.3-noclinfo.patch create mode 100644 gnu/packages/patches/rocm-opencl-runtime-4.3-nocltrace.patch create mode 100644 gnu/packages/patches/rocm-opencl-runtime-4.3-noopencl.patch create mode 100644 gnu/packages/rocm.scm
-- 2.31.1
L
L
Lars-Dominik Braun wrote on 6 Aug 09:11 +0200
Re: bug#49907: Acknowledgement ([PATCH 00/11] AMD ROCm OpenCL runtime)
(address . 49907@debbugs.gnu.org)
YQzgijKaVsWcCvQv@noor.fritz.box
Forgot to attach the actual patches, sorry.
From 4e4ca903ff7b94fceba9ba85c85f6d267baf71fc Mon Sep 17 00:00:00 2001From: Lars-Dominik Braun <lars@6xq.net>Date: Thu, 5 Aug 2021 15:33:47 +0200Subject: [PATCH 01/11] gnu: Add rocm-cmake.
* gnu/packages/rocm.scm (%rocm-version, rocm-cmake): New variables.--- gnu/packages/rocm.scm | 57 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 gnu/packages/rocm.scm
Toggle diff (65 lines)diff --git a/gnu/packages/rocm.scm b/gnu/packages/rocm.scmnew file mode 100644index 0000000000..b04eb8050e--- /dev/null+++ b/gnu/packages/rocm.scm@@ -0,0 +1,57 @@+;;;+;;; Copyright � 2021 Lars-Dominik Braun <lars@6xq.net>+;;;+;;; This program is free software; you can redistribute it and/or modify it+;;; under the terms of the GNU General Public License as published by+;;; the Free Software Foundation; either version 3 of the License, or (at+;;; your option) any later version.+;;;+;;; This program is distributed in the hope that it will be useful, but+;;; WITHOUT ANY WARRANTY; without even the implied warranty of+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+;;; GNU General Public License for more details.+;;;+;;; You should have received a copy of the GNU General Public License+;;; along with this program. If not, see <http://www.gnu.org/licenses/>.++(define-module (gnu packages rocm)+ #:use-module ((guix licenses) #:prefix license:)+ #:use-module (guix packages)+ #:use-module (guix download)+ #:use-module (guix utils)+ #:use-module (guix git-download)+ #:use-module (guix build-system cmake)+ #:use-module (gnu packages)+ #:use-module (gnu packages elf)+ #:use-module (gnu packages gl)+ #:use-module (gnu packages linux)+ #:use-module (gnu packages llvm)+ #:use-module (gnu packages opencl)+ #:use-module (gnu packages version-control)+ #:use-module (gnu packages vim))++;; The components are tightly integrated and can only be upgraded as a unit. If+;; you want to upgrade ROCm, bump this version number and update hashes below.+(define %rocm-version "4.3.0")++(define-public rocm-cmake+ (package+ (name "rocm-cmake")+ (version %rocm-version)+ (source (origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/RadeonOpenCompute/rocm-cmake.git")+ (commit (string-append "rocm-" version))))+ (file-name (git-file-name name version))+ (sha256+ (base32+ "0sic2zxmzl2pb2865vvq55mbpcr8pby8v19pjdlm08pypqw5h6h6"))))+ (build-system cmake-build-system)+ (arguments `(#:tests? #f)) ; Tests try to use git commit+ (native-inputs `(("git" ,git)))+ (home-page "https://github.com/RadeonOpenCompute/rocm-cmake")+ (synopsis "ROCm cmake modules")+ (description "ROCm cmake modules provides cmake modules for common build+tasks needed for the ROCM software stack.")+ (license license:ncsa)))-- 2.31.1
From e42d64d84a5901ab9177aab6d4dd02256fb95f63 Mon Sep 17 00:00:00 2001From: Lars-Dominik Braun <lars@6xq.net>Date: Fri, 6 Aug 2021 08:18:26 +0200Subject: [PATCH 02/11] gnu: Add llvm-for-rocm.
* gnu/packages/rocm.scm (llvm-for-rocm): New variable.* gnu/packages/patches/llvm-roc-3.0.0-add_libraries.patch,gnu/packages/patches/llvm-roc-4.0.0-remove-isystem-usr-include.patch,gnu/packages/patches/llvm-roc-4.2.0-add_Object.patch: New files.* gnu/local.mk (dist_patch_DATA): Add them.--- gnu/local.mk | 3 ++ .../llvm-roc-3.0.0-add_libraries.patch | 22 ++++++++++++ ...roc-4.0.0-remove-isystem-usr-include.patch | 29 +++++++++++++++ .../patches/llvm-roc-4.2.0-add_Object.patch | 13 +++++++ gnu/packages/rocm.scm | 35 +++++++++++++++++++ 5 files changed, 102 insertions(+) create mode 100644 gnu/packages/patches/llvm-roc-3.0.0-add_libraries.patch create mode 100644 gnu/packages/patches/llvm-roc-4.0.0-remove-isystem-usr-include.patch create mode 100644 gnu/packages/patches/llvm-roc-4.2.0-add_Object.patch
Toggle diff (141 lines)diff --git a/gnu/local.mk b/gnu/local.mkindex e8494806fd..ac9f1d6154 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -1414,6 +1414,9 @@ dist_patch_DATA = \ %D%/packages/patches/llvm-9-fix-bitcast-miscompilation.patch \ %D%/packages/patches/llvm-9-fix-lpad-miscompilation.patch \ %D%/packages/patches/llvm-9-fix-scev-miscompilation.patch \+ %D%/packages/patches/llvm-roc-3.0.0-add_libraries.patch \+ %D%/packages/patches/llvm-roc-4.0.0-remove-isystem-usr-include.patch \+ %D%/packages/patches/llvm-roc-4.2.0-add_Object.patch \ %D%/packages/patches/lm-sensors-hwmon-attrs.patch \ %D%/packages/patches/lrcalc-includes.patch \ %D%/packages/patches/lsh-fix-x11-forwarding.patch \diff --git a/gnu/packages/patches/llvm-roc-3.0.0-add_libraries.patch b/gnu/packages/patches/llvm-roc-3.0.0-add_libraries.patchnew file mode 100644index 0000000000..f6bc639f33--- /dev/null+++ b/gnu/packages/patches/llvm-roc-3.0.0-add_libraries.patch@@ -0,0 +1,22 @@+Taken from https://gitweb.gentoo.org/repo/gentoo.git/tree/sys-devel/llvm-roc/files++diff -Naur a/clang/lib/Basic/CMakeLists.txt b/clang/lib/Basic/CMakeLists.txt+--- a/clang/lib/Basic/CMakeLists.txt 2019-12-28 07:42:49.119055933 +0100++++ b/clang/lib/Basic/CMakeLists.txt 2019-12-28 07:42:13.265056070 +0100+@@ -2,6 +2,7 @@+ Core+ MC+ Support++ Option+ )+ + find_first_existing_vc_file("${LLVM_MAIN_SRC_DIR}" llvm_vc)+diff -Naur a/clang/lib/Driver/CMakeLists.txt b/clang/lib/Driver/CMakeLists.txt+--- a/clang/lib/Driver/CMakeLists.txt 2019-12-28 07:41:39.521056199 +0100++++ b/clang/lib/Driver/CMakeLists.txt 2019-12-28 07:40:23.998056487 +0100+@@ -79,4 +79,5 @@+ LINK_LIBS+ clangBasic+ ${system_libs}++ pthread+ )diff --git a/gnu/packages/patches/llvm-roc-4.0.0-remove-isystem-usr-include.patch b/gnu/packages/patches/llvm-roc-4.0.0-remove-isystem-usr-include.patchnew file mode 100644index 0000000000..f14ec4ac0d--- /dev/null+++ b/gnu/packages/patches/llvm-roc-4.0.0-remove-isystem-usr-include.patch@@ -0,0 +1,29 @@+Author: Wilfried (justxi) Holzke++Adopted from https://github.com/justxi/rocm/blob/master/sys-devel/llvm-roc/files/llvm-roc-4.0.0-remove-isystem-usr-include.patch++Index: llvm-project-rocm-4.0.0/clang/lib/Driver/ToolChains/AMDGPU.cpp+===================================================================+--- llvm-project-rocm-4.0.0.orig/clang/lib/Driver/ToolChains/AMDGPU.cpp++++ llvm-project-rocm-4.0.0/clang/lib/Driver/ToolChains/AMDGPU.cpp+@@ -326,11 +326,6 @@ void RocmInstallationDetector::AddHIPInc+ //+ // ROCm 3.5 does not fully support the wrapper headers. Therefore it needs+ // a workaround.+- SmallString<128> P(D.ResourceDir);+- if (UsesRuntimeWrapper)+- llvm::sys::path::append(P, "include", "cuda_wrappers");+- CC1Args.push_back("-internal-isystem");+- CC1Args.push_back(DriverArgs.MakeArgString(P));+ }+ + if (DriverArgs.hasArg(options::OPT_nogpuinc))+@@ -341,8 +336,6 @@ void RocmInstallationDetector::AddHIPInc+ return;+ }+ +- CC1Args.push_back("-internal-isystem");+- CC1Args.push_back(DriverArgs.MakeArgString(getIncludePath()));+ if (UsesRuntimeWrapper)+ CC1Args.append({"-include", "__clang_hip_runtime_wrapper.h"});+ }diff --git a/gnu/packages/patches/llvm-roc-4.2.0-add_Object.patch b/gnu/packages/patches/llvm-roc-4.2.0-add_Object.patchnew file mode 100644index 0000000000..f1762a558b--- /dev/null+++ b/gnu/packages/patches/llvm-roc-4.2.0-add_Object.patch@@ -0,0 +1,13 @@+Taken from https://gitweb.gentoo.org/repo/gentoo.git/tree/sys-devel/llvm-roc/files++diff --color -uprN orig/lib/Target/AMDGPU/Disassembler/CMakeLists.txt llvm/lib/Target/AMDGPU/Disassembler/CMakeLists.txt+--- a/llvm/lib/Target/AMDGPU/Disassembler/CMakeLists.txt 2021-06-14 11:57:54.222796911 +0800++++ b/llvm/lib/Target/AMDGPU/Disassembler/CMakeLists.txt 2021-06-14 11:58:35.206796875 +0800+@@ -11,6 +11,7 @@ add_llvm_component_library(LLVMAMDGPUDis+ MC+ MCDisassembler+ Support++ Object+ + ADD_TO_COMPONENT+ AMDGPUdiff --git a/gnu/packages/rocm.scm b/gnu/packages/rocm.scmindex b04eb8050e..473fbe21d0 100644--- a/gnu/packages/rocm.scm+++ b/gnu/packages/rocm.scm@@ -55,3 +55,38 @@ (description "ROCm cmake modules provides cmake modules for common build tasks needed for the ROCM software stack.") (license license:ncsa)))++(define-public llvm-for-rocm+ (hidden-package+ (package+ ;; Actually based on LLVM 13 as of v4.3, but llvm-12 works just fine.+ (inherit llvm-12)+ (name "llvm-for-rocm")+ (version %rocm-version)+ (source (origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/RadeonOpenCompute/llvm-project.git")+ (commit (string-append "rocm-" version))))+ (file-name (git-file-name name version))+ (sha256+ (base32+ "0p75nr1qpmy6crymdax5hm40wkimman4lnglz4x5cnbiqindya7s"))+ (patches+ (search-patches "llvm-roc-4.2.0-add_Object.patch"+ "llvm-roc-3.0.0-add_libraries.patch"+ "llvm-roc-4.0.0-remove-isystem-usr-include.patch"))))+ (arguments+ (substitute-keyword-arguments (package-arguments llvm-12)+ ((#:phases phases '%standard-phases)+ `(modify-phases ,phases+ (add-after 'unpack 'chdir+ (lambda _+ (chdir "llvm")))))+ ((#:configure-flags flags)+ ''("-DLLVM_ENABLE_PROJECTS=llvm;clang;lld"+ "-DLLVM_TARGETS_TO_BUILD=AMDGPU;X86"+ "-DCMAKE_SKIP_BUILD_RPATH=FALSE"+ "-DCMAKE_BUILD_WITH_INSTALL_RPATH=FALSE"+ "-DBUILD_SHARED_LIBS:BOOL=TRUE"+ "-DLLVM_VERSION_SUFFIX=")))))))-- 2.31.1
From 2dc8ac256a1994e3072b884c616d4aeb5bbdfa28 Mon Sep 17 00:00:00 2001From: Lars-Dominik Braun <lars@6xq.net>Date: Fri, 6 Aug 2021 08:19:34 +0200Subject: [PATCH 03/11] gnu: Add rocm-device-libs.
* gnu/packages/rocm.scm (rocm-device-libs): New variable.--- gnu/packages/rocm.scm | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
Toggle diff (35 lines)diff --git a/gnu/packages/rocm.scm b/gnu/packages/rocm.scmindex 473fbe21d0..14eaed9007 100644--- a/gnu/packages/rocm.scm+++ b/gnu/packages/rocm.scm@@ -90,3 +90,28 @@ tasks needed for the ROCM software stack.") "-DCMAKE_BUILD_WITH_INSTALL_RPATH=FALSE" "-DBUILD_SHARED_LIBS:BOOL=TRUE" "-DLLVM_VERSION_SUFFIX=")))))))++(define-public rocm-device-libs+ (package+ (name "rocm-device-libs")+ (version %rocm-version)+ (source (origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/RadeonOpenCompute/ROCm-Device-Libs.git")+ (commit (string-append "rocm-" version))))+ (file-name (git-file-name name version))+ (sha256+ (base32+ "1f8xsylfajpxqjk6ayjnrry53y8b0a6lh9d72pd41nffxfyzvw3w"))))+ (build-system cmake-build-system)+ (arguments+ `(#:configure-flags+ (list "-DCMAKE_SKIP_BUILD_RPATH=FALSE"+ "-DCMAKE_BUILD_WITH_INSTALL_RPATH=FALSE")))+ (inputs `(("llvm" ,llvm-for-rocm)))+ (home-page "https://github.com/RadeonOpenCompute/ROCm-Device-Libs")+ (synopsis "ROCm Device libraries")+ (description "AMD-specific device-side language runtime libraries, namely+oclc, ocml, ockl, opencl, hip and hc.")+ (license license:ncsa)))-- 2.31.1
From 588e8c6636fe6e9dad2845ff7ffa6a88cfe3bd07 Mon Sep 17 00:00:00 2001From: Lars-Dominik Braun <lars@6xq.net>Date: Fri, 6 Aug 2021 08:21:20 +0200Subject: [PATCH 04/11] gnu: Add rocm-comgr.
* gnu/packages/rocm.scm (rocm-comgr): New variable.* gnu/packages/patches/rocm-comgr-3.1.0-dependencies.patch: New file.* gnu/local.mk (dist_patch_DATA): Add it.--- gnu/local.mk | 1 + .../rocm-comgr-3.1.0-dependencies.patch | 52 +++++++++++++++++++ gnu/packages/rocm.scm | 33 ++++++++++++ 3 files changed, 86 insertions(+) create mode 100644 gnu/packages/patches/rocm-comgr-3.1.0-dependencies.patch
Toggle diff (113 lines)diff --git a/gnu/local.mk b/gnu/local.mkindex ac9f1d6154..db1f8e499d 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -1698,6 +1698,7 @@ dist_patch_DATA = \ %D%/packages/patches/rnp-add-version.cmake.patch \ %D%/packages/patches/rnp-disable-ruby-rnp-tests.patch \ %D%/packages/patches/rnp-unbundle-googletest.patch \+ %D%/packages/patches/rocm-comgr-3.1.0-dependencies.patch \ %D%/packages/patches/ruby-sanitize-system-libxml.patch \ %D%/packages/patches/rust-1.19-mrustc.patch \ %D%/packages/patches/rust-1.25-accept-more-detailed-gdb-lines.patch \diff --git a/gnu/packages/patches/rocm-comgr-3.1.0-dependencies.patch b/gnu/packages/patches/rocm-comgr-3.1.0-dependencies.patchnew file mode 100644index 0000000000..c91d273f92--- /dev/null+++ b/gnu/packages/patches/rocm-comgr-3.1.0-dependencies.patch@@ -0,0 +1,52 @@+https://github.com/RadeonOpenCompute/ROCm-CompilerSupport/pull/25++From c65cba2e73f9118e128b9ab7e655ee0f8a7798e7 Mon Sep 17 00:00:00 2001+From: Craig Andrews <candrews@integralblue.com>+Date: Sun, 1 Mar 2020 19:24:22 -0500+Subject: [PATCH] Link additional required LLVM libraries++Without these additional required dependencies, linking fails with errors such as:+`undefined reference to llvm::errs()'`+---+ CMakeLists.txt | 20 ++++++++++++++++++--+ 1 file changed, 18 insertions(+), 2 deletions(-)++diff --git a/CMakeLists.txt b/CMakeLists.txt+index 1794a07..c7b852a 100644+--- a/lib/comgr/CMakeLists.txt++++ b/lib/comgr/CMakeLists.txt+@@ -207,7 +207,11 @@ install(FILES+ DESTINATION "${AMD_COMGR_PACKAGE_PREFIX}")+ + set(CLANG_LIBS+- clangFrontendTool)++ clangFrontendTool++ clangFrontend++ clangBasic++ clangDriver++ clangSerialization)+ + set(LLD_LIBS+ lldELF+@@ -218,8 +222,20 @@ if (LLVM_LINK_LLVM_DYLIB)+ else()+ llvm_map_components_to_libnames(LLVM_LIBS+ ${LLVM_TARGETS_TO_BUILD}++ Option+ DebugInfoDWARF+- Symbolize)++ Symbolize++ Support++ Object++ BitWriter++ MC++ MCParser++ MCDisassembler++ Core++ IRReader++ CodeGen++ Linker++ BinaryFormat)+ endif()+ + target_link_libraries(amd_comgrdiff --git a/gnu/packages/rocm.scm b/gnu/packages/rocm.scmindex 14eaed9007..92530b5d83 100644--- a/gnu/packages/rocm.scm+++ b/gnu/packages/rocm.scm@@ -115,3 +115,36 @@ tasks needed for the ROCM software stack.") (description "AMD-specific device-side language runtime libraries, namely oclc, ocml, ockl, opencl, hip and hc.") (license license:ncsa)))++(define-public rocm-comgr+ (package+ (name "rocm-comgr")+ (version %rocm-version)+ (source (origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/RadeonOpenCompute/ROCm-CompilerSupport.git")+ (commit (string-append "rocm-" version))))+ (file-name (git-file-name name version))+ (sha256+ (base32+ "0bakbm7shr0l67lph44b5cnc9psd6rivg1mp79qizaawkn380x60"))+ (patches+ (search-patches "rocm-comgr-3.1.0-dependencies.patch"))))+ (build-system cmake-build-system)+ (arguments+ `(#:phases+ (modify-phases %standard-phases+ (add-after 'unpack 'chdir+ (lambda* (#:key inputs native-inputs #:allow-other-keys)+ (chdir "lib/comgr")+ #t)))))+ (inputs+ `(("rocm-device-libs" ,rocm-device-libs)+ ("llvm" ,llvm-for-rocm)+ ("lld" ,lld)))+ (home-page "https://github.com/RadeonOpenCompute/ROCm-CompilerSupport")+ (synopsis "ROCm Code Object Manager")+ (description "The Comgr library provides APIs for compiling and inspecting+AMDGPU code objects.")+ (license license:ncsa)))-- 2.31.1
From a2f768e7d53a58d3d94a072fc87f5516b8c84c86 Mon Sep 17 00:00:00 2001From: Lars-Dominik Braun <lars@6xq.net>Date: Fri, 6 Aug 2021 08:23:53 +0200Subject: [PATCH 05/11] gnu: Add roct-thunk-interface.
* gnu/packages/rocm.scm (roct-thunk-interface): New variable.--- gnu/packages/rocm.scm | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
Toggle diff (32 lines)diff --git a/gnu/packages/rocm.scm b/gnu/packages/rocm.scmindex 92530b5d83..21b7e43d2b 100644--- a/gnu/packages/rocm.scm+++ b/gnu/packages/rocm.scm@@ -148,3 +148,25 @@ oclc, ocml, ockl, opencl, hip and hc.") (description "The Comgr library provides APIs for compiling and inspecting AMDGPU code objects.") (license license:ncsa)))++(define-public roct-thunk-interface+ (package+ (name "roct-thunk-interface")+ (version %rocm-version)+ (source (origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/RadeonOpenCompute/ROCT-Thunk-Interface.git")+ (commit (string-append "rocm-" version))))+ (file-name (git-file-name name version))+ (sha256+ (base32+ "0ffqhrrscmcydfqf61dk58d7nnxk6n2k68jhqfj7a4hvhlphb74f"))))+ (build-system cmake-build-system)+ (arguments `(#:tests? #f)) ; Not sure how to run tests.+ (inputs `(("numactl" ,numactl)))+ (home-page "https://github.com/RadeonOpenCompute/ROCT-Thunk-Interface")+ (synopsis "Radeon Open Compute Thunk Interface")+ (description "User-mode API interfaces used to interact with the ROCk+driver.")+ (license license:ncsa)))-- 2.31.1
From d0537f8c11f59a0b0793f8763f913a9b88fe46df Mon Sep 17 00:00:00 2001From: Lars-Dominik Braun <lars@6xq.net>Date: Fri, 6 Aug 2021 08:24:44 +0200Subject: [PATCH 06/11] gnu: Add rocr-runtime.
* gnu/packages/rocm.scm (rocr-runtime): New variable.--- gnu/packages/rocm.scm | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+)
Toggle diff (50 lines)diff --git a/gnu/packages/rocm.scm b/gnu/packages/rocm.scmindex 21b7e43d2b..86b113ce42 100644--- a/gnu/packages/rocm.scm+++ b/gnu/packages/rocm.scm@@ -170,3 +170,43 @@ AMDGPU code objects.") (description "User-mode API interfaces used to interact with the ROCk driver.") (license license:ncsa)))++(define-public rocr-runtime+ (package+ (name "rocr-runtime")+ (version %rocm-version)+ (source (origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/RadeonOpenCompute/ROCR-Runtime.git")+ (commit (string-append "rocm-" version))))+ (file-name (git-file-name name version))+ (sha256+ (base32+ "0jqfqf5ymwlbpac065bhigmkgsk7mbyimdgvca7ymn38wpf80ka7"))))+ (build-system cmake-build-system)+ (arguments+ `(#:configure-flags+ `(,(string-append+ "-DBITCODE_DIR="+ (assoc-ref %build-inputs "rocm-device-libs")+ "/amdgcn/bitcode/"))+ #:tests? #f ; No tests.+ #:phases+ (modify-phases %standard-phases+ (add-after 'unpack 'chdir+ (lambda* (#:key inputs native-inputs #:allow-other-keys)+ (chdir "src")+ #t)))))+ (inputs+ `(("libelf" ,libelf)+ ("numactl" ,numactl)+ ("llvm" ,llvm-for-rocm)+ ("roct-thunk-interface" ,roct-thunk-interface)+ ("rocm-device-libs" ,rocm-device-libs))) ; For bitcode.+ (native-inputs `(("xxd" ,xxd)))+ (home-page "https://github.com/RadeonOpenCompute/ROCR-Runtime")+ (synopsis "ROCm Platform Runtime")+ (description "User-mode API interfaces and libraries necessary for host+applications to launch compute kernels to available HSA ROCm kernel agents.")+ (license license:ncsa)))-- 2.31.1
From 0b0b8ef438d2adf5900e0e4e7be4da032a502821 Mon Sep 17 00:00:00 2001From: Lars-Dominik Braun <lars@6xq.net>Date: Fri, 6 Aug 2021 08:25:38 +0200Subject: [PATCH 07/11] gnu: Add rocclr.
* gnu/packages/rocm.scm (rocclr): New variable.--- gnu/packages/rocm.scm | 45 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+)
Toggle diff (55 lines)diff --git a/gnu/packages/rocm.scm b/gnu/packages/rocm.scmindex 86b113ce42..3538208577 100644--- a/gnu/packages/rocm.scm+++ b/gnu/packages/rocm.scm@@ -210,3 +210,48 @@ driver.") (description "User-mode API interfaces and libraries necessary for host applications to launch compute kernels to available HSA ROCm kernel agents.") (license license:ncsa)))++(define-public rocclr+ (package+ (name "rocclr")+ (version %rocm-version)+ (source (origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/ROCm-Developer-Tools/ROCclr.git")+ (commit (string-append "rocm-" version))))+ (file-name (git-file-name name version))+ (sha256+ (base32+ "1pm1y020zriz7zmi95w0rcpka0jrsc7wwh81sssnysi8wxk3nnfy"))))+ (build-system cmake-build-system)+ (arguments+ `(#:tests? #f ; No tests.+ #:configure-flags+ `(,(string-append+ "-DOPENCL_DIR="+ (assoc-ref %build-inputs "rocm-opencl-runtime-src")))))+ (inputs+ `(("mesa" ,mesa)+ ("rocm-comgr" ,rocm-comgr)+ ("llvm" ,llvm-for-rocm)+ ("rocm-device-libs" ,rocm-device-libs)+ ("rocr-runtime" ,rocr-runtime)+ ("rocm-cmake" ,rocm-cmake)+ ;; rocclr depends on a few headers provided by rocm-opencl-runtime.+ ("rocm-opencl-runtime-src"+ ,(origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/RadeonOpenCompute/ROCm-OpenCL-Runtime.git")+ (commit (string-append "rocm-" version))))+ (file-name (git-file-name name version))+ (sha256+ (base32+ "1cglpiaj3ny1z74ssmy6j63vj92sfy4q38ix6qsga0mg3b2wvqz3"))))))+ (home-page "https://github.com/ROCm-Developer-Tools/ROCclr")+ (synopsis "Radeon Open Compute Common Language Runtime")+ (description "ROCclr is a virtual device interface that compute runtimes+interact with to different backends such as ROCr or PAL. This abstraction+allows runtimes to work on Windows as well as on Linux without much effort.")+ (license license:ncsa)))-- 2.31.1
From 179b4c74f80a6155a7f5d439acee60bc10585c74 Mon Sep 17 00:00:00 2001From: Lars-Dominik Braun <lars@6xq.net>Date: Fri, 6 Aug 2021 08:27:32 +0200Subject: [PATCH 08/11] gnu: Add rocm-opencl-runtime.
* gnu/packages/rocm.scm (rocm-opencl-runtime): New variable.* gnu/packages/patches/rocm-opencl-runtime-3.10.0-add-rocclr-include-directories.patch,gnu/packages/patches/rocm-opencl-runtime-4.3-noclinfo.patch,gnu/packages/patches/rocm-opencl-runtime-4.3-nocltrace.patch,gnu/packages/patches/rocm-opencl-runtime-4.3-noopencl.patch: New files.* gnu/local.mk (dist_patch_DATA): Add them.--- gnu/local.mk | 4 ++ ....10.0-add-rocclr-include-directories.patch | 14 +++++ .../rocm-opencl-runtime-4.3-noclinfo.patch | 28 +++++++++ .../rocm-opencl-runtime-4.3-nocltrace.patch | 25 ++++++++ .../rocm-opencl-runtime-4.3-noopencl.patch | 63 +++++++++++++++++++ gnu/packages/rocm.scm | 51 +++++++++++++++ 6 files changed, 185 insertions(+) create mode 100644 gnu/packages/patches/rocm-opencl-runtime-3.10.0-add-rocclr-include-directories.patch create mode 100644 gnu/packages/patches/rocm-opencl-runtime-4.3-noclinfo.patch create mode 100644 gnu/packages/patches/rocm-opencl-runtime-4.3-nocltrace.patch create mode 100644 gnu/packages/patches/rocm-opencl-runtime-4.3-noopencl.patch
Toggle diff (230 lines)diff --git a/gnu/local.mk b/gnu/local.mkindex db1f8e499d..6c73e02dd8 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -1699,6 +1699,10 @@ dist_patch_DATA = \ %D%/packages/patches/rnp-disable-ruby-rnp-tests.patch \ %D%/packages/patches/rnp-unbundle-googletest.patch \ %D%/packages/patches/rocm-comgr-3.1.0-dependencies.patch \+ %D%/packages/patches/rocm-opencl-runtime-3.10.0-add-rocclr-include-directories.patch+ %D%/packages/patches/rocm-opencl-runtime-4.3-noclinfo.patch+ %D%/packages/patches/rocm-opencl-runtime-4.3-nocltrace.patch+ %D%/packages/patches/rocm-opencl-runtime-4.3-noopencl.patch %D%/packages/patches/ruby-sanitize-system-libxml.patch \ %D%/packages/patches/rust-1.19-mrustc.patch \ %D%/packages/patches/rust-1.25-accept-more-detailed-gdb-lines.patch \diff --git a/gnu/packages/patches/rocm-opencl-runtime-3.10.0-add-rocclr-include-directories.patch b/gnu/packages/patches/rocm-opencl-runtime-3.10.0-add-rocclr-include-directories.patchnew file mode 100644index 0000000000..d81bb0747f--- /dev/null+++ b/gnu/packages/patches/rocm-opencl-runtime-3.10.0-add-rocclr-include-directories.patch@@ -0,0 +1,14 @@+Add missing include dirs for ROCclr.++--- a/amdocl/CMakeLists.txt 2020-12-05 22:05:55.838529158 +0100++++ b/amdocl/CMakeLists.txt 2020-12-05 22:07:35.677524507 +0100+@@ -23,6 +23,9 @@+ include_directories(${CMAKE_CURRENT_LIST_DIR}/../khronos)+ include_directories(${CMAKE_CURRENT_LIST_DIR}/../khronos/headers)+ include_directories(${CMAKE_CURRENT_LIST_DIR}/../khronos/headers/opencl2.2)++include_directories(${ROCclr_DIR}/../../../include)++include_directories(${ROCclr_DIR}/../../../include/compiler/lib/include/)++include_directories(${ROCclr_DIR}/../../../include/elf/)+ + if((CMAKE_CXX_COMPILER_ID STREQUAL "GNU") OR+ (CMAKE_${COMPILER}_COMPILER_ID MATCHES "Clang"))diff --git a/gnu/packages/patches/rocm-opencl-runtime-4.3-noclinfo.patch b/gnu/packages/patches/rocm-opencl-runtime-4.3-noclinfo.patchnew file mode 100644index 0000000000..5709e0d19a--- /dev/null+++ b/gnu/packages/patches/rocm-opencl-runtime-4.3-noclinfo.patch@@ -0,0 +1,28 @@+Do not build and install clinfo.++diff --git a/CMakeLists.txt.orig b/CMakeLists.txt+index 76847d3..3f62bfe 100644+--- a/CMakeLists.txt.orig++++ b/CMakeLists.txt+@@ -72,7 +72,7 @@ find_package(ROCclr REQUIRED CONFIG+ + add_subdirectory(khronos/icd)+ add_subdirectory(amdocl)+-add_subdirectory(tools/clinfo)++#add_subdirectory(tools/clinfo)+ add_subdirectory(tools/cltrace)+ if(BUILD_TESTS)+ add_subdirectory(tests/ocltst)+@@ -108,9 +108,9 @@ endif()+ ###--- Packaging ------------------------------------------------------------###+ + # MAIN package+-install(PROGRAMS $<TARGET_FILE:clinfo>+- DESTINATION bin+- COMPONENT MAIN)++#install(PROGRAMS $<TARGET_FILE:clinfo>++# DESTINATION bin++# COMPONENT MAIN)+ install(PROGRAMS $<TARGET_FILE:cltrace>+ DESTINATION lib+ COMPONENT MAIN)diff --git a/gnu/packages/patches/rocm-opencl-runtime-4.3-nocltrace.patch b/gnu/packages/patches/rocm-opencl-runtime-4.3-nocltrace.patchnew file mode 100644index 0000000000..e0328d7734--- /dev/null+++ b/gnu/packages/patches/rocm-opencl-runtime-4.3-nocltrace.patch@@ -0,0 +1,25 @@+Do not build and install cltrace.++diff --git a/CMakeLists.txt.orig b/CMakeLists.txt+index c449db4..9cff673 100644+--- a/CMakeLists.txt.orig++++ b/CMakeLists.txt+@@ -73,7 +73,7 @@ find_package(ROCclr REQUIRED CONFIG+ #add_subdirectory(khronos/icd)+ add_subdirectory(amdocl)+ #add_subdirectory(tools/clinfo)+-add_subdirectory(tools/cltrace)++#add_subdirectory(tools/cltrace)+ if(BUILD_TESTS)+ add_subdirectory(tests/ocltst)+ endif()+@@ -108,9 +108,6 @@ endif()+ #install(PROGRAMS $<TARGET_FILE:clinfo>+ # DESTINATION bin+ # COMPONENT MAIN)+-install(PROGRAMS $<TARGET_FILE:cltrace>+- DESTINATION lib+- COMPONENT MAIN)+ install(PROGRAMS $<TARGET_FILE:amdocl64>+ DESTINATION lib+ COMPONENT MAIN)diff --git a/gnu/packages/patches/rocm-opencl-runtime-4.3-noopencl.patch b/gnu/packages/patches/rocm-opencl-runtime-4.3-noopencl.patchnew file mode 100644index 0000000000..ed20f02503--- /dev/null+++ b/gnu/packages/patches/rocm-opencl-runtime-4.3-noopencl.patch@@ -0,0 +1,63 @@+Do not build and install libOpenCL.++--- b/CMakeLists.txt++++ b/CMakeLists.txt+@@ -70,7 +70,7 @@+ ${ROCclr_DIR}+ ${LIBROCclr_STATIC_DIR})+ +-add_subdirectory(khronos/icd)++#add_subdirectory(khronos/icd)+ add_subdirectory(amdocl)+ #add_subdirectory(tools/clinfo)+ add_subdirectory(tools/cltrace)+@@ -93,9 +93,6 @@+ OUTPUT_VARIABLE OPENCL_VERSION_GITDATE+ OUTPUT_STRIP_TRAILING_WHITESPACE)+ +-get_target_property(OPENCL_LIB_VERSION_MAJOR OpenCL SOVERSION)+-get_target_property(OPENCL_LIB_VERSION_STRING OpenCL VERSION)+-+ find_package(ROCM QUIET CONFIG PATHS /opt/rocm)+ + if(ROCM_FOUND)+@@ -117,25 +114,6 @@+ install(PROGRAMS $<TARGET_FILE:amdocl64>+ DESTINATION lib+ COMPONENT MAIN)+-install(PROGRAMS $<TARGET_FILE:OpenCL>+- DESTINATION lib+- COMPONENT MAIN)+-install(PROGRAMS $<TARGET_SONAME_FILE:OpenCL>+- DESTINATION lib+- COMPONENT MAIN)+-+-# DEV package+-install(DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/khronos/headers/opencl2.2/CL"+- DESTINATION include+- COMPONENT DEV+- USE_SOURCE_PERMISSIONS+- PATTERN cl_d3d10.h EXCLUDE+- PATTERN cl_d3d11.h EXCLUDE+- PATTERN cl_dx9_media_sharing.h EXCLUDE+- PATTERN cl_egl.h EXCLUDE)+-install(PROGRAMS $<TARGET_LINKER_FILE:OpenCL>+- DESTINATION lib+- COMPONENT DEV)+ + #############################+ # Packaging steps+--- a/khronos/icd/CMakeLists.txt 2020-06-07 16:05:32.425022904 +0200++++ b/khronos/icd/CMakeLists.txt 2020-06-07 16:06:03.273022786 +0200+@@ -132,7 +132,7 @@+ add_subdirectory (test)+ endif()+ +-install (TARGETS OpenCL+- RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}+- ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}+- LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR})++#install (TARGETS OpenCL++# RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}++# ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}++# LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR})diff --git a/gnu/packages/rocm.scm b/gnu/packages/rocm.scmindex 3538208577..8cc0eff9e3 100644--- a/gnu/packages/rocm.scm+++ b/gnu/packages/rocm.scm@@ -255,3 +255,54 @@ applications to launch compute kernels to available HSA ROCm kernel agents.") interact with to different backends such as ROCr or PAL. This abstraction allows runtimes to work on Windows as well as on Linux without much effort.") (license license:ncsa)))++(define-public rocm-opencl-runtime+ (package+ (name "rocm-opencl-runtime")+ (version %rocm-version)+ (source (origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/RadeonOpenCompute/ROCm-OpenCL-Runtime.git")+ (commit (string-append "rocm-" version))))+ (file-name (git-file-name name version))+ (sha256+ (base32+ "1cglpiaj3ny1z74ssmy6j63vj92sfy4q38ix6qsga0mg3b2wvqz3"))+ (patches+ (search-patches+ "rocm-opencl-runtime-3.10.0-add-rocclr-include-directories.patch"+ ;; Do not install libOpenCL, which ocl-icd provides.+ "rocm-opencl-runtime-4.3-noopencl.patch"+ ;; Guix includes a program clinfo already.+ "rocm-opencl-runtime-4.3-noclinfo.patch"+ ;; cltrace linking fails, remove it.+ "rocm-opencl-runtime-4.3-nocltrace.patch"))))+ (build-system cmake-build-system)+ (arguments+ `(#:tests? #f ; Not sure how to run them.+ #:phases+ (modify-phases %standard-phases+ (add-after 'install 'create-icd+ ;; Manually install ICD, which simply consists of dumping+ ;; the path of the .so into the correct file.+ (lambda* (#:key outputs #:allow-other-keys)+ (let* ((out (assoc-ref outputs "out"))+ (vendors (string-append out "/etc/OpenCL/vendors"))+ (sopath (string-append out "/lib/libamdocl64.so")))+ (mkdir-p vendors)+ (with-output-to-file (string-append vendors "/amdocl64.icd")+ (lambda _ (display sopath)))))))))+ (inputs+ `(("mesa" ,mesa)+ ("rocm-comgr" ,rocm-comgr)+ ("rocr-runtime" ,rocr-runtime)+ ("rocclr" ,rocclr)+ ("ocl-icd" ,ocl-icd)+ ("glew" ,glew)))+ (native-inputs `())+ (home-page "https://github.com/RadeonOpenCompute/ROCm-OpenCL-Runtime")+ (synopsis "ROCm OpenCL Runtime")+ (description "OpenCL 2.0 compatible language runtime, supporting offline+and in-process/in-memory compilation.")+ (license license:ncsa)))-- 2.31.1
From cdfbc09ae5ceb21357aeaaece3676afbcf3442b4 Mon Sep 17 00:00:00 2001From: Lars-Dominik Braun <lars@6xq.net>Date: Fri, 6 Aug 2021 08:29:12 +0200Subject: [PATCH 09/11] gnu: Add rocminfo.
* gnu/packages/rocm.scm (rocminfo): New variable.--- gnu/packages/rocm.scm | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)
Toggle diff (31 lines)diff --git a/gnu/packages/rocm.scm b/gnu/packages/rocm.scmindex 8cc0eff9e3..95b586c640 100644--- a/gnu/packages/rocm.scm+++ b/gnu/packages/rocm.scm@@ -306,3 +306,24 @@ allows runtimes to work on Windows as well as on Linux without much effort.") (description "OpenCL 2.0 compatible language runtime, supporting offline and in-process/in-memory compilation.") (license license:ncsa)))++(define-public rocminfo+ (package+ (name "rocminfo")+ (version %rocm-version)+ (source (origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/RadeonOpenCompute/rocminfo.git")+ (commit (string-append "rocm-" version))))+ (file-name (git-file-name name version))+ (sha256+ (base32+ "0pcm308vwkjrwnrk507iya20mkil8j0vx699w9jk2gas4n4jvkcz"))))+ (build-system cmake-build-system)+ (arguments `(#:tests? #f)) ; No tests.+ (inputs `(("rocr-runtime" ,rocr-runtime)))+ (home-page "https://github.com/RadeonOpenCompute/rocminfo")+ (synopsis "ROCm Application for Reporting System Info")+ (description #f)+ (license license:ncsa)))-- 2.31.1
From 79680d7d55c98212e6f397c133e74d52996e31a6 Mon Sep 17 00:00:00 2001From: Lars-Dominik Braun <lars@6xq.net>Date: Fri, 6 Aug 2021 08:29:39 +0200Subject: [PATCH 10/11] gnu: Add rocm-bandwidth-test.
* gnu/packages/rocm.scm (rocm-bandwidth-test): New variable.--- gnu/packages/rocm.scm | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
Toggle diff (36 lines)diff --git a/gnu/packages/rocm.scm b/gnu/packages/rocm.scmindex 95b586c640..eeea66526d 100644--- a/gnu/packages/rocm.scm+++ b/gnu/packages/rocm.scm@@ -327,3 +327,29 @@ and in-process/in-memory compilation.") (synopsis "ROCm Application for Reporting System Info") (description #f) (license license:ncsa)))++(define-public rocm-bandwidth-test+ (package+ (name "rocm-bandwidth-test")+ (version %rocm-version)+ (source (origin+ (method git-fetch)+ (uri (git-reference+ (url "https://github.com/RadeonOpenCompute/rocm_bandwidth_test.git")+ (commit (string-append "rocm-" version))))+ (file-name (git-file-name name version))+ (sha256+ (base32+ "0a14kwkjpiyljgzxblh031qibn6xgbxp6m12zdy1pmwb2c44jjmm"))))+ (build-system cmake-build-system)+ (arguments `(#:tests? #f)) ; No tests.+ (inputs `(("rocr-runtime" ,rocr-runtime)))+ (home-page "https://github.com/RadeonOpenCompute/rocm_bandwidth_test")+ (synopsis "Bandwidth test for ROCm")+ (description "RocBandwidthTest is designed to capture the performance+characteristics of buffer copying and kernel read/write operations. The help+screen of the benchmark shows various options one can use in initiating+cop/read/writer operations. In addition one can also query the topology of the+system in terms of memory pools and their agents.")+ (license license:ncsa)))+-- 2.31.1
From 164a786fe3bc322bdba549c6f3b8b49e7435d0f1 Mon Sep 17 00:00:00 2001From: Lars-Dominik Braun <lars@6xq.net>Date: Fri, 6 Aug 2021 08:30:13 +0200Subject: [PATCH 11/11] gnu: darktable: Add compatibility for ROCm.
darktable has a few tools, which can use OpenCL, not just the mainbinary. Instead of wrapping it, patch the search path. ROCm alsofails to build OpenCL kernels with just the inline keyword. Applyupstream fix.
* gnu/packages/photo.scm (darktable) [#:phases]: Add new phases, removeLD_LIBRARY_PATH from wrap-binary.--- gnu/packages/photo.scm | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-)
Toggle diff (44 lines)diff --git a/gnu/packages/photo.scm b/gnu/packages/photo.scmindex 3ef0fbf7f5..bf51e6885f 100644--- a/gnu/packages/photo.scm+++ b/gnu/packages/photo.scm@@ -486,6 +486,24 @@ photographic equipment.") "-DBUILD_TESTING=On") #:phases (modify-phases %standard-phases+ (add-after 'unpack 'libOpenCL-path+ (lambda* (#:key inputs #:allow-other-keys)+ ;; Statically link to libOpenCL.+ (substitute* "./src/common/dlopencl.c"+ (("\"libOpenCL\"")+ (string-append "\"" (assoc-ref inputs "ocl-icd") "/lib/libOpenCL.so\"")))+ #t))+ ;; The use of inline is wrong and darktable cannot compile its kernels+ ;; with ROCm. See upstream commit+ ;; https://github.com/darktable-org/darktable/commit/f0d8710f5ef34eb7e33b4064e022ebf3057b9e53+ (add-after 'unpack 'opencl-inline+ (lambda* (#:key inputs #:allow-other-keys)+ ;; This is a feature of OpenCL 1.2 and later.+ (substitute* "data/kernels/CMakeLists.txt"+ (("CL1\\.1") "CL1.2"))+ (substitute* (find-files "data/kernels" "\\.(cl|h)$")+ (("inline") "static inline"))+ #t)) (add-before 'configure 'prepare-build-environment (lambda* (#:key inputs #:allow-other-keys) ;; Rawspeed fails to build with GCC due to OpenMP error:@@ -516,11 +534,7 @@ photographic equipment.") ;; For GtkFileChooserDialog. `("GSETTINGS_SCHEMA_DIR" = (,(string-append (assoc-ref inputs "gtk+")- "/share/glib-2.0/schemas")))- ;; For libOpenCL.so.- `("LD_LIBRARY_PATH" =- (,(string-append (assoc-ref inputs "ocl-icd")- "/lib"))))+ "/share/glib-2.0/schemas")))) #t))))) (native-inputs `(("clang" ,clang-11)-- 2.31.1
L
L
Lars-Dominik Braun wrote on 8 Aug 16:59 +0200
Re: [PATCH 00/11] AMD ROCm OpenCL runtime
(address . 49907-done@debbugs.gnu.org)
YQ/xOfH0zI0qVX6L@noor.fritz.box
Hi,
pushed after review from Rekado and with minor changes ase81cf4e79a6e297db0ae2a9c39eab495e7e204f0 and before.
Cheers,Lars
Closed
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send email to 49907@debbugs.gnu.org