[PATCH] gnu: mandoc: Support zstd-compressed man pages.

  • Done
  • quality assurance status badge
Details
2 participants
  • Maxim Cournoyer
  • soeren
Owner
unassigned
Submitted by
soeren
Severity
normal

Debbugs page

soeren wrote 2 months ago
(address . guix-patches@gnu.org)
20250111190753.32110-2-soeren@soeren-tempel.net
From: Sören Tempel <soeren+git@soeren-tempel.net>

Since #68242 Guix uses zstd compression for man pages. Unfortunately,
upstream mandoc only supports gzip compressed man pages. Luckily, zstd
provides a wrapper library which easily allows adapting software using
zlib to zstd compression. This patch uses this wrapper library in
conjunction with mandoc to add support for zstd compression to it,
thereby allowing Guix man pages to be viewed with mandoc again.

* gnu/packages/man.scm (mandoc): Support zstd compression.
* gnu/local.mk: Add new patch.
* gnu/packages/patches/mandoc-support-zstd-compression.patch: New file.
---
Without this patch, mandoc is essentially defunct on Guix. You cannot
view any Guix man pages with mandoc presently because of this issue.

gnu/local.mk | 1 +
gnu/packages/man.scm | 47 +++++++++------
.../mandoc-support-zstd-compression.patch | 58 +++++++++++++++++++
3 files changed, 89 insertions(+), 17 deletions(-)
create mode 100644 gnu/packages/patches/mandoc-support-zstd-compression.patch

Toggle diff (148 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index f118fe4442..c8bfe47509 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1799,6 +1799,7 @@ dist_patch_DATA = \
%D%/packages/patches/lxc-no-static-bin.patch \
%D%/packages/patches/mactelnet-remove-init.patch \
%D%/packages/patches/mailutils-variable-lookup.patch \
+ %D%/packages/patches/mandoc-support-zstd-compression.patch \
%D%/packages/patches/make-impure-dirs.patch \
%D%/packages/patches/mariadb-rocksdb-atomic-linking.patch \
%D%/packages/patches/mathjax-disable-webpack.patch \
diff --git a/gnu/packages/man.scm b/gnu/packages/man.scm
index 3148fcc8a1..c62568e515 100644
--- a/gnu/packages/man.scm
+++ b/gnu/packages/man.scm
@@ -37,6 +37,7 @@ (define-module (gnu packages man)
#:use-module (guix build-system gnu)
#:use-module (guix build-system ruby)
#:use-module (guix utils)
+ #:use-module (gnu packages)
#:use-module (gnu packages base)
#:use-module (gnu packages compression)
#:use-module (gnu packages dbm)
@@ -273,30 +274,42 @@ (define-public mandoc
(method url-fetch)
(uri (string-append "https://mandoc.bsd.lv/snapshots/mandoc-"
version ".tar.gz"))
+ (patches (search-patches "mandoc-support-zstd-compression.patch"))
(sha256
(base32
"174x2x9ws47b14lm339j6rzm7mxy1j3qhh484khscw0yy1qdbw4b"))))
(build-system gnu-build-system)
(arguments
`(#:test-target "regress"
- #:phases (modify-phases %standard-phases
- (add-before 'configure 'set-prefix
- (lambda* (#:key outputs #:allow-other-keys)
- (substitute* "configure"
- (("^CC=.*")
- (string-append "CC=" ,(cc-for-target) "\n"))
- (("^DEFCFLAGS=\\\\\"")
- "DEFCFLAGS=\"-O2 ")
- (("^UTF8_LOCALE=.*") ;used for tests
- "UTF8_LOCALE=en_US.UTF-8\n")
- (("^MANPATH_(BASE|DEFAULT)=.*" _ which)
- (string-append "MANPATH_" which "="
- "/run/current-system/profile/share/man\n"))
- (("^PREFIX=.*")
- (string-append "PREFIX=" (assoc-ref outputs "out")
- "\n"))))))))
+ #:make-flags
+ (list "VPATH=./zstd-src/zlibWrapper"
+ (string-join
+ (list "CFLAGS=-DZWRAP_USE_ZSTD=1"
+ (string-append "-I./zstd-src/zlibWrapper"))
+ " "))
+ #:phases ,#~(modify-phases %standard-phases
+ (add-after 'unpack 'unpack-zstd
+ (lambda _
+ (mkdir "zstd-src")
+ (invoke "tar" "--strip-components=1" "-C" "zstd-src"
+ "-xf" #$(package-source zstd))))
+ (add-before 'configure 'set-prefix
+ (lambda* (#:key outputs #:allow-other-keys)
+ (substitute* "configure"
+ (("^CC=.*")
+ (string-append "CC=" #$(cc-for-target) "\n"))
+ (("^DEFCFLAGS=\\\\\"")
+ "DEFCFLAGS=\"-O2 ")
+ (("^UTF8_LOCALE=.*") ;used for tests
+ "UTF8_LOCALE=en_US.UTF-8\n")
+ (("^MANPATH_(BASE|DEFAULT)=.*" _ which)
+ (string-append "MANPATH_" which "="
+ "/run/current-system/profile/share/man\n"))
+ (("^PREFIX=.*")
+ (string-append "PREFIX=" (assoc-ref outputs "out")
+ "\n"))))))))
(native-inputs (list (libc-utf8-locales-for-target) perl)) ;used to run tests
- (inputs (list zlib))
+ (inputs (list zlib (list zstd "lib")))
(native-search-paths
(list (search-path-specification
(variable "MANPATH")
diff --git a/gnu/packages/patches/mandoc-support-zstd-compression.patch b/gnu/packages/patches/mandoc-support-zstd-compression.patch
new file mode 100644
index 0000000000..b8cbeb6782
--- /dev/null
+++ b/gnu/packages/patches/mandoc-support-zstd-compression.patch
@@ -0,0 +1,58 @@
+mandoc upstream does not support zstd compression. However, Guix uses zstd
+compression for its man pages, therefore—without support for this compression
+method—mandoc would be quite useless. Hence, this patchset uses zlibWrapper
+from the zstd project to add zstd compression support to mandoc.
+
+diff -upr mandoc-1.14.6.orig/Makefile mandoc-1.14.6/Makefile
+--- mandoc-1.14.6.orig/Makefile 2025-01-11 16:20:31.511129163 +0100
++++ mandoc-1.14.6/Makefile 2025-01-11 19:16:35.924788821 +0100
+@@ -251,7 +251,12 @@ LIBMANDOC_OBJS = $(LIBMAN_OBJS) \
+ msec.o \
+ preconv.o \
+ read.o \
+- tag.o
++ tag.o \
++ zstd_zlibwrapper.o \
++ gzclose.o \
++ gzlib.o \
++ gzread.o \
++ gzwrite.o
+
+ ALL_COBJS = compat_err.o \
+ compat_fts.o \
+Only in mandoc-1.14.6: Makefile.orig
+diff -upr mandoc-1.14.6.orig/configure mandoc-1.14.6/configure
+--- mandoc-1.14.6.orig/configure 2025-01-11 16:20:31.511129163 +0100
++++ mandoc-1.14.6/configure 2025-01-11 19:16:35.924788821 +0100
+@@ -430,7 +430,7 @@ fi
+ [ "${FATAL}" -eq 0 ] || exit 1
+
+ # --- LDADD ---
+-LDADD="${LDADD} ${LD_NANOSLEEP} ${LD_RECVMSG} ${LD_OHASH} -lz"
++LDADD="${LDADD} ${LD_NANOSLEEP} ${LD_RECVMSG} ${LD_OHASH} -lz -lzstd"
+ echo "selected LDADD=\"${LDADD}\"" 1>&2
+ echo "selected LDADD=\"${LDADD}\"" 1>&3
+ echo 1>&3
+Only in mandoc-1.14.6: configure.orig
+diff -upr mandoc-1.14.6.orig/read.c mandoc-1.14.6/read.c
+--- mandoc-1.14.6.orig/read.c 2025-01-11 16:35:03.825441715 +0100
++++ mandoc-1.14.6/read.c 2025-01-11 19:16:35.924788821 +0100
+@@ -37,7 +37,7 @@
+ #include <stdlib.h>
+ #include <string.h>
+ #include <unistd.h>
+-#include <zlib.h>
++#include <zstd_zlibwrapper.h>
+
+ #include "mandoc_aux.h"
+ #include "mandoc.h"
+@@ -627,7 +627,7 @@ mparse_open(struct mparse *curp, const char *file)
+ int fd, save_errno;
+
+ cp = strrchr(file, '.');
+- curp->gzip = (cp != NULL && ! strcmp(cp + 1, "gz"));
++ curp->gzip = (cp != NULL && (! strcmp(cp + 1, "gz") || ! strcmp(cp + 1, "zst")));
+
+ /* First try to use the filename as it is. */
+
+Only in mandoc-1.14.6: read.c.orig
Maxim Cournoyer wrote 2 months ago
(address . soeren@soeren-tempel.net)
87zfjsd5no.fsf@gmail.com
Hello!

soeren@soeren-tempel.net writes:

Toggle quote (9 lines)
> From: Sören Tempel <soeren+git@soeren-tempel.net>
>
> Since #68242 Guix uses zstd compression for man pages. Unfortunately,
> upstream mandoc only supports gzip compressed man pages. Luckily, zstd
> provides a wrapper library which easily allows adapting software using
> zlib to zstd compression. This patch uses this wrapper library in
> conjunction with mandoc to add support for zstd compression to it,
> thereby allowing Guix man pages to be viewed with mandoc again.

Oh! I have never used mandoc, only man-db.

Toggle quote (5 lines)
> * gnu/packages/man.scm (mandoc): Support zstd compression.
> * gnu/local.mk: Add new patch.
> * gnu/packages/patches/mandoc-support-zstd-compression.patch: New
> file.

[...]

Toggle quote (46 lines)
> (arguments
> `(#:test-target "regress"
> - #:phases (modify-phases %standard-phases
> - (add-before 'configure 'set-prefix
> - (lambda* (#:key outputs #:allow-other-keys)
> - (substitute* "configure"
> - (("^CC=.*")
> - (string-append "CC=" ,(cc-for-target) "\n"))
> - (("^DEFCFLAGS=\\\\\"")
> - "DEFCFLAGS=\"-O2 ")
> - (("^UTF8_LOCALE=.*") ;used for tests
> - "UTF8_LOCALE=en_US.UTF-8\n")
> - (("^MANPATH_(BASE|DEFAULT)=.*" _ which)
> - (string-append "MANPATH_" which "="
> - "/run/current-system/profile/share/man\n"))
> - (("^PREFIX=.*")
> - (string-append "PREFIX=" (assoc-ref outputs "out")
> - "\n"))))))))
> + #:make-flags
> + (list "VPATH=./zstd-src/zlibWrapper"
> + (string-join
> + (list "CFLAGS=-DZWRAP_USE_ZSTD=1"
> + (string-append "-I./zstd-src/zlibWrapper"))
> + " "))
> + #:phases ,#~(modify-phases %standard-phases
> + (add-after 'unpack 'unpack-zstd
> + (lambda _
> + (mkdir "zstd-src")
> + (invoke "tar" "--strip-components=1" "-C" "zstd-src"
> + "-xf" #$(package-source zstd))))
> + (add-before 'configure 'set-prefix
> + (lambda* (#:key outputs #:allow-other-keys)
> + (substitute* "configure"
> + (("^CC=.*")
> + (string-append "CC=" #$(cc-for-target) "\n"))
> + (("^DEFCFLAGS=\\\\\"")
> + "DEFCFLAGS=\"-O2 ")
> + (("^UTF8_LOCALE=.*") ;used for tests
> + "UTF8_LOCALE=en_US.UTF-8\n")
> + (("^MANPATH_(BASE|DEFAULT)=.*" _ which)
> + (string-append "MANPATH_" which "="
> + "/run/current-system/profile/share/man\n"))
> + (("^PREFIX=.*")
> + (string-append "PREFIX=" (assoc-ref outputs "out")
> + "\n"))))))))

While moving things around, I'd use a plain list for the arguments, and
format the phases under the #:phases argument (newline), to satisfy our
max 80 columns of width convention.

Toggle quote (17 lines)
> (native-inputs (list (libc-utf8-locales-for-target) perl)) ;used to run tests
> - (inputs (list zlib))
> + (inputs (list zlib (list zstd "lib")))
> (native-search-paths
> (list (search-path-specification
> (variable "MANPATH")
> diff --git a/gnu/packages/patches/mandoc-support-zstd-compression.patch b/gnu/packages/patches/mandoc-support-zstd-compression.patch
> new file mode 100644
> index 0000000000..b8cbeb6782
> --- /dev/null
> +++ b/gnu/packages/patches/mandoc-support-zstd-compression.patch
> @@ -0,0 +1,58 @@
> +mandoc upstream does not support zstd compression. However, Guix uses zstd
> +compression for its man pages, therefore—without support for this compression
> +method—mandoc would be quite useless. Hence, this patchset uses zlibWrapper
> +from the zstd project to add zstd compression support to mandoc.

Interesting solution! The issue should be ideally be brought upstream
though, and referenced here. With the leading implementation man-db
having gained zstd support, others should follow suite. They may be
interested in using your solution, or otherwise adding "native" support
for it.

Could you please create such an issue with them, cross-reference it in
your patch, and send a v2?

--
Thanks,
Maxim
soeren wrote 2 months ago
[PATCH v2] gnu: mandoc: Support zstd-compressed man pages.
(address . 75501@debbugs.gnu.org)
bd23f797a74d7132fb2ee7cbdc7d747fa267d37e.1737060248.git.soeren@soeren-tempel.net
From: Sören Tempel <soeren+git@soeren-tempel.net>

Since #68242 Guix uses zstd compression for man pages. Unfortunately,
upstream mandoc only supports gzip compressed man pages. Luckily, zstd
provides a wrapper library which easily allows adapting software using
zlib to zstd compression. This patch uses this wrapper library in
conjunction with mandoc to add support for zstd compression to it,
thereby allowing Guix man pages to be viewed with mandoc again.

Without this patch, mandoc is essentially defunct on Guix.

* gnu/packages/man.scm (mandoc): Support zstd compression.
* gnu/local.mk: Add new patch.
* gnu/packages/patches/mandoc-support-zstd-compression.patch: New file.
---
gnu/local.mk | 1 +
gnu/packages/man.scm | 54 ++++++++++------
.../mandoc-support-zstd-compression.patch | 63 +++++++++++++++++++
3 files changed, 100 insertions(+), 18 deletions(-)
create mode 100644 gnu/packages/patches/mandoc-support-zstd-compression.patch

Toggle diff (161 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 5fb354caae9..fef9a560284 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1807,6 +1807,7 @@ dist_patch_DATA = \
%D%/packages/patches/lxc-no-static-bin.patch \
%D%/packages/patches/mactelnet-remove-init.patch \
%D%/packages/patches/mailutils-variable-lookup.patch \
+ %D%/packages/patches/mandoc-support-zstd-compression.patch \
%D%/packages/patches/make-impure-dirs.patch \
%D%/packages/patches/mariadb-rocksdb-atomic-linking.patch \
%D%/packages/patches/mathjax-disable-webpack.patch \
diff --git a/gnu/packages/man.scm b/gnu/packages/man.scm
index 3148fcc8a16..1ff1fac991f 100644
--- a/gnu/packages/man.scm
+++ b/gnu/packages/man.scm
@@ -37,6 +37,7 @@ (define-module (gnu packages man)
#:use-module (guix build-system gnu)
#:use-module (guix build-system ruby)
#:use-module (guix utils)
+ #:use-module (gnu packages)
#:use-module (gnu packages base)
#:use-module (gnu packages compression)
#:use-module (gnu packages dbm)
@@ -273,30 +274,47 @@ (define-public mandoc
(method url-fetch)
(uri (string-append "https://mandoc.bsd.lv/snapshots/mandoc-"
version ".tar.gz"))
+ (patches (search-patches "mandoc-support-zstd-compression.patch"))
(sha256
(base32
"174x2x9ws47b14lm339j6rzm7mxy1j3qhh484khscw0yy1qdbw4b"))))
(build-system gnu-build-system)
(arguments
- `(#:test-target "regress"
- #:phases (modify-phases %standard-phases
- (add-before 'configure 'set-prefix
- (lambda* (#:key outputs #:allow-other-keys)
- (substitute* "configure"
- (("^CC=.*")
- (string-append "CC=" ,(cc-for-target) "\n"))
- (("^DEFCFLAGS=\\\\\"")
- "DEFCFLAGS=\"-O2 ")
- (("^UTF8_LOCALE=.*") ;used for tests
- "UTF8_LOCALE=en_US.UTF-8\n")
- (("^MANPATH_(BASE|DEFAULT)=.*" _ which)
- (string-append "MANPATH_" which "="
- "/run/current-system/profile/share/man\n"))
- (("^PREFIX=.*")
- (string-append "PREFIX=" (assoc-ref outputs "out")
- "\n"))))))))
+ (list
+ #:test-target "regress"
+ #:make-flags
+ #~(list "VPATH=./zstd-src/zlibWrapper"
+ (string-join
+ (list "CFLAGS=-DZWRAP_USE_ZSTD=1"
+ "-I./zstd-src/zlibWrapper")
+ " "))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'unpack-zstd
+ (lambda _
+ (mkdir "zstd-src")
+ (invoke "tar" "--strip-components=1" "-C"
+ "zstd-src" "-xf" #$(package-source zstd))))
+ (add-before 'configure 'set-prefix
+ (lambda* (#:key outputs #:allow-other-keys)
+ (substitute*
+ "configure"
+ (("^CC=.*")
+ (string-append "CC=" #$(cc-for-target) "\n"))
+ (("^DEFCFLAGS=\\\\\"")
+ "DEFCFLAGS=\"-O2 ")
+ (("^UTF8_LOCALE=.*") ;used for tests
+ "UTF8_LOCALE=en_US.UTF-8\n")
+ (("^MANPATH_(BASE|DEFAULT)=.*" _ which)
+ (string-append
+ "MANPATH_" which "="
+ "/run/current-system/profile/share/man\n"))
+ (("^PREFIX=.*")
+ (string-append "PREFIX="
+ (assoc-ref outputs "out")
+ "\n"))))))))
(native-inputs (list (libc-utf8-locales-for-target) perl)) ;used to run tests
- (inputs (list zlib))
+ (inputs (list zlib (list zstd "lib")))
(native-search-paths
(list (search-path-specification
(variable "MANPATH")
diff --git a/gnu/packages/patches/mandoc-support-zstd-compression.patch b/gnu/packages/patches/mandoc-support-zstd-compression.patch
new file mode 100644
index 00000000000..3577ff0891b
--- /dev/null
+++ b/gnu/packages/patches/mandoc-support-zstd-compression.patch
@@ -0,0 +1,63 @@
+mandoc upstream does not support zstd compression. However, Guix uses zstd
+compression for its man pages, therefore—without support for this compression
+method—mandoc would be quite useless. Hence, this patchset uses zlibWrapper
+from the zstd project to add zstd compression support to mandoc.
+
+Note that upstream is presently not interested in adding support for additional
+compression scheme, fourtunately, the patch is minimal and easy to maintain, see:
+
+ https://inbox.vuxu.org/mandoc-discuss/20201129201424.GI58187@athene.usta.de/
+
+diff -upr mandoc-1.14.6.orig/Makefile mandoc-1.14.6/Makefile
+--- mandoc-1.14.6.orig/Makefile 2025-01-11 16:20:31.511129163 +0100
++++ mandoc-1.14.6/Makefile 2025-01-11 19:16:35.924788821 +0100
+@@ -251,7 +251,12 @@ LIBMANDOC_OBJS = $(LIBMAN_OBJS) \
+ msec.o \
+ preconv.o \
+ read.o \
+- tag.o
++ tag.o \
++ zstd_zlibwrapper.o \
++ gzclose.o \
++ gzlib.o \
++ gzread.o \
++ gzwrite.o
+
+ ALL_COBJS = compat_err.o \
+ compat_fts.o \
+Only in mandoc-1.14.6: Makefile.orig
+diff -upr mandoc-1.14.6.orig/configure mandoc-1.14.6/configure
+--- mandoc-1.14.6.orig/configure 2025-01-11 16:20:31.511129163 +0100
++++ mandoc-1.14.6/configure 2025-01-11 19:16:35.924788821 +0100
+@@ -430,7 +430,7 @@ fi
+ [ "${FATAL}" -eq 0 ] || exit 1
+
+ # --- LDADD ---
+-LDADD="${LDADD} ${LD_NANOSLEEP} ${LD_RECVMSG} ${LD_OHASH} -lz"
++LDADD="${LDADD} ${LD_NANOSLEEP} ${LD_RECVMSG} ${LD_OHASH} -lz -lzstd"
+ echo "selected LDADD=\"${LDADD}\"" 1>&2
+ echo "selected LDADD=\"${LDADD}\"" 1>&3
+ echo 1>&3
+Only in mandoc-1.14.6: configure.orig
+diff -upr mandoc-1.14.6.orig/read.c mandoc-1.14.6/read.c
+--- mandoc-1.14.6.orig/read.c 2025-01-11 16:35:03.825441715 +0100
++++ mandoc-1.14.6/read.c 2025-01-11 19:16:35.924788821 +0100
+@@ -37,7 +37,7 @@
+ #include <stdlib.h>
+ #include <string.h>
+ #include <unistd.h>
+-#include <zlib.h>
++#include <zstd_zlibwrapper.h>
+
+ #include "mandoc_aux.h"
+ #include "mandoc.h"
+@@ -627,7 +627,7 @@ mparse_open(struct mparse *curp, const char *file)
+ int fd, save_errno;
+
+ cp = strrchr(file, '.');
+- curp->gzip = (cp != NULL && ! strcmp(cp + 1, "gz"));
++ curp->gzip = (cp != NULL && (! strcmp(cp + 1, "gz") || ! strcmp(cp + 1, "zst")));
+
+ /* First try to use the filename as it is. */
+
+Only in mandoc-1.14.6: read.c.orig

base-commit: da69a9e15115d6acc7e4a95cc6295f97c97f827e
Sören Tempel wrote 2 months ago
Re: [bug#75501] [PATCH] gnu: mandoc: Support zstd-compressed man pages.
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
3GW4TH6POR7DJ.393NEMAQAI01C@8pit.net
Hello Maxim,

thank you for having a look at the patch and your fast feedback!

Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
Toggle quote (4 lines)
> While moving things around, I'd use a plain list for the arguments, and
> format the phases under the #:phases argument (newline), to satisfy our
> max 80 columns of width convention.

Changed accordingly in the v2 that I just send.

Toggle quote (9 lines)
> Interesting solution! The issue should be ideally be brought upstream
> though, and referenced here. With the leading implementation man-db
> having gained zstd support, others should follow suite. They may be
> interested in using your solution, or otherwise adding "native" support
> for it.
>
> Could you please create such an issue with them, cross-reference it in
> your patch, and send a v2?

Having worked with upstream mandoc before, I doubt that they would be
interested in implementing additional compression schemes. mandoc is an
OpenBSD project, and OpenBSD doesn't compress its man pages. Usually,
its hard to convince them to add features that do not benefit OpenBSD.

Specifically, regarding compression there is a prior thread on adding
bzip2 where the mandoc maintainer states “compressing manual pages
makes absolutely no sense to me […]”:


The Guix patch is very small so that I don't see any issue with
maintaining it downstream for the foreseeable future. Nonetheless, I
added a link to the aforementioned thread to the patch description.

If it is a requirement, I can create another “compression support”
thread on their ML but I would prefer to use my time elsewhere.

Greetings
Sören
Maxim Cournoyer wrote 2 months ago
(name . Sören Tempel)(address . soeren@soeren-tempel.net)
877c6r9t1h.fsf@gmail.com
Hi Sören,

Sören Tempel <soeren@soeren-tempel.net> writes:

Toggle quote (25 lines)
> Hello Maxim,
>
> thank you for having a look at the patch and your fast feedback!
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>> While moving things around, I'd use a plain list for the arguments, and
>> format the phases under the #:phases argument (newline), to satisfy our
>> max 80 columns of width convention.
>
> Changed accordingly in the v2 that I just send.
>
>> Interesting solution! The issue should be ideally be brought upstream
>> though, and referenced here. With the leading implementation man-db
>> having gained zstd support, others should follow suite. They may be
>> interested in using your solution, or otherwise adding "native" support
>> for it.
>>
>> Could you please create such an issue with them, cross-reference it in
>> your patch, and send a v2?
>
> Having worked with upstream mandoc before, I doubt that they would be
> interested in implementing additional compression schemes. mandoc is an
> OpenBSD project, and OpenBSD doesn't compress its man pages. Usually,
> its hard to convince them to add features that do not benefit OpenBSD.

I see. I think just at least a mention of that issue and a pointer to
the solution devised for Guix would be useful to share with upstream, in
case it'd motivate them to address it themselves.

You could argue why mandoc is such a great piece of software that you
want to use it over man-db on other systems than OpenBSD, they may
soften a bit on their position, ha!

Toggle quote (6 lines)
> Specifically, regarding compression there is a prior thread on adding
> bzip2 where the mandoc maintainer states “compressing manual pages
> makes absolutely no sense to me […]”:
>
> https://inbox.vuxu.org/mandoc-discuss/20201129201424.GI58187@athene.usta.de/

Interesting position, but a bit extreme. It's true that a few megabytes
saved are not the end of the world, especially if users are using full
disk compression such as Btrfs + Zstd, but it's still a plus in my book.

I had done some limited number crunching in [0], which found zstd mildy
better at compressing man pages than zstd (about 10% size reduction),
and 1.5x faster at decompressing.


Toggle quote (3 lines)
> If it is a requirement, I can create another “compression support”
> thread on their ML but I would prefer to use my time elsewhere.

I'd encourage you to do so, whatever the outcome.

--
Thanks,
Maxim
soeren wrote 1 months ago
[PATCH v3] gnu: mandoc: Support zstd-compressed man pages.
(address . 75501@debbugs.gnu.org)(address . maxim.cournoyer@gmail.com)
b7eeef8c6184e46f2f70a9657ab9af02837af524.1738079747.git.soeren@soeren-tempel.net
From: Sören Tempel <soeren+git@soeren-tempel.net>

Since #68242 Guix uses zstd compression for man pages. Unfortunately,
upstream mandoc only supports gzip compressed man pages. Luckily, zstd
provides a wrapper library which easily allows adapting software using
zlib to zstd compression. This patch uses this wrapper library in
conjunction with mandoc to add support for zstd compression to it,
thereby allowing Guix man pages to be viewed with mandoc again.

Without this patch, mandoc is essentially defunct on Guix.

* gnu/packages/man.scm (mandoc): Support zstd compression.
* gnu/local.mk: Add new patch.
* gnu/packages/patches/mandoc-support-zstd-compression.patch: New file.
---
Change since v2: Update reference to the upstream ML in the patch file.

gnu/local.mk | 1 +
gnu/packages/man.scm | 54 ++++++++++------
.../mandoc-support-zstd-compression.patch | 63 +++++++++++++++++++
3 files changed, 100 insertions(+), 18 deletions(-)
create mode 100644 gnu/packages/patches/mandoc-support-zstd-compression.patch

Toggle diff (161 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 5fb354caae9..fef9a560284 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1807,6 +1807,7 @@ dist_patch_DATA = \
%D%/packages/patches/lxc-no-static-bin.patch \
%D%/packages/patches/mactelnet-remove-init.patch \
%D%/packages/patches/mailutils-variable-lookup.patch \
+ %D%/packages/patches/mandoc-support-zstd-compression.patch \
%D%/packages/patches/make-impure-dirs.patch \
%D%/packages/patches/mariadb-rocksdb-atomic-linking.patch \
%D%/packages/patches/mathjax-disable-webpack.patch \
diff --git a/gnu/packages/man.scm b/gnu/packages/man.scm
index 3148fcc8a16..1ff1fac991f 100644
--- a/gnu/packages/man.scm
+++ b/gnu/packages/man.scm
@@ -37,6 +37,7 @@ (define-module (gnu packages man)
#:use-module (guix build-system gnu)
#:use-module (guix build-system ruby)
#:use-module (guix utils)
+ #:use-module (gnu packages)
#:use-module (gnu packages base)
#:use-module (gnu packages compression)
#:use-module (gnu packages dbm)
@@ -273,30 +274,47 @@ (define-public mandoc
(method url-fetch)
(uri (string-append "https://mandoc.bsd.lv/snapshots/mandoc-"
version ".tar.gz"))
+ (patches (search-patches "mandoc-support-zstd-compression.patch"))
(sha256
(base32
"174x2x9ws47b14lm339j6rzm7mxy1j3qhh484khscw0yy1qdbw4b"))))
(build-system gnu-build-system)
(arguments
- `(#:test-target "regress"
- #:phases (modify-phases %standard-phases
- (add-before 'configure 'set-prefix
- (lambda* (#:key outputs #:allow-other-keys)
- (substitute* "configure"
- (("^CC=.*")
- (string-append "CC=" ,(cc-for-target) "\n"))
- (("^DEFCFLAGS=\\\\\"")
- "DEFCFLAGS=\"-O2 ")
- (("^UTF8_LOCALE=.*") ;used for tests
- "UTF8_LOCALE=en_US.UTF-8\n")
- (("^MANPATH_(BASE|DEFAULT)=.*" _ which)
- (string-append "MANPATH_" which "="
- "/run/current-system/profile/share/man\n"))
- (("^PREFIX=.*")
- (string-append "PREFIX=" (assoc-ref outputs "out")
- "\n"))))))))
+ (list
+ #:test-target "regress"
+ #:make-flags
+ #~(list "VPATH=./zstd-src/zlibWrapper"
+ (string-join
+ (list "CFLAGS=-DZWRAP_USE_ZSTD=1"
+ "-I./zstd-src/zlibWrapper")
+ " "))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'unpack-zstd
+ (lambda _
+ (mkdir "zstd-src")
+ (invoke "tar" "--strip-components=1" "-C"
+ "zstd-src" "-xf" #$(package-source zstd))))
+ (add-before 'configure 'set-prefix
+ (lambda* (#:key outputs #:allow-other-keys)
+ (substitute*
+ "configure"
+ (("^CC=.*")
+ (string-append "CC=" #$(cc-for-target) "\n"))
+ (("^DEFCFLAGS=\\\\\"")
+ "DEFCFLAGS=\"-O2 ")
+ (("^UTF8_LOCALE=.*") ;used for tests
+ "UTF8_LOCALE=en_US.UTF-8\n")
+ (("^MANPATH_(BASE|DEFAULT)=.*" _ which)
+ (string-append
+ "MANPATH_" which "="
+ "/run/current-system/profile/share/man\n"))
+ (("^PREFIX=.*")
+ (string-append "PREFIX="
+ (assoc-ref outputs "out")
+ "\n"))))))))
(native-inputs (list (libc-utf8-locales-for-target) perl)) ;used to run tests
- (inputs (list zlib))
+ (inputs (list zlib (list zstd "lib")))
(native-search-paths
(list (search-path-specification
(variable "MANPATH")
diff --git a/gnu/packages/patches/mandoc-support-zstd-compression.patch b/gnu/packages/patches/mandoc-support-zstd-compression.patch
new file mode 100644
index 00000000000..1c0ded56e4d
--- /dev/null
+++ b/gnu/packages/patches/mandoc-support-zstd-compression.patch
@@ -0,0 +1,63 @@
+mandoc upstream does not support zstd compression. However, Guix uses zstd
+compression for its man pages, therefore—without support for this compression
+method—mandoc would be quite useless. Hence, this patchset uses zlibWrapper
+from the zstd project to add zstd compression support to mandoc.
+
+Note that upstream is presently not interested in adding support for additional
+compression scheme, fourtunately, the patch is minimal and easy to maintain, see:
+
+ https://inbox.vuxu.org/mandoc-discuss/Z5i0H+XrKVrZqAXB@asta-kit.de/T/#t
+
+diff -upr mandoc-1.14.6.orig/Makefile mandoc-1.14.6/Makefile
+--- mandoc-1.14.6.orig/Makefile 2025-01-11 16:20:31.511129163 +0100
++++ mandoc-1.14.6/Makefile 2025-01-11 19:16:35.924788821 +0100
+@@ -251,7 +251,12 @@ LIBMANDOC_OBJS = $(LIBMAN_OBJS) \
+ msec.o \
+ preconv.o \
+ read.o \
+- tag.o
++ tag.o \
++ zstd_zlibwrapper.o \
++ gzclose.o \
++ gzlib.o \
++ gzread.o \
++ gzwrite.o
+
+ ALL_COBJS = compat_err.o \
+ compat_fts.o \
+Only in mandoc-1.14.6: Makefile.orig
+diff -upr mandoc-1.14.6.orig/configure mandoc-1.14.6/configure
+--- mandoc-1.14.6.orig/configure 2025-01-11 16:20:31.511129163 +0100
++++ mandoc-1.14.6/configure 2025-01-11 19:16:35.924788821 +0100
+@@ -430,7 +430,7 @@ fi
+ [ "${FATAL}" -eq 0 ] || exit 1
+
+ # --- LDADD ---
+-LDADD="${LDADD} ${LD_NANOSLEEP} ${LD_RECVMSG} ${LD_OHASH} -lz"
++LDADD="${LDADD} ${LD_NANOSLEEP} ${LD_RECVMSG} ${LD_OHASH} -lz -lzstd"
+ echo "selected LDADD=\"${LDADD}\"" 1>&2
+ echo "selected LDADD=\"${LDADD}\"" 1>&3
+ echo 1>&3
+Only in mandoc-1.14.6: configure.orig
+diff -upr mandoc-1.14.6.orig/read.c mandoc-1.14.6/read.c
+--- mandoc-1.14.6.orig/read.c 2025-01-11 16:35:03.825441715 +0100
++++ mandoc-1.14.6/read.c 2025-01-11 19:16:35.924788821 +0100
+@@ -37,7 +37,7 @@
+ #include <stdlib.h>
+ #include <string.h>
+ #include <unistd.h>
+-#include <zlib.h>
++#include <zstd_zlibwrapper.h>
+
+ #include "mandoc_aux.h"
+ #include "mandoc.h"
+@@ -627,7 +627,7 @@ mparse_open(struct mparse *curp, const char *file)
+ int fd, save_errno;
+
+ cp = strrchr(file, '.');
+- curp->gzip = (cp != NULL && ! strcmp(cp + 1, "gz"));
++ curp->gzip = (cp != NULL && (! strcmp(cp + 1, "gz") || ! strcmp(cp + 1, "zst")));
+
+ /* First try to use the filename as it is. */
+
+Only in mandoc-1.14.6: read.c.orig

base-commit: da69a9e15115d6acc7e4a95cc6295f97c97f827e
Sören Tempel wrote 1 months ago
Re: [bug#75501] [PATCH] gnu: mandoc: Support zstd-compressed man pages.
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
3L85LLP66SB0W.3KHGVB0OS294N@8pit.net
Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
Toggle quote (2 lines)
> Hi Sören,

Hello Maxim,

Toggle quote (4 lines)
> I see. I think just at least a mention of that issue and a pointer to
> the solution devised for Guix would be useful to share with upstream, in
> case it'd motivate them to address it themselves.

As requested, I did just that. There is a new thread on the mandoc
mailing list which discusses zstd compression support:


Feel free to add to it. I also send a v3 which updates the patch comment
accordingly to point to this thread. Is there anything else that needs
to be addressed?

Greetings
Sören
Maxim Cournoyer wrote 1 months ago
(name . Sören Tempel)(address . soeren@soeren-tempel.net)
87plk5pxl5.fsf@gmail.com
Pushed, thank you!

--
Maxim
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 75501@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 75501
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help