[bug#75501] gnu: mandoc: Support zstd-compressed man pages.
Commit Message
From: Sören Tempel <soeren+git@soeren-tempel.net>
Since #68242 Guix uses zstd compression for man pages. Unfortunately,
upstream mandoc only supports gzip compressed man pages. Luckily, zstd
provides a wrapper library which easily allows adapting software using
zlib to zstd compression. This patch uses this wrapper library in
conjunction with mandoc to add support for zstd compression to it,
thereby allowing Guix man pages to be viewed with mandoc again.
* gnu/packages/man.scm (mandoc): Support zstd compression.
* gnu/local.mk: Add new patch.
* gnu/packages/patches/mandoc-support-zstd-compression.patch: New file.
---
Without this patch, mandoc is essentially defunct on Guix. You cannot
view any Guix man pages with mandoc presently because of this issue.
gnu/local.mk | 1 +
gnu/packages/man.scm | 47 +++++++++------
.../mandoc-support-zstd-compression.patch | 58 +++++++++++++++++++
3 files changed, 89 insertions(+), 17 deletions(-)
create mode 100644 gnu/packages/patches/mandoc-support-zstd-compression.patch
Comments
Hello!
soeren@soeren-tempel.net writes:
> From: Sören Tempel <soeren+git@soeren-tempel.net>
>
> Since #68242 Guix uses zstd compression for man pages. Unfortunately,
> upstream mandoc only supports gzip compressed man pages. Luckily, zstd
> provides a wrapper library which easily allows adapting software using
> zlib to zstd compression. This patch uses this wrapper library in
> conjunction with mandoc to add support for zstd compression to it,
> thereby allowing Guix man pages to be viewed with mandoc again.
Oh! I have never used mandoc, only man-db.
> * gnu/packages/man.scm (mandoc): Support zstd compression.
> * gnu/local.mk: Add new patch.
> * gnu/packages/patches/mandoc-support-zstd-compression.patch: New
> file.
[...]
> (arguments
> `(#:test-target "regress"
> - #:phases (modify-phases %standard-phases
> - (add-before 'configure 'set-prefix
> - (lambda* (#:key outputs #:allow-other-keys)
> - (substitute* "configure"
> - (("^CC=.*")
> - (string-append "CC=" ,(cc-for-target) "\n"))
> - (("^DEFCFLAGS=\\\\\"")
> - "DEFCFLAGS=\"-O2 ")
> - (("^UTF8_LOCALE=.*") ;used for tests
> - "UTF8_LOCALE=en_US.UTF-8\n")
> - (("^MANPATH_(BASE|DEFAULT)=.*" _ which)
> - (string-append "MANPATH_" which "="
> - "/run/current-system/profile/share/man\n"))
> - (("^PREFIX=.*")
> - (string-append "PREFIX=" (assoc-ref outputs "out")
> - "\n"))))))))
> + #:make-flags
> + (list "VPATH=./zstd-src/zlibWrapper"
> + (string-join
> + (list "CFLAGS=-DZWRAP_USE_ZSTD=1"
> + (string-append "-I./zstd-src/zlibWrapper"))
> + " "))
> + #:phases ,#~(modify-phases %standard-phases
> + (add-after 'unpack 'unpack-zstd
> + (lambda _
> + (mkdir "zstd-src")
> + (invoke "tar" "--strip-components=1" "-C" "zstd-src"
> + "-xf" #$(package-source zstd))))
> + (add-before 'configure 'set-prefix
> + (lambda* (#:key outputs #:allow-other-keys)
> + (substitute* "configure"
> + (("^CC=.*")
> + (string-append "CC=" #$(cc-for-target) "\n"))
> + (("^DEFCFLAGS=\\\\\"")
> + "DEFCFLAGS=\"-O2 ")
> + (("^UTF8_LOCALE=.*") ;used for tests
> + "UTF8_LOCALE=en_US.UTF-8\n")
> + (("^MANPATH_(BASE|DEFAULT)=.*" _ which)
> + (string-append "MANPATH_" which "="
> + "/run/current-system/profile/share/man\n"))
> + (("^PREFIX=.*")
> + (string-append "PREFIX=" (assoc-ref outputs "out")
> + "\n"))))))))
While moving things around, I'd use a plain list for the arguments, and
format the phases under the #:phases argument (newline), to satisfy our
max 80 columns of width convention.
> (native-inputs (list (libc-utf8-locales-for-target) perl)) ;used to run tests
> - (inputs (list zlib))
> + (inputs (list zlib (list zstd "lib")))
> (native-search-paths
> (list (search-path-specification
> (variable "MANPATH")
> diff --git a/gnu/packages/patches/mandoc-support-zstd-compression.patch b/gnu/packages/patches/mandoc-support-zstd-compression.patch
> new file mode 100644
> index 0000000000..b8cbeb6782
> --- /dev/null
> +++ b/gnu/packages/patches/mandoc-support-zstd-compression.patch
> @@ -0,0 +1,58 @@
> +mandoc upstream does not support zstd compression. However, Guix uses zstd
> +compression for its man pages, therefore—without support for this compression
> +method—mandoc would be quite useless. Hence, this patchset uses zlibWrapper
> +from the zstd project to add zstd compression support to mandoc.
Interesting solution! The issue should be ideally be brought upstream
though, and referenced here. With the leading implementation man-db
having gained zstd support, others should follow suite. They may be
interested in using your solution, or otherwise adding "native" support
for it.
Could you please create such an issue with them, cross-reference it in
your patch, and send a v2?
Hello Maxim,
thank you for having a look at the patch and your fast feedback!
Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
> While moving things around, I'd use a plain list for the arguments, and
> format the phases under the #:phases argument (newline), to satisfy our
> max 80 columns of width convention.
Changed accordingly in the v2 that I just send.
> Interesting solution! The issue should be ideally be brought upstream
> though, and referenced here. With the leading implementation man-db
> having gained zstd support, others should follow suite. They may be
> interested in using your solution, or otherwise adding "native" support
> for it.
>
> Could you please create such an issue with them, cross-reference it in
> your patch, and send a v2?
Having worked with upstream mandoc before, I doubt that they would be
interested in implementing additional compression schemes. mandoc is an
OpenBSD project, and OpenBSD doesn't compress its man pages. Usually,
its hard to convince them to add features that do not benefit OpenBSD.
Specifically, regarding compression there is a prior thread on adding
bzip2 where the mandoc maintainer states “compressing manual pages
makes absolutely no sense to me […]”:
https://inbox.vuxu.org/mandoc-discuss/20201129201424.GI58187@athene.usta.de/
The Guix patch is very small so that I don't see any issue with
maintaining it downstream for the foreseeable future. Nonetheless, I
added a link to the aforementioned thread to the patch description.
If it is a requirement, I can create another “compression support”
thread on their ML but I would prefer to use my time elsewhere.
Greetings
Sören
Hi Sören,
Sören Tempel <soeren@soeren-tempel.net> writes:
> Hello Maxim,
>
> thank you for having a look at the patch and your fast feedback!
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>> While moving things around, I'd use a plain list for the arguments, and
>> format the phases under the #:phases argument (newline), to satisfy our
>> max 80 columns of width convention.
>
> Changed accordingly in the v2 that I just send.
>
>> Interesting solution! The issue should be ideally be brought upstream
>> though, and referenced here. With the leading implementation man-db
>> having gained zstd support, others should follow suite. They may be
>> interested in using your solution, or otherwise adding "native" support
>> for it.
>>
>> Could you please create such an issue with them, cross-reference it in
>> your patch, and send a v2?
>
> Having worked with upstream mandoc before, I doubt that they would be
> interested in implementing additional compression schemes. mandoc is an
> OpenBSD project, and OpenBSD doesn't compress its man pages. Usually,
> its hard to convince them to add features that do not benefit OpenBSD.
I see. I think just at least a mention of that issue and a pointer to
the solution devised for Guix would be useful to share with upstream, in
case it'd motivate them to address it themselves.
You could argue why mandoc is such a great piece of software that you
want to use it over man-db on other systems than OpenBSD, they may
soften a bit on their position, ha!
> Specifically, regarding compression there is a prior thread on adding
> bzip2 where the mandoc maintainer states “compressing manual pages
> makes absolutely no sense to me […]”:
>
> https://inbox.vuxu.org/mandoc-discuss/20201129201424.GI58187@athene.usta.de/
Interesting position, but a bit extreme. It's true that a few megabytes
saved are not the end of the world, especially if users are using full
disk compression such as Btrfs + Zstd, but it's still a plus in my book.
I had done some limited number crunching in [0], which found zstd mildy
better at compressing man pages than zstd (about 10% size reduction),
and 1.5x faster at decompressing.
[0] https://issues.guix.gnu.org/68242#16
> If it is a requirement, I can create another “compression support”
> thread on their ML but I would prefer to use my time elsewhere.
I'd encourage you to do so, whatever the outcome.
Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
> Hi Sören,
Hello Maxim,
> I see. I think just at least a mention of that issue and a pointer to
> the solution devised for Guix would be useful to share with upstream, in
> case it'd motivate them to address it themselves.
As requested, I did just that. There is a new thread on the mandoc
mailing list which discusses zstd compression support:
https://inbox.vuxu.org/mandoc-discuss/Z5i0H+XrKVrZqAXB@asta-kit.de/T/#t
Feel free to add to it. I also send a v3 which updates the patch comment
accordingly to point to this thread. Is there anything else that needs
to be addressed?
Greetings
Sören
@@ -1799,6 +1799,7 @@ dist_patch_DATA = \
%D%/packages/patches/lxc-no-static-bin.patch \
%D%/packages/patches/mactelnet-remove-init.patch \
%D%/packages/patches/mailutils-variable-lookup.patch \
+ %D%/packages/patches/mandoc-support-zstd-compression.patch \
%D%/packages/patches/make-impure-dirs.patch \
%D%/packages/patches/mariadb-rocksdb-atomic-linking.patch \
%D%/packages/patches/mathjax-disable-webpack.patch \
@@ -37,6 +37,7 @@ (define-module (gnu packages man)
#:use-module (guix build-system gnu)
#:use-module (guix build-system ruby)
#:use-module (guix utils)
+ #:use-module (gnu packages)
#:use-module (gnu packages base)
#:use-module (gnu packages compression)
#:use-module (gnu packages dbm)
@@ -273,30 +274,42 @@ (define-public mandoc
(method url-fetch)
(uri (string-append "https://mandoc.bsd.lv/snapshots/mandoc-"
version ".tar.gz"))
+ (patches (search-patches "mandoc-support-zstd-compression.patch"))
(sha256
(base32
"174x2x9ws47b14lm339j6rzm7mxy1j3qhh484khscw0yy1qdbw4b"))))
(build-system gnu-build-system)
(arguments
`(#:test-target "regress"
- #:phases (modify-phases %standard-phases
- (add-before 'configure 'set-prefix
- (lambda* (#:key outputs #:allow-other-keys)
- (substitute* "configure"
- (("^CC=.*")
- (string-append "CC=" ,(cc-for-target) "\n"))
- (("^DEFCFLAGS=\\\\\"")
- "DEFCFLAGS=\"-O2 ")
- (("^UTF8_LOCALE=.*") ;used for tests
- "UTF8_LOCALE=en_US.UTF-8\n")
- (("^MANPATH_(BASE|DEFAULT)=.*" _ which)
- (string-append "MANPATH_" which "="
- "/run/current-system/profile/share/man\n"))
- (("^PREFIX=.*")
- (string-append "PREFIX=" (assoc-ref outputs "out")
- "\n"))))))))
+ #:make-flags
+ (list "VPATH=./zstd-src/zlibWrapper"
+ (string-join
+ (list "CFLAGS=-DZWRAP_USE_ZSTD=1"
+ (string-append "-I./zstd-src/zlibWrapper"))
+ " "))
+ #:phases ,#~(modify-phases %standard-phases
+ (add-after 'unpack 'unpack-zstd
+ (lambda _
+ (mkdir "zstd-src")
+ (invoke "tar" "--strip-components=1" "-C" "zstd-src"
+ "-xf" #$(package-source zstd))))
+ (add-before 'configure 'set-prefix
+ (lambda* (#:key outputs #:allow-other-keys)
+ (substitute* "configure"
+ (("^CC=.*")
+ (string-append "CC=" #$(cc-for-target) "\n"))
+ (("^DEFCFLAGS=\\\\\"")
+ "DEFCFLAGS=\"-O2 ")
+ (("^UTF8_LOCALE=.*") ;used for tests
+ "UTF8_LOCALE=en_US.UTF-8\n")
+ (("^MANPATH_(BASE|DEFAULT)=.*" _ which)
+ (string-append "MANPATH_" which "="
+ "/run/current-system/profile/share/man\n"))
+ (("^PREFIX=.*")
+ (string-append "PREFIX=" (assoc-ref outputs "out")
+ "\n"))))))))
(native-inputs (list (libc-utf8-locales-for-target) perl)) ;used to run tests
- (inputs (list zlib))
+ (inputs (list zlib (list zstd "lib")))
(native-search-paths
(list (search-path-specification
(variable "MANPATH")
new file mode 100644
@@ -0,0 +1,58 @@
+mandoc upstream does not support zstd compression. However, Guix uses zstd
+compression for its man pages, therefore—without support for this compression
+method—mandoc would be quite useless. Hence, this patchset uses zlibWrapper
+from the zstd project to add zstd compression support to mandoc.
+
+diff -upr mandoc-1.14.6.orig/Makefile mandoc-1.14.6/Makefile
+--- mandoc-1.14.6.orig/Makefile 2025-01-11 16:20:31.511129163 +0100
++++ mandoc-1.14.6/Makefile 2025-01-11 19:16:35.924788821 +0100
+@@ -251,7 +251,12 @@ LIBMANDOC_OBJS = $(LIBMAN_OBJS) \
+ msec.o \
+ preconv.o \
+ read.o \
+- tag.o
++ tag.o \
++ zstd_zlibwrapper.o \
++ gzclose.o \
++ gzlib.o \
++ gzread.o \
++ gzwrite.o
+
+ ALL_COBJS = compat_err.o \
+ compat_fts.o \
+Only in mandoc-1.14.6: Makefile.orig
+diff -upr mandoc-1.14.6.orig/configure mandoc-1.14.6/configure
+--- mandoc-1.14.6.orig/configure 2025-01-11 16:20:31.511129163 +0100
++++ mandoc-1.14.6/configure 2025-01-11 19:16:35.924788821 +0100
+@@ -430,7 +430,7 @@ fi
+ [ "${FATAL}" -eq 0 ] || exit 1
+
+ # --- LDADD ---
+-LDADD="${LDADD} ${LD_NANOSLEEP} ${LD_RECVMSG} ${LD_OHASH} -lz"
++LDADD="${LDADD} ${LD_NANOSLEEP} ${LD_RECVMSG} ${LD_OHASH} -lz -lzstd"
+ echo "selected LDADD=\"${LDADD}\"" 1>&2
+ echo "selected LDADD=\"${LDADD}\"" 1>&3
+ echo 1>&3
+Only in mandoc-1.14.6: configure.orig
+diff -upr mandoc-1.14.6.orig/read.c mandoc-1.14.6/read.c
+--- mandoc-1.14.6.orig/read.c 2025-01-11 16:35:03.825441715 +0100
++++ mandoc-1.14.6/read.c 2025-01-11 19:16:35.924788821 +0100
+@@ -37,7 +37,7 @@
+ #include <stdlib.h>
+ #include <string.h>
+ #include <unistd.h>
+-#include <zlib.h>
++#include <zstd_zlibwrapper.h>
+
+ #include "mandoc_aux.h"
+ #include "mandoc.h"
+@@ -627,7 +627,7 @@ mparse_open(struct mparse *curp, const char *file)
+ int fd, save_errno;
+
+ cp = strrchr(file, '.');
+- curp->gzip = (cp != NULL && ! strcmp(cp + 1, "gz"));
++ curp->gzip = (cp != NULL && (! strcmp(cp + 1, "gz") || ! strcmp(cp + 1, "zst")));
+
+ /* First try to use the filename as it is. */
+
+Only in mandoc-1.14.6: read.c.orig