Thoughts on using Zstd compression instead of lzip

  • Done
  • quality assurance status badge
Details
3 participants
  • Ludovic Courtès
  • Tobias Geerinckx-Rice
  • romulasry
Owner
unassigned
Submitted by
romulasry
Severity
normal
R
R
romulasry wrote on 10 Jun 2020 21:22
(name . bug-guix@gnu.org)(address . bug-guix@gnu.org)
mQL02MBzUI0Or1OBC-6OI05eEgYrLxIalc2JKMZWQg6ysB93UfrP-rfzmFOKYvauVJ1lLmyGHVBz3drFUKRR6tSgTaaBP58Is7f0rX98UAE=@protonmail.com

Zstandard is a real-time compression algorithm, providing high compression ratios. It offers a very wide range of compression / speed trade-off, while being backed by a very fast decoder (see [benchmarks](http://facebook.github.io/zstd/#benchmarks)below). It also offers a special mode for small data, called [dictionary compression](http://facebook.github.io/zstd/#small-data), and can create dictionaries from any sample set. Zstandard library is provided as open source software using a BSD license.

Sent with [ProtonMail](https://protonmail.com) Secure Email.
Attachment: file
T
T
Tobias Geerinckx-Rice wrote on 10 Jun 2020 22:59
(name . romulasry)(address . romulasry@protonmail.com)
87eeqmr60v.fsf@nckx
Romulas,

It's not clear where you want to see zstd used. Here's a trivial
patch to add Zstd to ‘guix pack’, if that's what you mean.

I didn't submit it in the past because I dread bikeshedding the
compression level. -3 is the default, and compresses slightly
better than gzip in a fraction of the time.

Kind regards,

T G-R
From fa2ead6ac0f6c005e6392d6073122c7191e08944 Mon Sep 17 00:00:00 2001
From: Tobias Geerinckx-Rice <me@tobias.gr>
Date: Fri, 08 Nov 2019 09:03:27 +0100
Subject: [PATCH] pack: Add zstd compressor.

* guix/scripts/pack.scm (%compressors): Add ztsd.
---
guix/scripts/pack.scm | 3 +++
1 file changed, 3 insertions(+)

Toggle diff (23 lines)
diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index 518bf6e7e3..5e71ca9c21 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -4,6 +4,7 @@
;;; Copyright © 2018 Konrad Hinsen <konrad.hinsen@fastmail.net>
;;; Copyright © 2018 Chris Marusich <cmmarusich@gmail.com>
;;; Copyright © 2018 Efraim Flashner <efraim@flashner.co.il>
+;;; Copyright © 2019 Tobias Geerinckx-Rice <me@tobias.gr>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -81,6 +82,8 @@
#~(#+(file-append xz "/bin/xz") "-e"))
(compressor "bzip2" ".bz2"
#~(#+(file-append bzip2 "/bin/bzip2") "-9"))
+ (compressor "zstd" ".zst"
+ #~(#+(file-append zstd "/bin/zstd") "-3"))
(compressor "none" "" #f)))
;; This one is only for use in this module, so don't put it in %compressors.
--
2.26.2
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQT12iAyS4c9C3o4dnINsP+IT1VteQUCXuFJwAAKCRANsP+IT1Vt
eYoBAQD+Xz1vLVyGgIhRPL5EDcK/Dc87Ag8CLMfXoSuNXDXILwD/fAXb2MzWkYb/
FX3NMHpUGl9jklzKOelhgtcLQwDzOQ4=
=Ymr5
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 11 Jun 2020 23:21
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)
874krh9u4c.fsf@gnu.org
Hi,

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

Toggle quote (3 lines)
> It's not clear where you want to see zstd used. Here's a trivial
> patch to add Zstd to ‘guix pack’, if that's what you mean.

I suppose that was for substitutes, in reference to:


The idea is worth discussing, but IMO, romulasry, discussions should
happen on guix-devel rather than guix-patches… unless the discussion
comes with a patch, of course. :-)

Closing!

Ludo’.
L
L
Ludovic Courtès wrote on 11 Jun 2020 23:21
control message for bug #41794
(address . control@debbugs.gnu.org)
8736719u41.fsf@gnu.org
tags 41794 notabug
close 41794
quit
L
L
Ludovic Courtès wrote on 13 Jun 2020 22:30
Re: bug#41794: Thoughts on using Zstd compression instead of lzip
(name . Tobias Geerinckx-Rice via Bug reports for GNU Guix)(address . bug-guix@gnu.org)
875zbuyaho.fsf@gnu.org
Hi,

Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix@gnu.org>
skribis:

Toggle quote (7 lines)
> From fa2ead6ac0f6c005e6392d6073122c7191e08944 Mon Sep 17 00:00:00 2001
> From: Tobias Geerinckx-Rice <me@tobias.gr>
> Date: Fri, 08 Nov 2019 09:03:27 +0100
> Subject: [PATCH] pack: Add zstd compressor.
>
> * guix/scripts/pack.scm (%compressors): Add ztsd.

BTW, feel free to push it after mentioning zstd for -C under “Invoking
guix pack”.

Thanks,
Ludo’.
?