From debbugs-submit-bounces@debbugs.gnu.org Thu Jul 02 06:23:14 2020 Received: (at 42123) by debbugs.gnu.org; 2 Jul 2020 10:23:14 +0000 Received: from localhost ([127.0.0.1]:54422 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jqwN8-0000XW-4D for submit@debbugs.gnu.org; Thu, 02 Jul 2020 06:23:14 -0400 Received: from eggs.gnu.org ([209.51.188.92]:53362) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jqwN3-0000XF-T0 for 42123@debbugs.gnu.org; Thu, 02 Jul 2020 06:23:13 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58268) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jqwMy-0006lv-K9; Thu, 02 Jul 2020 06:23:04 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=53938 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jqwMx-0004JP-R7; Thu, 02 Jul 2020 06:23:04 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mathieu Othacehe Subject: Re: [bug#42123] [PATCH] linux-libre: Enable module compression. References: <20200629142434.21308-1-othacehe@gnu.org> Date: Thu, 02 Jul 2020 12:23:01 +0200 In-Reply-To: <20200629142434.21308-1-othacehe@gnu.org> (Mathieu Othacehe's message of "Mon, 29 Jun 2020 16:24:34 +0200") Message-ID: <87366atdve.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 42123 Cc: Mathieu Othacehe , 42123@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Mathieu Othacehe skribis: > This commit enables GZIP compression for linux-libre kernel modules, redu= cing > the size of linux-libre by 63% (165MB). The initrd modules are kept > uncompressed as the initrd is already compressed as a whole. > > The linux-libre kernel also supports XZ compression, but as Guix does not= have > any available bindings for now, and the compression time is far more > significant, GZIP seems to be a better option. > > * gnu/packages/aux-files/linux-libre/5.4-arm.conf: Enable GZ compression. > * gnu/packages/aux-files/linux-libre/5.4-arm64.conf: Ditto. > * gnu/packages/aux-files/linux-libre/5.4-i686.conf: Ditto. > * gnu/packages/aux-files/linux-libre/5.4-x86_64.conf: Ditto. > * gnu/build/linux-modules.scm (modinfo-section-contents): Use > 'call-with-gzip-input-port' to read from a module file using '.gz' extens= ion, > (strip-extension): new procedure, > (dot-ko): adapt to support compression, > (ensure-dot-ko): ditto, > (file-name->module-name): ditto, > (find-module-file): ditto, > (load-linux-module*): ditto, > (module-name->file-name/guess): ditto, > (module-name-lookup): ditto, > (write-module-name-database): ditto, > (write-module-alias-database): ditto, > (write-module-device-database): ditto. > * gnu/system/linux-initrd.scm (flat-linux-module-directory): Make sure th= at > zlib bindings are available because they may be used in > 'write-module-device-database'. Also make sure that the initrd only conta= ins > uncompressed module files. Nice! I do think that gzip is more appropriate than xz here, also in terms of memory requirements. Perhaps you can do this in two patches: first the linux-initrd bits, and 2nd the linux-libre changes. > diff --git a/gnu/build/linux-modules.scm b/gnu/build/linux-modules.scm > index aa1c7cfeae..7c6945a881 100644 > --- a/gnu/build/linux-modules.scm > +++ b/gnu/build/linux-modules.scm > @@ -21,6 +21,7 @@ > (define-module (gnu build linux-modules) > #:use-module (guix elf) > #:use-module (guix glob) > + #:use-module (guix zlib) > #:use-module (guix build syscalls) > #:use-module ((guix build utils) #:select (find-files invoke)) > #:use-module (guix build union) > @@ -97,7 +98,16 @@ string list." > (define (modinfo-section-contents file) > "Return the contents of the '.modinfo' section of FILE as a list of > key/value pairs.." > - (let* ((bv (call-with-input-file file get-bytevector-all)) > + (define (get-bytevector file) > + (cond > + ((string-contains file ".ko.gz") =E2=80=98string-suffix?=E2=80=99 would be more accurate. > + (call-with-input-file file > + (lambda (port) > + (call-with-gzip-input-port port get-bytevector-all)))) =E2=80=98call-with-input-file=E2=80=99 creates a buffered port, which could= be problematic, although =E2=80=98make-gzip-input-port=E2=80=99 checks that. To be safe, you=E2=80=99d do (open-file file "r0") with a dynwind. > +(define* (module-name->file-name/guess directory name > + #:key compression) > "Guess the file name corresponding to NAME, a module name. That doesn= 't > always work because sometimes underscores in NAME map to hyphens (e.g., > \"input-leds.ko\"), sometimes not (e.g., \"mac_hid.ko\")." Please mention COMPRESSION in the docstring. > (define (write-module-alias-database directory) > - "Traverse the '.ko' files in DIRECTORY and create the corresponding > + "Traverse the '.ko[.gz|.xz]' files in DIRECTORY and create the corresp= onding > 'modules.alias' file." > (define aliases > (map (lambda (file) > (cons (file-name->module-name file) (module-aliases file))) > - (find-files directory "\\.ko$"))) > + (find-files directory "\\.ko.*$"))) Should we refine this regexp (there are a couple of places like this)? There other Scheme bits LGTM! (I don=E2=80=99t really know about Linux-libre but you do!) Ludo=E2=80=99.