From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 03 22:15:51 2023 Received: (at submit) by debbugs.gnu.org; 4 Mar 2023 03:15:51 +0000 Received: from localhost ([127.0.0.1]:35013 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pYINB-0001fN-HK for submit@debbugs.gnu.org; Fri, 03 Mar 2023 22:15:51 -0500 Received: from lists.gnu.org ([209.51.188.17]:37098) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pYIN9-0001fF-Mn for submit@debbugs.gnu.org; Fri, 03 Mar 2023 22:15:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYIN9-0005If-Eh for guix-patches@gnu.org; Fri, 03 Mar 2023 22:15:47 -0500 Received: from mail-qv1-xf31.google.com ([2607:f8b0:4864:20::f31]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pYIN3-0000hy-H1 for guix-patches@gnu.org; Fri, 03 Mar 2023 22:15:46 -0500 Received: by mail-qv1-xf31.google.com with SMTP id g9so1388016qvt.8 for ; Fri, 03 Mar 2023 19:15:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Ca7UgmTmyEZ0Gb1P9L25PYwbuWbEkWkYryWTD6PEB18=; b=P+b0ggMvmQxzA/hdccGEK9x0P/Qv/ucU9mrg7As96xHmgbx2AT9BVj4rlHrbGsCAZp aqKCVqEX//hSe/HMkcAsPASyYtpw5i+84QFpOGXNoJ7AtSGWAkoYXT+Ft0sMxu18ob9f 6+Otuwqc7fFHFeSFBSy5+u7VcnBboq9VznycMOT3E65s+sVG0PPrz/u2c8dwB8mC3Cs8 iCo4zHXyeWM66sRc2U7MaGbjv32omTz9rB5DKLLUbjufWl4jK/bDi3c53SOgSIDxWdjz +wH0uY2QH62U5OaMf9ZcpCeeicrxPCazNXraFoKFOV06Af9Mj/xWrRcERIoVBk2N3Neb 7FfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Ca7UgmTmyEZ0Gb1P9L25PYwbuWbEkWkYryWTD6PEB18=; b=J3dwBvtViskenYoS86J/WhW+u25cmqer4JMz3Rs4FjGPKsZm5OZ9HNXGQ1ke4KLV// v3mF0pmItYpop59K7BLV1TyHlbM7Ca4q9KEIcwmA8QzGSBdfRgtsJ4A2724OKtEHvFmj jI56S16Y5QV9BBmxPH/LgxQRPEUt4v71JUxZA7iYF/aSp5aY9IS+qPB14jSi8QoqzqsW ZR99oiEJUGFrK0Tl2icDEhMRgyXbJwmALXFb8Tge5duqLrQiVZsKBNG/hfD+akNgyxiN 6AbCVeL7FjXQWWU7whTBoGqzivzUHds7Ll0MQOW07Nu3a5jACKqhaXQYg3tf/69ln+o3 koSQ== X-Gm-Message-State: AO0yUKXFdl4QicXsga6uhgFV42XfVLbFWsixaE8Hw6fX6k39rkc9Bhq1 bU0P37hgYa1zQz1q3qfWMKOa3HPzXysqRQ== X-Google-Smtp-Source: AK7set+kpxknzXED64uwY9MM447GH7aUA1sb+lkXM/VTocd355+Qs/qQBe+L77zHIwbgIZ+rZ4VBYA== X-Received: by 2002:a05:6214:4109:b0:56f:378:954 with SMTP id kc9-20020a056214410900b0056f03780954mr13524122qvb.3.1677899738922; Fri, 03 Mar 2023 19:15:38 -0800 (PST) Received: from localhost.localdomain (dsl-10-129-180.b2b2c.ca. [72.10.129.180]) by smtp.gmail.com with ESMTPSA id t33-20020a05622a182100b003b646123691sm3057169qtc.31.2023.03.03.19.15.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Mar 2023 19:15:38 -0800 (PST) From: Maxim Cournoyer To: guix-patches@gnu.org Subject: [PATCH] pack: Move common build code to (guix build pack). Date: Fri, 3 Mar 2023 22:15:23 -0500 Message-Id: <20230304031523.24102-1-maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.39.1 MIME-Version: 1.0 X-Debbugs-Cc: zimon.toutoune@gmail.com X-Debbugs-Cc: dev@jpoiret.xyz X-Debbugs-Cc: mail@cbaines.net X-Debbugs-Cc: rekado@elephly.net X-Debbugs-Cc: othacehe@gnu.org X-Debbugs-Cc: me@tobias.gr X-Debbugs-Cc: ludo@gnu.org Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=2607:f8b0:4864:20::f31; envelope-from=maxim.cournoyer@gmail.com; helo=mail-qv1-xf31.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: Maxim Cournoyer X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) The rationale is to reduce the number of derivations built per pack to ideally one, to minimize storage requirements. The number of derivations had gone up with 68380db4 ("pack: Extract populate-profile-root from self-contained-tarball/builder.") as a side effect to improving code reuse. * guix/scripts/pack.scm (guix): Add commentary comment. (populate-profile-root, self-contained-tarball/builder): Extract to... * guix/build/pack.scm (populate-profile-root!): ... this, and... (build-self-contained-tarball): ... that, adjusting for use on the build side. (assert-utf8-locale): New procedure. (self-contained-tarball, debian-archive, rpm-archive): Adjust accordingly. --- guix/build/pack.scm | 115 +++++++++++++- guix/scripts/pack.scm | 341 +++++++++++++++--------------------------- tests/pack.scm | 104 ++++++------- 3 files changed, 284 insertions(+), 276 deletions(-) diff --git a/guix/build/pack.scm b/guix/build/pack.scm index 3b73d1b227..fa9a5f5905 100644 --- a/guix/build/pack.scm +++ b/guix/build/pack.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2021 Maxim Cournoyer +;;; Copyright © 2021, 2023 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -16,9 +16,26 @@ ;;; You should have received a copy of the GNU General Public License ;;; along with GNU Guix. If not, see . +;;; Commentary: + +;;; This module contains build-side common procedures used by the host-side +;;; (guix scripts pack) module, mostly to allow for code reuse. Due to making +;;; use of the (guix build store-copy) module, it transitively requires the +;;; sqlite and gcrypt extensions to be available. + +;;; Code: + (define-module (guix build pack) + #:use-module (gnu build install) #:use-module (guix build utils) - #:export (tar-base-options)) + #:use-module (guix build store-copy) + #:use-module ((guix build union) #:select (relative-file-name)) + #:use-module (ice-9 match) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:export (tar-base-options + populate-profile-root! + build-self-contained-tarball)) (define* (tar-base-options #:key tar compressor) "Return the base GNU tar options required to produce deterministic archives @@ -52,3 +69,97 @@ (define (tar-supports-sort? tar) ;; process. Use '--hard-dereference' to eliminate it. "--hard-dereference" "--check-links")) + +(define (assert-utf8-locale) + "Verify the current process is using the en_US.utf8 locale." + (unless (string=? "unset for tests" (getenv "GUIX_LOCPATH")) + (unless (false-if-exception (setlocale LC_ALL "en_US.utf8")) + (error "environment not configured for en_US.utf8 locale")))) + +(define* (populate-profile-root! profile + #:key (profile-name "guix-profile") + localstatedir? + store-database + deduplicate? + (symlinks '())) + "Populate the root profile directory with SYMLINKS and a Guix database, when +LOCALSTATEDIR? is set, and a pre-computed STORE-DATABASE is provided. The +directory is created as \"root\" in the current working directory. When +DEDUPLICATE? is true, deduplicate the store items, which relies on hard +links. It needs to run in an environment where " + (when localstatedir? + (unless store-database + (error "missing STORE-DATABASE argument"))) + + (define symlink->directives + ;; Return "populate directives" to make the given symlink and its + ;; parent directories. + (match-lambda + ((source '-> target) + (let ((target (string-append profile "/" target)) + (parent (dirname source))) + ;; Never add a 'directory' directive for "/" so as to + ;; preserve its ownership when extracting the archive (see + ;; below), and also because this would lead to adding the + ;; same entries twice in the tarball. + `(,@(if (string=? parent "/") + '() + `((directory ,parent))) + ;; Use a relative file name for compatibility with + ;; relocatable packs. + (,source -> ,(relative-file-name parent target))))))) + + (define directives + ;; Fully-qualified symlinks. + (append-map symlink->directives symlinks)) + + (define %root "root") + + (assert-utf8-locale) + + ;; Note: there is not much to gain here with deduplication and there + ;; is the overhead of the '.links' directory, so turn it off by + ;; default. Furthermore GNU tar < 1.30 sometimes fails to extract + ;; tarballs with hard links: + ;; . + (populate-store (list "profile") %root #:deduplicate? deduplicate?) + + (when localstatedir? + (install-database-and-gc-roots %root store-database + profile #:profile-name profile-name)) + + ;; Create SYMLINKS. + (for-each (cut evaluate-populate-directive <> %root) directives)) + +(define* (build-self-contained-tarball profile + tarball-file-name + #:key (profile-name "guix-profile") + target + localstatedir? + store-database + deduplicate? + symlinks + compressor-command + archiver) + "Create a self-contained tarball TARBALL-FILE-NAME from PROFILE, optionally +compressing it with COMPRESSOR-COMMAND, the complete command-line string to +use for the compressor." + (assert-utf8-locale) + + (populate-profile-root! profile + #:profile-name profile-name + #:localstatedir? localstatedir? + #:store-database store-database + #:deduplicate? deduplicate? + #:symlinks symlinks) + + (define tar (string-append archiver "/bin/tar")) + + ;; GNU Tar recurses directories by default. Simply add the whole root + ;; directory, which contains all the files to be archived. This avoids + ;; creating duplicate files in the archives that would be stored as hard + ;; links by GNU Tar. + (apply invoke tar "-cvf" tarball-file-name "-C" "root" "." + (tar-base-options + #:tar tar + #:compressor compressor-command))) diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index eb41eb5563..984622bd16 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -24,6 +24,14 @@ ;;; You should have received a copy of the GNU General Public License ;;; along with GNU Guix. If not, see . +;;; Commentary: + +;;; This module implements the 'guix pack' command and the various supported +;;; formats. Where feasible, the builders of the packs should be implemented +;;; as single derivations to minimize storage requirements. + +;;; Code: + (define-module (guix scripts pack) #:use-module (guix scripts) #:use-module (guix ui) @@ -199,153 +207,18 @@ (define (set-utf8-locale profile) "Configure the environment to use the \"en_US.utf8\" locale provided by the GLIBC-UT8-LOCALES package." ;; Arrange to not depend on 'glibc-utf8-locales' when using '--bootstrap'. - (and (or (not (profile? profile)) - (profile-locales? profile)) - #~(begin - (setenv "GUIX_LOCPATH" - #+(file-append glibc-utf8-locales "/lib/locale")) - (setlocale LC_ALL "en_US.utf8")))) - -(define* (populate-profile-root profile - #:key (profile-name "guix-profile") - target - localstatedir? - deduplicate? - (symlinks '())) - "Populate the root profile directory with SYMLINKS and a Guix database, when -LOCALSTATEDIR? is set. When DEDUPLICATE? is true, deduplicate the store -items, which relies on hard links." - (define database - (and localstatedir? - (file-append (store-database (list profile)) - "/db/db.sqlite"))) - - (define bootstrap? - ;; Whether a '--bootstrap' environment is needed, for testing purposes. - ;; XXX: Infer that from available info. - (and (not database) (not (profile-locales? profile)))) - - (define (import-module? module) - ;; Since we don't use deduplication support in 'populate-store', don't - ;; import (guix store deduplication) and its dependencies, which includes - ;; Guile-Gcrypt, unless DEDUPLICATE? is #t. This makes it possible to run - ;; tests with '--bootstrap'. - (and (not-config? module) - (or deduplicate? (not (equal? '(guix store deduplication) module))))) - - (computed-file "profile-directory" - (with-imported-modules (source-module-closure - `((guix build pack) - (guix build store-copy) - (guix build utils) - (guix build union) - (gnu build install)) - #:select? import-module?) + (if (or (not (profile? profile)) + (profile-locales? profile)) #~(begin - (use-modules (guix build pack) - (guix build store-copy) - (guix build utils) - ((guix build union) #:select (relative-file-name)) - (gnu build install) - (srfi srfi-1) - (srfi srfi-26) - (ice-9 match)) - - (define symlink->directives - ;; Return "populate directives" to make the given symlink and its - ;; parent directories. - (match-lambda - ((source '-> target) - (let ((target (string-append #$profile "/" target)) - (parent (dirname source))) - ;; Never add a 'directory' directive for "/" so as to - ;; preserve its ownership when extracting the archive (see - ;; below), and also because this would lead to adding the - ;; same entries twice in the tarball. - `(,@(if (string=? parent "/") - '() - `((directory ,parent))) - ;; Use a relative file name for compatibility with - ;; relocatable packs. - (,source -> ,(relative-file-name parent target))))))) - - (define directives - ;; Fully-qualified symlinks. - (append-map symlink->directives '#$symlinks)) - - ;; Make sure non-ASCII file names are properly handled. - #+(set-utf8-locale profile) - - ;; Note: there is not much to gain here with deduplication and there - ;; is the overhead of the '.links' directory, so turn it off by - ;; default. Furthermore GNU tar < 1.30 sometimes fails to extract - ;; tarballs with hard links: - ;; . - (populate-store (list "profile") #$output - #:deduplicate? #$deduplicate?) - - (when #+localstatedir? - (install-database-and-gc-roots #$output #+database #$profile - #:profile-name #$profile-name)) - - ;; Create SYMLINKS. - (for-each (cut evaluate-populate-directive <> #$output) - directives))) - #:local-build? #f - #:guile (if bootstrap? %bootstrap-guile (default-guile)) - #:options (list #:references-graphs `(("profile" ,profile)) - #:target target))) + (setenv "GUIX_LOCPATH" + #+(file-append glibc-utf8-locales "/lib/locale")) + (setlocale LC_ALL "en_US.utf8")) + #~(setenv "GUIX_LOCPATH" "unset for tests"))) ;;; ;;; Tarball format. ;;; -(define* (self-contained-tarball/builder profile - #:key (profile-name "guix-profile") - target - localstatedir? - deduplicate? - symlinks - compressor - archiver) - "Return a GEXP that can build a self-contained tarball." - - (define root (populate-profile-root profile - #:profile-name profile-name - #:target target - #:localstatedir? localstatedir? - #:deduplicate? deduplicate? - #:symlinks symlinks)) - - (with-imported-modules (source-module-closure '((guix build pack) - (guix build utils))) - #~(begin - (use-modules (guix build pack) - (guix build utils)) - - ;; Make sure non-ASCII file names are properly handled. - #+(set-utf8-locale profile) - - (define tar #+(file-append archiver "/bin/tar")) - - (define %root (if #$localstatedir? "." #$root)) - - (when #$localstatedir? - ;; Fix the permission of the Guix database file, which was made - ;; read-only when copied to the store in populate-profile-root. - (copy-recursively #$root %root) - (chmod (string-append %root "/var/guix/db/db.sqlite") #o644)) - - (with-directory-excursion %root - ;; GNU Tar recurses directories by default. Simply add the whole - ;; current directory, which contains all the files to be archived. - ;; This avoids creating duplicate files in the archives that would - ;; be stored as hard links by GNU Tar. - (apply invoke tar "-cvf" #$output "." - (tar-base-options - #:tar tar - #:compressor #+(and=> compressor compressor-command))))))) - (define* (self-contained-tarball name profile #:key target (profile-name "guix-profile") @@ -367,16 +240,39 @@ (define* (self-contained-tarball name profile (warning (G_ "entry point not supported in the '~a' format~%") 'tarball)) + (define database + (and localstatedir? + (file-append (store-database (list profile)) + "/db/db.sqlite"))) + (gexp->derivation (string-append name ".tar" (compressor-extension compressor)) - (self-contained-tarball/builder profile - #:profile-name profile-name - #:target target - #:localstatedir? localstatedir? - #:deduplicate? deduplicate? - #:symlinks symlinks - #:compressor compressor - #:archiver archiver))) + (with-extensions (list guile-gcrypt) + (with-imported-modules `(((guix config) => ,(make-config.scm)) + ,@(source-module-closure + `((guix build pack) + (guix build utils)) + #:select? not-config?)) + #~(begin + (use-modules (guix build pack) + (guix build utils)) + + ;; Make sure non-ASCII file names are properly handled. + #+(set-utf8-locale profile) + + (build-self-contained-tarball #$profile + #$output + #:profile-name #$profile-name + #:target #$target + #:localstatedir? #$localstatedir? + #:store-database #+database + #:deduplicate? #$deduplicate? + #:symlinks '#$symlinks + #:compressor-command + #+(and=> compressor compressor-command) + #:archiver #+archiver)))) + #:target target + #:references-graphs `(("profile" ,profile)))) ;;; @@ -721,20 +617,10 @@ (define %valid-compressors '("gzip" "xz" "none")) (warning (G_ "entry point not supported in the '~a' format~%") 'deb)) - (define data-tarball - (computed-file (string-append "data.tar" (compressor-extension - compressor)) - (self-contained-tarball/builder profile - #:target target - #:profile-name profile-name - #:localstatedir? localstatedir? - #:deduplicate? deduplicate? - #:symlinks symlinks - #:compressor compressor - #:archiver archiver) - #:local-build? #f ;allow offloading - #:options (list #:references-graphs `(("profile" ,profile)) - #:target target))) + (define database + (and localstatedir? + (file-append (store-database (list profile)) + "/db/db.sqlite"))) (define build (with-extensions (list guile-gcrypt) @@ -752,6 +638,9 @@ (define build (ice-9 optargs) (srfi srfi-1)) + ;; Make sure non-ASCII file names are properly handled. + #+(set-utf8-locale profile) + (define machine-type ;; Extract the machine type from the specified target, else from the ;; current system. @@ -805,10 +694,25 @@ (define debian-format-version "2.0") (lambda (port) (format port "~a~%" debian-format-version))) - (define data-tarball-file-name (strip-store-file-name - #+data-tarball)) + (define compressor-command + #+(and=> compressor compressor-command)) - (copy-file #+data-tarball data-tarball-file-name) + (define compressor-extension + #+(compressor-extension compressor)) + + (define data-tarball-file-name + (string-append "data.tar" compressor-extension)) + + (build-self-contained-tarball #$profile + data-tarball-file-name + #:profile-name #$profile-name + #:localstatedir? #$localstatedir? + #:store-database #+database + #:deduplicate? #$deduplicate? + #:symlinks '#$symlinks + #:compressor-command + compressor-command + #:archiver #+archiver) ;; Generate the control archive. (let-keywords '#$extra-options #f @@ -817,8 +721,7 @@ (define data-tarball-file-name (strip-store-file-name (triggers-file #f)) (define control-tarball-file-name - (string-append "control.tar" - #$(compressor-extension compressor))) + (string-append "control.tar" compressor-extension)) ;; Write the compressed control tarball. Only the control file is ;; mandatory (see: 'man deb' and 'man deb-control'). @@ -848,7 +751,7 @@ (define tar (string-append #+archiver "/bin/tar")) (apply invoke tar `(,@(tar-base-options #:tar tar - #:compressor #+(and=> compressor compressor-command)) + #:compressor compressor-command) "-cvf" ,control-tarball-file-name "control" ,@(if postinst-file '("postinst") '()) @@ -859,7 +762,9 @@ (define tar (string-append #+archiver "/bin/tar")) "debian-binary" control-tarball-file-name data-tarball-file-name)))))) - (gexp->derivation (string-append name ".deb") build)) + (gexp->derivation (string-append name ".deb") build + #:target target + #:references-graphs `(("profile" ,profile)))) ;;; @@ -883,66 +788,27 @@ (define* (rpm-archive name profile (when entry-point (warning (G_ "entry point not supported in the '~a' format~%") 'rpm)) - (define root (populate-profile-root profile - #:profile-name profile-name - #:target target - #:localstatedir? localstatedir? - #:deduplicate? deduplicate? - #:symlinks symlinks)) - - (define payload - (let* ((raw-cpio-file-name "payload.cpio") - (compressed-cpio-file-name (string-append raw-cpio-file-name - (compressor-extension - compressor)))) - (computed-file compressed-cpio-file-name - (with-imported-modules (source-module-closure - '((guix build utils) - (guix cpio) - (guix rpm))) - #~(begin - (use-modules (guix build utils) - (guix cpio) - (guix rpm) - (srfi srfi-1)) - - ;; Make sure non-ASCII file names are properly handled. - #+(set-utf8-locale profile) - - (define %root (if #$localstatedir? "." #$root)) - - (when #$localstatedir? - ;; Fix the permission of the Guix database file, which was made - ;; read-only when copied to the store in populate-profile-root. - (copy-recursively #$root %root) - (chmod (string-append %root "/var/guix/db/db.sqlite") #o644)) - - (call-with-output-file #$raw-cpio-file-name - (lambda (port) - (with-directory-excursion %root - ;; The first "." entry is discarded. - (write-cpio-archive - (remove fhs-directory? - (cdr (find-files "." #:directories? #t))) - port)))) - (when #+(compressor-command compressor) - (apply invoke (append #+(compressor-command compressor) - (list #$raw-cpio-file-name)))) - (copy-file #$compressed-cpio-file-name #$output))) - #:local-build? #f))) ;allow offloading + (define database + (and localstatedir? + (file-append (store-database (list profile)) + "/db/db.sqlite"))) (define build (with-extensions (list guile-gcrypt) (with-imported-modules `(((guix config) => ,(make-config.scm)) ,@(source-module-closure `((gcrypt hash) + (guix build pack) (guix build utils) + (guix cpio) (guix profiles) (guix rpm)) #:select? not-config?)) #~(begin (use-modules (gcrypt hash) + (guix build pack) (guix build utils) + (guix cpio) (guix profiles) (guix rpm) (ice-9 binary-ports) @@ -954,6 +820,35 @@ (define build ;; Make sure non-ASCII file names are properly handled. #+(set-utf8-locale profile) + (define %root "root") + + (populate-profile-root! #$profile + #:profile-name #$profile-name + #:localstatedir? #$localstatedir? + #:store-database #+database + #:deduplicate? #$deduplicate? + #:symlinks '#$symlinks) + + (define raw-cpio-file-name "payload.cpio") + + ;; Generate CPIO payload. + (call-with-output-file raw-cpio-file-name + (lambda (port) + (with-directory-excursion %root + ;; The first "." entry is discarded. + (write-cpio-archive + (remove fhs-directory? + (cdr (find-files "." #:directories? #t))) + port)))) + + (when #+(compressor-command compressor) + (apply invoke (append #+(compressor-command compressor) + (list raw-cpio-file-name)))) + + (define cpio-file-name + (string-append "payload.cpio" + #$(compressor-extension compressor))) + (define machine-type (and=> (or #$target %host-type) (lambda (triplet) @@ -981,7 +876,7 @@ (define lead #:target (or #$target %host-type))) (define payload-digest - (bytevector->hex-string (file-sha256 #$payload))) + (bytevector->hex-string (file-sha256 cpio-file-name))) (let-keywords '#$extra-options #f ((relocatable? #f) (prein-file #f) @@ -991,7 +886,7 @@ (define payload-digest (let ((header (generate-header name version payload-digest - #$root + %root #$(compressor-name compressor) #:target (or #$target %host-type) #:relocatable? relocatable? @@ -1003,7 +898,7 @@ (define payload-digest (define header-sha256 (bytevector->hex-string (sha256 (u8-list->bytevector header)))) - (define payload-size (stat:size (stat #$payload))) + (define payload-size (stat:size (stat cpio-file-name))) (define header+compressed-payload-size (+ (length header) payload-size)) @@ -1013,7 +908,7 @@ (define signature header+compressed-payload-size)) ;; Serialize the archive components to a file. - (call-with-input-file #$payload + (call-with-input-file cpio-file-name (lambda (in) (call-with-output-file #$output (lambda (out) @@ -1022,7 +917,9 @@ (define signature header)) (sendfile out in payload-size))))))))))) - (gexp->derivation (string-append name ".rpm") build)) + (gexp->derivation (string-append name ".rpm") build + #:target target + #:references-graphs `(("profile" ,profile)))) ;;; diff --git a/tests/pack.scm b/tests/pack.scm index 87187bb62c..397fb37b12 100644 --- a/tests/pack.scm +++ b/tests/pack.scm @@ -76,64 +76,64 @@ (define rpm-for-tests (test-begin "pack") -(unless (network-reachable?) (test-skip 1)) -(test-assertm "self-contained-tarball" %store - (mlet* %store-monad - ((profile -> (profile - (content (packages->manifest (list %bootstrap-guile))) - (hooks '()) - (locales? #f))) - (tarball (self-contained-tarball "pack" profile - #:symlinks '(("/bin/Guile" - -> "bin/guile")) - #:compressor %gzip-compressor - #:archiver %tar-bootstrap)) - (check (gexp->derivation "check-tarball" - (with-imported-modules '((guix build utils)) - #~(begin - (use-modules (guix build utils) - (srfi srfi-1)) - - (define store - ;; The unpacked store. - (string-append "." (%store-directory) "/")) - - (define (canonical? file) - ;; Return #t if FILE is read-only and its mtime is 1. - (let ((st (lstat file))) - (or (not (string-prefix? store file)) - (eq? 'symlink (stat:type st)) - (and (= 1 (stat:mtime st)) - (zero? (logand #o222 - (stat:mode st))))))) - - (define bin - (string-append "." #$profile "/bin")) - - (setenv "PATH" - (string-append #$%tar-bootstrap "/bin")) - (system* "tar" "xvf" #$tarball) - (mkdir #$output) - (exit - (and (file-exists? (string-append bin "/guile")) - (file-exists? store) - (every canonical? - (find-files "." (const #t) - #:directories? #t)) - (string=? (string-append #$%bootstrap-guile "/bin") - (readlink bin)) - (string=? (string-append ".." #$profile - "/bin/guile") - (readlink "bin/Guile"))))))))) - (built-derivations (list check)))) - ;; The following test needs guile-sqlite3, libgcrypt, etc. as a consequence of ;; commit c45477d2a1a651485feede20fe0f3d15aec48b39 and related changes. Thus, ;; run it on the user's store, if it's available, on the grounds that these ;; dependencies may be already there, or we can get substitutes or build them ;; quite inexpensively; see . - (with-external-store store + (unless store (test-skip 1)) + (test-assertm "self-contained-tarball" store + (mlet* %store-monad + ((guile (set-guile-for-build (default-guile))) + (profile -> (profile + (content (packages->manifest (list %bootstrap-guile))) + (hooks '()) + (locales? #f))) + (tarball (self-contained-tarball "pack" profile + #:symlinks '(("/bin/Guile" + -> "bin/guile")) + #:compressor %gzip-compressor + #:archiver %tar-bootstrap)) + (check (gexp->derivation "check-tarball" + (with-imported-modules '((guix build utils)) + #~(begin + (use-modules (guix build utils) + (srfi srfi-1)) + + (define store + ;; The unpacked store. + (string-append "." (%store-directory) "/")) + + (define (canonical? file) + ;; Return #t if FILE is read-only and its mtime is 1. + (let ((st (lstat file))) + (or (not (string-prefix? store file)) + (eq? 'symlink (stat:type st)) + (and (= 1 (stat:mtime st)) + (zero? (logand #o222 + (stat:mode st))))))) + + (define bin + (string-append "." #$profile "/bin")) + + (setenv "PATH" + (string-append #$%tar-bootstrap "/bin")) + (system* "tar" "xvf" #$tarball) + (mkdir #$output) + (exit + (and (file-exists? (string-append bin "/guile")) + (file-exists? store) + (every canonical? + (find-files "." (const #t) + #:directories? #t)) + (string=? (string-append #$%bootstrap-guile "/bin") + (readlink bin)) + (string=? (string-append ".." #$profile + "/bin/guile") + (readlink "bin/Guile"))))))))) + (built-derivations (list check)))) + (unless store (test-skip 1)) (test-assertm "self-contained-tarball + localstatedir" store (mlet* %store-monad base-commit: 89e5f3f3847b3bfd507ea9f0874a73f99a53cbf9 -- 2.39.1