From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 15 10:33:32 2022 Received: (at 52828) by debbugs.gnu.org; 15 Jan 2022 15:33:32 +0000 Received: from localhost ([127.0.0.1]:41530 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n8l3V-00058s-En for submit@debbugs.gnu.org; Sat, 15 Jan 2022 10:33:32 -0500 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:34419) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n8l3Q-00058Z-MC for 52828@debbugs.gnu.org; Sat, 15 Jan 2022 10:33:24 -0500 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 9165E5C00CD; Sat, 15 Jan 2022 10:33:15 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Sat, 15 Jan 2022 10:33:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=e95IXO CkHa+rxlUG96Bi/CYgeIdUiLEGzysQqCYdVrA=; b=OKg46fBBvmk2kugvmslFk9 gCAB0Dv0A2jeFNpdAKPA2T8h6zdBcOTBf/B5gwWQcLbi662qIVKTkt4nytqmTcrT s3LNwP45n1y29mpnVD1SDWoBvxcKVuuYCEvTlXM+o7pH0pvC7SrmA3rMlIzBrEtV HhvFt5GwMYDqb4gKrSXhs947nY8TKznjQ4U+ZamrgVQSDkF+qsM39nwDTYFW/VWB aplqh/pB0aK9V61/VIsUYN0SAXvjKvD+3+JcfXqY5hGbJ8Y30usawuaX8fFqhs3Q kz+YqP2MO1LuJ2NFw7d2Lssni3vQO4YhduFdK4vNoGtXQzFSJF9ATUjpxEd+wBHg == X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvvddrtdejgdejkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufhffjgfkfgggtgesmhdtreertderjeenucfhrhhomhepvfhimhhothhh hicuufgrmhhplhgvuceoshgrmhhplhgvthesnhhghihrohdrtghomheqnecuggftrfgrth htvghrnhepvdetjeevkefggfefgfevteehgffgledugfetudfggeeltdfhfeeuleejffeu ueffnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepsh grmhhplhgvthesnhhghihrohdrtghomh X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 15 Jan 2022 10:33:15 -0500 (EST) From: Timothy Sample To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#52828: [PATCH v2] swh: Do not rely on $PATH for tar and gzip. References: <877dbpuc2n.fsf@ngyro.com> <87mtk9kiol.fsf@gnu.org> Date: Sat, 15 Jan 2022 10:33:14 -0500 In-Reply-To: <87mtk9kiol.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Wed, 05 Jan 2022 21:54:02 +0100") Message-ID: <8735lpxbd1.fsf_-_@ngyro.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Ludovic, Ludovic Courtès writes: > What about uses of guix-daemon on other distros? Perhaps we assume tar > and gzip are already on PATH? Content analysis details: (1.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: yoctocell.xyz (xyz)] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [66.111.4.28 listed in wl.mailspike.net] -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [66.111.4.28 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-Debbugs-Envelope-To: 52828 Cc: 52828@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludovic, Ludovic Court=C3=A8s writes: > What about uses of guix-daemon on other distros? Perhaps we assume tar > and gzip are already on PATH? That=E2=80=99s my guess. And if those other weird distros don=E2=80=99t ha= ve =E2=80=98tar=E2=80=99 and =E2=80=98gzip=E2=80=99 in =E2=80=9C/usr/bin=E2=80=9D, I=E2=80=99m sure they= =E2=80=99re plenty accustomed to liberally patching their packages. ;) > I=E2=80=99d feel more comfortable with a solution at the package level. = In the > =E2=80=98guix=E2=80=99 package, or perhaps in a Makefile.am, we could har= d-code absolute > file names of tar and gzip in (guix scripts perform-download) and set > PATH from there. We=E2=80=99d need to do the same in (guix self) though. > > Alternatively, we could change (guix swh) to use Guile-Zlib and the tar > implementation of Gash-Utils or that of Disarchive. > > WDYT? I=E2=80=99ve attached a new patch that mixes those two suggestions but gets= the first one a little wrong. It uses the absolute path for =E2=80=98tar=E2=80= =99, but uses Guile-zlib for decompression. I honestly don=E2=80=99t have a strong opinion about when and where to set =E2=80=98$PATH=E2=80=99 vs. using a configured, absolute path. My original= patch assumed that it=E2=80=99s the user=E2=80=99s job to make sure that =E2=80= =98tar=E2=80=99 and =E2=80=98gzip=E2=80=99 are available to Guix at runtime. This patch assumes that that linkage happens at configure time. The main benefit I could see to setting =E2=80=98$PATH=E2=80=99 in =E2=80=98(guix scripts perform-download)=E2=80= =99 is that we could add our =E2=80=98tar=E2=80=99 and =E2=80=98gzip=E2=80=99 as a suffix. This makes i= t work while allowing users to choose whatever =E2=80=98tar=E2=80=99 and =E2=80=98gzip=E2=80=99 they pr= efer. The downside is that whenever we use =E2=80=98(guix swh)=E2=80=99 have to remember to make sure = that =E2=80=98tar=E2=80=99 and =E2=80=98gzip=E2=80=99 are available. Basically, I can to change this to do the setup in =E2=80=98perform-downloa= d=E2=80=99, but I really want to understand why. Thanks! --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=v2-0001-swh-Do-not-rely-on-PATH-for-tar-and-gzip.patch Content-Transfer-Encoding: quoted-printable From 010666376890b6adf6c14253b1e2651b5c2861e8 Mon Sep 17 00:00:00 2001 From: Timothy Sample Date: Fri, 14 Jan 2022 18:03:10 -0500 Subject: [PATCH v2] swh: Do not rely on $PATH for tar and gzip. Fixes . * configure.ac: Find the path of the tar utility. * guix/config.scm.in (%tar): New variable. * guix/self.scm (specification->package): Add "tar". (make-config.scm): Add a 'tar' keyword parameter and use it to set the '%tar' variable. (compiled-guix): Add a 'tar' keyword parameter, and pass it to 'make-config.scm'; add 'guile-zlib' as an extension for "guix-core". * guix/swh.scm (swh-download-archive): Use Guile-zlib to decompress "flat" archives, and use an absolute path when invoking 'tar'. --- configure.ac | 4 ++++ guix/config.scm.in | 7 ++++++- guix/self.scm | 18 ++++++++++++++---- guix/swh.scm | 13 ++++++++----- 4 files changed, 32 insertions(+), 10 deletions(-) diff --git a/configure.ac b/configure.ac index 341cff8fbd..78f599b2f4 100644 --- a/configure.ac +++ b/configure.ac @@ -198,6 +198,10 @@ AC_SUBST([GZIP]) AC_SUBST([BZIP2]) AC_SUBST([XZ]) =20 +dnl The '(guix swh)' module uses 'tar'. +AC_PATH_PROG([TAR], [tar]) +AC_SUBST([TAR]) + LIBGCRYPT_LIBDIR=3D"no" LIBGCRYPT_PREFIX=3D"no" =20 diff --git a/guix/config.scm.in b/guix/config.scm.in index d582d91d74..7b400a9ff8 100644 --- a/guix/config.scm.in +++ b/guix/config.scm.in @@ -37,7 +37,9 @@ (define-module (guix config) %system %gzip %bzip2 - %xz)) + %xz + + %tar)) =20 ;;; Commentary: ;;; @@ -118,4 +120,7 @@ (define %bzip2 (define %xz "@XZ@") =20 +(define %tar + "@TAR@") + ;;; config.scm ends here diff --git a/guix/self.scm b/guix/self.scm index 943bb0b498..3944f1a98d 100644 --- a/guix/self.scm +++ b/guix/self.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2017, 2018, 2019, 2020, 2021 Ludovic Court=C3=A8s ;;; Copyright =C2=A9 2020 Martin Becze +;;; Copyright =C2=A9 2022 Timothy Sample ;;; ;;; This file is part of GNU Guix. ;;; @@ -68,6 +69,7 @@ (define specification->package ("gzip" (ref '(gnu packages compression) 'gzip)) ("bzip2" (ref '(gnu packages compression) 'bzip2)) ("xz" (ref '(gnu packages compression) 'xz)) + ("tar" (ref '(gnu packages base) 'tar)) ("po4a" (ref '(gnu packages gettext) 'po4a)) ("gettext" (ref '(gnu packages gettext) 'gettext-minimal)) ("gcc-toolchain" (ref '(gnu packages commencement) 'gcc-toolchain)) @@ -749,6 +751,7 @@ (define* (compiled-guix source #:key (gzip (specification->package "gzip")) (bzip2 (specification->package "bzip2")) (xz (specification->package "xz")) + (tar (specification->package "tar")) (guix (specification->package "guix"))) "Return a file-like object that contains a compiled Guix." (define guile-avahi @@ -832,7 +835,9 @@ (define* (compiled-guix source #:key ,(local-file "../guix/store/schema.sql"))) =20 #:extensions (list guile-gcrypt - guile-json) ;for (guix swh) + ;; The following are for (guix swh) + guile-json + guile-zlib) #:guile-for-build guile-for-build)) =20 (define *extra-modules* @@ -964,6 +969,7 @@ (define* (compiled-guix source #:key =3D> ,(make-config.scm #:gzip gzip #:bzip2 bzip2 #:xz xz + #:tar tar #:package-name %guix-package-name #:package-version @@ -1071,7 +1077,7 @@ (define %default-config-variables (%storedir . "/gnu/store") (%sysconfdir . "/etc"))) =20 -(define* (make-config.scm #:key gzip xz bzip2 +(define* (make-config.scm #:key gzip xz bzip2 tar (package-name "GNU Guix") (package-version "0") (channel-metadata #f) @@ -1097,7 +1103,8 @@ (define* (make-config.scm #:key gzip xz bzip2 %config-directory %gzip %bzip2 - %xz)) + %xz + %tar)) =20 (define %system #$(%current-system)) @@ -1142,7 +1149,10 @@ (define* (make-config.scm #:key gzip xz bzip2 (define %bzip2 #+(and bzip2 (file-append bzip2 "/bin/bzip2"))) (define %xz - #+(and xz (file-append xz "/bin/xz")))) + #+(and xz (file-append xz "/bin/xz"))) + + (define %tar + #+(and tar (file-append tar "/bin/tar")))) =20 ;; Guile 2.0 *requires* the 'define-module' to be at the ;; top-level or the 'toplevel-ref' in the resulting .go fil= e are diff --git a/guix/swh.scm b/guix/swh.scm index c7c1c873a2..93bb023192 100644 --- a/guix/swh.scm +++ b/guix/swh.scm @@ -3,6 +3,7 @@ ;;; Copyright =C2=A9 2020 Jakub K=C4=85dzio=C5=82ka ;;; Copyright =C2=A9 2021 Xinglu Chen ;;; Copyright =C2=A9 2021 Simon Tournier +;;; Copyright =C2=A9 2022 Timothy Sample ;;; ;;; This file is part of GNU Guix. ;;; @@ -23,6 +24,7 @@ (define-module (guix swh) #:use-module (guix base16) #:use-module (guix build utils) #:use-module ((guix build syscalls) #:select (mkdtemp!)) + #:use-module (guix config) #:use-module (web uri) #:use-module (web client) #:use-module (web response) @@ -35,6 +37,7 @@ (define-module (guix swh) #:use-module (ice-9 regex) #:use-module (ice-9 popen) #:use-module ((ice-9 ftw) #:select (scandir)) + #:use-module (zlib) #:export (%swh-base-url %verify-swh-certificate? %allow-request? @@ -674,11 +677,11 @@ (define* (swh-download-archive swhid output swhid) #f) ((? port? input) - (let ((tar (open-pipe* OPEN_WRITE "tar" "-C" directory - (match archive-type - ('flat "-xzvf") ;gzipped - ('git-bare "-xvf")) ;uncompressed - "-"))) + (let ((input (match archive-type + ;; "flat" archives are compressed. + ('flat (make-zlib-input-port input #:format 'gzip)) + ('git-bare input))) + (tar (open-pipe* OPEN_WRITE %tar "-C" directory "-xvf" "-"))) (dump-port input tar) (close-port input) (let ((status (close-pipe tar))) --=20 2.34.0 --=-=-=--