[PATCH] guix: packages: Consider 'patches' by 'package-direct-sources'.

  • Done
  • quality assurance status badge
Details
3 participants
  • Josselin Poiret
  • Ludovic Courtès
  • Simon Tournier
Owner
unassigned
Submitted by
Simon Tournier
Severity
normal
S
S
Simon Tournier wrote on 7 Mar 2023 18:18
(address . guix-patches@gnu.org)(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
20230307171833.4170067-1-zimon.toutoune@gmail.com
* guix/packages.scm (package-direct-sources): Return 'origin' from 'patches'.
* tests/packages.scm: Test it.
---
guix/packages.scm | 10 ++++++++--
tests/packages.scm | 17 ++++++++++++++++-
2 files changed, 24 insertions(+), 3 deletions(-)

Hi,

This patch improves some coverage when listing all the origins (fixed outputs)
of some packages. The procedure 'packages-direct-sources' already lists the
'origin' as inputs but is missing the 'origin' as 'patches'. For an instance
of such, see the package 'ntp'.

Note that it is not recursive. If an 'origin' is listed in 'patches' or
'inputs' and that origin also contains 'patches' which is another 'origin'
then it will be missed. For now, it is not covered because it somehow adds
complexity without an instance (yet) of such case.

Cheers,
simon




Toggle diff (79 lines)
diff --git a/guix/packages.scm b/guix/packages.scm
index 041a872f9d..0f88564ab4 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -9,6 +9,7 @@
;;; Copyright © 2021 Chris Marusich <cmmarusich@gmail.com>
;;; Copyright © 2022 Maxime Devos <maximedevos@telenet.be>
;;; Copyright © 2022 jgart <jgart@dismail.de>
+;;; Copyright © 2023 Simon Tournier <zimon.toutoune@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -1239,8 +1240,13 @@ (define-syntax modify-inputs
(define (package-direct-sources package)
"Return all source origins associated with PACKAGE; including origins in
-PACKAGE's inputs."
- `(,@(or (and=> (package-source package) list) '())
+PACKAGE's inputs and patches."
+ (define (expand source)
+ (append
+ (list source)
+ (filter origin? (origin-patches source))))
+
+ `(,@(or (and=> (package-source package) expand) '())
,@(filter-map (match-lambda
((_ (? origin? orig) _ ...)
orig)
diff --git a/tests/packages.scm b/tests/packages.scm
index f58c47817b..27fb918f90 100644
--- a/tests/packages.scm
+++ b/tests/packages.scm
@@ -3,6 +3,7 @@
;;; Copyright © 2018 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
;;; Copyright © 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;; Copyright © 2021 Maxime Devos <maximedevos@telenet.be>
+;;; Copyright © 2023 Simon Tournier <zimon.toutoune@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -418,12 +419,15 @@ (define read-at
(let* ((o (dummy-origin))
(u (dummy-origin))
(i (dummy-origin))
+ (j (dummy-origin (patches (list o))))
(a (dummy-package "a"))
(b (dummy-package "b" (inputs (list a i))))
(c (package (inherit b) (source o)))
(d (dummy-package "d"
(build-system trivial-build-system)
- (source u) (inputs (list c)))))
+ (source u) (inputs (list c))))
+ (e (dummy-package "e" (source j)))
+ (f (package (inherit e) (inputs (list u)))))
(test-assert "package-direct-sources, no source"
(null? (package-direct-sources a)))
(test-equal "package-direct-sources, #f source"
@@ -437,6 +441,17 @@ (define read-at
(and (= (length (pk 's-sources s)) 2)
(member o s)
(member i s))))
+ (test-assert "package-direct-sources, with patches"
+ (let ((s (package-direct-sources e)))
+ (and (= (length (pk 's-sources s)) 2)
+ (member o s)
+ (member j s))))
+ (test-assert "package-direct-sources, with patches and inputs"
+ (let ((s (package-direct-sources f)))
+ (and (= (length (pk 's-sources s)) 3)
+ (member o s)
+ (member j s)
+ (member u s))))
(test-assert "package-transitive-sources"
(let ((s (package-transitive-sources d)))
(and (= (length (pk 'd-sources s)) 3)

base-commit: 723fc5df3e964fcecb09c7c6fd48f00f97e2e806
--
2.38.1
J
J
Josselin Poiret wrote on 9 Mar 2023 20:43
87lek5d82p.fsf@jpoiret.xyz
Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> writes:

Toggle quote (10 lines)
> (define (package-direct-sources package)
> "Return all source origins associated with PACKAGE; including origins in
> -PACKAGE's inputs."
> - `(,@(or (and=> (package-source package) list) '())
> +PACKAGE's inputs and patches."
> + (define (expand source)
> + (append
> + (list source)
> + (filter origin? (origin-patches source))))

* cough * (cons source (filter ...)) * cough *

Other than that, LGTM! Tests worked fine on my end. No idea what this
is used for though :p

Best,
--
Josselin Poiret
-----BEGIN PGP SIGNATURE-----

iQHEBAEBCAAuFiEEOSSM2EHGPMM23K8vUF5AuRYXGooFAmQKNt4QHGRldkBqcG9p
cmV0Lnh5egAKCRBQXkC5FhcaigLjC/4z42MulqY4FuyhGqP9YC0JLljEf82HPn02
KLk17Hqv+iyjrchcaSzbk1LdzUEuQOoScy0tfWH1cNVtQ8ixZ+CU6YahP3SkErD4
x5wthA030DacfhZR9jPgHpqxGFNqVjMOaFx9cUa4fG4W2RTw5/UbGvdZ28yeZ8Wm
hL+YD5V71h7T+ShRwIY4Ky/uHMV+ehQ2F5ENQDQ5vsyKCkVRfqkeWB+U8wrd8ja8
zS5spTxSY3DUR1HTiSmb9mFJVE/AfHrQtPBK6yEZhEnF5m94F6ZqhJvbYEvxe+yr
TTesEgy/kpDIlsukILSreA8lQULmETikpjWAWXDtTMqLcLjTP8ojOoldUYlwX2ev
CjVDgoVGiP0Xf3s1Erkua98GRvukXCOEp/x5k3xbHJnmR2Pcbox8q4uCn5qS0mSX
t95Yh0nD+5s7biczROPQ4rdS/dTB321YpAUS+3u0CKd9f8BLUujZx/o0rwq1KwDB
2HZok+oderfr2PcX9HogFb7O6Mclj+Q=
=sU7q
-----END PGP SIGNATURE-----

S
S
Simon Tournier wrote on 10 Mar 2023 12:33
87jzzoon7x.fsf@gmail.com
Hi Josselin,

On jeu., 09 mars 2023 at 20:43, Josselin Poiret via Guix-patches via <guix-patches@gnu.org> wrote:

Toggle quote (6 lines)
>> + (append
>> + (list source)
>> + (filter origin? (origin-patches source))))
>
> * cough * (cons source (filter ...)) * cough *

Ahah! Somehow I removed from my mental toolbox cons, car and cdr
because I am spending too much time explaining to non-lispers. :-)

Thanks for showing me the light. ;-)


Toggle quote (3 lines)
> Other than that, LGTM! Tests worked fine on my end. No idea what this
> is used for though :p

For instance, it can be used to list all the ’origin’ of a package.
Consider the package ’tensorflow’, it reads,

Toggle snippet (13 lines)
(native-inputs
[...]
;; The commit hashes and URLs for third-party source code are taken
;; from "tensorflow/workspace.bzl".
("boringssl-src"
,(let ((commit "ee7aa02")
(revision "1"))
(origin
(method git-fetch)
(uri (git-reference
(url "https://boringssl.googlesource.com/boringssl")

where some inputs are not packages but just ’origin’. Therefore, the
procedure allows to get all the ’origin’, the one from the field
’origin’ and also the ones from inputs.

Toggle snippet (7 lines)
scheme@(guix-user)> ,use(gnu packages machine-learning)
scheme@(guix-user)> ,pp (map origin-uri (package-direct-sources tensorflow))
$1 = (#<<git-reference> url: "https://github.com/tensorflow/tensorflow" commit: "v1.9.0" recursive?: #f>
#<<git-reference> url: "https://boringssl.googlesource.com/boringssl" commit: "ee7aa02" recursive?: #f>
[...]

For some packages as ’ntp’, the patches are also a list of ’origin’,

Toggle snippet (13 lines)
(origin
(method url-fetch)
(uri (list (string-append
"https://www.eecis.udel.edu/~ntp/ntp_spool/ntp4/ntp-"
[...]
;; Add an upstream patch to fix build with GCC 10. Taken from
;; <https://bugs.ntp.org/show_bug.cgi?id=3688>.
(patches (list (origin
(method url-fetch)
(uri "https://bugs.ntp.org/attachment.cgi?id=1760&action=diff&context=patch&collapsed=&headers=1&format=raw")
[...]

and the patch allows to also extract them:

Toggle snippet (9 lines)
scheme@(guix-user)> ,use(gnu packages ntp)
scheme@(guix-user)> ,pp (map origin-uri (package-direct-sources ntp))
$2 = (("https://www.eecis.udel.edu/~ntp/ntp_spool/ntp4/ntp-4.2/ntp-4.2.8p15.tar.gz"
"http://archive.ntp.org/ntp4/ntp-4.2/ntp-4.2.8p15.tar.gz")
"https://bugs.ntp.org/attachment.cgi?id=1760&action=diff&context=patch&collapsed=&headers=1&format=raw"
"https://bugs.ntp.org/attachment.cgi?id=1814&action=diff&collapsed=&headers=1&format=raw")


This way it improves the coverage with Software Heritage. All the
source code (origin) is extracted and feed some SWH loader. The code is
there:


Cheers,
simon
J
J
Josselin Poiret wrote on 10 Mar 2023 13:12
87jzzoby9n.fsf@jpoiret.xyz
Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> writes:

Toggle quote (6 lines)
> This way it improves the coverage with Software Heritage. All the
> source code (origin) is extracted and feed some SWH loader. The code is
> there:
>
> https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/build-package-metadata.scm#n58

Ahh, that's the part I was missing. Great work!

Best,
--
Josselin Poiret
-----BEGIN PGP SIGNATURE-----

iQHEBAEBCAAuFiEEOSSM2EHGPMM23K8vUF5AuRYXGooFAmQLHsQQHGRldkBqcG9p
cmV0Lnh5egAKCRBQXkC5FhcaihUXDACfgl9R1VQfRYZd/hND/0GZFQ6pMCuYb7m8
FyO6RuuL6yt6mR0LeT/9vESJVgTWaNiBJxzX+EPLOcJN2Mhua5qRcvscQJ+JTUCH
g65Wj9qD/fyqB7s9Qdov2UHAyfvptUAliB0YJRUt2AjjtzpThbX7e8TUvopwNrvf
2Jd7OobKYhPjsas1WFkNWszTOKZN38oqPV1N2tOXls0/wSX7gUryyqlBguTPH21x
tntlkF6YLZH+x72NOLyW6doPn1cIm6paiVFkxHNM0R5Z2Utb7e6vZTwdmCM5RiwT
JCY/WQZi0LH7ZdoH2e28zUJ9qvev8X64CbyQV49wUP0BDoZRRbd6GXPkHp0VXHOh
5/1j+aaKjfPbX0fJ6MlAZgkB7YNLNimn4OXG3+NEEyU8cxUfhUdALOd+gFfl08L7
MEzuP8DE1+4GBEzv/WZyjHZLe0/UEMdNlNIUKLCpPpqzynLxSOlyFyQ0OY3enu+V
XYnngirofSox6cNwe1IOCS5BZdfoSK4=
=Y9iy
-----END PGP SIGNATURE-----

S
S
Simon Tournier wrote on 11 Mar 2023 17:51
[PATCH v2] guix: packages: Consider 'patches' by 'package-direct-sources'.
(address . 62036@debbugs.gnu.org)
20230311165110.726639-1-zimon.toutoune@gmail.com
* guix/packages.scm (package-direct-sources): Return 'origin' from 'patches'.
* tests/packages.scm: Test it.
---
guix/packages.scm | 10 ++++++++--
tests/packages.scm | 17 ++++++++++++++++-
2 files changed, 24 insertions(+), 3 deletions(-)

Toggle diff (79 lines)
diff --git a/guix/packages.scm b/guix/packages.scm
index 041a872f9d..7b098e21f0 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -9,6 +9,7 @@
;;; Copyright © 2021 Chris Marusich <cmmarusich@gmail.com>
;;; Copyright © 2022 Maxime Devos <maximedevos@telenet.be>
;;; Copyright © 2022 jgart <jgart@dismail.de>
+;;; Copyright © 2023 Simon Tournier <zimon.toutoune@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -1239,8 +1240,13 @@ (define-syntax modify-inputs
(define (package-direct-sources package)
"Return all source origins associated with PACKAGE; including origins in
-PACKAGE's inputs."
- `(,@(or (and=> (package-source package) list) '())
+PACKAGE's inputs and patches."
+ (define (expand source)
+ (cons
+ source
+ (filter origin? (origin-patches source))))
+
+ `(,@(or (and=> (package-source package) expand) '())
,@(filter-map (match-lambda
((_ (? origin? orig) _ ...)
orig)
diff --git a/tests/packages.scm b/tests/packages.scm
index f58c47817b..27fb918f90 100644
--- a/tests/packages.scm
+++ b/tests/packages.scm
@@ -3,6 +3,7 @@
;;; Copyright © 2018 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
;;; Copyright © 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;; Copyright © 2021 Maxime Devos <maximedevos@telenet.be>
+;;; Copyright © 2023 Simon Tournier <zimon.toutoune@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -418,12 +419,15 @@ (define read-at
(let* ((o (dummy-origin))
(u (dummy-origin))
(i (dummy-origin))
+ (j (dummy-origin (patches (list o))))
(a (dummy-package "a"))
(b (dummy-package "b" (inputs (list a i))))
(c (package (inherit b) (source o)))
(d (dummy-package "d"
(build-system trivial-build-system)
- (source u) (inputs (list c)))))
+ (source u) (inputs (list c))))
+ (e (dummy-package "e" (source j)))
+ (f (package (inherit e) (inputs (list u)))))
(test-assert "package-direct-sources, no source"
(null? (package-direct-sources a)))
(test-equal "package-direct-sources, #f source"
@@ -437,6 +441,17 @@ (define read-at
(and (= (length (pk 's-sources s)) 2)
(member o s)
(member i s))))
+ (test-assert "package-direct-sources, with patches"
+ (let ((s (package-direct-sources e)))
+ (and (= (length (pk 's-sources s)) 2)
+ (member o s)
+ (member j s))))
+ (test-assert "package-direct-sources, with patches and inputs"
+ (let ((s (package-direct-sources f)))
+ (and (= (length (pk 's-sources s)) 3)
+ (member o s)
+ (member j s)
+ (member u s))))
(test-assert "package-transitive-sources"
(let ((s (package-transitive-sources d)))
(and (= (length (pk 'd-sources s)) 3)

base-commit: 7376424844e0dc1103a9d86799c2fb9bd4aa35eb
--
2.38.1
L
L
Ludovic Courtès wrote on 13 Mar 2023 12:14
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
87o7ow52dw.fsf@gnu.org
Hello,

Simon Tournier <zimon.toutoune@gmail.com> skribis:

Toggle quote (3 lines)
> * guix/packages.scm (package-direct-sources): Return 'origin' from 'patches'.
> * tests/packages.scm: Test it.

Applied, thanks!

Ludo’.
Closed
?