[PATCH 0/2] Fix the CPAN importer

DoneSubmitted by Christopher Baines.
Details
2 participants
  • Ludovic Courtès
  • Christopher Baines
Owner
unassigned
Severity
normal
C
C
Christopher Baines wrote on 6 Aug 2019 21:07
(address . guix-patches@gnu.org)
87k1bq864w.fsf@cbaines.net
Christopher Baines (2): import: utils: Add hash-ref*. import: cpan: Adapt for the change to guile-json version 3.
guix/import/cpan.scm | 30 ++++++++++++++++-------------- guix/import/utils.scm | 10 ++++++++++ tests/cpan.scm | 13 ++++++++----- 3 files changed, 34 insertions(+), 19 deletions(-)
-----BEGIN PGP SIGNATURE-----
iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl1Jz+9fFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNFODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE9Xey4xAAi3zgdTxzfHC63C+/UCKocaIN/HAYjZ8JLraeXurr+HhCPp1OAMK3h/T59tYtypgTWXetSG14XD3UcK75Bpbvc4A3lHRpO4nq8LAIsMsWPjmuj1eq36dXnOmOnQfapj0VTzV/V7Crw0z6fLqwOzJJ3Xa7S5UCBEPTYGV/ZvpWBKlVMPrBFK0S5hwgT/7aW8Ntmeikv4Am/CLm9FLKhXNBQ1fIctVmUXoAApLgFRMamjZudaoiqps83Wt9KYDkvHBxS+TCP9RZbgACPoQghrfXIbsxs+3hN030VeF6Lc8Ts7D/2F2SKKt0+zTvnWpDdtYcpBna08QQJiNXLlbXp4mb39uI5X242vpTBAl+Dtx5nJk9bHp/Or2b2DgWyXQuXHKwql5slNz78e0C+YtTYUVWHRo3miwUZBxwogMdJ0vW+/12p28xsVz66cf6v+z7ZFJGjdDsVcQYf8TfA+SDyAlxgElfsOgwsD26WtsZdTj3ZqKsy9Mr5CFzZVBwUBlr0FaVAmkfiKANeM66Xuw+95Bdnqhbm1byL4RZG9mTQNaLxBTev7dPbFqO5qkQvw+MRUd5vXURqiu6F7b6DMvFhe6+HCrJkcONPdXLo1pGBU4PjxdAUhXlr2DkUrm+KOnVjCQgzkfjLXkEFqAFwcR68knlEJuCXivDd3Wl0A6Y4Jsvfo8==Iwo+-----END PGP SIGNATURE-----
C
C
Christopher Baines wrote on 6 Aug 2019 21:17
[PATCH 1/2] import: utils: Add hash-ref*.
(address . 36948@debbugs.gnu.org)
20190806191728.22923-1-mail@cbaines.net
With the change to guile-json version 3, JSON objects are represented as hashtables, rather than alists. The cpan importer uses assoc-ref* on a hash table,so add an equivalent function for hash tables.
* guix/import/utils.scm (hash-ref*): New procedure.--- guix/import/utils.scm | 10 ++++++++++ 1 file changed, 10 insertions(+)
Toggle diff (30 lines)diff --git a/guix/import/utils.scm b/guix/import/utils.scmindex 2a3b7341fb..ed6c3ce6af 100644--- a/guix/import/utils.scm+++ b/guix/import/utils.scm@@ -47,6 +47,7 @@ flatten assoc-ref*+ hash-ref* url-fetch guix-hash-url@@ -116,6 +117,15 @@ recursively apply the procedure to the sub-list." (assoc-ref alist key) (apply assoc-ref* (assoc-ref alist key) rest))) +(define (hash-ref* hash-table key . rest)+ "Return the value for KEY from HASH-TABLE. For each additional key specified,+recursively apply the procedure to the sub-hash-table."+ (if (hash-table? hash-table)+ (if (null? rest)+ (hash-ref hash-table key)+ (apply hash-ref* (hash-ref hash-table key) rest))+ #f)) ; For consistency with assoc-ref*+ (define (url-fetch url file-name) "Save the contents of URL to FILE-NAME. Return #f on failure." (parameterize ((current-output-port (current-error-port)))-- 2.22.0
C
C
Christopher Baines wrote on 6 Aug 2019 21:17
[PATCH 2/2] import: cpan: Adapt for the change to guile-json version 3.
(address . 36948@debbugs.gnu.org)
20190806191728.22923-2-mail@cbaines.net
In guile-json version 3, JSON objects are represented as hash tables, ratherthan alists.
* guix/import/cpan.scm (string->license): Change the match expression to matchon lists, rather than vectors.(module->dist-name, cpan-source-url, cpan-version): Change assoc-ref tohash-ref.(cpan-module->sexp): Change assoc-ref to hash-ref, and assoc-ref* tohash-ref*.* tests/cpan.scm ("source-url-http", "source-url-https"): Convert the alist toa hash table.--- guix/import/cpan.scm | 30 ++++++++++++++++-------------- tests/cpan.scm | 13 ++++++++----- 2 files changed, 24 insertions(+), 19 deletions(-)
Toggle diff (125 lines)diff --git a/guix/import/cpan.scm b/guix/import/cpan.scmindex ec86f11743..0be37e715e 100644--- a/guix/import/cpan.scm+++ b/guix/import/cpan.scm@@ -34,7 +34,7 @@ #:use-module (guix ui) #:use-module ((guix download) #:select (download-to-store url-fetch)) #:use-module ((guix import utils) #:select (factorize-uri- flatten assoc-ref*))+ flatten hash-ref*)) #:use-module (guix import json) #:use-module (guix packages) #:use-module (guix upstream)@@ -76,8 +76,8 @@ ;; ssleay ;; sun ("zlib" 'zlib)- (#(x) (string->license x))- (#(lst ...) `(list ,@(map string->license lst)))+ ((x) (string->license x))+ ((lst ...) `(list ,@(map string->license lst))) (_ #f))) (define (module->name module)@@ -88,11 +88,11 @@ "Return the base distribution module for a given module. E.g. the 'ok' module is distributed with 'Test::Simple', so (module->dist-name \"ok\") would return \"Test-Simple\""- (assoc-ref (json-fetch (string-append- "https://fastapi.metacpan.org/v1/module/"- module- "?fields=distribution"))- "distribution"))+ (hash-ref (json-fetch (string-append+ "https://fastapi.metacpan.org/v1/module/"+ module+ "?fields=distribution"))+ "distribution")) (define (package->upstream-name package) "Return the CPAN name of PACKAGE."@@ -122,12 +122,12 @@ or #f on failure. MODULE should be e.g. \"Test::Script\"" (define (cpan-source-url meta) "Return the download URL for a module's source tarball." (regexp-substitute/global #f "http[s]?://cpan.metacpan.org"- (assoc-ref meta "download_url")+ (hash-ref meta "download_url") 'pre "mirror://cpan" 'post)) (define (cpan-version meta) "Return the version number from META."- (match (assoc-ref meta "version")+ (match (hash-ref meta "version") ((? number? version) ;; version is sometimes not quoted in the module json, so it gets ;; imported into Guile as a number, so convert it to a string.@@ -183,7 +183,7 @@ depend on (gnu packages perl)." "Return the `package' s-expression for a CPAN module from the metadata in META." (define name- (assoc-ref meta "distribution"))+ (hash-ref meta "distribution")) (define (guix-name name) (if (string-prefix? "perl-" name)@@ -198,7 +198,9 @@ META." (match (flatten (map (lambda (ph) (filter-map (lambda (t)- (assoc-ref* meta "metadata" "prereqs" ph t))+ (and=> (hash-ref* meta "metadata" "prereqs" ph t)+ (lambda (h)+ (hash-map->list cons h)))) '("requires" "recommends" "suggests"))) phases)) (#f@@ -251,9 +253,9 @@ META." ,@(maybe-inputs 'propagated-inputs (convert-inputs '("runtime"))) (home-page ,(cpan-home name))- (synopsis ,(assoc-ref meta "abstract"))+ (synopsis ,(hash-ref meta "abstract")) (description fill-in-yourself!)- (license ,(string->license (assoc-ref meta "license"))))))+ (license ,(string->license (hash-ref meta "license")))))) (define (cpan->guix-package module-name) "Fetch the metadata for PACKAGE-NAME from metacpan.org, and return thediff --git a/tests/cpan.scm b/tests/cpan.scmindex 189dd027e6..cdd6c0e76a 100644--- a/tests/cpan.scm+++ b/tests/cpan.scm@@ -24,7 +24,8 @@ #:use-module (guix tests) #:use-module (guix grafts) #:use-module (srfi srfi-64)- #:use-module (ice-9 match))+ #:use-module (ice-9 match)+ #:use-module (ice-9 hash-table)) ;; Globally disable grafts because they can trigger early builds. (%graft? #f)@@ -109,14 +110,16 @@ (test-equal "source-url-http" ((@@ (guix import cpan) cpan-source-url)- `(("download_url" .- "http://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")))+ (alist->hash-table+ `(("download_url" .+ "http://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")))) "mirror://cpan/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz") (test-equal "source-url-https" ((@@ (guix import cpan) cpan-source-url)- `(("download_url" .- "https://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")))+ (alist->hash-table+ `(("download_url" .+ "https://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")))) "mirror://cpan/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz") (test-end "cpan")-- 2.22.0
C
C
Christopher Baines wrote on 21 Aug 2019 19:30
Re: [bug#36948] [PATCH 0/2] Fix the CPAN importer
(address . 36948-done@debbugs.gnu.org)
875zmq8lz1.fsf@cbaines.net
Christopher Baines <mail@cbaines.net> writes:
Toggle quote (9 lines)> Christopher Baines (2):> import: utils: Add hash-ref*.> import: cpan: Adapt for the change to guile-json version 3.>> guix/import/cpan.scm | 30 ++++++++++++++++--------------> guix/import/utils.scm | 10 ++++++++++> tests/cpan.scm | 13 ++++++++-----> 3 files changed, 34 insertions(+), 19 deletions(-)
I've pushed these patches as 01ce7af25add55514f737af48ea6c127bedfde67now.
-----BEGIN PGP SIGNATURE-----
iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl1df8JfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNFODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE9Xc7vRAAtV1Pil0G9JCqIYbdzcv2H4Q7DkhZyx4VO/nbY6pjWltY21IrLDEjYJi7B9614FHoMIql2YCBn3NrgqvYN9VmZlGeIGooydJVhSqsNW83XU/w6xUuYQuXwed+9EkJQofp/MT9DCPblUC4Y4BLV1mNIkjPgvRoTnNXLNnZOkveOoARL6Go9vxVw/HkxxHm7x2mvby5suZdvq2eX8Bdv0BKNe+7NHAGOPeL1AlkiJSis9gVMmAOEcmV9cweg+sI4NYnWhGW26LYCFkA05f8wlQt3H2z1N7126/Njcy4GFPhlOz8mkXfJF/itiu8NtRQvcmu5WllpKCYV4fITaPTIaxeuuoJe5nMqNrDk5Xpv0ZOJwJ/BdszsUIeTAi2f4H2p0lyPc7/Mp79IbsV5xSVP6JR65/WLEUnNU5X0o2mEAmBIKyZ2/xcCryy0i2Xewf8gGkxvI44tR4VYtoAPOJx87fkV9f0jExt5ZucidkFxd+W/k8mOdr2GwReI/2IhWgYj1ZNj293Db4Y0ayDyffHryAxpZO4L4MhX49re1q/Vn9ei9qsC/xUTC5NWPsTtfffZJ7z61KicdB3s3T5hvBPGp0PN52fpfNij+rcbasRV6I3baqWfErjJBXl/p5Hg/vbKQe+iTUmCexwTeY3PYENxlc3/jdCRv/awGxxPCAX0vnNGq0==ThD/-----END PGP SIGNATURE-----
Closed
L
L
Ludovic Courtès wrote on 22 Aug 2019 14:04
Re: [bug#36948] [PATCH 1/2] import: utils: Add hash-ref*.
(name . Christopher Baines)(address . mail@cbaines.net)(address . 36948@debbugs.gnu.org)
8736htqucz.fsf@gnu.org
Hello Chris,
Christopher Baines <mail@cbaines.net> skribis:
Toggle quote (6 lines)> With the change to guile-json version 3, JSON objects are represented as hash> tables, rather than alists. The cpan importer uses assoc-ref* on a hash table,> so add an equivalent function for hash tables.>> * guix/import/utils.scm (hash-ref*): New procedure.
[...]
Toggle quote (3 lines)> In guile-json version 3, JSON objects are represented as hash tables, rather> than alists.
I seems to me that this is adapting for Guile-JSON v1, not v3: in v3,JSON arrays map to Scheme vectors, and JSON dictionaries map to alists;JSON dictionaries used to map to hash tables in v1.
Indeed, ‘tests/cpan.scm’ now fails for me:
Toggle snippet (14 lines)actual-error:+ (wrong-type-arg+ "scm_hash_fn_get_handle"+ "Wrong type argument in position ~A (expecting ~A): ~S"+ (1+ "hash-table"+ (("version" . "0.1")+ ("author" . "Guix")+ ("download_url"+ .+ "http://example.com/Foo-Bar-0.1.tar.gz")[…]
Could it be that you were testing this in an environment containing v1and not v3?
Sorry for not noticing earlier!
Thanks,Ludo’.
?
Your comment

This issue is archived.

To comment on this conversation send email to 36948@debbugs.gnu.org