[PATCH 0/2] Fix the CPAN importer

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Christopher Baines
Owner
unassigned
Submitted by
Christopher Baines
Severity
normal

Debbugs page

Christopher Baines wrote 6 years ago
(address . guix-patches@gnu.org)
87k1bq864w.fsf@cbaines.net
Christopher Baines (2):
import: utils: Add hash-ref*.
import: cpan: Adapt for the change to guile-json version 3.

guix/import/cpan.scm | 30 ++++++++++++++++--------------
guix/import/utils.scm | 10 ++++++++++
tests/cpan.scm | 13 ++++++++-----
3 files changed, 34 insertions(+), 19 deletions(-)
-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl1Jz+9fFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE
9Xey4xAAi3zgdTxzfHC63C+/UCKocaIN/HAYjZ8JLraeXurr+HhCPp1OAMK3h/T5
9tYtypgTWXetSG14XD3UcK75Bpbvc4A3lHRpO4nq8LAIsMsWPjmuj1eq36dXnOmO
nQfapj0VTzV/V7Crw0z6fLqwOzJJ3Xa7S5UCBEPTYGV/ZvpWBKlVMPrBFK0S5hwg
T/7aW8Ntmeikv4Am/CLm9FLKhXNBQ1fIctVmUXoAApLgFRMamjZudaoiqps83Wt9
KYDkvHBxS+TCP9RZbgACPoQghrfXIbsxs+3hN030VeF6Lc8Ts7D/2F2SKKt0+zTv
nWpDdtYcpBna08QQJiNXLlbXp4mb39uI5X242vpTBAl+Dtx5nJk9bHp/Or2b2DgW
yXQuXHKwql5slNz78e0C+YtTYUVWHRo3miwUZBxwogMdJ0vW+/12p28xsVz66cf6
v+z7ZFJGjdDsVcQYf8TfA+SDyAlxgElfsOgwsD26WtsZdTj3ZqKsy9Mr5CFzZVBw
UBlr0FaVAmkfiKANeM66Xuw+95Bdnqhbm1byL4RZG9mTQNaLxBTev7dPbFqO5qkQ
vw+MRUd5vXURqiu6F7b6DMvFhe6+HCrJkcONPdXLo1pGBU4PjxdAUhXlr2DkUrm+
KOnVjCQgzkfjLXkEFqAFwcR68knlEJuCXivDd3Wl0A6Y4Jsvfo8=
=Iwo+
-----END PGP SIGNATURE-----

Christopher Baines wrote 6 years ago
[PATCH 1/2] import: utils: Add hash-ref*.
(address . 36948@debbugs.gnu.org)
20190806191728.22923-1-mail@cbaines.net
With the change to guile-json version 3, JSON objects are represented as hash
tables, rather than alists. The cpan importer uses assoc-ref* on a hash table,
so add an equivalent function for hash tables.

* guix/import/utils.scm (hash-ref*): New procedure.
---
guix/import/utils.scm | 10 ++++++++++
1 file changed, 10 insertions(+)

Toggle diff (30 lines)
diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index 2a3b7341fb..ed6c3ce6af 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -47,6 +47,7 @@
flatten
assoc-ref*
+ hash-ref*
url-fetch
guix-hash-url
@@ -116,6 +117,15 @@ recursively apply the procedure to the sub-list."
(assoc-ref alist key)
(apply assoc-ref* (assoc-ref alist key) rest)))
+(define (hash-ref* hash-table key . rest)
+ "Return the value for KEY from HASH-TABLE. For each additional key specified,
+recursively apply the procedure to the sub-hash-table."
+ (if (hash-table? hash-table)
+ (if (null? rest)
+ (hash-ref hash-table key)
+ (apply hash-ref* (hash-ref hash-table key) rest))
+ #f)) ; For consistency with assoc-ref*
+
(define (url-fetch url file-name)
"Save the contents of URL to FILE-NAME. Return #f on failure."
(parameterize ((current-output-port (current-error-port)))
--
2.22.0
Christopher Baines wrote 6 years ago
[PATCH 2/2] import: cpan: Adapt for the change to guile-json version 3.
(address . 36948@debbugs.gnu.org)
20190806191728.22923-2-mail@cbaines.net
In guile-json version 3, JSON objects are represented as hash tables, rather
than alists.

* guix/import/cpan.scm (string->license): Change the match expression to match
on lists, rather than vectors.
(module->dist-name, cpan-source-url, cpan-version): Change assoc-ref to
hash-ref.
(cpan-module->sexp): Change assoc-ref to hash-ref, and assoc-ref* to
hash-ref*.
* tests/cpan.scm ("source-url-http", "source-url-https"): Convert the alist to
a hash table.
---
guix/import/cpan.scm | 30 ++++++++++++++++--------------
tests/cpan.scm | 13 ++++++++-----
2 files changed, 24 insertions(+), 19 deletions(-)

Toggle diff (125 lines)
diff --git a/guix/import/cpan.scm b/guix/import/cpan.scm
index ec86f11743..0be37e715e 100644
--- a/guix/import/cpan.scm
+++ b/guix/import/cpan.scm
@@ -34,7 +34,7 @@
#:use-module (guix ui)
#:use-module ((guix download) #:select (download-to-store url-fetch))
#:use-module ((guix import utils) #:select (factorize-uri
- flatten assoc-ref*))
+ flatten hash-ref*))
#:use-module (guix import json)
#:use-module (guix packages)
#:use-module (guix upstream)
@@ -76,8 +76,8 @@
;; ssleay
;; sun
("zlib" 'zlib)
- (#(x) (string->license x))
- (#(lst ...) `(list ,@(map string->license lst)))
+ ((x) (string->license x))
+ ((lst ...) `(list ,@(map string->license lst)))
(_ #f)))
(define (module->name module)
@@ -88,11 +88,11 @@
"Return the base distribution module for a given module. E.g. the 'ok'
module is distributed with 'Test::Simple', so (module->dist-name \"ok\") would
return \"Test-Simple\""
- (assoc-ref (json-fetch (string-append
- "https://fastapi.metacpan.org/v1/module/"
- module
- "?fields=distribution"))
- "distribution"))
+ (hash-ref (json-fetch (string-append
+ "https://fastapi.metacpan.org/v1/module/"
+ module
+ "?fields=distribution"))
+ "distribution"))
(define (package->upstream-name package)
"Return the CPAN name of PACKAGE."
@@ -122,12 +122,12 @@ or #f on failure. MODULE should be e.g. \"Test::Script\""
(define (cpan-source-url meta)
"Return the download URL for a module's source tarball."
(regexp-substitute/global #f "http[s]?://cpan.metacpan.org"
- (assoc-ref meta "download_url")
+ (hash-ref meta "download_url")
'pre "mirror://cpan" 'post))
(define (cpan-version meta)
"Return the version number from META."
- (match (assoc-ref meta "version")
+ (match (hash-ref meta "version")
((? number? version)
;; version is sometimes not quoted in the module json, so it gets
;; imported into Guile as a number, so convert it to a string.
@@ -183,7 +183,7 @@ depend on (gnu packages perl)."
"Return the `package' s-expression for a CPAN module from the metadata in
META."
(define name
- (assoc-ref meta "distribution"))
+ (hash-ref meta "distribution"))
(define (guix-name name)
(if (string-prefix? "perl-" name)
@@ -198,7 +198,9 @@ META."
(match (flatten
(map (lambda (ph)
(filter-map (lambda (t)
- (assoc-ref* meta "metadata" "prereqs" ph t))
+ (and=> (hash-ref* meta "metadata" "prereqs" ph t)
+ (lambda (h)
+ (hash-map->list cons h))))
'("requires" "recommends" "suggests")))
phases))
(#f
@@ -251,9 +253,9 @@ META."
,@(maybe-inputs 'propagated-inputs
(convert-inputs '("runtime")))
(home-page ,(cpan-home name))
- (synopsis ,(assoc-ref meta "abstract"))
+ (synopsis ,(hash-ref meta "abstract"))
(description fill-in-yourself!)
- (license ,(string->license (assoc-ref meta "license"))))))
+ (license ,(string->license (hash-ref meta "license"))))))
(define (cpan->guix-package module-name)
"Fetch the metadata for PACKAGE-NAME from metacpan.org, and return the
diff --git a/tests/cpan.scm b/tests/cpan.scm
index 189dd027e6..cdd6c0e76a 100644
--- a/tests/cpan.scm
+++ b/tests/cpan.scm
@@ -24,7 +24,8 @@
#:use-module (guix tests)
#:use-module (guix grafts)
#:use-module (srfi srfi-64)
- #:use-module (ice-9 match))
+ #:use-module (ice-9 match)
+ #:use-module (ice-9 hash-table))
;; Globally disable grafts because they can trigger early builds.
(%graft? #f)
@@ -109,14 +110,16 @@
(test-equal "source-url-http"
((@@ (guix import cpan) cpan-source-url)
- `(("download_url" .
- "http://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")))
+ (alist->hash-table
+ `(("download_url" .
+ "http://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz"))))
"mirror://cpan/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")
(test-equal "source-url-https"
((@@ (guix import cpan) cpan-source-url)
- `(("download_url" .
- "https://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")))
+ (alist->hash-table
+ `(("download_url" .
+ "https://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz"))))
"mirror://cpan/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")
(test-end "cpan")
--
2.22.0
Christopher Baines wrote 6 years ago
Re: [bug#36948] [PATCH 0/2] Fix the CPAN importer
(address . 36948-done@debbugs.gnu.org)
875zmq8lz1.fsf@cbaines.net
Christopher Baines <mail@cbaines.net> writes:

Toggle quote (9 lines)
> Christopher Baines (2):
> import: utils: Add hash-ref*.
> import: cpan: Adapt for the change to guile-json version 3.
>
> guix/import/cpan.scm | 30 ++++++++++++++++--------------
> guix/import/utils.scm | 10 ++++++++++
> tests/cpan.scm | 13 ++++++++-----
> 3 files changed, 34 insertions(+), 19 deletions(-)

I've pushed these patches as 01ce7af25add55514f737af48ea6c127bedfde67
now.
-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl1df8JfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE
9Xc7vRAAtV1Pil0G9JCqIYbdzcv2H4Q7DkhZyx4VO/nbY6pjWltY21IrLDEjYJi7
B9614FHoMIql2YCBn3NrgqvYN9VmZlGeIGooydJVhSqsNW83XU/w6xUuYQuXwed+
9EkJQofp/MT9DCPblUC4Y4BLV1mNIkjPgvRoTnNXLNnZOkveOoARL6Go9vxVw/Hk
xxHm7x2mvby5suZdvq2eX8Bdv0BKNe+7NHAGOPeL1AlkiJSis9gVMmAOEcmV9cwe
g+sI4NYnWhGW26LYCFkA05f8wlQt3H2z1N7126/Njcy4GFPhlOz8mkXfJF/itiu8
NtRQvcmu5WllpKCYV4fITaPTIaxeuuoJe5nMqNrDk5Xpv0ZOJwJ/BdszsUIeTAi2
f4H2p0lyPc7/Mp79IbsV5xSVP6JR65/WLEUnNU5X0o2mEAmBIKyZ2/xcCryy0i2X
ewf8gGkxvI44tR4VYtoAPOJx87fkV9f0jExt5ZucidkFxd+W/k8mOdr2GwReI/2I
hWgYj1ZNj293Db4Y0ayDyffHryAxpZO4L4MhX49re1q/Vn9ei9qsC/xUTC5NWPsT
tfffZJ7z61KicdB3s3T5hvBPGp0PN52fpfNij+rcbasRV6I3baqWfErjJBXl/p5H
g/vbKQe+iTUmCexwTeY3PYENxlc3/jdCRv/awGxxPCAX0vnNGq0=
=ThD/
-----END PGP SIGNATURE-----

Closed
Ludovic Courtès wrote 6 years ago
Re: [bug#36948] [PATCH 1/2] import: utils: Add hash-ref*.
(name . Christopher Baines)(address . mail@cbaines.net)(address . 36948@debbugs.gnu.org)
8736htqucz.fsf@gnu.org
Hello Chris,

Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (6 lines)
> With the change to guile-json version 3, JSON objects are represented as hash
> tables, rather than alists. The cpan importer uses assoc-ref* on a hash table,
> so add an equivalent function for hash tables.
>
> * guix/import/utils.scm (hash-ref*): New procedure.

[...]

Toggle quote (3 lines)
> In guile-json version 3, JSON objects are represented as hash tables, rather
> than alists.

I seems to me that this is adapting for Guile-JSON v1, not v3: in v3,
JSON arrays map to Scheme vectors, and JSON dictionaries map to alists;
JSON dictionaries used to map to hash tables in v1.

Indeed, ‘tests/cpan.scm’ now fails for me:

Toggle snippet (14 lines)
actual-error:
+ (wrong-type-arg
+ "scm_hash_fn_get_handle"
+ "Wrong type argument in position ~A (expecting ~A): ~S"
+ (1
+ "hash-table"
+ (("version" . "0.1")
+ ("author" . "Guix")
+ ("download_url"
+ .
+ "http://example.com/Foo-Bar-0.1.tar.gz")
[…]

Could it be that you were testing this in an environment containing v1
and not v3?

Sorry for not noticing earlier!

Thanks,
Ludo’.
?
Your comment

This issue is archived.

To comment on this conversation send an email to 36948@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 36948
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help