[PATCH 0/2] Fix the CPAN importer

Done

Details

2 participants

Ludovic Courtès
Christopher Baines

Owner: unassigned

Submitted by: Christopher Baines

Severity: normal

Christopher Baines wrote on 6 Aug 2019 21:07

Recipients:(address . guix-patches@gnu.org)

Message-ID:87k1bq864w.fsf@cbaines.net

Christopher Baines (2):
  import: utils: Add hash-ref*.
  import: cpan: Adapt for the change to guile-json version 3.

 guix/import/cpan.scm  | 30 ++++++++++++++++--------------
 guix/import/utils.scm | 10 ++++++++++
 tests/cpan.scm        | 13 ++++++++-----
 3 files changed, 34 insertions(+), 19 deletions(-)

Christopher Baines wrote on 6 Aug 2019 21:17

[PATCH 1/2] import: utils: Add hash-ref*.

Recipients:(address . 36948@debbugs.gnu.org)

Message-ID:20190806191728.22923-1-mail@cbaines.net

With the change to guile-json version 3, JSON objects are represented as hash

tables, rather than alists. The cpan importer uses assoc-ref* on a hash table,

so add an equivalent function for hash tables.

* guix/import/utils.scm (hash-ref*): New procedure.

---

guix/import/utils.scm | 10 ++++++++++

1 file changed, 10 insertions(+)

Toggle diff (30 lines)diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index 2a3b7341fb..ed6c3ce6af 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -47,6 +47,7 @@
 
             flatten
             assoc-ref*
+            hash-ref*
 
             url-fetch
             guix-hash-url
@@ -116,6 +117,15 @@ recursively apply the procedure to the sub-list."
       (assoc-ref alist key)
       (apply assoc-ref* (assoc-ref alist key) rest)))
 
+(define (hash-ref* hash-table key . rest)
+  "Return the value for KEY from HASH-TABLE.  For each additional key specified,
+recursively apply the procedure to the sub-hash-table."
+  (if (hash-table? hash-table)
+      (if (null? rest)
+          (hash-ref hash-table key)
+          (apply hash-ref* (hash-ref hash-table key) rest))
+      #f))                              ; For consistency with assoc-ref*
+
 (define (url-fetch url file-name)
   "Save the contents of URL to FILE-NAME.  Return #f on failure."
   (parameterize ((current-output-port (current-error-port)))
-- 
2.22.0

Christopher Baines wrote on 6 Aug 2019 21:17

[PATCH 2/2] import: cpan: Adapt for the change to guile-json version 3.

Recipients:(address . 36948@debbugs.gnu.org)

Message-ID:20190806191728.22923-2-mail@cbaines.net

In guile-json version 3, JSON objects are represented as hash tables, rather
than alists.

* guix/import/cpan.scm (string->license): Change the match expression to match
on lists, rather than vectors.
(module->dist-name, cpan-source-url, cpan-version): Change assoc-ref to
hash-ref.
(cpan-module->sexp): Change assoc-ref to hash-ref, and assoc-ref* to
hash-ref*.
* tests/cpan.scm ("source-url-http", "source-url-https"): Convert the alist to
a hash table.
---
 guix/import/cpan.scm | 30 ++++++++++++++++--------------
 tests/cpan.scm       | 13 ++++++++-----
 2 files changed, 24 insertions(+), 19 deletions(-)

Toggle diff (125 lines)diff --git a/guix/import/cpan.scm b/guix/import/cpan.scm
index ec86f11743..0be37e715e 100644
--- a/guix/import/cpan.scm
+++ b/guix/import/cpan.scm
@@ -34,7 +34,7 @@
   #:use-module (guix ui)
   #:use-module ((guix download) #:select (download-to-store url-fetch))
   #:use-module ((guix import utils) #:select (factorize-uri
-                                              flatten assoc-ref*))
+                                              flatten hash-ref*))
   #:use-module (guix import json)
   #:use-module (guix packages)
   #:use-module (guix upstream)
@@ -76,8 +76,8 @@
    ;; ssleay
    ;; sun
    ("zlib" 'zlib)
-   (#(x) (string->license x))
-   (#(lst ...) `(list ,@(map string->license lst)))
+   ((x) (string->license x))
+   ((lst ...) `(list ,@(map string->license lst)))
    (_ #f)))
 
 (define (module->name module)
@@ -88,11 +88,11 @@
   "Return the base distribution module for a given module.  E.g. the 'ok'
 module is distributed with 'Test::Simple', so (module->dist-name \"ok\") would
 return \"Test-Simple\""
-  (assoc-ref (json-fetch (string-append
-                          "https://fastapi.metacpan.org/v1/module/"
-                          module
-                          "?fields=distribution"))
-             "distribution"))
+  (hash-ref (json-fetch (string-append
+                         "https://fastapi.metacpan.org/v1/module/"
+                         module
+                         "?fields=distribution"))
+            "distribution"))
 
 (define (package->upstream-name package)
   "Return the CPAN name of PACKAGE."
@@ -122,12 +122,12 @@ or #f on failure.  MODULE should be e.g. \"Test::Script\""
 (define (cpan-source-url meta)
   "Return the download URL for a module's source tarball."
   (regexp-substitute/global #f "http[s]?://cpan.metacpan.org"
-                            (assoc-ref meta "download_url")
+                            (hash-ref meta "download_url")
                             'pre "mirror://cpan" 'post))
 
 (define (cpan-version meta)
   "Return the version number from META."
-  (match (assoc-ref meta "version")
+  (match (hash-ref meta "version")
     ((? number? version)
      ;; version is sometimes not quoted in the module json, so it gets
      ;; imported into Guile as a number, so convert it to a string.
@@ -183,7 +183,7 @@ depend on (gnu packages perl)."
   "Return the `package' s-expression for a CPAN module from the metadata in
 META."
   (define name
-    (assoc-ref meta "distribution"))
+    (hash-ref meta "distribution"))
 
   (define (guix-name name)
     (if (string-prefix? "perl-" name)
@@ -198,7 +198,9 @@ META."
     (match (flatten
             (map (lambda (ph)
                    (filter-map (lambda (t)
-                                 (assoc-ref* meta "metadata" "prereqs" ph t))
+                                 (and=> (hash-ref* meta "metadata" "prereqs" ph t)
+                                        (lambda (h)
+                                          (hash-map->list cons h))))
                                '("requires" "recommends" "suggests")))
                  phases))
       (#f
@@ -251,9 +253,9 @@ META."
        ,@(maybe-inputs 'propagated-inputs
                        (convert-inputs '("runtime")))
        (home-page ,(cpan-home name))
-       (synopsis ,(assoc-ref meta "abstract"))
+       (synopsis ,(hash-ref meta "abstract"))
        (description fill-in-yourself!)
-       (license ,(string->license (assoc-ref meta "license"))))))
+       (license ,(string->license (hash-ref meta "license"))))))
 
 (define (cpan->guix-package module-name)
   "Fetch the metadata for PACKAGE-NAME from metacpan.org, and return the
diff --git a/tests/cpan.scm b/tests/cpan.scm
index 189dd027e6..cdd6c0e76a 100644
--- a/tests/cpan.scm
+++ b/tests/cpan.scm
@@ -24,7 +24,8 @@
   #:use-module (guix tests)
   #:use-module (guix grafts)
   #:use-module (srfi srfi-64)
-  #:use-module (ice-9 match))
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 hash-table))
 
 ;; Globally disable grafts because they can trigger early builds.
 (%graft? #f)
@@ -109,14 +110,16 @@
 
 (test-equal "source-url-http"
   ((@@ (guix import cpan) cpan-source-url)
-   `(("download_url" .
-      "http://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")))
+   (alist->hash-table
+    `(("download_url" .
+       "http://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz"))))
   "mirror://cpan/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")
 
 (test-equal "source-url-https"
   ((@@ (guix import cpan) cpan-source-url)
-   `(("download_url" .
-      "https://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")))
+   (alist->hash-table
+    `(("download_url" .
+       "https://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz"))))
   "mirror://cpan/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")
 
 (test-end "cpan")
-- 
2.22.0

Christopher Baines wrote on 21 Aug 2019 19:30

Re: [bug#36948] [PATCH 0/2] Fix the CPAN importer

Recipients:(address . 36948-done@debbugs.gnu.org)

Message-ID:875zmq8lz1.fsf@cbaines.net

Christopher Baines <mail@cbaines.net> writes:

Toggle quote (9 lines)

> Christopher Baines (2):

> import: utils: Add hash-ref*.

> import: cpan: Adapt for the change to guile-json version 3.

> guix/import/cpan.scm | 30 ++++++++++++++++--------------

> guix/import/utils.scm | 10 ++++++++++

> tests/cpan.scm | 13 ++++++++-----

> 3 files changed, 34 insertions(+), 19 deletions(-)

I've pushed these patches as 01ce7af25add55514f737af48ea6c127bedfde67

now.

Closed

Ludovic Courtès wrote on 22 Aug 2019 14:04

Re: [bug#36948] [PATCH 1/2] import: utils: Add hash-ref*.

Recipients:(name . Christopher Baines)(address . mail@cbaines.net)(address . 36948@debbugs.gnu.org)

Message-ID:8736htqucz.fsf@gnu.org

Hello Chris,

Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (6 lines)

> With the change to guile-json version 3, JSON objects are represented as hash

> tables, rather than alists. The cpan importer uses assoc-ref* on a hash table,

> so add an equivalent function for hash tables.

> * guix/import/utils.scm (hash-ref*): New procedure.

[...]

Toggle quote (3 lines)

> In guile-json version 3, JSON objects are represented as hash tables, rather

> than alists.

I seems to me that this is adapting for Guile-JSON v1, not v3: in v3,

JSON arrays map to Scheme vectors, and JSON dictionaries map to alists;

JSON dictionaries used to map to hash tables in v1.

Indeed, ‘tests/cpan.scm’ now fails for me:

Toggle snippet (14 lines)actual-error:
+ (wrong-type-arg
+   "scm_hash_fn_get_handle"
+   "Wrong type argument in position ~A (expecting ~A): ~S"
+   (1
+    "hash-table"
+    (("version" . "0.1")
+     ("author" . "Guix")
+     ("download_url"
+      .
+      "http://example.com/Foo-Bar-0.1.tar.gz")
[…]

Could it be that you were testing this in an environment containing v1

and not v3?

Sorry for not noticing earlier!

Thanks,

Ludo’.

Your comment

This issue is archived.

To comment on this conversation send an email to 36948@debbugs.gnu.org

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date