From debbugs-submit-bounces@debbugs.gnu.org Sun Feb 06 16:42:17 2022 Received: (at submit) by debbugs.gnu.org; 6 Feb 2022 21:42:17 +0000 Received: from localhost ([127.0.0.1]:39975 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nGpIW-0006I6-NA for submit@debbugs.gnu.org; Sun, 06 Feb 2022 16:42:17 -0500 Received: from lists.gnu.org ([209.51.188.17]:37360) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nGpIV-0006Hy-3W for submit@debbugs.gnu.org; Sun, 06 Feb 2022 16:42:15 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55344) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nGpIU-0007wM-ML for guix-patches@gnu.org; Sun, 06 Feb 2022 16:42:14 -0500 Received: from [2a00:5884:8208::1] (port=35922 helo=lepiller.eu) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nGpI2-0006Lw-3K for guix-patches@gnu.org; Sun, 06 Feb 2022 16:41:49 -0500 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id 5c328a32 for ; Sun, 6 Feb 2022 21:41:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date:from :to:subject:message-id:mime-version:content-type; s=dkim; bh=V5n maPMXPc32Hoqgk9j6lNhM0FesZDhSCkXEZ9qBakE=; b=mV+E8Q7kAyLAfWLMHhW Kt8i/eZwvldjss7CJtovIG3pTffPTRxj+JCTYj0hFkE+7cU/YW4oosngn3idopVX lBe8CTz8N/SDr9NZ1qv2SJYTXBSHn9XFo3y16eK4sLLR016C+X6pz6aZrb1Dmqyk MEXzL3Gq1Eqv4l3RQ4AizbV6u/fPVquBr1KAfBNErIy8AIxNny3glWUkURpHTh75 zVBH0tD5WMCPYDOCRaGr5vcnbpdvzz/MuE4qmk4GEVJe70icrlxlvfCxPTCRZH1q jusdlQ4PX6fGArL9D7AfUsGDoaUJBY0cF38BMptWRdwdLzxCkFO9SPUIEK/wPoLt yVA== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id 7855a846 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO) for ; Sun, 6 Feb 2022 21:41:39 +0000 (UTC) Date: Sun, 6 Feb 2022 22:41:30 +0100 From: Julien Lepiller To: guix-patches@gnu.org Subject: [PATCH] guix: opam: Allow importing local files. Message-ID: <20220206224130.2220aae9@tachikoma.lepiller.eu> X-Mailer: Claws Mail 4.0.0 (GTK+ 3.24.30; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="MP_/GxuYmKHeuMF5J7IXm2T6L90" X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:5884:8208::1 (failed) Received-SPF: pass client-ip=2a00:5884:8208::1; envelope-from=julien@lepiller.eu; helo=lepiller.eu X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RDNS_NONE=0.793, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) --MP_/GxuYmKHeuMF5J7IXm2T6L90 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi Guix! This patch lets you import opam packages from a local file. This is a feature that was requested a few times, and there it is :) Now, if you have an OCaml package sources that provides an opam file (usually called opam, or with the .opam extension), you can import the package, recursively or not with something like: guix import opam foo -f foo.opam -r instead of fetching foo from the repositories, it will fetch the data directly from the given file. WDYT? --MP_/GxuYmKHeuMF5J7IXm2T6L90 Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0001-guix-opam-Allow-importing-local-files.patch From b027391357eef4d6fa1bb8741528cf9650aef568 Mon Sep 17 00:00:00 2001 Message-Id: From: Julien Lepiller Date: Sun, 6 Feb 2022 22:35:14 +0100 Subject: [PATCH] guix: opam: Allow importing local files. * guix/scripts/import/opam.scm (guix-import-opam): Support `--file` flag. * guix/import/utils.scm (recursive-import): Add optional file argument. Pass it to the first iteration when available. * guix/import/opam.scm (opam-fetch): Fetch from file when available. (opam->guix-source): Factorize source out of opam->guix-package. (opam->guix-package): Add file argument. Ensure it does not fail when name, version and url are not available. --- guix/import/opam.scm | 101 +++++++++++++++++++---------------- guix/import/utils.scm | 17 +++--- guix/scripts/import/opam.scm | 11 +++- 3 files changed, 75 insertions(+), 54 deletions(-) diff --git a/guix/import/opam.scm b/guix/import/opam.scm index a6f6fe8c9f..05f79110f8 100644 --- a/guix/import/opam.scm +++ b/guix/import/opam.scm @@ -310,8 +310,11 @@ (define (dependency-list->inputs lst) (map string->symbol (ocaml-names->guix-names lst))) -(define* (opam-fetch name #:optional (repositories-specs '("opam"))) - (or (fold (lambda (repository others) +(define* (opam-fetch name #:optional (repositories-specs '("opam")) (file #f)) + (or (and file (let ((metadata (get-metadata file))) + `(("metadata" . ,metadata) + ("version" . #f)))) + (fold (lambda (repository others) (match (find-latest-version name repository) ((_ version file) `(("metadata" ,@(get-metadata file)) ("version" . ,version))) @@ -320,17 +323,29 @@ (define* (opam-fetch name #:optional (repositories-specs '("opam"))) (filter-map get-opam-repository repositories-specs)) (warning (G_ "opam: package '~a' not found~%") name))) -(define* (opam->guix-package name #:key (repo 'opam) version) +(define (opam->guix-source url-dict) + (let ((source-url (and url-dict + (or (metadata-ref url-dict "src") + (metadata-ref url-dict "archive"))))) + (if source-url + (call-with-temporary-output-file + (lambda (temp port) + (and (url-fetch source-url temp) + `(origin + (method url-fetch) + (uri ,source-url) + (sha256 (base32 ,(guix-hash-url temp))))))) + 'no-source-information))) + +(define* (opam->guix-package name #:key (repo '("opam")) (file #f) version) "Import OPAM package NAME from REPOSITORY (a directory name) or, if -REPOSITORY is #f, from the official OPAM repository. Return a 'package' sexp +REPOSITORY is #f, from the official OPAM repository. When FILE is defined, +import the packaged defined in the opam FILE instead. Return a 'package' sexp or #f on failure." (and-let* ((with-opam (if (member "opam" repo) repo (cons "opam" repo))) - (opam-file (opam-fetch name with-opam)) - (version (assoc-ref opam-file "version")) + (opam-file (opam-fetch name with-opam file)) (opam-content (assoc-ref opam-file "metadata")) - (url-dict (metadata-ref opam-content "url")) - (source-url (or (metadata-ref url-dict "src") - (metadata-ref url-dict "archive"))) + (source (opam->guix-source (metadata-ref opam-content "url"))) (requirements (metadata-ref opam-content "depends")) (names (dependency-list->names requirements)) (dependencies (filter-dependencies names)) @@ -344,47 +359,41 @@ (define* (opam->guix-package name #:key (repo 'opam) version) (not (member name '("dune" "jbuilder")))) native-dependencies)))) (let ((use-dune? (member "dune" names))) - (call-with-temporary-output-file - (lambda (temp port) - (and (url-fetch source-url temp) - (values - `(package - (name ,(ocaml-name->guix-name name)) - (version ,version) - (source - (origin - (method url-fetch) - (uri ,source-url) - (sha256 (base32 ,(guix-hash-url temp))))) - (build-system ,(if use-dune? - 'dune-build-system - 'ocaml-build-system)) - ,@(if (null? inputs) - '() - `((propagated-inputs (list ,@inputs)))) - ,@(if (null? native-inputs) - '() - `((native-inputs (list ,@native-inputs)))) - ,@(if (equal? name (guix-name->opam-name (ocaml-name->guix-name name))) - '() - `((properties - ,(list 'quasiquote `((upstream-name . ,name)))))) - (home-page ,(metadata-ref opam-content "homepage")) - (synopsis ,(metadata-ref opam-content "synopsis")) - (description ,(beautify-description - (metadata-ref opam-content "description"))) - (license ,(spdx-string->license - (metadata-ref opam-content "license")))) - (filter - (lambda (name) - (not (member name '("dune" "jbuilder")))) - dependencies)))))))) + (values + `(package + (name ,(and name (ocaml-name->guix-name name))) + (version ,(assoc-ref opam-file "version")) + (source ,source) + (build-system ,(if use-dune? + 'dune-build-system + 'ocaml-build-system)) + ,@(if (null? inputs) + '() + `((propagated-inputs (list ,@inputs)))) + ,@(if (null? native-inputs) + '() + `((native-inputs (list ,@native-inputs)))) + ,@(if (and name (equal? name (guix-name->opam-name (ocaml-name->guix-name name)))) + '() + `((properties + ,(list 'quasiquote `((upstream-name . ,name)))))) + (home-page ,(metadata-ref opam-content "homepage")) + (synopsis ,(metadata-ref opam-content "synopsis")) + (description ,(beautify-description + (metadata-ref opam-content "description"))) + (license ,(spdx-string->license + (metadata-ref opam-content "license")))) + (filter + (lambda (name) + (not (member name '("dune" "jbuilder")))) + dependencies))))) -(define* (opam-recursive-import package-name #:key repo) +(define* (opam-recursive-import package-name #:key repo file) (recursive-import package-name #:repo->guix-package opam->guix-package #:guix-name ocaml-name->guix-name - #:repo repo)) + #:repo repo + #:file file)) (define (guix-name->opam-name name) (if (string-prefix? "ocaml-" name) diff --git a/guix/import/utils.scm b/guix/import/utils.scm index 1c3cfa3e0b..ab35b8a4fc 100644 --- a/guix/import/utils.scm +++ b/guix/import/utils.scm @@ -470,7 +470,7 @@ (define (topological-sort nodes (set-insert (node-name head) visited)))))))) (define* (recursive-import package-name - #:key repo->guix-package guix-name version repo + #:key repo->guix-package guix-name version repo file #:allow-other-keys) "Return a list of package expressions for PACKAGE-NAME and all its dependencies, sorted in topological order. For each package, @@ -488,10 +488,15 @@ (define* (recursive-import package-name (define (exists? name version) (not (null? (find-packages-by-name (guix-name name) version)))) - (define (lookup-node name version) - (let* ((package dependencies (repo->guix-package name - #:version version - #:repo repo)) + (define* (lookup-node name version #:optional file) + (let* ((package dependencies (if file + (repo->guix-package name + #:version version + #:repo repo + #:file file) + (repo->guix-package name + #:version version + #:repo repo))) (normalized-deps (map (match-lambda ((name version) (list name version)) (name (list name #f))) dependencies))) @@ -499,7 +504,7 @@ (define* (recursive-import package-name (filter-map node-package - (topological-sort (list (lookup-node package-name version)) + (topological-sort (list (lookup-node package-name version file)) (lambda (node) (map (lambda (name-version) (apply lookup-node name-version)) diff --git a/guix/scripts/import/opam.scm b/guix/scripts/import/opam.scm index 834ac34cb0..b58c6eab3d 100644 --- a/guix/scripts/import/opam.scm +++ b/guix/scripts/import/opam.scm @@ -50,6 +50,8 @@ (define (show-help) --repo import packages from this opam repository (name, URL or local path) can be used more than once")) (display (G_ " + -f, --file import the package defined from a .opam file")) + (display (G_ " -V, --version display version information and exit")) (newline) (show-bug-report-information)) @@ -66,6 +68,9 @@ (define %options (option '(#f "repo") #t #f (lambda (opt name arg result) (alist-cons 'repo arg result))) + (option '(#\f "file") #t #f + (lambda (opt name arg result) + (alist-cons 'file arg result))) (option '(#\r "recursive") #f #f (lambda (opt name arg result) (alist-cons 'recursive #t result))) @@ -100,9 +105,11 @@ (define (guix-import-opam . args) `(define-public ,(string->symbol name) ,pkg)) (_ #f)) - (opam-recursive-import package-name #:repo repo)) + (opam-recursive-import package-name #:repo repo + #:file (assoc-ref opts 'file))) ;; Single import - (let ((sexp (opam->guix-package package-name #:repo repo))) + (let ((sexp (opam->guix-package package-name #:repo repo + #:file (assoc-ref opts 'file)))) (unless sexp (leave (G_ "failed to download meta-data for package '~a'~%") package-name)) -- 2.34.0 --MP_/GxuYmKHeuMF5J7IXm2T6L90--