[PATCH 0/3] Add the '--with-branch' package transformation option

DoneSubmitted by Ludovic Courtès.
Details
2 participants
  • Ludovic Courtès
  • Ludovic Courtès
Owner
unassigned
Severity
normal
L
L
Ludovic Courtès wrote on 27 Nov 2018 16:31
(address . guix-patches@gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20181127153147.21247-1-ludo@gnu.org
Hello,
This patch series adds a ‘git-checkout’ file-like construct along witha ‘--with-branch’ package transformation option.
The ‘git-checkout’ construct may be used like this:
(define-public guile-git (package (inherit guile-next) (version "git") (source (git-checkout (url "https://git.savannah.gnu.org/git/guile.git") (branch "master")))))
Then you can do this:
Toggle snippet (7 lines)$ guix build -L ~/.config/guix/personal/ guile-next@gitupdating checkout of 'https://git.savannah.gnu.org/git/guile.git'...retrieved commit 1f6786db06fee5f0bbdbd637acc6aa46aecd4169building /gnu/store/7l168l4lgpjjg9jq7lh2416cc884r3s9-guile-next-git.drv...[…]
IOW, every time you build the package, it gets built off the latestcommit of the specified branch.
The next patch add ‘--with-branch’ which makes that functionalityavailable from the command line. For example:
guix build --with-branch=guile-gcrypt=master cuirass
builds the latest guile-gcrypt, guix, and cuirass, using the latestguile-gcrypt commit on ‘master’.
The ‘--with-branch’ option only works with packages that have a‘git-fetch’ origin or a ‘git-checkout’ as their source.
I find it rather neat. WDYT? :-)
Ludo’.
Ludovic Courtès (3): git: 'latest-repository-commit' logs its progress. git: Add <git-checkout> record type. guix build: Add '--with-branch' transformation option.
Makefile.am | 1 + doc/guix.texi | 27 ++++++++++++++++++ guix/git.scm | 41 ++++++++++++++++++++++++++-- guix/scripts/build.scm | 56 ++++++++++++++++++++++++++++++++++++-- tests/guix-build-branch.sh | 48 ++++++++++++++++++++++++++++++++ 5 files changed, 168 insertions(+), 5 deletions(-) create mode 100644 tests/guix-build-branch.sh
-- 2.19.2
L
L
Ludovic Courtès wrote on 27 Nov 2018 16:34
[PATCH 1/3] git: 'latest-repository-commit' logs its progress.
(address . 33522@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
20181127153402.21357-1-ludo@gnu.org
From: Ludovic Courtès <ludovic.courtes@inria.fr>
* guix/git.scm (latest-repository-commit): Add #:log-port and honor it.--- guix/git.scm | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
Toggle diff (38 lines)diff --git a/guix/git.scm b/guix/git.scmindex d007916662..78645024c9 100644--- a/guix/git.scm+++ b/guix/git.scm@@ -154,6 +154,7 @@ data, respectively [<branch name> | <sha1> | <tag name>]." (define* (latest-repository-commit store url #:key+ (log-port (%make-void-port "w")) (cache-directory (%repository-cache-directory)) (ref '(branch . "master")))@@ -164,11 +165,14 @@ REF is pair whose key is [branch | commit | tag] and value the associated data, respectively [<branch name> | <sha1> | <tag name>]. Git repositories are kept in the cache directory specified by-%repository-cache-directory parameter."+%repository-cache-directory parameter.++Log progress and checkout info to LOG-PORT." (define (dot-git? file stat) (and (string=? (basename file) ".git") (eq? 'directory (stat:type stat)))) + (format log-port "updating checkout of '~a'...~%" url) (let*-values (((checkout commit) (update-cached-checkout url@@ -177,6 +181,7 @@ Git repositories are kept in the cache directory specified by (url-cache-directory url cache-directory))) ((name) (url+commit->name url commit)))+ (format log-port "retrieved commit ~a~%" commit) (values (add-to-store store name #t "sha256" checkout #:select? (negate dot-git?)) commit)))-- 2.19.2
L
L
Ludovic Courtès wrote on 27 Nov 2018 16:34
[PATCH 2/3] git: Add <git-checkout> record type.
(address . 33522@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
20181127153402.21357-2-ludo@gnu.org
From: Ludovic Courtès <ludovic.courtes@inria.fr>
* guix/git.scm (<git-checkout>): New record type.(latest-repository-commit*): New procedure.(git-checkout-compiler): New gexp compiler.--- guix/git.scm | 34 +++++++++++++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-)
Toggle diff (58 lines)diff --git a/guix/git.scm b/guix/git.scmindex 78645024c9..56cebb06ed 100644--- a/guix/git.scm+++ b/guix/git.scm@@ -25,6 +25,8 @@ #:use-module ((guix build utils) #:select (mkdir-p)) #:use-module (guix store) #:use-module (guix utils)+ #:use-module (guix records)+ #:use-module (guix gexp) #:use-module (rnrs bytevectors) #:use-module (ice-9 match) #:use-module (srfi srfi-1)@@ -33,7 +35,12 @@ #:use-module (srfi srfi-35) #:export (%repository-cache-directory update-cached-checkout- latest-repository-commit))+ latest-repository-commit++ git-checkout+ git-checkout?+ git-checkout-url+ git-checkout-branch)) (define %repository-cache-directory (make-parameter (string-append (cache-directory #:ensure? #f)@@ -185,3 +192,28 @@ Log progress and checkout info to LOG-PORT." (values (add-to-store store name #t "sha256" checkout #:select? (negate dot-git?)) commit)))++ +;;;+;;; Checkouts.+;;;++;; Representation of the "latest" checkout of a branch.+(define-record-type* <git-checkout>+ git-checkout make-git-checkout+ git-checkout?+ (url git-checkout-url)+ (branch git-checkout-branch (default "master")))++(define latest-repository-commit*+ (store-lift latest-repository-commit))++(define-gexp-compiler (git-checkout-compiler (checkout <git-checkout>)+ system target)+ ;; "Compile" CHECKOUT by updating the local checkout and adding it to the+ ;; store.+ (match checkout+ (($ <git-checkout> url branch)+ (latest-repository-commit* url+ #:ref `(branch . ,branch)+ #:log-port (current-error-port)))))-- 2.19.2
L
L
Ludovic Courtès wrote on 27 Nov 2018 16:34
[PATCH 3/3] guix build: Add '--with-branch' transformation option.
(address . 33522@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
20181127153402.21357-3-ludo@gnu.org
From: Ludovic Courtès <ludovic.courtes@inria.fr>
* guix/scripts/build.scm (evaluate-git-replacement-specs)(transform-package-source-branch): New procedures.(%transformations, %transformation-options): Add 'with-branch'.(show-transformation-options-help): Likewise.* tests/guix-build-branch.sh: New file.* Makefile.am (SH_TESTS): Add it.* doc/guix.texi (Package Transformation Options): Document it.--- Makefile.am | 1 + doc/guix.texi | 27 ++++++++++++++++++ guix/scripts/build.scm | 56 ++++++++++++++++++++++++++++++++++++-- tests/guix-build-branch.sh | 48 ++++++++++++++++++++++++++++++++ 4 files changed, 129 insertions(+), 3 deletions(-) create mode 100644 tests/guix-build-branch.sh
Toggle diff (201 lines)diff --git a/Makefile.am b/Makefile.amindex c5676b0b94..4f134ab174 100644--- a/Makefile.am+++ b/Makefile.am@@ -407,6 +407,7 @@ endif SH_TESTS = \ tests/guix-build.sh \+ tests/guix-build-branch.sh \ tests/guix-download.sh \ tests/guix-gc.sh \ tests/guix-hash.sh \diff --git a/doc/guix.texi b/doc/guix.texiindex 917a3e9d57..18bc3508c1 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -6447,6 +6447,33 @@ must be compatible. If @var{replacement} is somehow incompatible with @var{package}, then the resulting package may be unusable. Use with care! +@item --with-branch=@var{package}=@var{branch}+@cindex Git, using the latest commit+@cindex latest commit, building+Build @var{package} from the latest commit of @var{branch}. The @code{source}+field of @var{package} must be an origin with the @code{git-fetch} method+(@pxref{origin Reference}) or a @code{git-checkout} object; the repository URL+is taken from that @code{source}.++For instance, the following command builds @code{guile-sqlite3} from the+latest commit of its @code{master} branch, and then builds @code{guix} (which+depends on it) and @code{cuirass} (which depends on @code{guix}) against this+specific @code{guile-sqlite3} build:++@example+guix build --with-branch=guile-sqlite3=master cuirass+@end example++@cindex continuous integration+Obviously, since it uses the latest commit of the given branch, the result of+such a command varies over time. Nevertheless it is a convenient way to+rebuild entire software stacks against the latest commit of one or more+packages. This is particularly useful in the context of continuous+integration (CI).++Checkouts are kept in a cache under @file{~/.cache/guix/checkouts} to speed up+consecutive accesses to the same repository. You may want to clean it up once+in a while to save disk space. @end table @node Additional Build Optionsdiff --git a/guix/scripts/build.scm b/guix/scripts/build.scmindex 13978abb77..e8f2fe973d 100644--- a/guix/scripts/build.scm+++ b/guix/scripts/build.scm@@ -45,6 +45,8 @@ #:use-module (srfi srfi-37) #:autoload (gnu packages) (specification->package %package-module-path) #:autoload (guix download) (download-to-store)+ #:autoload (guix git-download) (git-reference?)+ #:autoload (guix git) (git-checkout?) #:use-module (guix status) #:use-module ((guix progress) #:select (current-terminal-columns)) #:use-module ((guix build syscalls) #:select (terminal-columns))@@ -270,6 +272,48 @@ current 'gnutls' package, after which version 3.5.4 is grafted onto them." (rewrite obj) obj)))) +(define (evaluate-git-replacement-specs specs)+ "Parse SPECS, a list of strings like \"guile=stable-2.2\", and return a list+of package pairs. Raise an error if an element of SPECS uses invalid syntax,+or if a package it refers to could not be found."+ (define not-equal+ (char-set-complement (char-set #\=)))++ (map (lambda (spec)+ (match (string-tokenize spec not-equal)+ ((name branch)+ (let* ((old (specification->package name))+ (source (package-source old))+ (url (cond ((and (origin? source)+ (git-reference? (origin-uri source)))+ (git-reference-url (origin-uri source)))+ ((git-checkout? source)+ (git-checkout-url source))+ (else+ (leave (G_ "the source of ~a is not a Git \+reference~%")+ (package-full-name old))))))+ (cons old+ (package+ (inherit old)+ (version (string-append "git." branch))+ (source (git-checkout (url url) (branch branch)))))))+ (x+ (leave (G_ "invalid replacement specification: ~s~%") spec))))+ specs))++(define (transform-package-source-branch replacement-specs)+ "Return a procedure that, when passed a package, replaces its direct+dependencies according to REPLACEMENT-SPECS. REPLACEMENT-SPECS is a list of+strings like \"guile-next=stable-3.0\" meaning that packages are built using+'guile-next' from the latest commit on its 'stable-3.0' branch."+ (let* ((replacements (evaluate-git-replacement-specs replacement-specs))+ (rewrite (package-input-rewriting replacements)))+ (lambda (store obj)+ (if (package? obj)+ (rewrite obj)+ obj))))+ (define %transformations ;; Transformations that can be applied to things to build. The car is the ;; key used in the option alist, and the cdr is the transformation@@ -277,7 +321,8 @@ current 'gnutls' package, after which version 3.5.4 is grafted onto them." ;; things to build. `((with-source . ,transform-package-source) (with-input . ,transform-package-inputs)- (with-graft . ,transform-package-inputs/graft)))+ (with-graft . ,transform-package-inputs/graft)+ (with-branch . ,transform-package-source-branch))) (define %transformation-options ;; The command-line interface to the above transformations.@@ -291,7 +336,9 @@ current 'gnutls' package, after which version 3.5.4 is grafted onto them." (option '("with-input") #t #f (parser 'with-input)) (option '("with-graft") #t #f- (parser 'with-graft)))))+ (parser 'with-graft))+ (option '("with-branch") #t #f+ (parser 'with-branch))))) (define (show-transformation-options-help) (display (G_ "@@ -302,7 +349,10 @@ current 'gnutls' package, after which version 3.5.4 is grafted onto them." replace dependency PACKAGE by REPLACEMENT")) (display (G_ " --with-graft=PACKAGE=REPLACEMENT- graft REPLACEMENT on packages that refer to PACKAGE")))+ graft REPLACEMENT on packages that refer to PACKAGE"))+ (display (G_ "+ --with-branch=PACKAGE=BRANCH+ build PACKAGE from the latest commit of BRANCH"))) (define (options->transformation opts)diff --git a/tests/guix-build-branch.sh b/tests/guix-build-branch.shnew file mode 100644index 0000000000..bc50d9c0ef--- /dev/null+++ b/tests/guix-build-branch.sh@@ -0,0 +1,48 @@+# GNU Guix --- Functional package management for GNU+# Copyright © 2018 Ludovic Courtès <ludo@gnu.org>+#+# This file is part of GNU Guix.+#+# GNU Guix is free software; you can redistribute it and/or modify it+# under the terms of the GNU General Public License as published by+# the Free Software Foundation; either version 3 of the License, or (at+# your option) any later version.+#+# GNU Guix is distributed in the hope that it will be useful, but+# WITHOUT ANY WARRANTY; without even the implied warranty of+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+# GNU General Public License for more details.+#+# You should have received a copy of the GNU General Public License+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++#+# Test 'guix build --with-branch'.+#++guix build --version++# 'guix build --with-branch' requires access to the network to clone the+# Git repository below.++if ! guile -c '(getaddrinfo "www.gnu.org" "80" AI_NUMERICSERV)' 2> /dev/null+then+ # Skipping.+ exit 77+fi++orig_drv="`guix build guile-gcrypt -d`"+latest_drv="`guix build guile-gcrypt --with-branch=guile-gcrypt=master -d`"+test -n "$latest_drv"+test "$orig_drv" != "$latest_drv"++# FIXME: '-S' currently doesn't work with non-derivation source.+# checkout="`guix build guile-gcrypt --with-branch=guile-gcrypt=master -S`"+checkout="`guix gc --references "$latest_drv" | grep guile-gcrypt | grep -v -E '(-builder|\.drv)'`"+test -d "$checkout"+test -f "$checkout/COPYING"++orig_drv="`guix build guix -d`"+latest_drv="`guix build guix --with-branch=guile-gcrypt=master -d`"+guix gc -R "$latest_drv" | grep guile-gcrypt-git.master+test "$orig_drv" != "$latest_drv"-- 2.19.2
L
L
Ludovic Courtès wrote on 30 Nov 2018 17:04
Re: [bug#33522] [PATCH 0/3] Add the '--with-branch' package transformation option
(address . 33522-done@debbugs.gnu.org)
871s72wnz3.fsf@gnu.org
Hi there!
Ludovic Courtès <ludo@gnu.org> skribis:
Toggle quote (5 lines)> Ludovic Courtès (3):> git: 'latest-repository-commit' logs its progress.> git: Add <git-checkout> record type.> guix build: Add '--with-branch' transformation option.
I pushed these 3 commits and added a 4th one that implements‘--with-commit’ along the same lines.
Enjoy!
Ludo’.
Closed
?
Your comment

This issue is archived.

To comment on this conversation send email to 33522@debbugs.gnu.org