Cuirass: Allow to use authenticated Git repositories as inputs

  • Open
  • quality assurance status badge
Details
4 participants
  • Clément Lassieur
  • Erik Edrosa
  • Ludovic Courtès
  • Mathieu Othacehe
Owner
unassigned
Submitted by
Clément Lassieur
Severity
wishlist
C
C
Clément Lassieur wrote on 22 Nov 2019 10:51
(address . bug-guix@gnu.org)
875zjc8ciz.fsf@lassieur.org
Hi,

I'd like to use private Git repositories as inputs, authenticated with
SSH.

I think it is important because it would encourage users to use Cuirass
for other purposes than just a CI tool to build Guix packages. I'd like
to see it as an alternative to Jenkins, it would make its user base
broader.

My use of Cuirass, for example, is to do continuous integration on my
own personal projects. So each of my inputs is a personnal private
repository, and each time one of those inputs has a new commit, the
'proc' will give Cuirass a derivation of the Guix package corresponding
to the updated input, with a modified 'origin' so that it matches the
new input.

My workaround for authentication is to use local repositories, updated
with an external cron task.

I think there are small modifications to do to (guix git) and (git
clone).

Cheers,
Clément
C
C
Clément Lassieur wrote on 22 Nov 2019 10:52
control message for bug #38320
(address . control@debbugs.gnu.org)
874kyw8ch5.fsf@lassieur.org
severity 38320 wishlist
quit
M
M
Mathieu Othacehe wrote on 25 Nov 2019 14:42
Re: bug#38320: Cuirass: Allow to use authenticated Git repositories as inputs
(address . bug-guix@gnu.org)(address . 38320@debbugs.gnu.org)
878so4t6mk.fsf@gmail.com
Hello Clément,

Toggle quote (3 lines)
> I think there are small modifications to do to (guix git) and (git
> clone).

I did integrate a part of libgit2 ssh authentification mechanism in
Guile-Git in 2017. You can find it in (git fetch) module.

It is currently broken, because of a regression. See

What would be missing to have support for authenticated Git repositories
as Cuirass inputs is:

* Fix the regression mentionned above.

* Add support for a fetch-options argument in clone method of (git clone).

* In (guix git), "latest-repository-commit" method would take parameters
to setup ssh authentication (such as ssh private key path at least) and
pass them to "fetch" and "clone" methods of Guile-Git.

* Finally in Cuirass, the ssh authentication parameters could be
specified in the specification file (maybe for each input?) and passed
to "latest-repository-commit" method accordingly.

So there's still some work to do :)

Mathieu
M
M
Mathieu Othacehe wrote on 25 Nov 2019 14:46
(address . bug-guix@gnu.org)(address . 38320@debbugs.gnu.org)
877e3ot6f0.fsf@gmail.com
Toggle quote (2 lines)
> * Fix the regression mentionned above.

I would need some help for this regression I don't understand, but I
will take care of the work needed in Guile-Git and (guix git) once this
is fixed.

I think too that extending Cuirass to support new use-cases would be
really great :).

Thanks,

Mathieu
L
L
Ludovic Courtès wrote on 26 Nov 2019 11:05
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)(address . 38320@debbugs.gnu.org)
87r21v9cmi.fsf@gnu.org
Hi Mathieu,

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (9 lines)
>> I think there are small modifications to do to (guix git) and (git
>> clone).
>
> I did integrate a part of libgit2 ssh authentification mechanism in
> Guile-Git in 2017. You can find it in (git fetch) module.
>
> It is currently broken, because of a regression. See
> https://lists.gnu.org/archive/html/guix-devel/2019-11/msg00415.html.

Oh I missed that message of yours. Do you have a complete example using
that functionality that I could use as a test?

It would be great to have a test for that in Guile-Git. We could use
Guile-SSH, when it’s available, to spawn an SSH server.

Toggle quote (15 lines)
> What would be missing to have support for authenticated Git repositories
> as Cuirass inputs is:
>
> * Fix the regression mentionned above.
>
> * Add support for a fetch-options argument in clone method of (git clone).
>
> * In (guix git), "latest-repository-commit" method would take parameters
> to setup ssh authentication (such as ssh private key path at least) and
> pass them to "fetch" and "clone" methods of Guile-Git.
>
> * Finally in Cuirass, the ssh authentication parameters could be
> specified in the specification file (maybe for each input?) and passed
> to "latest-repository-commit" method accordingly.

I’d like to see that happen!

Thanks,
Ludo’.
C
C
Clément Lassieur wrote on 29 Nov 2019 00:46
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)(address . 38320@debbugs.gnu.org)
87eexrtvhf.fsf@lassieur.org
Mathieu Othacehe <m.othacehe@gmail.com> writes:

Toggle quote (6 lines)
>> * Fix the regression mentionned above.
>
> I would need some help for this regression I don't understand, but I
> will take care of the work needed in Guile-Git and (guix git) once this
> is fixed.

Thank you Mathieu for your replies! I'm looking forward to your work :)

Toggle quote (3 lines)
> I think too that extending Cuirass to support new use-cases would be
> really great :).

And some of the new use-cases are low-hanging fruits actually.
M
M
Mathieu Othacehe wrote on 9 Dec 2019 17:41
87h829sb73.fsf@gmail.com
Hello,

Here's a patch that add support for ssh authenticated repositories in
"clone" and "remote-fetch" methods of Guile-Git.

At first, I used Guile-SSH in the tests to start an SSH server, but as
"make-server" call of Guile-SSH is really low level, this is not very
realistic. I just ended up with a half-broken ssh server, poorly
implemented, after (too many hours) spent reading ssh dumps.

So the strategy is to spawn an openssh server for the tests. It seems to
work alright, using key based or ssh-agent authentication.

WDYT?

Mathieu
Attachment: 0001-Add-ssh-authentication-support.patch
Ludovic Courtès writes:

Toggle quote (38 lines)
> Hi Mathieu,
>
> Mathieu Othacehe <m.othacehe@gmail.com> skribis:
>
>>> I think there are small modifications to do to (guix git) and (git
>>> clone).
>>
>> I did integrate a part of libgit2 ssh authentification mechanism in
>> Guile-Git in 2017. You can find it in (git fetch) module.
>>
>> It is currently broken, because of a regression. See
>> https://lists.gnu.org/archive/html/guix-devel/2019-11/msg00415.html.
>
> Oh I missed that message of yours. Do you have a complete example using
> that functionality that I could use as a test?
>
> It would be great to have a test for that in Guile-Git. We could use
> Guile-SSH, when it’s available, to spawn an SSH server.
>
>> What would be missing to have support for authenticated Git repositories
>> as Cuirass inputs is:
>>
>> * Fix the regression mentionned above.
>>
>> * Add support for a fetch-options argument in clone method of (git clone).
>>
>> * In (guix git), "latest-repository-commit" method would take parameters
>> to setup ssh authentication (such as ssh private key path at least) and
>> pass them to "fetch" and "clone" methods of Guile-Git.
>>
>> * Finally in Cuirass, the ssh authentication parameters could be
>> specified in the specification file (maybe for each input?) and passed
>> to "latest-repository-commit" method accordingly.
>
> I’d like to see that happen!
>
> Thanks,
> Ludo’.
L
L
Ludovic Courtès wrote on 10 Dec 2019 15:28
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
877e34z24m.fsf@gnu.org
Hi!

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (3 lines)
> Here's a patch that add support for ssh authenticated repositories in
> "clone" and "remote-fetch" methods of Guile-Git.

Woow, awesome!

Toggle quote (5 lines)
> At first, I used Guile-SSH in the tests to start an SSH server, but as
> "make-server" call of Guile-SSH is really low level, this is not very
> realistic. I just ended up with a half-broken ssh server, poorly
> implemented, after (too many hours) spent reading ssh dumps.

Oh, I thought it’d be easier to scrap bits from the example SSH server
that’s in Guile-SSH, perhaps a wishlist item for them.

Toggle quote (3 lines)
> So the strategy is to spawn an openssh server for the tests. It seems to
> work alright, using key based or ssh-agent authentication.

Anyway, if it works with sshd, that’s great.

Toggle quote (29 lines)
> From ae3c5a9851b02e78096963616d4e2f999119fc4d Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <m.othacehe@gmail.com>
> Date: Mon, 9 Dec 2019 16:16:45 +0100
> Subject: [PATCH] Add ssh authentication support.
>
> * Makefile.am (SOURCES): Add git/auth.scm,
> (TESTS): add tests/clone.scm.
> * configure.ac: Check for git and ssh binaries.
> * git.scm (%public-modules): Add (git auth) and (git bindings).
> * git/auth.scm: New file.
> * git/clone.scm (clone): Add an auth-method argument. Pass it to
> new init-fetch-options call, before proceeding to clone.
> * git/remote.scm (remote-fetch): Add an auth-method. Pass it to
> init-fetch-options before proceeding to fetch.
> * git/structs.scm (clone-options-fetch-options): Do not return a copy of
> fetch-options nested inside clone-options. Instead, find the offset of
> fetch-options and use it to create a pointer to fetch-options.
> * git/fetch.scm (init-fetch-options): New exported procedure,
> (make-fetch-options): call the procedure above to initialize fetch-options,
> (set-fetch-auth-with-ssh-agent!): handle the case where username is not set
> and libgit2 asks for one.
> (set-fetch-auth-with-default-ssh-key!): remove this procedure,
> (set-fetch-auth-with-ssh-key): new procedure.
> * tests/.ssh/id_rsa_client: New file.
> * tests/.ssh/id_rsa_client.pub: New file.
> * tests/.ssh/id_rsa_server: New file.
> * tests/clone.scm: New file.
> * tests/ssh.scm.in: New file.

[...]

Toggle quote (13 lines)
> (define-module (git fetch)
> #:use-module (system foreign)
> + #:use-module (git auth)
> #:use-module (git bindings)
> #:use-module (git cred)
> #:use-module (git structs)
> #:use-module (git types)
> #:use-module (srfi srfi-26)
>
> - #:export (make-fetch-options
> + #:export (init-fetch-options
> + make-fetch-options

I think we should keep ‘init-fetch-options’ private.

Toggle quote (2 lines)
> fetch-init-options ;deprecated!

‘init-fetch-options’, ‘fetch-init-options’, hmm… o_O

Toggle quote (5 lines)
> new file mode 100644
> index 0000000..7e16000
> --- /dev/null
> +++ b/tests/.ssh/id_rsa_client

I wonder if we should generate those upon ‘make check’. Thoughts?
(It shouldn’t be a blocker though.)

Toggle quote (22 lines)
> +(with-sshd-server ssh-server-port
> + (with-repository "simple-bare" directory
> + (test-equal "clone-auth-ssh-credentials"
> + "3f848a1a52416ac99a5c5bf2e6bd55eb7b99d55b"
> + (clone-test directory (make-client-ssh-auth))))
> +
> + (with-repository "simple-bare" directory
> + (test-equal "clone-auth-ssh-agent"
> + "3f848a1a52416ac99a5c5bf2e6bd55eb7b99d55b"
> + (with-ssh-agent
> + (clone-test directory (%make-auth-ssh-agent)))))
> +
> + (with-repository "simple-bare" directory
> + (test-assert "clone-and-fetch-auth-ssh-credentials"
> + (let* ((auth (make-client-ssh-auth))
> + (do-clone (clone-test directory auth))
> + (clone-dir (in-vicinity directory "out"))
> + (repository (repository-open clone-dir))
> + (remote (remote-lookup repository "origin")))
> + (remote-fetch remote #:auth-method auth)
> + #t))))

I think we should add something like:

(define (sshd-available?)
;; Return #t if sshd is available (it does not support
;; ‘--version’ or anything similar though).
(not (= 127 (system* sshd "--something-not-supported"))))

(unless (sshd-available?) (test-skip 1))
;; …

Apart from this detail, it looks great to me!

You have push access, right?

Speaking of which, we really need to push a release at some point.
Erik, would you be available to do that, or would you like to delegate?

Thanks,
Ludo’.
E
E
Erik Edrosa wrote on 11 Dec 2019 01:28
e473fddd-3a62-3b50-e9fc-4c3a8492e06d@gmail.com
Toggle quote (8 lines)
>
> Speaking of which, we really need to push a release at some point.
> Erik, would you be available to do that, or would you like to delegate?
>
> Thanks,
> Ludo’.
>

Sure, I was planning on creating a release soon. So once this is in I
can start testing on some different distros to make sure there isn't any
issues.
M
M
Mathieu Othacehe wrote on 11 Dec 2019 12:53
(name . Ludovic Courtès)(address . ludo@gnu.org)
87wob3xepy.fsf@gmail.com
Hey!

Toggle quote (7 lines)
> Apart from this detail, it looks great to me!
>
> You have push access, right?
>
> Speaking of which, we really need to push a release at some point.
> Erik, would you be available to do that, or would you like to delegate?

Great, thanks for reviewing :). I couldn't get it to work with a generated
client rsa key for an unknown reason, but pushed anyway.

Now regarding (guix git) integration, I have a question. It would be nice
to have "guix pull" and Cuirass support ssh authenticated
directories.

So "latest-repository-commit" could be call with ssh authentication
parameters. However, the guix-daemon won't be able to communicate with the
user ssh-agent, and storing an unencrypted private ssh key in the store
doesn't feel great to me.

Do you see any workaround?

Thanks,

Mathieu
C
C
Clément Lassieur wrote on 11 Dec 2019 16:36
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
87zhfyvppi.fsf@lassieur.org
Hi everyone,

Whoo, nice, thank you so much Mathieu! I'll test everything this
week-end probably, and start working on the (guix git) / Cuirass
counterpart (which is 1% of the work :D).

Mathieu Othacehe <m.othacehe@gmail.com> writes:

Toggle quote (4 lines)
> Now regarding (guix git) integration, I have a question. It would be nice
> to have "guix pull" and Cuirass support ssh authenticated
> directories.

Indeed :) Almost there!

Toggle quote (7 lines)
> So "latest-repository-commit" could be call with ssh authentication
> parameters. However, the guix-daemon won't be able to communicate with the
> user ssh-agent, and storing an unencrypted private ssh key in the store
> doesn't feel great to me.
>
> Do you see any workaround?

As far as I understand, LATEST-REPOSITORY-COMMIT is never called by the
daemon, it downloads stuff first and then calls ADD-TO-STORE. So both
using the SSH agent or passing a private SSH key should be
straightforward.

Clément
L
L
Ludovic Courtès wrote on 12 Dec 2019 14:13
(name . Erik Edrosa)(address . erik.edrosa@gmail.com)
87a77x8z54.fsf@gnu.org
Hi Erik,

Erik Edrosa <erik.edrosa@gmail.com> skribis:

Toggle quote (12 lines)
>>
>> Speaking of which, we really need to push a release at some point.
>> Erik, would you be available to do that, or would you like to delegate?
>>
>> Thanks,
>> Ludo’.
>>
>
> Sure, I was planning on creating a release soon. So once this is in I
> can start testing on some different distros to make sure there isn't any
> issues.

Awesome, sounds great!

I was interested in getting support for progress reports (for ‘clone’ in
particular), but I don’t think I’ll work on it soon enough, and that can
surely wait until the next release.

Thank you!

Ludo’.
L
L
Ludovic Courtès wrote on 12 Dec 2019 14:15
(name . Clément Lassieur)(address . clement@lassieur.org)
8736dp8z2p.fsf@gnu.org
Hey ho!

Clément Lassieur <clement@lassieur.org> skribis:

Toggle quote (4 lines)
> Whoo, nice, thank you so much Mathieu! I'll test everything this
> week-end probably, and start working on the (guix git) / Cuirass
> counterpart (which is 1% of the work :D).

Neat!

Toggle quote (12 lines)
>> So "latest-repository-commit" could be call with ssh authentication
>> parameters. However, the guix-daemon won't be able to communicate with the
>> user ssh-agent, and storing an unencrypted private ssh key in the store
>> doesn't feel great to me.
>>
>> Do you see any workaround?
>
> As far as I understand, LATEST-REPOSITORY-COMMIT is never called by the
> daemon, it downloads stuff first and then calls ADD-TO-STORE. So both
> using the SSH agent or passing a private SSH key should be
> straightforward.

Indeed. ‘guix pull --url’ and ‘guix build --with-git-url’ (and similar)
should work just fine.

Thanks!

Ludo’.
M
M
Mathieu Othacehe wrote on 4 Feb 2020 10:16
(name . Ludovic Courtès)(address . ludo@gnu.org)
87tv4667b4.fsf@gmail.com
Hello,

Here's a small patch to (guix git) so that cloning/fetching from ssh
authenticated repositories is supported using ssh agent.

I tested:

* guix pull --url=git@gitlab.com:mothacehe/private.git
* guix pull with the following channel configuration

Toggle snippet (6 lines)
(cons* (channel
(name 'gitlab)
(url "git@gitlab.com:mothacehe/test-channel.git"))
%default-channels)

This works fine, but we still need to see how it works for Cuirass
inputs and (guix git-download) module.

Mathieu
From ae380c15f1c37e2c94e0954975f5f712e76340ac Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <m.othacehe@gmail.com>
Date: Mon, 3 Feb 2020 18:05:02 +0100
Subject: [PATCH] git: Add ssh authentication support.

SSH agent authentication method is used.

* guix/git.scm (auth-method): New variable,
(clone*): pass previous variable in clone options,
(update-cached-checkout): pass previous variable in fetch options.
---
guix/git.scm | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)

Toggle diff (44 lines)
diff --git a/guix/git.scm b/guix/git.scm
index a12f1eec8e..aee7b325e0 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
+;;; Copyright © 2017, 2020 Mathieu Othacehe <m.othacehe@gmail.com>
;;; Copyright © 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
;;;
;;; This file is part of GNU Guix.
@@ -108,6 +108,9 @@ the 'SSL_CERT_FILE' and 'SSL_CERT_DIR' environment variables."
(string-append "R:" url)
url))))))
+;; Default authentication method.
+(define auth-method (%make-auth-ssh-agent))
+
(define (clone* url directory)
"Clone git repository at URL into DIRECTORY. Upon failure,
make sure no empty directory is left behind."
@@ -119,7 +122,9 @@ make sure no empty directory is left behind."
;; value in Guile-Git: <https://bugs.gnu.org/29238>.
(if (module-defined? (resolve-interface '(git))
'clone-init-options)
- (clone url directory (clone-init-options))
+ (clone url directory
+ (make-clone-options
+ #:fetch-options (make-fetch-options auth-method)))
(clone url directory)))
(lambda _
(false-if-exception (rmdir directory)))))
@@ -281,7 +286,8 @@ When RECURSIVE? is true, check out submodules as well, if any."
;; Only fetch remote if it has not been cloned just before.
(when (and cache-exists?
(not (reference-available? repository ref)))
- (remote-fetch (remote-lookup repository "origin")))
+ (remote-fetch (remote-lookup repository "origin")
+ #:fetch-options (make-fetch-options auth-method)))
(when recursive?
(update-submodules repository #:log-port log-port))
(let ((oid (switch-to-ref repository canonical-ref)))
--
2.25.0
Ludovic Courtès writes:

Toggle quote (28 lines)
> Hey ho!
>
> Clément Lassieur <clement@lassieur.org> skribis:
>
>> Whoo, nice, thank you so much Mathieu! I'll test everything this
>> week-end probably, and start working on the (guix git) / Cuirass
>> counterpart (which is 1% of the work :D).
>
> Neat!
>
>>> So "latest-repository-commit" could be call with ssh authentication
>>> parameters. However, the guix-daemon won't be able to communicate with the
>>> user ssh-agent, and storing an unencrypted private ssh key in the store
>>> doesn't feel great to me.
>>>
>>> Do you see any workaround?
>>
>> As far as I understand, LATEST-REPOSITORY-COMMIT is never called by the
>> daemon, it downloads stuff first and then calls ADD-TO-STORE. So both
>> using the SSH agent or passing a private SSH key should be
>> straightforward.
>
> Indeed. ‘guix pull --url’ and ‘guix build --with-git-url’ (and similar)
> should work just fine.
>
> Thanks!
>
> Ludo’.
L
L
Ludovic Courtès wrote on 4 Feb 2020 13:58
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
87v9omy0es.fsf@gnu.org
Hello!

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (13 lines)
> Here's a small patch to (guix git) so that cloning/fetching from ssh
> authenticated repositories is supported using ssh agent.
>
> I tested:
>
> * guix pull --url=git@gitlab.com:mothacehe/private.git
> * guix pull with the following channel configuration
>
> (cons* (channel
> (name 'gitlab)
> (url "git@gitlab.com:mothacehe/test-channel.git"))
> %default-channels)

Woohoo! Really nice.

Toggle quote (3 lines)
> This works fine, but we still need to see how it works for Cuirass
> inputs and (guix git-download) module.

Ah yes, ‘git-fetch’.

Toggle quote (11 lines)
> From ae380c15f1c37e2c94e0954975f5f712e76340ac Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <m.othacehe@gmail.com>
> Date: Mon, 3 Feb 2020 18:05:02 +0100
> Subject: [PATCH] git: Add ssh authentication support.
>
> SSH agent authentication method is used.
>
> * guix/git.scm (auth-method): New variable,
> (clone*): pass previous variable in clone options,
> (update-cached-checkout): pass previous variable in fetch options.

[...]

Toggle quote (25 lines)
> +;; Default authentication method.
> +(define auth-method (%make-auth-ssh-agent))
> +
> (define (clone* url directory)
> "Clone git repository at URL into DIRECTORY. Upon failure,
> make sure no empty directory is left behind."
> @@ -119,7 +122,9 @@ make sure no empty directory is left behind."
> ;; value in Guile-Git: <https://bugs.gnu.org/29238>.
> (if (module-defined? (resolve-interface '(git))
> 'clone-init-options)
> - (clone url directory (clone-init-options))
> + (clone url directory
> + (make-clone-options
> + #:fetch-options (make-fetch-options auth-method)))
> (clone url directory)))
> (lambda _
> (false-if-exception (rmdir directory)))))
> @@ -281,7 +286,8 @@ When RECURSIVE? is true, check out submodules as well, if any."
> ;; Only fetch remote if it has not been cloned just before.
> (when (and cache-exists?
> (not (reference-available? repository ref)))
> - (remote-fetch (remote-lookup repository "origin")))
> + (remote-fetch (remote-lookup repository "origin")
> + #:fetch-options (make-fetch-options auth-method)))

It LGTM, and I like that it’s actually a small patch.

Until now, we had conditionals like the ‘module-defined?’ thing above to
allow for a smooth transition from older Guile-Git versions. Do we want
to keep doing that?

If we do, then perhaps you should arrange so that uses of the new
Guile-Git APIs that appeared in 0.3.0 are conditional.

I’d say we should do it if it’s easy to do and not too intrusive.
Otherwise, let’s just require 0.3.0 and be done with it. (‘guix pull’
gets 0.3.0 anyway.)

Thoughts?

Ludo’.
M
M
Mathieu Othacehe wrote on 5 Feb 2020 09:45
(name . Ludovic Courtès)(address . ludo@gnu.org)
87v9oltobu.fsf@gmail.com
Hey Ludo,

Thanks for reviewing :)

Toggle quote (13 lines)
> It LGTM, and I like that it’s actually a small patch.
>
> Until now, we had conditionals like the ‘module-defined?’ thing above to
> allow for a smooth transition from older Guile-Git versions. Do we want
> to keep doing that?
>
> If we do, then perhaps you should arrange so that uses of the new
> Guile-Git APIs that appeared in 0.3.0 are conditional.
>
> I’d say we should do it if it’s easy to do and not too intrusive.
> Otherwise, let’s just require 0.3.0 and be done with it. (‘guix pull’
> gets 0.3.0 anyway.)

Here's a version with optional authentication support. I do not find it
too intrusive, but let me know what you think!

Thanks,

Mathieu
From 4554baf59564eb0c31cfe235acd078d54afef6d7 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <m.othacehe@gmail.com>
Date: Mon, 3 Feb 2020 18:05:02 +0100
Subject: [PATCH] git: Add ssh authentication support.

SSH agent authentication method is used.

* guix/git.scm (auth-method): New variable,
(clone*): pass previous variable in clone options,
(update-cached-checkout): pass previous variable in fetch options.
---
guix/git.scm | 21 ++++++++++++++++++---
1 file changed, 18 insertions(+), 3 deletions(-)

Toggle diff (53 lines)
diff --git a/guix/git.scm b/guix/git.scm
index a12f1eec8e..2165e612f1 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
+;;; Copyright © 2017, 2020 Mathieu Othacehe <m.othacehe@gmail.com>
;;; Copyright © 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
;;;
;;; This file is part of GNU Guix.
@@ -108,6 +108,14 @@ the 'SSL_CERT_FILE' and 'SSL_CERT_DIR' environment variables."
(string-append "R:" url)
url))))))
+;; Authentication appeared in Guile-Git 0.3.0, check if it is available.
+(define auth-supported?
+ (false-if-exception (resolve-interface '(git auth))))
+
+;; Default authentication method.
+(define auth-method (and auth-supported?
+ (%make-auth-ssh-agent)))
+
(define (clone* url directory)
"Clone git repository at URL into DIRECTORY. Upon failure,
make sure no empty directory is left behind."
@@ -119,7 +127,11 @@ make sure no empty directory is left behind."
;; value in Guile-Git: <https://bugs.gnu.org/29238>.
(if (module-defined? (resolve-interface '(git))
'clone-init-options)
- (clone url directory (clone-init-options))
+ (clone url directory
+ (if auth-supported?
+ (make-clone-options
+ #:fetch-options (make-fetch-options auth-method))
+ (clone-init-options)))
(clone url directory)))
(lambda _
(false-if-exception (rmdir directory)))))
@@ -281,7 +293,10 @@ When RECURSIVE? is true, check out submodules as well, if any."
;; Only fetch remote if it has not been cloned just before.
(when (and cache-exists?
(not (reference-available? repository ref)))
- (remote-fetch (remote-lookup repository "origin")))
+ (if auth-supported?
+ (remote-fetch (remote-lookup repository "origin")
+ #:fetch-options (make-fetch-options auth-method))
+ (remote-fetch (remote-lookup repository "origin"))))
(when recursive?
(update-submodules repository #:log-port log-port))
(let ((oid (switch-to-ref repository canonical-ref)))
--
2.25.0
L
L
Ludovic Courtès wrote on 5 Feb 2020 22:24
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
87mu9wlobp.fsf@gnu.org
Hi!

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (3 lines)
> Here's a version with optional authentication support. I do not find it
> too intrusive, but let me know what you think!

Yup, looks good!

Toggle quote (11 lines)
> From 4554baf59564eb0c31cfe235acd078d54afef6d7 Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <m.othacehe@gmail.com>
> Date: Mon, 3 Feb 2020 18:05:02 +0100
> Subject: [PATCH] git: Add ssh authentication support.
>
> SSH agent authentication method is used.
>
> * guix/git.scm (auth-method): New variable,
> (clone*): pass previous variable in clone options,
> (update-cached-checkout): pass previous variable in fetch options.

[...]

Toggle quote (4 lines)
> +;; Default authentication method.
> +(define auth-method (and auth-supported?
> + (%make-auth-ssh-agent)))

Perhaps we should not call it at the top level, in case it throws an
exception or has unwanted side effects? I’d suggest moving it to
the procedures where it’s used. Thoughts?

Apart from that it LGTM, thanks!

Ludo’.
M
M
Mathieu Othacehe wrote on 6 Feb 2020 16:16
(name . Ludovic Courtès)(address . ludo@gnu.org)
87tv43zqxn.fsf@gmail.com
Hey,

Toggle quote (6 lines)
> Perhaps we should not call it at the top level, in case it throws an
> exception or has unwanted side effects? I’d suggest moving it to
> the procedures where it’s used. Thoughts?
>
> Apart from that it LGTM, thanks!

Fixed and pushed! Shall we inform users (guix pull news?) that they can
now use ssh authenticated repositories for guix pull + channels if they
have a running ssh-agent?

Mathieu
L
L
Ludovic Courtès wrote on 6 Feb 2020 18:17
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
87a75v7i0c.fsf@gnu.org
Hi!

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (10 lines)
>> Perhaps we should not call it at the top level, in case it throws an
>> exception or has unwanted side effects? I’d suggest moving it to
>> the procedures where it’s used. Thoughts?
>>
>> Apart from that it LGTM, thanks!
>
> Fixed and pushed! Shall we inform users (guix pull news?) that they can
> now use ssh authenticated repositories for guix pull + channels if they
> have a running ssh-agent?

Yes, good idea, could you add an entry in etc/news.txt? That’d be great.

(Double-check you got the commit ID and syntax right before pushing, it’s
easy to make mistakes there.)

Thank you!

Ludo’.
?