services: cuirass: Add "simple-cuirass-services".

  • Done
  • quality assurance status badge
Details
3 participants
  • Jonathan Brielmaier
  • Ludovic Courtès
  • Mathieu Othacehe
Owner
unassigned
Submitted by
Mathieu Othacehe
Severity
normal
M
M
Mathieu Othacehe wrote on 22 Jan 2021 09:50
(address . guix-patches@gnu.org)
87czxxz767.fsf@gnu.org
Hello,

Here is a service that provides some syntactic sugar over the (complex)
Cuirass configuration. It uses Guix Channels to declare Cuirass inputs.

In the future, it would be nice if Cuirass could operate directly on
Channels. For now, this service only act as a translation layer for
people willing to setup a simple Cuirass instance.

Thanks,

Mathieu
J
J
Jonathan Brielmaier wrote on 22 Jan 2021 17:37
d1433f17-2e6b-d1c9-1048-dd747410e128@web.de
On 22.01.21 09:50, Mathieu Othacehe wrote:
Toggle quote (3 lines)
> Here is a service that provides some syntactic sugar over the (complex)
> Cuirass configuration. It uses Guix Channels to declare Cuirass inputs.

On my server I stopped cuirass && cuirass-web. Removed any signs from
the config.scm and tried this barebone example from your branch:

(use-modules (gnu)

(guix channels))
[...]
(simple-cuirass-services

(simple-cuirass-configuration

(build 'all)

(channels (cons (channel

(name 'my-guix)


%default-channels))))


M
M
Mathieu Othacehe wrote on 22 Jan 2021 18:55
(name . Jonathan Brielmaier)(address . jonathan.brielmaier@web.de)(address . 46031@debbugs.gnu.org)
87h7n8vosx.fsf@gnu.org
Hello Jonathan,

Toggle quote (2 lines)
Thanks for testing! Be careful, "simple-cuirass-services" returns a list
of services and has to be appended to the list of other services.

Could you paste a bit more of your configuration?

Thanks,

Mathieu
J
J
Jonathan Brielmaier wrote on 24 Jan 2021 14:47
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 46031@debbugs.gnu.org)
88959b06-fee9-e786-c105-d4766709281c@web.de
On 22.01.21 18:55, Mathieu Othacehe wrote:
Toggle quote (8 lines)
>
> Hello Jonathan,
>
>> It gave me this error: https://paste.opensuse.org/view/raw/110509
>
> Thanks for testing! Be careful, "simple-cuirass-services" returns a list
> of services and has to be appended to the list of other services.

That is the problem, because of changes to guix-daemon. I have following
construct:

(services
(cons*
(service a-service-type)
[...]
(service z-service-type)
(modify-services %base-services
(guix-service-type config =>
(guix-configuration
(inherit config)
(extra-options '("--cores=x")))))))
M
M
Mathieu Othacehe wrote on 29 Jan 2021 12:39
(name . Jonathan Brielmaier)(address . jonathan.brielmaier@web.de)(address . 46031@debbugs.gnu.org)
875z3gj7k1.fsf@gnu.org
Hello Jonathan,

Toggle quote (11 lines)
> (services
> (cons*
> (service a-service-type)
> [...]
> (service z-service-type)
> (modify-services %base-services
> (guix-service-type config =>
> (guix-configuration
> (inherit config)
> (extra-options '("--cores=x")))))))

That would impose to write something like:

Toggle snippet (20 lines)
(services
(append
(cons*
(service a-service-type)
[...]
(service z-service-type)
(modify-services %base-services
(guix-service-type config =>
(guix-configuration
(inherit config)
(extra-options '("--cores=x"))))))
(simple-cuirass-services
(simple-cuirass-configuration
(build 'all)
(channels (cons (channel
(name 'my-guix)
(url "https://my-git-repo/guix.git"))
%default-channels))))))

which is admittedly not that elegant. The problem I'd like to hide that
"postgresql-service-type" and "postgresql-role-service-type" are
required by Cuirass. Any idea?

Thanks,

Mathieu
J
J
Jonathan Brielmaier wrote on 2 Feb 2021 23:43
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 46031@debbugs.gnu.org)
9b90045c-8d02-34bc-1fcf-ff2061d87bb4@web.de
On 29.01.21 12:39, Mathieu Othacehe wrote:
Toggle quote (37 lines)
>
> Hello Jonathan,
>
>> (services
>> (cons*
>> (service a-service-type)
>> [...]
>> (service z-service-type)
>> (modify-services %base-services
>> (guix-service-type config =>
>> (guix-configuration
>> (inherit config)
>> (extra-options '("--cores=x")))))))
>
> That would impose to write something like:
>
> --8<---------------cut here---------------start------------->8---
> (services
> (append
> (cons*
> (service a-service-type)
> [...]
> (service z-service-type)
> (modify-services %base-services
> (guix-service-type config =>
> (guix-configuration
> (inherit config)
> (extra-options '("--cores=x"))))))
> (simple-cuirass-services
> (simple-cuirass-configuration
> (build 'all)
> (channels (cons (channel
> (name 'my-guix)
> (url "https://my-git-repo/guix.git"))
> %default-channels))))))
> --8<---------------cut here---------------end--------------->8---

Thanks Mathieu. I think we can merge and I'll try it then. If we found a
nicer solution we can still change it...
M
M
Mathieu Othacehe wrote on 19 Feb 2021 20:16
Re: bug#46031: services: cuirass: Add "simple-cuirass-services".
(name . Jonathan Brielmaier)(address . jonathan.brielmaier@web.de)(address . 46031-done@debbugs.gnu.org)
87zgzz7ts8.fsf_-_@gnu.org
Hello,

Toggle quote (3 lines)
> Thanks Mathieu. I think we can merge and I'll try it then. If we found a
> nicer solution we can still change it...

I added a system test and pushed, thanks for testing it.

Mathieu
Closed
J
J
Jonathan Brielmaier wrote on 19 Feb 2021 22:35
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 46031@debbugs.gnu.org)
3ecc3bc1-0ef6-8932-1c59-262053ef2722@web.de
On 19.02.21 20:16, Mathieu Othacehe wrote:
Toggle quote (8 lines)
>
> Hello,
>
>> Thanks Mathieu. I think we can merge and I'll try it then. If we found a
>> nicer solution we can still change it...
>
> I added a system test and pushed, thanks for testing it.

Merci Mathieu.

Is it intended that I need to import `(guix channels)`? Shouldn't that
come with the cuirass service module?
L
L
Ludovic Courtès wrote on 20 Feb 2021 11:59
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 46031@debbugs.gnu.org)
874ki7j97p.fsf@gnu.org
Hi,

Mathieu Othacehe <othacehe@gnu.org> skribis:

Toggle quote (7 lines)
> Here is a service that provides some syntactic sugar over the (complex)
> Cuirass configuration. It uses Guix Channels to declare Cuirass inputs.
>
> In the future, it would be nice if Cuirass could operate directly on
> Channels. For now, this service only act as a translation layer for
> people willing to setup a simple Cuirass instance.

I had overlooked these patches; that’s really nice!

Toggle quote (17 lines)
> +@lisp
> +(simple-cuirass-services
> + (simple-cuirass-configuration
> + (build (list
> + (build-manifest
> + (channel-name 'conf)
> + (manifest "guix/manifest.scm"))))
> + (channels (cons* (channel
> + (name 'my-guix)
> + (url "https://my-git-repo/guix.git"))
> + (channel
> + (name 'conf)
> + (url "https://my-git-repo/conf.git"))
> + %default-channels))
> + (non-package-channels '(conf))))
> +@end lisp

I wonder if it would make sense to allow users to pass directly a
manifest, as in (pseudo syntax):

(simple-cuirass-configuration
(build (list (build-manifest … (local-file "my-manifest.scm"))))
(channels …))

It’d be less expressive (you’d have to reconfigure when you change the
manifest), but perhaps easier to set up. WDYT?

Toggle quote (26 lines)
> + (define (config->spec config)
> + (match-record config <simple-cuirass-configuration>
> + (build channels non-package-channels systems)
> + `((#:name . "simple-config")
> + (#:load-path-inputs . ("guix"))
> + (#:package-path-inputs . ,(package-path channels
> + non-package-channels))
> + (#:proc-input . "guix")
> + (#:proc-file . "build-aux/cuirass/gnu-system.scm")
> + (#:proc . cuirass-jobs)
> + (#:proc-args . ((systems . ,systems)
> + ,@(if (eq? build 'all)
> + '()
> + `((subset . "manifests")
> + (manifests . ,(format-manifests build))))))
> + (#:inputs . ,(map channel->input channels))
> + (#:build-outputs . ())
> + (#:priority . 1))))
> +
> + (list
> + (service cuirass-service-type
> + (cuirass-configuration
> + (inherit cuirass)
> + (specifications #~(list
> + '#$(config->spec config)))))

What about exposing ‘simple-cuirass-configuration->specs’, and document
it such that one can do:

(service cuirass-service-type
(cuirass-configuration
(specifications
(simple-cuirass-configuration->specs config))))

or even:

(service cuirass-service-type
(compile-simple-cuirass-configuration config))

?

That way, the relationship between “simple” and “not simple” would be
clearer.

Thanks,
Ludo’.
M
M
Mathieu Othacehe wrote on 22 Feb 2021 09:50
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 46031@debbugs.gnu.org)
87k0r0iizg.fsf@gnu.org
Hey Ludo!

Toggle quote (2 lines)
> I had overlooked these patches; that’s really nice!

Thanks :)

Toggle quote (7 lines)
> I wonder if it would make sense to allow users to pass directly a
> manifest, as in (pseudo syntax):
>
> (simple-cuirass-configuration
> (build (list (build-manifest … (local-file "my-manifest.scm"))))
> (channels …))

It would be nice but Cuirass doesn't support specification update for
now. Passing a manifest this way would require to drop the database
content at each reconfiguration.

Toggle quote (8 lines)
> What about exposing ‘simple-cuirass-configuration->specs’, and document
> it such that one can do:
>
> (service cuirass-service-type
> (cuirass-configuration
> (specifications
> (simple-cuirass-configuration->specs config))))

I agree it feels nicer, however with this service I'd like to hide the
dependencies to postgresql-service-type and
postgresql-role-service-type. That's why "simple-cuirass-services"
returns three services. Maybe you see another way?

Thanks,

Mathieu
M
M
Mathieu Othacehe wrote on 22 Feb 2021 09:53
(name . Jonathan Brielmaier)(address . jonathan.brielmaier@web.de)(address . 46031@debbugs.gnu.org)
87ft1oiiud.fsf@gnu.org
Hello Jonathan,

Toggle quote (3 lines)
> Is it intended that I need to import `(guix channels)`? Shouldn't that
> come with the cuirass service module?

Yes you need it to declare the channels that are passed in the
"channels" field of "simple-cuirass-configuration". Maybe I should add a
comment about it in the documentation.

Thanks,

Mathieu
L
L
Ludovic Courtès wrote on 22 Feb 2021 13:59
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 46031@debbugs.gnu.org)
87eeh8b6mq.fsf@gnu.org
Hi!

Mathieu Othacehe <othacehe@gnu.org> skribis:

Toggle quote (11 lines)
>> I wonder if it would make sense to allow users to pass directly a
>> manifest, as in (pseudo syntax):
>>
>> (simple-cuirass-configuration
>> (build (list (build-manifest … (local-file "my-manifest.scm"))))
>> (channels …))
>
> It would be nice but Cuirass doesn't support specification update for
> now. Passing a manifest this way would require to drop the database
> content at each reconfiguration.

Ah yes, that wouldn’t be convenient.

Toggle quote (13 lines)
>> What about exposing ‘simple-cuirass-configuration->specs’, and document
>> it such that one can do:
>>
>> (service cuirass-service-type
>> (cuirass-configuration
>> (specifications
>> (simple-cuirass-configuration->specs config))))
>
> I agree it feels nicer, however with this service I'd like to hide the
> dependencies to postgresql-service-type and
> postgresql-role-service-type. That's why "simple-cuirass-services"
> returns three services. Maybe you see another way?

When a service extends a service type, an instance of that service type
is automatically added if it’s missing (provided that service type has a
default value). This happens in ‘instantiate-missing-services’.

So, if postgresql-role-service-type and postgresql-service-type have a
default value, simple-cuirass-service could extend them both (possibly
with a dummy value) and it would just work.

HTH!

Ludo’.
M
M
Mathieu Othacehe wrote on 23 Feb 2021 09:48
(name . Ludovic Courtès)(address . ludo@gnu.org)
87blcbkw52.fsf@gnu.org
Hey,

Toggle quote (4 lines)
> When a service extends a service type, an instance of that service type
> is automatically added if it’s missing (provided that service type has a
> default value). This happens in ‘instantiate-missing-services’.

Oh! Didn't know about that one!

Toggle quote (4 lines)
> So, if postgresql-role-service-type and postgresql-service-type have a
> default value, simple-cuirass-service could extend them both (possibly
> with a dummy value) and it would just work.

With 8163f74542300720f6ee5dc061b79ddf0c345bb8 and
108e2c6116f01c3b0a98498717d65a96c1857a51, I made sure that those
services are automatically instantiated when missing.

This means simple-cuirass-configuration->specs makes way more sense, as
you suggested. Took care of it with
bebcf97600b2fa65482ae8ee870800dafa34d3f8.

Now, as suggested by Jonathan on IRC yesterday, it would be nice to
be able to build only the packages of a given channel. For that, we
would need to be able to figure out which channel is providing a
package, with something like a package-channel field.

Do you think that would make sense?

Thanks,

Mathieu
L
L
Ludovic Courtès wrote on 23 Feb 2021 10:29
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87v9aj5dyl.fsf@gnu.org
Hello,

Mathieu Othacehe <othacehe@gnu.org> skribis:

Toggle quote (12 lines)
>> So, if postgresql-role-service-type and postgresql-service-type have a
>> default value, simple-cuirass-service could extend them both (possibly
>> with a dummy value) and it would just work.
>
> With 8163f74542300720f6ee5dc061b79ddf0c345bb8 and
> 108e2c6116f01c3b0a98498717d65a96c1857a51, I made sure that those
> services are automatically instantiated when missing.
>
> This means simple-cuirass-configuration->specs makes way more sense, as
> you suggested. Took care of it with
> bebcf97600b2fa65482ae8ee870800dafa34d3f8.

Yay!

Toggle quote (7 lines)
> Now, as suggested by Jonathan on IRC yesterday, it would be nice to
> be able to build only the packages of a given channel. For that, we
> would need to be able to figure out which channel is providing a
> package, with something like a package-channel field.
>
> Do you think that would make sense?

One could filter packages from the manifest.

Currently there’s no definite way to know which channel a given package
comes from. ‘package-provenance’ in (guix describe) approximates that.
Perhaps we should try something along these lines.

I guess the first step would be to provide an a ‘package-channel(s)’
procedure that does like ‘package-provenance’, but returns a list of
channels.

WDYT?

Ludo’.
M
M
Mathieu Othacehe wrote on 23 Feb 2021 14:30
(name . Ludovic Courtès)(address . ludo@gnu.org)
875z2ilxnl.fsf@gnu.org
Hey,

Toggle quote (4 lines)
> I guess the first step would be to provide an a ‘package-channel(s)’
> procedure that does like ‘package-provenance’, but returns a list of
> channels.

Seems fine to me. Here's an implementation attached. If it works for
you, the next step will be to make the "cuirass-jobs" procedure of
"gnu-system.scm" operate on channels I guess.

WDYT?

Thanks,

Mathieu
From d44dcd5d153ba0a4627c205f24a0741384f3d301 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe@gnu.org>
Date: Tue, 23 Feb 2021 14:24:39 +0100
Subject: [PATCH] describe: Add package-channels.

* guix/describe.scm (package-channels): New procedure.
---
guix/describe.scm | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)

Toggle diff (41 lines)
diff --git a/guix/describe.scm b/guix/describe.scm
index 03569b1db4..65e5772856 100644
--- a/guix/describe.scm
+++ b/guix/describe.scm
@@ -33,6 +33,7 @@
package-path-entries
package-provenance
+ package-channels
manifest-entry-with-provenance
manifest-entry-provenance))
@@ -178,6 +179,26 @@ property of manifest entries, or #f if it could not be determined."
`(,main
,@(if extra (list extra) '()))))))))))
+(define (package-channels package)
+ "Return the list of channels providing PACKAGE or an empty list if it could
+not be determined."
+ (match (and=> (package-location package) location-file)
+ (#f #f)
+ (file
+ (let ((file (if (string-prefix? "/" file)
+ file
+ (search-path %load-path file))))
+ (and file
+ (string-prefix? (%store-prefix) file)
+
+ (filter-map
+ (lambda (entry)
+ (let ((item (manifest-entry-item entry)))
+ (and (or (string-prefix? item file)
+ (string=? "guix" (manifest-entry-name entry)))
+ (manifest-entry-channel entry))))
+ (current-profile-entries)))))))
+
(define (manifest-entry-with-provenance entry)
"Return ENTRY with an additional 'provenance' property if it's not already
there."
--
2.30.1
L
L
Ludovic Courtès wrote on 23 Feb 2021 18:42
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
875z2i4r5c.fsf@gnu.org
Hi,

Mathieu Othacehe <othacehe@gnu.org> skribis:

Toggle quote (8 lines)
>> I guess the first step would be to provide an a ‘package-channel(s)’
>> procedure that does like ‘package-provenance’, but returns a list of
>> channels.
>
> Seems fine to me. Here's an implementation attached. If it works for
> you, the next step will be to make the "cuirass-jobs" procedure of
> "gnu-system.scm" operate on channels I guess.

But by definition, “gnu-system.scm” is about the 'guix channel.
Intuitively I’d expect channel handling to happen in user code: either
in the user-provided manifest, or in some helper code in Cuirass. WDYT?

Toggle quote (7 lines)
> From d44dcd5d153ba0a4627c205f24a0741384f3d301 Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <othacehe@gnu.org>
> Date: Tue, 23 Feb 2021 14:24:39 +0100
> Subject: [PATCH] describe: Add package-channels.
>
> * guix/describe.scm (package-channels): New procedure.

[...]

Toggle quote (6 lines)
> +(define (package-channels package)
> + "Return the list of channels providing PACKAGE or an empty list if it could
> +not be determined."
> + (match (and=> (package-location package) location-file)
> + (#f #f)

The #f return value doesn’t match the docstring.

Toggle quote (15 lines)
> + (file
> + (let ((file (if (string-prefix? "/" file)
> + file
> + (search-path %load-path file))))
> + (and file
> + (string-prefix? (%store-prefix) file)
> +
> + (filter-map
> + (lambda (entry)
> + (let ((item (manifest-entry-item entry)))
> + (and (or (string-prefix? item file)
> + (string=? "guix" (manifest-entry-name entry)))
> + (manifest-entry-channel entry))))
> + (current-profile-entries)))))))

To avoid duplication, perhaps you should rewrite ‘package-provenance’ in
terms of ‘package-channels’?

Otherwise LGTM, thanks!

Thanks,
Ludo’.
?