[PATCH] doc: Add 'Working on Shepherd' section.

OpenSubmitted by Attila Lendvai.
Details
3 participants
  • Attila Lendvai
  • Maxime Devos
  • pelzflorian (Florian Pelz)
Owner
unassigned
Severity
normal
A
A
Attila Lendvai wrote on 28 Feb 19:51 +0100
(address . guix-patches@gnu.org)(name . Attila Lendvai)(address . attila@lendvai.name)
20220228185115.28042-1-attila@lendvai.name
---
doc/contributing.texi | 91 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 91 insertions(+)

Toggle diff (111 lines)
diff --git a/doc/contributing.texi b/doc/contributing.texi
index 207efc4ee6..d36b6e66e0 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -29,6 +29,7 @@ choice.
 * Tracking Bugs and Patches::   Keeping it all organized.
 * Commit Access::               Pushing to the official repository.
 * Updating the Guix Package::   Updating the Guix package definition.
+* Working on Shepherd::         Modifying and testing Shepherd.
 * Translating Guix::            Make Guix speak your native language.
 @end menu
 
@@ -1697,6 +1698,96 @@ This check can be disabled, @emph{at your own peril}, by setting the
 this variable is set, the updated package source is also added to the
 store.  This is used as part of the release process of Guix.
 
+@node Working on Shepherd
+@section Working on Shepherd
+
+This chapter documents how to modify and test GNU@tie{}Shepherd
+(@pxref{Shepherd Services}) in the Guix environment.
+
+There are two @emph{manifestations} of Shepherd in a Guix System:
+
+@table @code
+
+@item @emph{The Shepherd process}
+The init process first started by the kernel (i.e. running as PID 1).
+This is a Guile executable that is executing the @code{main} function of
+the Shepherd codebase.  Among other things, this is the process
+responsible for starting and stopping Guix System services (i.e. daemon
+processes).
+
+@item @emph{The Shepherd API}
+The Scheme code of Shepherd, which is a dependency of certain packages
+and the Guix codebase itself.  A typical example of this is the Scheme
+code implementing a Guix System service, e.g. the OpenSSH server service
+(see @code{openssh-shepherd-service}).
+
+@end table
+
+Modifying the latter results in the recompilation of several dependant
+packages, and it takes too long to be a reasonable edit-compile-test
+cycle.  But starting up a VM that merely uses a customized Shepherd init
+process is a relatively quick operation.
+
+Luckily, not all changes to Shepherd require the recompilation of all
+its dependencies.  The rule of thumb here is that:
+
+@itemize
+
+@item
+if you are making changes to the public API of Shepherd (i.e. anything
+that may have compile-time effects on dependant packages, like adding or
+removing public functions, or changing public macros, etc.), then you
+will need to go through a full recompilation, so that the the Guix
+codebase, and the dependant packages can observe the changes while they
+are being compiled.
+
+@item
+if you're only working on Shepherd's implementation (e.g. making
+Shepherd's error handling more bullet proof), then it's enough to only
+recompile Shepherd itself, and use the resulting package as the one that
+gets started as the init process.
+
+@end itemize
+
+The @ref{Shepherd Services, @code{shepherd-configuration}} section
+documents how you can replace the Shepherd process by specifying a
+custom Shepherd package for an @code{operating-system} object.  To get a
+customized Shepherd package, you can simply make a copy of it in
+@file{gnu/packages/admin.scm}, and change the @code{source} and
+@code{version} field along these lines:
+
+@lisp
+(define-public shepherd-dev-pid-1
+  (package
+    (name "shepherd")
+    (version "dev-pid-1")
+    (source (git-checkout
+             (url "file:///my/path/shepherd/")))
+    ...
+    ))
+@end lisp
+
+To modify and use a new Shepherd API, you can change the @code{source}
+and @code{version} field of the @code{shepherd} package in
+@file{gnu/packages/admin.scm} along these lines:
+
+@lisp
+(define-public shepherd     ; do not change this
+  (package
+    (name "shepherd")
+    (version "dev")
+    (source (git-checkout
+             (url "file:///my/path/shepherd/")
+             (commit "[a commit hash]")))
+    ...
+    ))
+@end lisp
+
+To avoid excessive recompilation times, we pick a specific commit in the
+latter, and only update it as needed.  But the former will pick up any
+newly recorded commit when we issue a @command{guix system vm
+/path/to/my-test.scm}.
+
 @cindex translation
 @cindex l10n
 @cindex i18n
-- 
2.34.0
M
M
Maxime Devos wrote on 28 Feb 20:31 +0100
73e2b7cba0065c3b6b16bb14438dd3a8ea218808.camel@telenet.be
Attila Lendvai schreef op ma 28-02-2022 om 19:51 [+0100]:
Toggle quote (3 lines)
> +Luckily, not all changes to Shepherd require the recompilation of all
> +its dependencies.  The rule of thumb here is that:

‘not all changes require’ -> ‘most changes do not require’?

Toggle quote (10 lines)
> +@itemize
> +
> +@item
> +if you are making changes to the public API of Shepherd (i.e. anything
> +that may have compile-time effects on dependant packages, like adding or
> +removing public functions, or changing public macros, etc.), then you
> +will need to go through a full recompilation, so that the the Guix
> +codebase, and the dependant packages can observe the changes while they
> +are being compiled.

I don't think this is necessary? If I change the API a little, then
why would packages only using 'sbin/halt', 'sbin/reboot' and
'sbin/shutdown' of the 'shepherd' package have to be recompiled?

Even if the API is changed, the package still uses the old shepherd
package with the old API, so no recompilation necessary.

Also, even if the API is changed, then 'guix system reconfigure ...'
would pick up the modified shepherd, and shepherd services modules
would be compiled against the shepherd from the shepherd-configuration
record (see 'shepherd-boot-gexp', 'shepherd-configuration-file' and
'scm->go' in (gnu services shepherd)).

Toggle quote (7 lines)
> +@item
> +if you're only working on Shepherd's implementation (e.g. making
> +Shepherd's error handling more bullet proof), then it's enough to only
> +recompile Shepherd itself, and use the resulting package as the one that
> +gets started as the init process.
> +

So I don't think the distinction between API and implementation is
necessary here. (Feel free to correct in you have observed the
contrary!)

Greetings,
Maxime
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYh0jDRccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7rChAQDtnfhTsxl+cA3h9U/XjghagtLC
o+qwfRtTTdSzyE+jUgD+MZV8nOX7u+3w+bCFHwhGlJKrDD28Ejybfvj4seIt/wM=
=6Hjl
-----END PGP SIGNATURE-----


M
M
Maxime Devos wrote on 28 Feb 20:38 +0100
cfb14eda0845cab93da71a68172eda67be1a36e7.camel@telenet.be
Attila Lendvai schreef op ma 28-02-2022 om 19:51 [+0100]:
Toggle quote (16 lines)
> +custom Shepherd package for an @code{operating-system} object.  To get a
> +customized Shepherd package, you can simply make a copy of it in
> +@file{gnu/packages/admin.scm}, and change the @code{source} and
> +@code{version} field along these lines:
> +
> +@lisp
> +(define-public shepherd-dev-pid-1
> +  (package
> +    (name "shepherd")
> +    (version "dev-pid-1")
> +    (source (git-checkout
> +             (url "file:///my/path/shepherd/")))
> +    ...
> +    ))


There's no need to make a copy, we can use inheritance here
to reduce duplication:

(define my-shepherd
(package
(inherit shepherd)
(source (git-checkout
(url "file:///my/path/shepherd/"))))

Also, we're missing autoconf & friends here?
To help with shepherd hacking, I think it would be ok to add them to
the standard 'shepherd' package with a nice comment. If not, a
'shepherd-next' package or the like could be defined, a variant
of 'shepherd' for building from git?

Also, the documentation is being inconsistent with versions here.
In one place, you're using "dev-pid-1", and in another, "dev".
These versions also seem a bit odd for not being version _numbers_,
which doesn't mesh well with '17.4.3 Version Numbers'.

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYh0kvBccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7hfNAQCaaBZjyT4VeIfz8cA/w5RXsTLO
izsCCVwdbgea75GkVAEAkj6bO9m8Hn2y1ZF6Sp/1hggsj/3SJFxu1mCrV2D+cQ8=
=KLzT
-----END PGP SIGNATURE-----


P
P
pelzflorian (Florian Pelz) wrote on 1 Mar 11:14 +0100
(name . Attila Lendvai)(address . attila@lendvai.name)(address . 54199@debbugs.gnu.org)
20220301101414.pvf2wmnfbys3hw62@pelzflorian.localdomain
Thank you Attila.

On Mon, Feb 28, 2022 at 07:51:16PM +0100, Attila Lendvai wrote:
Toggle quote (6 lines)
> +@item @emph{The Shepherd API}
> +The Scheme code of Shepherd, which is a dependency of certain packages
> +and the Guix codebase itself. A typical example of this is the Scheme
> +code implementing a Guix System service, e.g. the OpenSSH server service
> +(see @code{openssh-shepherd-service}).

The last line reads like a @ref clickable link.


Toggle quote (21 lines)
> +To modify and use a new Shepherd API, you can change the @code{source}
> +and @code{version} field of the @code{shepherd} package in
> +@file{gnu/packages/admin.scm} along these lines:
> +
> +@lisp
> +(define-public shepherd ; do not change this
> + (package
> + (name "shepherd")
> + (version "dev")
> + (source (git-checkout
> + (url "file:///my/path/shepherd/")
> + (commit "[a commit hash]")))
> + ...
> + ))
> +@end lisp
> +
> +To avoid excessive recompilation times, we pick a specific commit in the
> +latter, and only update it as needed. But the former will pick up any
> +newly recorded commit when we issue a @command{guix system vm
> +/path/to/my-test.scm}.

Don’t you need to update the sha256 hash so Guix picks up that it
needs to rebuild shepherd? Unless you change the version each time to
a new value.

I wonder if you can just graft, i.e. introduce a
(replacement my-shepherd) field in the

(define-public shepherd ; do not change this
(package
(name "shepherd")
(replacement my-shepherd)

I have not tested and grafting may be unreliable.

Regards,
Florian
A
A
Attila Lendvai wrote on 2 May 13:11 +0200
Re: [PATCH] doc: Add 'Working on Shepherd' section.
(name . 54199@debbugs.gnu.org)(address . 54199@debbugs.gnu.org)
Xz2EpTg1VwJDI6Yff7sfGzq-GHqYC5jerjJbK4THtVYxyN2dBAjAxZDLnw_e2BQU1TVhM51gn7uy7ZPpkbtF_RrJe29MEFm3liWDUFrIpnA=@lendvai.name
FTR, this is waiting for the fate of:


i'll update this patch after that one is settled.

--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Reason is not automatic. Those who deny it cannot be conquered by it. Do not count on them. Leave them alone.”
— Ayn Rand (1905–1982)
?