Support stable uids and gids for system accounts in a container

  • Open
  • quality assurance status badge
Details
6 participants
  • Arun Isaac
  • Brendan Tildesley
  • Jason Conroy
  • Danny Milosavljevic
  • Leo Prikler
  • Ludovic Courtès
Owner
unassigned
Submitted by
Jason Conroy
Severity
normal
J
J
Jason Conroy wrote on 31 Dec 2020 19:18
(address . bug-guix@gnu.org)
CABWzUjULfU-uvBMB13oS=xD9Jv4u8R6TrDErTKVRbSVfwmUgmQ@mail.gmail.com
If I understand correctly, the container script produced by "guix system
container" will allocate the same uid and gid for a service on each
execution, but only if the corresponding entry in the service list has the
same absolute position as it did before. I.e., if the services are
reordered or if there are additions and removals, it's unlikely that the id
allocations will be the same.

As long as a container's filesystems don't outlive the container itself,
this works fine. But when host filesystems are bind-mounted inside the
container with the --share or --expose options, it's important that each
incarnation of a service uses the same uid and gid, because the bind mounts
might be used to hold persistent state for those services.

At first, I thought that I could just define static uids and gids for these
system accounts by adding corresponding user-account and user-group
entries. But this doesn't work: rather than changing how the system
accounts are defined for these services, it results in /etc files with
duplicate entries. (See https://issues.guix.gnu.org/45570for details.)
Attachment: file
D
D
Danny Milosavljevic wrote on 1 Jan 2021 15:47
Support stable uids and gids for all accounts
(name . Jason Conroy)(address . conjaroy@gmail.com)
20210101154504.28a18674@scratchpost.org
Hi,

I agree that user ids and group ids should be made stable, even in general.

I, too, have been bitten by this. (So would everyone else if Guix touched
existing UNIX accounts in general)

The right way to make them stable is for Guix ot default each uid to the hash
of the user name.

That said, we'd want to leave free some range of the integer uids for the usual
suspects (yp, samba) to allocate domain users there.

The place to change is gnu/system/accounts.scm. It would need to be changed
to do something similar for the "uid" field that it already does for the
"home-directory" field.

According to the source code of "useradd" in the package "shadow", it uses
the following range to use for automatic uid assignment:

Range starts at SYS_UID_MIN (default 1) for system user account uids, and stops
at SYS_UID_MAX (default (UID_MIN - 1)).
For non-system user account uids, it starts at UID_MIN (default 1000) and
stops at 60000 (UID_MAX).

See /etc/login.defs for the configured values.

Note that Linux has no problem using 32 bit uids.

If we want to make it possible for Guix to distinguish system from non-system
accounts by having different uid ranges for each, "system?" in the
<user-account> record would need to be moved to the front.
Then, in order to be backward compatible, custom procedures/macros
"make-user-account" and "user-account" would need to be provided with the
parameters in the previous order.

Should not be difficult to do--as always, the main work is in agreeing what
should be done, and in testing it after it's done. The actual change is like
10 lines of source code.

(An easier workaround would be to make the uid mandatory, with the default
being failure. But that would be the "punting" solution)
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl/vNgAACgkQ5xo1VCww
uqWm/Qf7BKjeacyamWrwQD+Jcs9/iRyRbQKhRYks2uG7PbLGVsLs8j0Vv0cLGKVu
IVp/22wuhs0gbwNul3lAHOaYIO1EuawaOwmIGlFt0SywSbzGfMPjPUpfVKwKsOLC
rzFAQcaZgWiwOz2urPhJEONm47q6uKGCuHLqGoV58ABEFS5r/RhV/xWlSBPva+dD
EX/nslH1SJB2/LHZV0UfwXD8yDOYmygYFiPDoInYf9rEZJ9DBX+jEjZtyI3i8Kzw
hjWhTk7YFxcSAi0HaRRYmzVJDy99EiW/TAW4C4ThVmVws+6jP//+xdVVilYgbwey
i9V78V86uC3y9uQ3oH3R3CJdmscKZQ==
=FyvA
-----END PGP SIGNATURE-----


L
L
Leo Prikler wrote on 1 Jan 2021 17:20
(address . dannym@scratchpost.org)(address . 45571@debbugs.gnu.org)
dfa026bb208f02d2e6e99c13b109bce911ffe368.camel@student.tugraz.at
Hello Danny,

I don't think changing the way UIDs are allocated by default is a good
solution as that will break many running installations on real
hardware, that default to those. A better solution would be to make
user accounts and groups explicit configuration wherever account-
service-type is used, so that it's possible to override them if needed.

Regards,
Leo
J
J
Jason Conroy wrote on 1 Jan 2021 17:26
CABWzUjXg6rZhniTdruFA=TZQhjRR+Ft5CSYqVNrS-PrdD6rxyA@mail.gmail.com
Hi Danny,

Your idea has a definite elegance to it. :) I did not realize that Linux
supported 32-bit UIDs out-of-the-box. Still, I wonder if this could
introduce support challenges for packages that incorrectly assume UIDs are
16 bits wide, since they traditionally were that way in UNIX, and since
other Linux distros still seem to prefer small UIDs in their packaging. By
comparison, my earlier idea of declaring static UIDs/GIDs in the
operating-system is decidedly less elegant, but it avoids this particular
risk. Can we be confident that this class of integer width bugs is
extremely rare?

On Fri, Jan 1, 2021 at 9:49 AM Danny Milosavljevic <dannym@scratchpost.org>
wrote:

Toggle quote (50 lines)
> Hi,
>
> I agree that user ids and group ids should be made stable, even in general.
>
> I, too, have been bitten by this. (So would everyone else if Guix touched
> existing UNIX accounts in general)
>
> The right way to make them stable is for Guix ot default each uid to the
> hash
> of the user name.
>
> That said, we'd want to leave free some range of the integer uids for the
> usual
> suspects (yp, samba) to allocate domain users there.
>
> The place to change is gnu/system/accounts.scm. It would need to be
> changed
> to do something similar for the "uid" field that it already does for the
> "home-directory" field.
>
> According to the source code of "useradd" in the package "shadow", it uses
> the following range to use for automatic uid assignment:
>
> Range starts at SYS_UID_MIN (default 1) for system user account uids, and
> stops
> at SYS_UID_MAX (default (UID_MIN - 1)).
>
> For non-system user account uids, it starts at UID_MIN (default 1000) and
> stops at 60000 (UID_MAX).
>
> See /etc/login.defs for the configured values.
>
> Note that Linux has no problem using 32 bit uids.
>
> If we want to make it possible for Guix to distinguish system from
> non-system
> accounts by having different uid ranges for each, "system?" in the
> <user-account> record would need to be moved to the front.
> Then, in order to be backward compatible, custom procedures/macros
> "make-user-account" and "user-account" would need to be provided with the
> parameters in the previous order.
>
> Should not be difficult to do--as always, the main work is in agreeing what
> should be done, and in testing it after it's done. The actual change is
> like
> 10 lines of source code.
>
> (An easier workaround would be to make the uid mandatory, with the default
> being failure. But that would be the "punting" solution)
>
Attachment: file
D
D
Danny Milosavljevic wrote on 1 Jan 2021 18:36
(name . Jason Conroy)(address . conjaroy@gmail.com)(address . 45571@debbugs.gnu.org)
20210101183637.6ff62412@scratchpost.org
Hi Jason,

Toggle quote (4 lines)
>Still, I wonder if this could introduce support challenges for packages that
>incorrectly assume UIDs are 16 bits wide, since they traditionally were that
>way in UNIX,

I don't think that these 16 bit problems are common at all since all the
getuid() syscalls I've ever seen, even in traditional UNIXes, have "int" as
return value--and nowadays that's at least 32 bits; and also because UNIX
configuration is usually in text files, which would read and store the stuff
using "%d". That said, it's possible that it could happen in some seriously
strange configurations (I doubt it).

Toggle quote (5 lines)
> and since other Linux distros still seem to prefer small UIDs in
> their packaging. By comparison, my earlier idea of declaring static UIDs/GIDs
> in the operating-system is decidedly less elegant, but it avoids this particular
> risk.

If we did something like that, I would prefer for guix services to just specify
a fixed UID/GID for each of the entries in gnu/services/*.scm. We can even
reuse existing system uids and gids that have been statically allocated
already by FreeBSD and Debian and be even uid/gid compatible to those.

For example, change Guix to do (similarly for all Guix services using user
accounts):

Toggle diff (31 lines)
diff --git a/gnu/services/monitoring.scm b/gnu/services/monitoring.scm
index 5123a8c441..45d3f4ad17 100644
--- a/gnu/services/monitoring.scm
+++ b/gnu/services/monitoring.scm
@@ -71,6 +71,7 @@
(define %darkstat-accounts
(list (user-account
+ (uid 4711)
(name "darkstat")
(group "darkstat")
(system? #t)
@@ -78,6 +79,7 @@
(home-directory "/var/lib/darkstat")
(shell (file-append shadow "/sbin/nologin")))
(user-group
+ (id 4711)
(name "darkstat")
(system? #t))))

And other constants for other user names.

Or use hashes. Whichever. The point is that the ids are deterministic--not
necessarily that we need to use hashes. The advantage of using a hash is that
we don't need a central registry and this problem will be out of our life once
and for all.

Anyway, as a workaround for your immediate problem, I suggest to preserve
/etc/passwd, /etc/group and /etc/shadow--present entries in those will be
preferred by Guix to even its own configuration. (I'm not sure whether that's
currently easy to do with guix system container, however)
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl/vXaUACgkQ5xo1VCww
uqV4twf/SE9DMqS+8viTnEak25kEV7V5xCU7eRyiBDqI1XyrXuL55Si03mkCrmbI
gbP0EC7/qR4JDyapRyoAlVif4Z0R9zgVD4jBxDi/8Sz7+mjnvwj1KfMxCIeQhbqg
FcGiO5smE1yQHupNXWkTW04dzqzgl3FP0vQtAcAHht1z3Hc0REHfhNJ/aYu6GbyQ
YdcPh7qBL9gY1B1CY2Ap0pdoFF4pFEDUYCg3lW4VWKwCngmCD+D0mifxevXLAjix
b9Z/lh+4nkQJvzoHssZJLFGHdhsLBXyvR5ok+7bWM0er0z1+ChYoAsVhCOTQVM13
8lSeASGEFMcrt7X5bqL71f6WSwKn8g==
=SB/f
-----END PGP SIGNATURE-----


D
D
Danny Milosavljevic wrote on 1 Jan 2021 18:50
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)(address . 45571@debbugs.gnu.org)
20210101184838.21869359@scratchpost.org
Hello,

On Fri, 01 Jan 2021 17:20:58 +0100
Leo Prikler <leo.prikler@student.tugraz.at> wrote:

Toggle quote (4 lines)
> I don't think changing the way UIDs are allocated by default is a good
> solution as that will break many running installations on real
> hardware, that default to those.

(gnu build accounts), allocate-passwd defaults to keep using the uids of
existing /etc/passwd entries. So running installations on real hardware
will not be affected when we change the defaults--otherwise those
installations on real hardware would already have had big problems regarding
user accounts with the current version of Guix when someone adds a user account
(system account or not) to them. Then, depending on the order of the declared
user-accounts, the uids would have swapped.

Toggle quote (4 lines)
>A better solution would be to make
> user accounts and groups explicit configuration wherever account-
> service-type is used, so that it's possible to override them if needed.

They already are explicit.

For example, (gnu services monitoring) has:

Toggle quote (12 lines)
>(define %darkstat-accounts
> (list (user-account
> (name "darkstat")
> (group "darkstat")
> (system? #t)
> (comment "darkstat daemon user")
> (home-directory "/var/lib/darkstat")
> (shell (file-append shadow "/sbin/nologin")))
> (user-group
> (name "darkstat")
> (system? #t))))

and then

Toggle quote (4 lines)
> (extensions
> (list (service-extension account-service-type
> (const %darkstat-accounts))

As you can see, the user and group accounts are already explicit. What's
more, the uid (and group id) can already be specified right there, and I
argue that it should be specified right there, and that there should be a
stable default if it's not specified.

So to summarize:

(1) This bug is a real problem, and something should be done about it.

(2) The reason it doesn't currently affect Guix system users is because there
is logic preferring existing /etc/passwd, /etc/shadow and /etc/group entries
(which I agree is a good idea). Doesn't help for guix system container, though.

Or for NFS network file systems. Any time you have more than one computer
(even with Guix on it) with a filesystem in the network and regular users,
you have to have consistent uids or have a lot of weird uid maps per computer
that someone has to maintain, or worse, an extra service just mapping them.
Why do that to yourself?

(3) For /etc/shadow, it also makes sense to keep the existing entry (the user
name is the key for the search for it btw) because of the password set.

(4) It makes sense to keep the existing /etc/{passwd,shadow,group} entries both
for backward compatibility and also for extensibility of guix with user & group
accounts guix knows nothing about.

(5) Also for not having this bug with containers, it would still be better to
just make uid and gid mandatory for "user-account" records.

(6) Since (5) would move the burden to the user, it would be better usability
to generate uid and gid in a deterministic manner as a default.

Both (5) and (6) can be done even now without problems because of (2).
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl/vYNQACgkQ5xo1VCww
uqXjIgf/a3oYUVFXIxWvOQuPlfCUIqmNducup/1Ak0xgM7vc5NYdxFfLef1FResi
l3meMnL4ZbmECInGrQ9NicXPuod94wU30wHc20dB3ZvGxgifqvyXaKi8fo+oruEb
e8GVi8jS+MzypsrzF0sGihb0DUHI54TCvFhUZAg3pXeZPgK/9P/5NXbQEhsqnEuU
IbBcufV5YrzI6ngpDERyfIikIblJhhP7EgDiSJd6rcRRTN9bkvuKD97lcDK1RVe2
hioRw/G7ic5EvDJ2gKucaT6YIHQJRXriQ1emLyPSNkuj+9HFGRWM1QydhPhbQMA3
XcJIgPZjHaNLFLG9kQ745EatiTN9nw==
=ZiyK
-----END PGP SIGNATURE-----


L
L
Leo Prikler wrote on 1 Jan 2021 19:44
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)(address . 45571@debbugs.gnu.org)
2f2fd3d66066b23f31f7db465aea65478ef81e87.camel@student.tugraz.at
Hi,

Am Freitag, den 01.01.2021, 18:50 +0100 schrieb Danny Milosavljevic:
Toggle quote (22 lines)
> Hello,
>
> On Fri, 01 Jan 2021 17:20:58 +0100
> Leo Prikler <leo.prikler@student.tugraz.at> wrote:
>
> > I don't think changing the way UIDs are allocated by default is a
> > good
> > solution as that will break many running installations on real
> > hardware, that default to those.
>
> (gnu build accounts), allocate-passwd defaults to keep using the uids
> of
> existing /etc/passwd entries. So running installations on real
> hardware
> will not be affected when we change the defaults--otherwise those
> installations on real hardware would already have had big problems
> regarding
> user accounts with the current version of Guix when someone adds a
> user account
> (system account or not) to them. Then, depending on the order of the
> declared
> user-accounts, the uids would have swapped.
Ah, that puts things into perspective. In other words, the problem is
not, that Guix doesn't read /etc/passwd at all, but that it reads the
wrong one (the host instead of the guest, so to speak). Should this
perhaps be a parameter instead?

Toggle quote (34 lines)
> > A better solution would be to make
> > user accounts and groups explicit configuration wherever account-
> > service-type is used, so that it's possible to override them if
> > needed.
>
> They already are explicit.
>
> For example, (gnu services monitoring) has:
>
> > (define %darkstat-accounts
> > (list (user-account
> > (name "darkstat")
> > (group "darkstat")
> > (system? #t)
> > (comment "darkstat daemon user")
> > (home-directory "/var/lib/darkstat")
> > (shell (file-append shadow "/sbin/nologin")))
> > (user-group
> > (name "darkstat")
> > (system? #t))))
>
> and then
>
> > (extensions
> > (list (service-extension account-service-type
> > (const %darkstat-accounts))
>
> As you can see, the user and group accounts are already
> explicit. What's
> more, the uid (and group id) can already be specified right there,
> and I
> argue that it should be specified right there, and that there should
> be a
> stable default if it's not specified.
How is that explicit? The % even implies, that it's considered
internal to the definition. Instead, you'd have (darkstat-accounts
config), which default to the current value of %darkstat-accounts, but
are configurable at least in the way that they allow you to set their
ids.

Toggle quote (4 lines)
> So to summarize:
>
> (1) This bug is a real problem, and something should be done about
> it.
Naturally.

Toggle quote (16 lines)
> (2) The reason it doesn't currently affect Guix system users is
> because there
> is logic preferring existing /etc/passwd, /etc/shadow and /etc/group
> entries
> (which I agree is a good idea). Doesn't help for guix system
> container, though.
>
> Or for NFS network file systems. Any time you have more than one
> computer
> (even with Guix on it) with a filesystem in the network and regular
> users,
> you have to have consistent uids or have a lot of weird uid maps per
> computer
> that someone has to maintain, or worse, an extra service just mapping
> them.
> Why do that to yourself?
In the realm of Guix system, this could be resolved by allowing the
user to choose the "seeds" for those files, so to speak (in commands
such as init, vm, deploy, etc.), could it not?
Especially for (3), carrying over the old shadow from the guest rather
than generating a new one with initial passwords sounds like it'd be a
necessary precondition for using them with persistent storage.

Toggle quote (4 lines)
> (3) For /etc/shadow, it also makes sense to keep the existing entry
> (the user
> name is the key for the search for it btw) because of the password
> set.
See above.

Toggle quote (5 lines)
> (4) It makes sense to keep the existing /etc/{passwd,shadow,group}
> entries both
> for backward compatibility and also for extensibility of guix with
> user & group
> accounts guix knows nothing about.
Not knowing about such accounts sounds somewhat antithetical to Guix,
but whatever.

Toggle quote (7 lines)
> (5) Also for not having this bug with containers, it would still be
> better to
> just make uid and gid mandatory for "user-account" records.
>
> (6) Since (5) would move the burden to the user, it would be better
> usability
> to generate uid and gid in a deterministic manner as a default.
Is the current logic non-deterministic in any way other than supporting
the reuse of old entries (which you yourself agree is a good thing)?
As far as I understand it, same config.scm + same
/etc/{passwd,group,shadow} => same /etc/{passwd,group,shadow}.

Regards,
Leo
D
D
Danny Milosavljevic wrote on 1 Jan 2021 21:22
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
20210101212242.00252cac@scratchpost.org
Hi Leo,

On Fri, 01 Jan 2021 19:44:12 +0100
Leo Prikler <leo.prikler@student.tugraz.at> wrote:

Toggle quote (5 lines)
> Ah, that puts things into perspective. In other words, the problem is
> not, that Guix doesn't read /etc/passwd at all, but that it reads the
> wrong one (the host instead of the guest, so to speak). Should this
> perhaps be a parameter instead?

Considering the goal of Guix, it's weird that with Guix, one needs to
store&restore /etc/passwd at all. It's state, but not very useful one.
I mean that's how it is right now--but it's still weird.

With /etc/shadow maybe there's a slightly better case, but note that the key
to find stuff in /etc/shadow can't be the uid--the uid isn't even in there!

Toggle quote (3 lines)
> How is that explicit? The % even implies, that it's considered
> internal to the definition.

Explicit means that the user-account record is initialized right there (every
time account-service-type is extended), by a literal.

And it is. You can see it plain as day in the guix git repo where
account-service-type is used in gnu/services/ .

Implicit would be if some code would generate this <user-account> record
on-the-fly, usually leaving stuff hard to change by the maintainer.

'(user-account (name "x") ...)' is about as explicit as it gets for a record.

The "%" in the name of the binding changes nothing in the literal value.

And it indeed is possible to add (uid 4711) in the literal and it will work
just fine.

Toggle quote (5 lines)
> Instead, you'd have (darkstat-accounts
> config), which default to the current value of %darkstat-accounts, but
> are configurable at least in the way that they allow you to set their
> ids.

Oh, you want internal service users to be USER-configurable. Indeed that is
also what Jason suggested in the initial mail.

But I think that that would put undue burden on each user and is really just
a workaround.
I would really like to caution against us doing a "whack a mole" development
approach, where workarounds like that are introduced to work around bugs
without understanding the underlying causes. So I disagree that having
internal service users be user-configurable is a good idea.

Toggle quote (4 lines)
> In the realm of Guix system, this could be resolved by allowing the
> user to choose the "seeds" for those files, so to speak (in commands
> such as init, vm, deploy, etc.), could it not?

Sure, but that's a last resort. Better is to eliminate state if possible.

Toggle quote (4 lines)
> Especially for (3), carrying over the old shadow from the guest rather
> than generating a new one with initial passwords sounds like it'd be a
> necessary precondition for using them with persistent storage.

It depends on what it is used for, really.

Toggle quote (11 lines)
> > (5) Also for not having this bug with containers, it would still be
> > better to
> > just make uid and gid mandatory for "user-account" records.
> >
> > (6) Since (5) would move the burden to the user, it would be better
> > usability
> > to generate uid and gid in a deterministic manner as a default.

> Is the current logic non-deterministic in any way other than supporting
> the reuse of old entries (which you yourself agree is a good thing)?

It generates uids using a counter, so it depends on what order
user-accounts are created in by Guix, which depends on the order the user
specifies services in /etc/config.scm and on the order to user accounts
are specified in gnu/services/ by guix maintainers. Then the service
executable (potentially) goes on to create files using those uids.

That means that if you remove or reorder service references in /etc/config.scm,
the uids "want" to change. The only reason they don't change is because the
logic prefers the existing /etc/passwd's uids--a stopgap measure at the last
second to prevent total chaos.

Does any of this sound good to you?

I mean, strictly speaking, it's better than the alternative--but that's a low
bar.

Better would be a making the uid field mandatory and/or generating each uid
from the respective name.

Toggle quote (3 lines)
> As far as I understand it, same config.scm + same
> /etc/{passwd,group,shadow} => same /etc/{passwd,group,shadow}.

That is the intention of (gnu system shadow), I think. I can't say whether
that's the case in practice now or not. It certainly was not the case a few
years ago where I did run into the same problem (a necessary condition for
the problem to manifest is that the services change--but my /etc/config.scm
services forms have been stable for a long time now, and Guix upstream also
doesn't change service definitions a lot anymore. So who knows?).
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl/vhJIACgkQ5xo1VCww
uqXjHAgAihpWdLfb4CmXKF0tamt5EkLtW+QXf8gX8vgn74mV6Qj8OwE2G3ZkNsOp
xNR994Ri8W2AOdYOGPlxzEr84hCSZtoiseXeIimCayq2uYINkNnSWvvkHTI6+Xce
GJ7oMDKtQYFAd6rAMRQ1BtBSpnRg9H9zdtnYryxgsDeQd/U+TceVia3/jLn1XfDP
/NuegKSLRwl5QHLIyTk7OMnl+3cXcFCmLx8hbCBus0oq1ufe33nv0K2q5iwKF/js
GrBFekj59ALgSDgsUkRekgcXKwIFTr297Jgav2DhiwBGFced64Yrvbn/ToRq3MDj
w7gz1qOc2jR2ZBpWIMhqsnvUr0Nkmw==
=VfF5
-----END PGP SIGNATURE-----


L
L
Leo Prikler wrote on 2 Jan 2021 01:25
(address . 45571@debbugs.gnu.org)
90ec1e8c2daab55d0e41b0fcd61706418789b2a8.camel@student.tugraz.at
Forgot to CC the ML.
Attachment: file
D
D
Danny Milosavljevic wrote on 2 Jan 2021 02:30
bug#45571: Support stable uids and gids for all accounts
(address . 45571@debbugs.gnu.org)
20210102023020.3e4186d3@scratchpost.org
Hi Leo,

On Fri, 01 Jan 2021 19:44:12 +0100
Leo Prikler <leo.prikler@student.tugraz.at> wrote:

Toggle quote (5 lines)
> Ah, that puts things into perspective. In other words, the problem is
> not, that Guix doesn't read /etc/passwd at all, but that it reads the
> wrong one (the host instead of the guest, so to speak). Should this
> perhaps be a parameter instead?

Considering the goal of Guix, it's weird that with Guix, one needs to
store&restore /etc/passwd at all. It's state, but not very useful one.
I mean that's how it is right now--but it's still weird.

With /etc/shadow maybe there's a slightly better case, but note that the key
to find stuff in /etc/shadow can't be the uid--the uid isn't even in there!

Toggle quote (3 lines)
> How is that explicit? The % even implies, that it's considered
> internal to the definition.

Explicit means that the user-account record is initialized right there (every
time account-service-type is extended), by a literal.

And it is. You can see it plain as day in the guix git repo where
account-service-type is used in gnu/services/ .

Implicit would be if some code would generate this <user-account> record
on-the-fly, usually leaving stuff hard to change by the maintainer.

'(user-account (name "x") ...)' is about as explicit as it gets for a record.

The "%" in the name of the binding changes nothing in the literal value.

And it indeed is possible to add (uid 4711) in the literal and it will work
just fine.

Toggle quote (5 lines)
> Instead, you'd have (darkstat-accounts
> config), which default to the current value of %darkstat-accounts, but
> are configurable at least in the way that they allow you to set their
> ids.

Oh, you want internal service users to be USER-configurable. Indeed that is
also what Jason suggested in the initial mail.

But I think that that would put undue burden on each user and is really just
a workaround.
I would really like to caution against us doing a "whack a mole" development
approach, where workarounds like that are introduced to work around bugs
without understanding the underlying causes. So I disagree that having
internal service users be user-configurable is a good idea.

Toggle quote (4 lines)
> In the realm of Guix system, this could be resolved by allowing the
> user to choose the "seeds" for those files, so to speak (in commands
> such as init, vm, deploy, etc.), could it not?

Sure, but that's a last resort. Better is to eliminate state if possible.

Toggle quote (4 lines)
> Especially for (3), carrying over the old shadow from the guest rather
> than generating a new one with initial passwords sounds like it'd be a
> necessary precondition for using them with persistent storage.

It depends on what it is used for, really.

Toggle quote (11 lines)
> > (5) Also for not having this bug with containers, it would still be
> > better to
> > just make uid and gid mandatory for "user-account" records.
> >
> > (6) Since (5) would move the burden to the user, it would be better
> > usability
> > to generate uid and gid in a deterministic manner as a default.

> Is the current logic non-deterministic in any way other than supporting
> the reuse of old entries (which you yourself agree is a good thing)?

It generates uids using a counter, so it depends on what order
user-accounts are created in by Guix, which depends on the order the user
specifies services in /etc/config.scm and on the order to user accounts
are specified in gnu/services/ by guix maintainers. Then the service
executable (potentially) goes on to create files using those uids.

That means that if you remove or reorder service references in /etc/config.scm,
the uids "want" to change. The only reason they don't change is because the
logic prefers the existing /etc/passwd's uids--a stopgap measure at the last
second to prevent total chaos.

Does any of this sound good to you?

I mean, strictly speaking, it's better than the alternative--but that's a low
bar.

Better would be a making the uid field mandatory and/or generating each uid
from the respective name.

Toggle quote (3 lines)
> As far as I understand it, same config.scm + same
> /etc/{passwd,group,shadow} => same /etc/{passwd,group,shadow}.

That is the intention of (gnu system shadow), I think. I can't say whether
that's the case in practice now or not. It certainly was not the case a few
years ago where I did run into the same problem (a necessary condition for
the problem to manifest is that the services change--but my /etc/config.scm
services forms have been stable for a long time now, and Guix upstream also
doesn't change service definitions a lot anymore. So who knows?).


--
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl/vzK0ACgkQ5xo1VCww
uqU9ZAgAo6m6zX2woiXZJdWfwSQ60B43JfyXRGuh16kI4fKcfAYCfgKjgs/Znr/M
EWjRMqHZjVAAfyHog9bBOMicqMG2y6K1zAWNnGgDaSWOiOdmq+A1Doy1K8qrr+GA
/C0wsqSh/WCA4+gPlJozM4m7GLy2L7KMeBZwlGL2jsQOWuR+cfAO+6UrJ1NzvWrq
weArWrj6q9cCO0D47yXs1rTSSJY3Q5X2q6VeTK3hcyvnMsnuOvRe0sddDMMU7Ok0
GLiUXQoXGwwuYNm712Gso2/+ZcicYs2x3GGPWaKFv2qYIFB4ImX4Zw96iLuQos7k
C8oT0ppHH/Y/N4fiicBQyB6hIcvbnQ==
=b+TB
-----END PGP SIGNATURE-----


D
D
Danny Milosavljevic wrote on 2 Jan 2021 02:40
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)(address . 45571@debbugs.gnu.org)
20210102024054.158bb3ba@scratchpost.org
Hi Leo,

On Sat, 02 Jan 2021 00:16:45 +0100
Leo Prikler <leo.prikler@student.tugraz.at> wrote:

Toggle quote (5 lines)
> > And it indeed is possible to add (uid 4711) in the literal and it
> > will work
> > just fine.
> I'm aware you're joking, or at least I hope you are,

What? It's perfectly reasonable for a distribution to have stable system
user ids.

That's what Debian supports, too:


Toggle quote (5 lines)
>0-99:
>Globally allocated by the Debian project, the same on every Debian system. These ids will appear in the passwd and group files of all Debian systems, new ids in this range being added automatically as the base-passwd package is updated.

>Packages which need a single statically allocated uid or gid should use one of these; their maintainers should ask the base-passwd maintainer for ids.

[...]

Toggle quote (4 lines)
>60000-64999:
>Globally allocated by the Debian project, but only created on demand. The ids are allocated centrally and statically, but the actual accounts are only >created on users’ systems on demand.
>[...]

And so does FreeBSD,

For that matter, IANA does this for ports and many other things. And so on.

Stable defaults are *good*.

Right now, the Guix service user user-account record specifies 99% of the
/etc/passwd entry. I indeed propose to make it 100% for system users for Guix
system services.

Toggle quote (3 lines)
>but I shouldn't have to point out why hardcoding ids into those literals is a
>bad idea.

You have to point that out to us--especially since Guix service user accounts
of the account-service-type extension can only be instantiated once anyway.
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl/vzyYACgkQ5xo1VCww
uqUbsAgAjLphtZlJJ10DEg8c6typg0Rq5EWdyRhaMoSFRDfCO2wXX9vOdAQBiN23
/8bjCJ9Lh5so8E3jrY1xjCKtrkI+oNL9JjhJ/1rBwPCeq9EX2k73uzaXmWndDbE1
vMo7rwrKDxde6DsOAivgLdHWRnxwOxbOK+2IMNjWqPX7rJiPGpNntVTbZauZCTB4
CTkoHbHIdwws8VPIhfoRH1nT5kb6jVQHcjOjuB57lkhhzmb33RCXvS/yGbuazxSH
m1VGWYN5mJAg0TFMWR4K1AANHASECnF7k0jKGGB21WV+JgFKwpspCdvcv77IHMB4
RE7k53vfolCQ/VQ/g+78BbA8LB8gUw==
=6948
-----END PGP SIGNATURE-----


L
L
Leo Prikler wrote on 2 Jan 2021 04:10
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)(address . 45571@debbugs.gnu.org)
0d5ecae08ad352669fab46858eeefff7f6446998.camel@student.tugraz.at
Hi Danny,
Am Samstag, den 02.01.2021, 02:40 +0100 schrieb Danny Milosavljevic:
Toggle quote (34 lines)
> Hi Leo,
>
> On Sat, 02 Jan 2021 00:16:45 +0100
> Leo Prikler <leo.prikler@student.tugraz.at> wrote:
>
> > > And it indeed is possible to add (uid 4711) in the literal and it
> > > will work
> > > just fine.
> > I'm aware you're joking, or at least I hope you are,
>
> What? It's perfectly reasonable for a distribution to have stable
> system
> user ids.
>
> That's what Debian supports, too:
>
> https://www.debian.org/doc/debian-policy/ch-opersys.html#uid-and-gid-classes
>
> > 0-99:
> > Globally allocated by the Debian project, the same on every Debian
> > system. These ids will appear in the passwd and group files of all
> > Debian systems, new ids in this range being added automatically as
> > the base-passwd package is updated.
> > Packages which need a single statically allocated uid or gid should
> > use one of these; their maintainers should ask the base-passwd
> > maintainer for ids.
>
> [...]
>
> > 60000-64999:
> > Globally allocated by the Debian project, but only created on
> > demand. The ids are allocated centrally and statically, but the
> > actual accounts are only >created on users’ systems on demand.
> > [...]
You do know, that services such as gdm, pulseaudio, avahi, sshd, mpd,
and others fall into neither region, do you?

Toggle quote (5 lines)
> And so does FreeBSD,
> see
> https://www.freebsd.org/doc/en/books/porters-handbook/users-and-groups.html
> and https://github.com/freebsd/freebsd-ports/blob/master/UIDs for the
> actual registry.
If I had a guixbuilder for every account in that list, that I didn't
need, I'd have a lot of guixbuilders. Probably more than I could
allocate into a contiguous block under FreeBSD.

Toggle quote (4 lines)
> For that matter, IANA does this for ports and many other things. And
> so on.
>
> Stable defaults are *good*.
So is leaving room for other configurations. Some of the bindings we
now consider "default" were only made because other ports were already
claimed. Not to mention overlaps, such as port 465.

Toggle quote (5 lines)
> Right now, the Guix service user user-account record specifies 99% of
> the
> /etc/passwd entry. I indeed propose to make it 100% for system users
> for Guix
> system services.
What's the remaining 1%?

Toggle quote (8 lines)
> > but I shouldn't have to point out why hardcoding ids into those
> > literals is a
> > bad idea.
>
> You have to point that out to us--especially since Guix service user
> accounts
> of the account-service-type extension can only be instantiated once
> anyway.
Unlike in other systems, where you'd expect people to manually fiddle
around with such files and tragically fail, in Guix your OS config.scm
should reflect the actual state of the system (modulo secrets, that
can't be expressed currently). If you claim UID 92 for GDM like
FreeBSD does, but people live on installations, that have the old
default of 983 (or any other, depending on the number of guixbuilders
you have), that's going to cause problems. Perhaps not the same
problems that led to the creation of its activation-service, but still.

That's not to say, that claiming such IDs is *always* bad, just that
it's bad to do so without leaving room for configuration. I should
likely have worded that better, but at the same time there was context
from which one could have inferred, that I meant hardcoding IDs into
unchanging constants.

From the solutions we do have so far, I believe that making user
accounts an explicit part of service configuration (in what shape may
still be up for debate), with reasonable defaults including numeric
UIDs and GIDs (at least) for essential services such as GDM sounds like
the best option. WDYT?

Regards,
Leo
J
J
Jason Conroy wrote on 2 Jan 2021 15:02
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
CABWzUjVCW=-yrKQk-KU0sYdLkDuafNod6W4G-htKKw_jQyE1hw@mail.gmail.com
On Fri, Jan 1, 2021 at 10:11 PM Leo Prikler <leo.prikler@student.tugraz.at>
wrote:

Toggle quote (17 lines)
> Hi Danny,
> Am Samstag, den 02.01.2021, 02:40 +0100 schrieb Danny Milosavljevic:
> > Hi Leo,
> >
> > On Sat, 02 Jan 2021 00:16:45 +0100
> > Leo Prikler <leo.prikler@student.tugraz.at> wrote:
> >
> > > > And it indeed is possible to add (uid 4711) in the literal and it
> > > > will work
> > > > just fine.
> > > I'm aware you're joking, or at least I hope you are,
> >
> > What? It's perfectly reasonable for a distribution to have stable
> > system
> > user ids.
>

My reaction to this was not that defaults are bad, but that dispersing
numeric literals throughout the code is. Collectively these values specify
the contents of a registry, so that registry might as well be located
centrally. Or at least, there should be some mechanism to ensure that two
services can't claim the same default ID, otherwise the collision will not
manifest until somebody instantiates a system with the colliding services.

From the solutions we do have so far, I believe that making user
Toggle quote (9 lines)
> accounts an explicit part of service configuration (in what shape may
> still be up for debate), with reasonable defaults including numeric
> UIDs and GIDs (at least) for essential services such as GDM sounds like
> the best option. WDYT?
>
> Regards,
> Leo
>

That seems reasonable to me. As for representation, I think there's value
decoupling these settings from a service's own config so that support for
custom UIDs/GIDs remains consistent across services.
Attachment: file
L
L
Leo Prikler wrote on 2 Jan 2021 15:29
(name . Jason Conroy)(address . conjaroy@gmail.com)
250d033edc273b0f89e2bd37a331a3f961a8d785.camel@student.tugraz.at
Hi Jason,
Am Samstag, den 02.01.2021, 09:02 -0500 schrieb Jason Conroy:
Toggle quote (28 lines)
> On Fri, Jan 1, 2021 at 10:11 PM Leo Prikler <
> leo.prikler@student.tugraz.at> wrote:
> > Hi Danny,
> > Am Samstag, den 02.01.2021, 02:40 +0100 schrieb Danny
> > Milosavljevic:
> > > Hi Leo,
> > >
> > > On Sat, 02 Jan 2021 00:16:45 +0100
> > > Leo Prikler <leo.prikler@student.tugraz.at> wrote:
> > >
> > > > > And it indeed is possible to add (uid 4711) in the literal
> > and it
> > > > > will work
> > > > > just fine.
> > > > I'm aware you're joking, or at least I hope you are,
> > >
> > > What? It's perfectly reasonable for a distribution to have
> > stable
> > > system
> > > user ids.
>
> My reaction to this was not that defaults are bad, but that
> dispersing numeric literals throughout the code is. Collectively
> these values specify the contents of a registry, so that registry
> might as well be located centrally. Or at least, there should be some
> mechanism to ensure that two services can't claim the same default
> ID, otherwise the collision will not manifest until somebody
> instantiates a system with the colliding services.
I think it would suffice to raise a condition from shadow.scm, much
like my proposed fix for #45570. As far as development is concerned,
one could add a check to see, that no conflicts exist between services
extending account-service-type.

Toggle quote (14 lines)
> > From the solutions we do have so far, I believe that making user
> > accounts an explicit part of service configuration (in what shape
> > may
> > still be up for debate), with reasonable defaults including numeric
> > UIDs and GIDs (at least) for essential services such as GDM sounds
> > like
> > the best option. WDYT?
> >
> > Regards,
> > Leo
>
> That seems reasonable to me. As for representation, I think there's
> value decoupling these settings from a service's own config so that
> support for custom UIDs/GIDs remains consistent across services.
In this case I'd agree with Danny, that asking users to update two
fields to get one service working puts an excessive burden on them. To
be fair, I don't even necessarily think, that making the full account
configurable is a good idea, unless someone wants to argue, that
there's value in potentially giving those accounts shell access. At
the very least, there should be syntax like

(user-account
(inherit sane-system-account-defaults)
(name "gdm")
(id 92))

Perhaps there could also be a procedure (system-account+group name
#:key comment uid gid).

Regards,
Leo
D
D
Danny Milosavljevic wrote on 2 Jan 2021 15:50
(name . Jason Conroy)(address . conjaroy@gmail.com)
20210102155002.2360e505@scratchpost.org
Hi Jason,

On Sat, 2 Jan 2021 09:02:18 -0500
Jason Conroy <conjaroy@gmail.com> wrote:

Toggle quote (3 lines)
> My reaction to this was not that defaults are bad, but that dispersing
> numeric literals throughout the code is.

In general that is not exactly true. What you want is some way to check
the uids for collisions--and putting them into one file only and manually
checking is only one way to do that. And it's not ideal because then the uid
to use for an account would be in some random file far away from the actual
point of use. If you mean having both the central registry, and the numeric
literal throughout the code, carry on--I agree.

We also disperse shepherd service names and many other similar things across
literals in the source code of Guix. Those can cause similar problems.

I agree that it would be better to have a checker, and central registry, for
cross-checking--but right now we don't do that for a lot of other things,
among which are the shepherd service names, dbus service names and dbus
interface names.

This is guile--it could find and lint user account definitions perfectly well,
no matter where they are. There could be an automated check that the uids are
not duplicated, at compile or lint time. It would be nice to have such a
checker for the dbus things, too.

But seriously, at this point I don't think any of this in the
currently-discussed form adds enough value to be worth the complexity
and the risk of changing it in the first place.

Toggle quote (4 lines)
>Collectively these values specify
> the contents of a registry, so that registry might as well be located
> centrally.

I agree, if in addition.

(Or if a registry is undesireable, calculate the uids by user name.
These are the possibilities. If hashing user names is too uncommon on
UNIX elsewhere, I say that we have no /usr or /lib, so we are not exactly doing
common things in Guix because they are common--what we do really depends on the
merits)

Toggle quote (4 lines)
>Or at least, there should be some mechanism to ensure that two
> services can't claim the same default ID, otherwise the collision will not
> manifest until somebody instantiates a system with the colliding services.

I agree--that mechanism would need to be added.

Toggle quote (4 lines)
> >From the solutions we do have so far, I believe that making user
> > accounts an explicit part of service configuration (in what shape may
> > still be up for debate),

Not everything needs to be user configurable. You suggest having these kinds
of things especially user-configurable as a work-around for them not being
stable, right? Or do you want it in general?

I would like to avoid them here, if only needed for that reason.

(Also, if they are user-configurable, then it's much easier for uid collisions
to happen than if Guix manages them)

So after thinking about it some more, I disagree that that is the best option
for this specific problem.

In my opinion, the best option here is all of these things:

(1) To document that you need to seed /etc/passwd (for Docker etc)
(2) For guix system container to default to the host's /etc/passwd and
(3) For guix system container to add and retain (!) its own /etc/passwd for
accounts only it has (merged together with the host's /etc/passwd for ease
of implementation)

The suggestions I made before that would obsolete /etc/passwd storage got
watered down to a version where they don't obsolete /etc/passwd storage and
thus I oppose doing them in that form. It would be making the stuff more
complicated--and now you'd have TWO /etc/passwd:

* one /etc/template-passwd (for the uid registry)
(might as well integrate that into guix as a scheme file, though),
* and one /etc/passwd with the currently created users.

This seems to be excessive just for making uids stable.

Toggle quote (4 lines)
> That seems reasonable to me. As for representation, I think there's value
> decoupling these settings from a service's own config so that support for
> custom UIDs/GIDs remains consistent across services.

Is there a need for using custom service UIDs?
I think if so, that is independent of this bug report, which asked for
stable UIDs and GIDs.
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl/wiBoACgkQ5xo1VCww
uqWENQf+I93c+mzIWTXh79ee4vIqnSYmnjLzr2GVA57Odi4jmJYUte7Xg3Tvsi8f
8FkVjN4MWg7Bu6vbLinJlH//DJrxJ09I/0rdU2EGtHbVTMn4XPcj/PByJOyFkyTc
z7FRXidaOUOZALDKU/2ZWjP38EukUvrBGfMbr7AqHdK80tAeCCFcDt8G73y/GJkP
Hc7oLwbAfYKuqEOwavcuuvzhlfmd94W/pZCNgATZjMwNKJ+J5hy8RxA7Ot0yXlcO
deRzX0PGebi/uSEweL+SkETnsl3S9jlSwMaukWqwA8KLVa1vUzIKbNXhZZC/qSgq
Nh1gJEeZpvVc8fs7pjMI321/TWK1OA==
=2IDa
-----END PGP SIGNATURE-----


J
J
Jason Conroy wrote on 2 Jan 2021 15:52
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
CABWzUjV5oAq2RPAY3e3T9qz7p7m-ANNDF6tKqUU+sEfsPOv5-g@mail.gmail.com
On Sat, Jan 2, 2021 at 9:29 AM Leo Prikler <leo.prikler@student.tugraz.at>
wrote:

Toggle quote (36 lines)
> Hi Jason,
> Am Samstag, den 02.01.2021, 09:02 -0500 schrieb Jason Conroy:
> > On Fri, Jan 1, 2021 at 10:11 PM Leo Prikler <
> > leo.prikler@student.tugraz.at> wrote:
> > > Hi Danny,
> > > Am Samstag, den 02.01.2021, 02:40 +0100 schrieb Danny
> > > Milosavljevic:
> > > > Hi Leo,
> > > >
> > > > On Sat, 02 Jan 2021 00:16:45 +0100
> > > > Leo Prikler <leo.prikler@student.tugraz.at> wrote:
> > > >
> > > > > > And it indeed is possible to add (uid 4711) in the literal
> > > and it
> > > > > > will work
> > > > > > just fine.
> > > > > I'm aware you're joking, or at least I hope you are,
> > > >
> > > > What? It's perfectly reasonable for a distribution to have
> > > stable
> > > > system
> > > > user ids.
> >
> > My reaction to this was not that defaults are bad, but that
> > dispersing numeric literals throughout the code is. Collectively
> > these values specify the contents of a registry, so that registry
> > might as well be located centrally. Or at least, there should be some
> > mechanism to ensure that two services can't claim the same default
> > ID, otherwise the collision will not manifest until somebody
> > instantiates a system with the colliding services.
> I think it would suffice to raise a condition from shadow.scm, much
> like my proposed fix for #45570. As far as development is concerned,
> one could add a check to see, that no conflicts exist between services
> extending account-service-type.
>

Assuming that authors of new services tend to choose the lowest free ID, is
this validation sufficient to ensure that two services checked around the
same time by different authors won't collide?

Toggle quote (16 lines)
>
> > > From the solutions we do have so far, I believe that making user
> > > accounts an explicit part of service configuration (in what shape
> > > may
> > > still be up for debate), with reasonable defaults including numeric
> > > UIDs and GIDs (at least) for essential services such as GDM sounds
> > > like
> > > the best option. WDYT?
> > >
> > > Regards,
> > > Leo
> >
> > That seems reasonable to me. As for representation, I think there's
> > value decoupling these settings from a service's own config so that
> > support for custom UIDs/GIDs remains consistent across services.
>
In this case I'd agree with Danny, that asking users to update two
Toggle quote (3 lines)
> fields to get one service working puts an excessive burden on them.


Leo, I'm not sure what you mean by updating two fields. Are you saying that
some services already permit manual selection of UIDs? I was proposing
setting that value in the "users" section of operating-system (or
elsewhere) rather than setting it in the "services" section, but not both
places.

Since Guix already uses a central allocator for UIDs and GIDs (implemented
using simple counters), I was imagining a model where the decision is still
made centrally, but with different inputs: 1) a global mapping from
user/group name to default ID; and 2) a similar name-to-ID mapping in
operating-system where users specify their overrides.

Toggle quote (7 lines)
>
> To
> be fair, I don't even necessarily think, that making the full account
> configurable is a good idea, unless someone wants to argue, that
> there's value in potentially giving those accounts shell access.


The thought of making other parts of the account configurable occurred to
me, but I couldn't come up with any serious use cases either.


Toggle quote (15 lines)
> At
> the very least, there should be syntax like
>
> (user-account
> (inherit sane-system-account-defaults)
> (name "gdm")
> (id 92))
>
> Perhaps there could also be a procedure (system-account+group name
> #:key comment uid gid).
>
> Regards,
> Leo
>
>
Attachment: file
D
D
Danny Milosavljevic wrote on 2 Jan 2021 16:04
Re: bug#45571: Support stable uids and gids for all accounts
20210102160415.30fcb7e8@scratchpost.org
Hi Leo,

Toggle quote (11 lines)
> > Considering the goal of Guix, it's weird that with Guix, one needs to
> > store&restore /etc/passwd at all. It's state, but not very useful
> > one.
> > I mean that's how it is right now--but it's still weird.
> > With /etc/shadow maybe there's a slightly better case, but note that
> > the key
> > to find stuff in /etc/shadow can't be the uid--the uid isn't even in
> > there!

> AFAIU yes, it's state, but not one that Guix can simply do away with.

It's easily possible to recreate /etc/passwd from scratch if the uids are
always specified in <user-account>s and thus /etc/passwd would not need to
be persistent state anymore. Right now everything from /etc/passwd except
the uid and the comment is already specified in <user-account>.

So Guix can indeed simply do away with the persistent state of
/etc/passwd--that's why I suggested specifying the uids in the first place.

(By now I don't think that's the best way to make UIDs stable, but it's
factually incorrect to assert that Guix can't simply do away with that
persistent state specifically. It can.)

Toggle quote (3 lines)
> There is not yet a syntax for keeping secrets, which would be needed to
> fully populate /etc from config.scm. Perhaps we'll get there some day.

/etc/passwd does not contain secrets. Neither does /etc/group.

And /etc/shadow doesn't contain uids.

So there is no conflict.
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl/wi28ACgkQ5xo1VCww
uqW8Uwf+OF0uHzOoKtVm4ZVuoAzXUetEX4xil+pfwjtQc/k0oAmCNzgXFYasASVs
cM7reZLqix/lWXk6FmIgMYdvgF6M9SS78aTjCWVcHTJtVuc55XrPIRVtn/P2fPwK
sVd3+DkpGN7LXsIJm9DsISU9W4vNMlwgiXLpG4rUqldwwSPmrjfvfpTwMpfuBnRO
Uc227svfgQS77AYk06SjyMo1JMQisrSE2x5CzkFs2a+0ceV+jy3Js8xSMXSo67RM
1L7KxGsWgeJKM87/EPP0gzuHFxJIylGysqpChLqEXtX1vIKez6I56OhuRuziodG2
9ewAyJU4vdcNCUbRTHdTGNz1b3G6qA==
=IHzF
-----END PGP SIGNATURE-----


J
J
Jason Conroy wrote on 2 Jan 2021 16:03
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
CABWzUjUZbB==QvXWG-ZVJOj8FTK9zo6oCQZe6b87VN44u=n+0g@mail.gmail.com
Hi Danny,

On Sat, Jan 2, 2021 at 9:50 AM Danny Milosavljevic <dannym@scratchpost.org>
wrote:

Toggle quote (16 lines)
> > >From the solutions we do have so far, I believe that making user
> > > accounts an explicit part of service configuration (in what shape may
> > > still be up for debate),
>
> Not everything needs to be user configurable. You suggest having these
> kinds
> of things especially user-configurable as a work-around for them not being
> stable, right? Or do you want it in general?
>
> I would like to avoid them here, if only needed for that reason.
>
> Is there a need for using custom service UIDs?
> I think if so, that is independent of this bug report, which asked for
> stable UIDs and GIDs.
>

As far as I know, stability is sufficient for me, but Leo raised some cases
where specific values would ease migration. An /etc/passwd seed file seems
like another reasonable way to handle that, but I haven't investigated how
difficult this change would be compared adding new knobs in
operating-system.
Attachment: file
L
L
Leo Prikler wrote on 2 Jan 2021 16:18
(address . 45571@debbugs.gnu.org)
fd0727b3a49d267354a1d35b697477302babb0e0.camel@student.tugraz.at
Hi Danny,
Am Samstag, den 02.01.2021, 15:50 +0100 schrieb Danny Milosavljevic:
Toggle quote (11 lines)
> [...]
> I agree, if in addition.
>
> (Or if a registry is undesireable, calculate the uids by user name.
> These are the possibilities. If hashing user names is too uncommon
> on
> UNIX elsewhere, I say that we have no /usr or /lib, so we are not
> exactly doing
> common things in Guix because they are common--what we do really
> depends on the
> merits)
I don't think hashing has much merit if you have the range of 100-1000
to work with. Using the full 32 bit integer range also feels like a
hack just to enable hashing, and even then hash functions targeting 32
bit integers are not known to be the most secure in this world. In
other words, hashing user names into IDs is little more than theatre
and while it can certainly mitigate the underlying issue in *some*
cases, it is not by itself a solution.

Toggle quote (10 lines)
> > > From the solutions we do have so far, I believe that making user
> > > accounts an explicit part of service configuration (in what shape
> > > may
> > > still be up for debate),
>
> Not everything needs to be user configurable. You suggest having
> these kinds
> of things especially user-configurable as a work-around for them not
> being
> stable, right? Or do you want it in general?
I believe there might be a general utility for that if we're not going
to use an existing passwd as oracle. In that case, you would need a
way of claiming those IDs from the config.scm.

Other than that, I believe you pointed out an NFS example, where it
could also be beneficial to sync up user/system accounts with the IDs
they have on other machines within the network. Of course, if all
machines within the network use Guix, that point is moot, but there
might be a case when you want to play nice with that one old Debian
server.

Toggle quote (5 lines)
> I would like to avoid them here, if only needed for that reason.
>
> (Also, if they are user-configurable, then it's much easier for uid
> collisions
> to happen than if Guix manages them)
Sure, but either way we'll need a collision checker, will we not?

Toggle quote (14 lines)
> So after thinking about it some more, I disagree that that is the
> best option
> for this specific problem.
>
> In my opinion, the best option here is all of these things:
>
> (1) To document that you need to seed /etc/passwd (for Docker etc)
> (2) For guix system container to default to the host's /etc/passwd
> and
> (3) For guix system container to add and retain (!) its own
> /etc/passwd for
> accounts only it has (merged together with the host's /etc/passwd for
> ease
> of implementation)
That's also a solution and I think I'd prefer that over spamming IDs
throughout the codebase.

Toggle quote (13 lines)
> The suggestions I made before that would obsolete /etc/passwd storage
> got
> watered down to a version where they don't obsolete /etc/passwd
> storage and
> thus I oppose doing them in that form. It would be making the stuff
> more
> complicated--and now you'd have TWO /etc/passwd:
>
> * one /etc/template-passwd (for the uid registry)
> (might as well integrate that into guix as a scheme file, though),
> * and one /etc/passwd with the currently created users.
>
> This seems to be excessive just for making uids stable.
FWIW I think taking the passwd-seed as a file-like object, that
defaults to using the host's /etc/passwd might work here, but I'm not
completely sure.

Toggle quote (10 lines)
> > That seems reasonable to me. As for representation, I think there's
> > value
> > decoupling these settings from a service's own config so that
> > support for
> > custom UIDs/GIDs remains consistent across services.
>
> Is there a need for using custom service UIDs?
> I think if so, that is independent of this bug report, which asked
> for
> stable UIDs and GIDs.
I agree, the discourse regarding that has been muddled a bit, but since
custom implies stable I don't think this option can be completely
discarded. Of course, both stability and customization are also given
by a passwd-seed, so if we choose that implementation, other means of
customization might not be needed.

Regards,
Leo
L
L
Leo Prikler wrote on 2 Jan 2021 16:25
Re: bug#45571: Support stable uids and gids for all accounts
95f76be4dfebc473e8f4436464978a26296d2f57.camel@student.tugraz.at
Hi Danny,
Am Samstag, den 02.01.2021, 16:04 +0100 schrieb Danny Milosavljevic:
Toggle quote (45 lines)
> Hi Leo,
>
> > > Considering the goal of Guix, it's weird that with Guix, one
> > > needs to
> > > store&restore /etc/passwd at all. It's state, but not very
> > > useful
> > > one.
> > > I mean that's how it is right now--but it's still weird.
> > > With /etc/shadow maybe there's a slightly better case, but note
> > > that
> > > the key
> > > to find stuff in /etc/shadow can't be the uid--the uid isn't even
> > > in
> > > there!
> > AFAIU yes, it's state, but not one that Guix can simply do away
> > with.
>
> It's easily possible to recreate /etc/passwd from scratch if the uids
> are
> always specified in <user-account>s and thus /etc/passwd would not
> need to
> be persistent state anymore. Right now everything from /etc/passwd
> except
> the uid and the comment is already specified in <user-account>.
>
> So Guix can indeed simply do away with the persistent state of
> /etc/passwd--that's why I suggested specifying the uids in the first
> place.
>
> (By now I don't think that's the best way to make UIDs stable, but
> it's
> factually incorrect to assert that Guix can't simply do away with
> that
> persistent state specifically. It can.)
>
> > There is not yet a syntax for keeping secrets, which would be
> > needed to
> > fully populate /etc from config.scm. Perhaps we'll get there some
> > day.
>
> /etc/passwd does not contain secrets. Neither does /etc/group.
>
> And /etc/shadow doesn't contain uids.
>
> So there is no conflict.
Point taken, it is indeed possibly to do away with one of those files,
but looking at them as a trio (as one ought to imo), I don't think
removing one while keeping the other(s) is the way to go.

Also if you do go that route, you would need a way to specify that your
passwd has hitherto been different to all other Guix installations;
hence forcing you to make system account [GU]IDs configurable once
again.

Regards,
Leo
L
L
Leo Prikler wrote on 2 Jan 2021 16:35
(name . Jason Conroy)(address . conjaroy@gmail.com)
f74a8e0dc78c7d336487076743d5b4b496106c81.camel@student.tugraz.at
Hello Jason,
Am Samstag, den 02.01.2021, 09:52 -0500 schrieb Jason Conroy:
Toggle quote (45 lines)
> On Sat, Jan 2, 2021 at 9:29 AM Leo Prikler <
> leo.prikler@student.tugraz.at> wrote:
> > Hi Jason,
> > Am Samstag, den 02.01.2021, 09:02 -0500 schrieb Jason Conroy:
> > > On Fri, Jan 1, 2021 at 10:11 PM Leo Prikler <
> > > leo.prikler@student.tugraz.at> wrote:
> > > > Hi Danny,
> > > > Am Samstag, den 02.01.2021, 02:40 +0100 schrieb Danny
> > > > Milosavljevic:
> > > > > Hi Leo,
> > > > >
> > > > > On Sat, 02 Jan 2021 00:16:45 +0100
> > > > > Leo Prikler <leo.prikler@student.tugraz.at> wrote:
> > > > >
> > > > > > > And it indeed is possible to add (uid 4711) in the
> > literal
> > > > and it
> > > > > > > will work
> > > > > > > just fine.
> > > > > > I'm aware you're joking, or at least I hope you are,
> > > > >
> > > > > What? It's perfectly reasonable for a distribution to have
> > > > stable
> > > > > system
> > > > > user ids.
> > >
> > > My reaction to this was not that defaults are bad, but that
> > > dispersing numeric literals throughout the code is. Collectively
> > > these values specify the contents of a registry, so that registry
> > > might as well be located centrally. Or at least, there should be
> > some
> > > mechanism to ensure that two services can't claim the same
> > default
> > > ID, otherwise the collision will not manifest until somebody
> > > instantiates a system with the colliding services.
> > I think it would suffice to raise a condition from shadow.scm, much
> > like my proposed fix for #45570. As far as development is
> > concerned,
> > one could add a check to see, that no conflicts exist between
> > services
> > extending account-service-type.
>
> Assuming that authors of new services tend to choose the lowest free
> ID, is this validation sufficient to ensure that two services checked
> around the same time by different authors won't collide?
No, you'd need to lint the services in the pre-push hook. That would
not be the biggest deal though, we already authenticate the commits and
check whether the NEWS file is broken before pushing, for instance. I
believe it could also be checked without actually instantiating that
system, but don't quote me on that.

Toggle quote (29 lines)
> > > > From the solutions we do have so far, I believe that making
> > user
> > > > accounts an explicit part of service configuration (in what
> > shape
> > > > may
> > > > still be up for debate), with reasonable defaults including
> > numeric
> > > > UIDs and GIDs (at least) for essential services such as GDM
> > sounds
> > > > like
> > > > the best option. WDYT?
> > > >
> > > > Regards,
> > > > Leo
> > >
> > > That seems reasonable to me. As for representation, I think
> > there's
> > > value decoupling these settings from a service's own config so
> > that
> > > support for custom UIDs/GIDs remains consistent across services.
> > In this case I'd agree with Danny, that asking users to update two
> > fields to get one service working puts an excessive burden on
> > them.
>
> Leo, I'm not sure what you mean by updating two fields. Are you
> saying that some services already permit manual selection of UIDs? I
> was proposing setting that value in the "users" section of operating-
> system (or elsewhere) rather than setting it in the "services"
> section, but not both places.
As far as I'm aware, no service so far do that, but there are some,
that don't set up the user (e.g. mpd). However, I don't think, that's
the way to go for services like gdm. If you decoupled the gdm user and
group from its service specification, you'd need to modify three
operating-system fields to add gdm as opposed to one. If the gdm user
and group were configuration, you could instead specify them with
modify-services or gdm-service-type itself.

Toggle quote (6 lines)
> Since Guix already uses a central allocator for UIDs and GIDs
> (implemented using simple counters), I was imagining a model where
> the decision is still made centrally, but with different inputs: 1) a
> global mapping from user/group name to default ID; and 2) a similar
> name-to-ID mapping in operating-system where users specify their
> overrides.
I'm not sure how well that'd work together with account-service-type.
You'd have to find a novel way of extending it, that's for sure.

Toggle quote (9 lines)
> > To
> > be fair, I don't even necessarily think, that making the full
> > account
> > configurable is a good idea, unless someone wants to argue, that
> > there's value in potentially giving those accounts shell access.
>
> The thought of making other parts of the account configurable
> occurred to me, but I couldn't come up with any serious use cases
> either.
As far as I'm aware, nologin exists for a good reason.

Regards,
Leo
J
J
Jason Conroy wrote on 2 Jan 2021 16:58
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
CABWzUjXZ3FADpc6pKF+n3a2AB0BRJzSL0-pAY-ykN5ia9iSz6w@mail.gmail.com
Hi Leo,

On Sat, Jan 2, 2021 at 10:35 AM Leo Prikler <leo.prikler@student.tugraz.at>
wrote:

Toggle quote (54 lines)
> Hello Jason,
> Am Samstag, den 02.01.2021, 09:52 -0500 schrieb Jason Conroy:
> > On Sat, Jan 2, 2021 at 9:29 AM Leo Prikler <
> > leo.prikler@student.tugraz.at> wrote:
> > > Hi Jason,
> > > Am Samstag, den 02.01.2021, 09:02 -0500 schrieb Jason Conroy:
> > > > On Fri, Jan 1, 2021 at 10:11 PM Leo Prikler <
> > > > leo.prikler@student.tugraz.at> wrote:
> > > > > Hi Danny,
> > > > > Am Samstag, den 02.01.2021, 02:40 +0100 schrieb Danny
> > > > > Milosavljevic:
> > > > > > Hi Leo,
> > > > > >
> > > > > > On Sat, 02 Jan 2021 00:16:45 +0100
> > > > > > Leo Prikler <leo.prikler@student.tugraz.at> wrote:
> > > > > >
> > > > > > > > And it indeed is possible to add (uid 4711) in the
> > > literal
> > > > > and it
> > > > > > > > will work
> > > > > > > > just fine.
> > > > > > > I'm aware you're joking, or at least I hope you are,
> > > > > >
> > > > > > What? It's perfectly reasonable for a distribution to have
> > > > > stable
> > > > > > system
> > > > > > user ids.
> > > >
> > > > My reaction to this was not that defaults are bad, but that
> > > > dispersing numeric literals throughout the code is. Collectively
> > > > these values specify the contents of a registry, so that registry
> > > > might as well be located centrally. Or at least, there should be
> > > some
> > > > mechanism to ensure that two services can't claim the same
> > > default
> > > > ID, otherwise the collision will not manifest until somebody
> > > > instantiates a system with the colliding services.
> > > I think it would suffice to raise a condition from shadow.scm, much
> > > like my proposed fix for #45570. As far as development is
> > > concerned,
> > > one could add a check to see, that no conflicts exist between
> > > services
> > > extending account-service-type.
> >
> > Assuming that authors of new services tend to choose the lowest free
> > ID, is this validation sufficient to ensure that two services checked
> > around the same time by different authors won't collide?
> No, you'd need to lint the services in the pre-push hook. That would
> not be the biggest deal though, we already authenticate the commits and
> check whether the NEWS file is broken before pushing, for instance. I
> believe it could also be checked without actually instantiating that
> system, but don't quote me on that.
>

Ok, that seems achievable. I would only point out that with a central UID
registry you get that validation "for free" in the form of a Git merge
conflict.


Toggle quote (58 lines)
> > > > > From the solutions we do have so far, I believe that making
> > > user
> > > > > accounts an explicit part of service configuration (in what
> > > shape
> > > > > may
> > > > > still be up for debate), with reasonable defaults including
> > > numeric
> > > > > UIDs and GIDs (at least) for essential services such as GDM
> > > sounds
> > > > > like
> > > > > the best option. WDYT?
> > > > >
> > > > > Regards,
> > > > > Leo
> > > >
> > > > That seems reasonable to me. As for representation, I think
> > > there's
> > > > value decoupling these settings from a service's own config so
> > > that
> > > > support for custom UIDs/GIDs remains consistent across services.
> > > In this case I'd agree with Danny, that asking users to update two
> > > fields to get one service working puts an excessive burden on
> > > them.
> >
> > Leo, I'm not sure what you mean by updating two fields. Are you
> > saying that some services already permit manual selection of UIDs? I
> > was proposing setting that value in the "users" section of operating-
> > system (or elsewhere) rather than setting it in the "services"
> > section, but not both places.
> As far as I'm aware, no service so far do that, but there are some,
> that don't set up the user (e.g. mpd). However, I don't think, that's
> the way to go for services like gdm. If you decoupled the gdm user and
> group from its service specification, you'd need to modify three
> operating-system fields to add gdm as opposed to one. If the gdm user
> and group were configuration, you could instead specify them with
> modify-services or gdm-service-type itself.
>
> > Since Guix already uses a central allocator for UIDs and GIDs
> > (implemented using simple counters), I was imagining a model where
> > the decision is still made centrally, but with different inputs: 1) a
> > global mapping from user/group name to default ID; and 2) a similar
> > name-to-ID mapping in operating-system where users specify their
> > overrides.
> I'm not sure how well that'd work together with account-service-type.
> You'd have to find a novel way of extending it, that's for sure.
>
> > > To
> > > be fair, I don't even necessarily think, that making the full
> > > account
> > > configurable is a good idea, unless someone wants to argue, that
> > > there's value in potentially giving those accounts shell access.
> >
> > The thought of making other parts of the account configurable
> > occurred to me, but I couldn't come up with any serious use cases
> > either.
> As far as I'm aware, nologin exists for a good reason.
>

No arguments there. :) I thought your point was that we don't have a
compelling reason to let the end user replace it with something else.
Attachment: file
L
L
Ludovic Courtès wrote on 6 Jan 2021 11:03
Re: bug#45571: Support stable uids and gids for all accounts
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
87pn2io013.fsf@gnu.org
Hi Danny,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

Toggle quote (8 lines)
> It's easily possible to recreate /etc/passwd from scratch if the uids are
> always specified in <user-account>s and thus /etc/passwd would not need to
> be persistent state anymore. Right now everything from /etc/passwd except
> the uid and the comment is already specified in <user-account>.
>
> So Guix can indeed simply do away with the persistent state of
> /etc/passwd--that's why I suggested specifying the uids in the first place.

I don’t think it can: UID/GID allocation is stateful by nature because
you don’t want to reuse a recently-used ID right away. See the
allocation strategy in (gnu build accounts).

Ludo’.
B
B
Brendan Tildesley wrote on 7 Apr 2021 09:13
Support stable uids and gids for system accounts in a container
(name . 45571@debbugs.gnu.org)(address . 45571@debbugs.gnu.org)
887823078.43817.1617779630834@office.mailbox.org
It may be useful to see what Nix does.

Nix has many static id, with the comment:

# IMPORTANT!
# We only add static uids and gids for services where it is not feasible
# to change uids/gids on service start, in example a service with a lot of
# files. Please also check if the service is applicable for systemd's

Attachment: file
A
A
Arun Isaac wrote on 10 Jun 2021 08:02
(address . 45571@debbugs.gnu.org)
878s3ijlzi.fsf@systemreboot.net
Hi all,

For many services, extending the activation-service-type with a gexp
that can chown the necessary files is a good enough solution. This is
how I work around the problem of unstable uids/gids in my guix system
containers.

Regards,
Arun
-----BEGIN PGP SIGNATURE-----

iQFPBAEBCAA5FiEEf3MDQ/Lwnzx3v3nTLiXui2GAK7MFAmDBqvEbHGFydW5pc2Fh
Y0BzeXN0ZW1yZWJvb3QubmV0AAoJEC4l7othgCuzer4H/2xANi0NJxY2hZEsSFUA
xAzOhxWRBSWe/orc6z+1q2+HdbRWJ35ijuc124Ob+Zu9iiO8217/P2oEvAzHB38g
nAZEj+c8yWYjsR2KQUq1xYT74DtoxqmaazwjKDgqogXfl+mXbvTYMWasIubrG55f
+OZxBZkst9utK1xIcPHYi2MQXBNyZrf8aYpwDATudxohNmwraMakjsJPoJDjxs+5
u+MrjH/5eYlCoey16ItNP/UrGf2/l34M7NzMNlgybqKlM/5uSOXgKmIVE6kEDj3S
2DvvxwqjJoaTv4kWAURbvkax/mGZ8/Jo7EFv8/qer8dT6P4Pjbqfp32fExWvW8ZX
akc=
=87w6
-----END PGP SIGNATURE-----

?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 45571@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 45571
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch