gitolite broken: created repositories keep references to /usr/bin for hooks

OpenSubmitted by ng0.
Details
6 participants
  • ng0
  • Danny Milosavljevic
  • Thompson, David
  • Efraim Flashner
  • Maxime Devos
  • zimoun
Owner
unassigned
Severity
normal
N
(address . bug-guix@gnu.org)
20170303215819.bttmrfsbhlxyipmy@abyayala
Our gitolite package currently creates all
(including gitolite-admin.git) git repositories with references to
"/usr/bin/perl" as shebang, which makes it completely useless on
serverside.
Given that the server side in the case of a gitolite from Guix runs an
environment where you will not run perl from /usr/bin/, you will have to
change all hooks manually currently.

When you install perl into the profile of the user which hosts the
repositories and change the shebangs, gitolite can be used.
N
Re: bug#25957: Acknowledgement (gitolite broken: created repositories keep references to /usr/bin for hooks)
(address . 25957@debbugs.gnu.org)
20170303222743.wf777eedaauuof3f@abyayala
What makes this worse, with every update (push) of gitolite-admin
repository the shebang of "hooks/update" is reset.
Other repositories seem to keep changes in the hooks shebangs so
far.
N
(address . 25957@debbugs.gnu.org)
20170304133242.towlmzdcm6x43hvi@abyayala
On 17-03-03 22:27:43, ng0 wrote:
Toggle quote (8 lines)
> What makes this worse, with every update (push) of gitolite-admin
> repository the shebang of "hooks/update" is reset.
> Other repositories seem to keep changes in the hooks shebangs so
> far.
>
>
>

When I build gitolite from guix, this looks trivial to fix.

[user@abyayala /gnu/store/jw252kw9blfh1lrrib3yk4fkbj5mvdpm-gitolite-3.6.5/share/gitolite]$ egrep -nr "/usr/"
commands/svnserve:9:$svnserve ||= "/usr/bin/svnserve -r /var/svn/ -t --tunnel-user=%u";
lib/Gitolite/Test/Tsh.pm:42:# path when cwd is [...] at /usr/share/perl5/File/Temp.pm line 902".
lib/Gitolite/Hooks/PostUpdate.pm:62:#!/usr/bin/perl
lib/Gitolite/Hooks/Update.pm:158:#!/usr/bin/perl
lib/Gitolite/Cache.pm:127: open( REDIS, "|-", "/usr/sbin/redis-server", "-" ) or die "start redis server fail
ed: $!";


The parts I want to fix as my immediately affect every user, are in
the directory "lib/Gitolite/Hooks/", I have no idea about redis, but
I think there should be a reference to /gnu/store/ reddis and not
"/usr/sbin/redis-server". Different problem, related bug.. This can be
solved in a commit after this bug.
D
D
Danny Milosavljevic wrote on 4 Mar 2017 16:43
(name . ng0)(address . contact.ng0@cryptolab.net)(address . 25957@debbugs.gnu.org)
20170304164309.08e43b4c@scratchpost.org
Hi ng0,

Toggle quote (4 lines)
> I think there should be a reference to /gnu/store/ reddis and not
> "/usr/sbin/redis-server". Different problem, related bug.. This can be
> solved in a commit after this bug.

Yeah.

I would question why a normal application needs to start a redis *server* in the first place. Sounds strange to me. But I agree that if it wants to do that it should use a store reference.

https://redis.io/topics/admin says "Redis is designed to be a very long running process in your server" so that definitely reads to me that a normal program shouldn't just start redis-server when it feels like it (and I hope it stops it again later? After reading the source code it doesn't appear that way...).

http://gitolite.com/gitolite/cache.html says "WARNING: this has not been tested in a while. YMMV". Uhhhh. Not confidence-inspiring.
N
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)(address . 25957@debbugs.gnu.org)
20170304173339.ubumkmfdfrkbascj@abyayala
On 17-03-04 16:43:09, Danny Milosavljevic wrote:
Toggle quote (14 lines)
> Hi ng0,
>
> > I think there should be a reference to /gnu/store/ reddis and not
> > "/usr/sbin/redis-server". Different problem, related bug.. This can be
> > solved in a commit after this bug.
>
> Yeah.
>
> I would question why a normal application needs to start a redis *server* in the first place. Sounds strange to me. But I agree that if it wants to do that it should use a store reference.
>
> <https://redis.io/topics/admin> says "Redis is designed to be a very long running process in your server" so that definitely reads to me that a normal program shouldn't just start redis-server when it feels like it (and I hope it stops it again later? After reading the source code it doesn't appear that way...).
>
> <http://gitolite.com/gitolite/cache.html> says "WARNING: this has not been tested in a while. YMMV". Uhhhh. Not confidence-inspiring.

It was the first time I read about reddis in gitolite context, and in
all the time I used gitolite I never really needed it when building or
running.
I disregard this as not very important and not really important at all
to fix. It should be fixed in the long run, but my main concern was
usability of gitolite, which has been addressed in one of the two
patches I've sent.
Z
Z
zimoun wrote on 5 Jan 01:07 +0100
Re: bug#25957: gitolite broken: created repositories keep references to /usr/bin for hooks
(address . 25957@debbugs.gnu.org)
86k0ff9has.fsf_-_@gmail.com
Hi,

This old bug [1] is still relevant.


On Sat, 04 Mar 2017 at 13:32, ng0 <contact.ng0@cryptolab.net> wrote:
Toggle quote (2 lines)
> On 17-03-03 22:27:43, ng0 wrote:

This previous…

Toggle quote (8 lines)
> [user@abyayala /gnu/store/jw252kw9blfh1lrrib3yk4fkbj5mvdpm-gitolite-3.6.5/share/gitolite]$ egrep -nr "/usr/"
> commands/svnserve:9:$svnserve ||= "/usr/bin/svnserve -r /var/svn/ -t --tunnel-user=%u";
> lib/Gitolite/Test/Tsh.pm:42:# path when cwd is [...] at /usr/share/perl5/File/Temp.pm line 902".
> lib/Gitolite/Hooks/PostUpdate.pm:62:#!/usr/bin/perl
> lib/Gitolite/Hooks/Update.pm:158:#!/usr/bin/perl
> lib/Gitolite/Cache.pm:127: open( REDIS, "|-", "/usr/sbin/redis-server", "-" ) or die "start redis server fail
> ed: $!";

…is now…

Toggle snippet (5 lines)
share/gitolite/lib/Gitolite/Test/Tsh.pm:42:# path when cwd is [...] at /usr/share/perl5/File/Temp.pm line 902".
share/gitolite/lib/Gitolite/Cache.pm:127: open( REDIS, "|-", "/usr/sbin/redis-server", "-" ) or die "start redis server failed: $!";
share/gitolite/commands/svnserve:9:$svnserve ||= "/usr/bin/svnserve -r /var/svn/ -t --tunnel-user=%u";

…but…

Toggle quote (6 lines)
> The parts I want to fix as my immediately affect every user, are in
> the directory "lib/Gitolite/Hooks/", I have no idea about redis, but
> I think there should be a reference to /gnu/store/ reddis and not
> "/usr/sbin/redis-server". Different problem, related bug.. This can be
> solved in a commit after this bug.

…the package redis is not a dependency of gitolite. Therefore, the
question is: is our Gitolite package working with Redis? Even using the
/usr/bin one? Idem for SVN.

Otherwise, I am favor to remove the 2 “problematic” references. WDYT?


Cheers,
simon
E
E
Efraim Flashner wrote on 12 Jan 19:07 +0100
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 25957@debbugs.gnu.org)
Yd8Y6ScvOqyczm2A@3900XT
On Wed, Jan 05, 2022 at 01:07:07AM +0100, zimoun wrote:
Toggle quote (41 lines)
> Hi,
>
> This old bug [1] is still relevant.
>
> 1: <http://issues.guix.gnu.org/issue/25957>
>
> On Sat, 04 Mar 2017 at 13:32, ng0 <contact.ng0@cryptolab.net> wrote:
> > On 17-03-03 22:27:43, ng0 wrote:
>
> This previous…
>
> > [user@abyayala /gnu/store/jw252kw9blfh1lrrib3yk4fkbj5mvdpm-gitolite-3.6.5/share/gitolite]$ egrep -nr "/usr/"
> > commands/svnserve:9:$svnserve ||= "/usr/bin/svnserve -r /var/svn/ -t --tunnel-user=%u";
> > lib/Gitolite/Test/Tsh.pm:42:# path when cwd is [...] at /usr/share/perl5/File/Temp.pm line 902".
> > lib/Gitolite/Hooks/PostUpdate.pm:62:#!/usr/bin/perl
> > lib/Gitolite/Hooks/Update.pm:158:#!/usr/bin/perl
> > lib/Gitolite/Cache.pm:127: open( REDIS, "|-", "/usr/sbin/redis-server", "-" ) or die "start redis server fail
> > ed: $!";
>
> …is now…
>
> --8<---------------cut here---------------start------------->8---
> share/gitolite/lib/Gitolite/Test/Tsh.pm:42:# path when cwd is [...] at /usr/share/perl5/File/Temp.pm line 902".
> share/gitolite/lib/Gitolite/Cache.pm:127: open( REDIS, "|-", "/usr/sbin/redis-server", "-" ) or die "start redis server failed: $!";
> share/gitolite/commands/svnserve:9:$svnserve ||= "/usr/bin/svnserve -r /var/svn/ -t --tunnel-user=%u";
> --8<---------------cut here---------------end--------------->8---
>
> …but…
>
> > The parts I want to fix as my immediately affect every user, are in
> > the directory "lib/Gitolite/Hooks/", I have no idea about redis, but
> > I think there should be a reference to /gnu/store/ reddis and not
> > "/usr/sbin/redis-server". Different problem, related bug.. This can be
> > solved in a commit after this bug.
>
> …the package redis is not a dependency of gitolite. Therefore, the
> question is: is our Gitolite package working with Redis? Even using the
> /usr/bin one? Idem for SVN.
>
> Otherwise, I am favor to remove the 2 “problematic” references. WDYT?

Or change it to search the $PATH for the binary, so it would just be
'redis-server' or 'svnserve'

--
Efraim Flashner <efraim@flashner.co.il> רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmHfGOgACgkQQarn3Mo9
g1H5RQ//UJ90vtM+nXuz+3mhuFap57Df4c1llA3rHCpg9OI/o4ZgBdqF1pO8Shcp
iy8S/MVOsfpOzTC3Xu88wA5Z03sEmEGRNTPnZKi1dEng5SYIczy9BIsQcxhL/xuy
XEJg7FuFWLX2N1beREWlAAcWyB12mu2I0AeRS+5yqY4AClUeHdJ+iR8kN1BrOTy/
1hTeGJDPNbxMqDAfdLtWcdX4G4PRhpfykAOyGZmj6DrFNczgs+RB/vCMWTAoEKcx
uuvinG/WFohduA4hbtgczQPbJ3P1cgQQl5o/1qUdgzWOK6bD9EUXhUhiQFpRyDXg
lu0pnGTRXJ+Ac6yp7/N5oRPiSqJY8bR43FSbjr6dSuXTt71HXzL0bDtjroOLdHi7
S/72adidesoXFtYiGUnMu2vGYrr3mDEUHRqMiccbNifaPthJwSpI1vbTe5pHUWw5
sT0s48DXCy7MDqHZ4gEpVpk8NFBBxMCVAPHERxSQihg45ca6DnIptwGSilntm0Xz
zI474GcJi2XdD1pQgJ3QsS40G1/RlgfEjgGgd/Y+gILDFS2BQKI4LjmTvNwVVn4B
ONjL1tvwAabnKnmeDgEQSvQz1IedgJVdSqPCIts/0qMAkFASygzS14TL171lzfCR
GoLH1rjcBujj0iF683SGET38r30kGqsnCo34vjhSAxeFoal2zO0=
=caCW
-----END PGP SIGNATURE-----


Z
Z
zimoun wrote on 3 Feb 03:49 +0100
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 25957@debbugs.gnu.org)
8635l01x7a.fsf@gmail.com
Hi Efraim,

On Wed, 12 Jan 2022 at 20:07, Efraim Flashner <efraim@flashner.co.il> wrote:

Toggle quote (9 lines)
>> …the package redis is not a dependency of gitolite. Therefore, the
>> question is: is our Gitolite package working with Redis? Even using the
>> /usr/bin one? Idem for SVN.
>>
>> Otherwise, I am favor to remove the 2 “problematic” references. WDYT?
>
> Or change it to search the $PATH for the binary, so it would just be
> 'redis-server' or 'svnserve'

Is our Gitolite package working with Redis? If not, why try to fix. ;-)


Cheers,
simon
Z
Z
zimoun wrote on 23 Mar 11:45 +0100
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 25957@debbugs.gnu.org)
86lex10wwr.fsf@gmail.com
Hi,

On Thu, 03 Feb 2022 at 03:49, zimoun <zimon.toutoune@gmail.com> wrote:
Toggle quote (13 lines)
> On Wed, 12 Jan 2022 at 20:07, Efraim Flashner <efraim@flashner.co.il> wrote:
>
>>> …the package redis is not a dependency of gitolite. Therefore, the
>>> question is: is our Gitolite package working with Redis? Even using the
>>> /usr/bin one? Idem for SVN.
>>>
>>> Otherwise, I am favor to remove the 2 “problematic” references. WDYT?
>>
>> Or change it to search the $PATH for the binary, so it would just be
>> 'redis-server' or 'svnserve'
>
> Is our Gitolite package working with Redis? If not, why try to fix. ;-)

What is the status of this old bug [1]? Is it actionable? If yes, what
is the action? If no, let close it. :-) WDYT?




Cheers,
simon
M
M
Maxime Devos wrote on 23 Mar 13:44 +0100
(address . 25957@debbugs.gnu.org)
6a325301e7cc55ee08652c67e49c3eb8a0802baa.camel@telenet.be
zimoun schreef op wo 23-03-2022 om 11:45 [+0100]:
Toggle quote (23 lines)
> > On Wed, 12 Jan 2022 at 20:07, Efraim Flashner
> > <efraim@flashner.co.il> wrote:
> >
> > > > …the package redis is not a dependency of gitolite.  Therefore,
> > > > the
> > > > question is: is our Gitolite package working with Redis?  Even
> > > > using the
> > > > /usr/bin one?  Idem for SVN.
> > > >
> > > > Otherwise, I am favor to remove the 2 “problematic”
> > > > references.   WDYT?
> > >
> > > Or change it to search the $PATH for the binary, so it would just
> > > be
> > > 'redis-server' or 'svnserve'
> >
> > Is our Gitolite package working with Redis?  If not, why try to
> > fix. ;-)
>
> What is the status of this old bug [1]?  Is it actionable?  If yes,
> what
> is the action?  If no, let close it. :-)  WDYT?

Seems like all we have to do is 'substitute*' a '/usr/bin/svnserve'
into a '/gnu/store/...' (untested), so seems actionable to me.
Alternatively, as Efraim wrote, let it search the $PATH (that might be
useful if adding svnserve would increase the closure too much and it is
an optional dependency in practice?).

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYjsWLRccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7i9eAQC7udOOdgxpn6huFy0Pq6TulGQr
rM8xLyTjPS/y6olwAgEAvrKQqgpEJv4qd5J0k59AI3N34xLGk68jzBMn2eQrMQg=
=JfJY
-----END PGP SIGNATURE-----


E
E
Efraim Flashner wrote on 28 Mar 08:49 +0200
(name . Maxime Devos)(address . maximedevos@telenet.be)
YkFaedjhpznFqbNM@3900XT
On Wed, Mar 23, 2022 at 01:44:29PM +0100, Maxime Devos wrote:
Toggle quote (30 lines)
> zimoun schreef op wo 23-03-2022 om 11:45 [+0100]:
> > > On Wed, 12 Jan 2022 at 20:07, Efraim Flashner
> > > <efraim@flashner.co.il> wrote:
> > >
> > > > > …the package redis is not a dependency of gitolite.  Therefore,
> > > > > the
> > > > > question is: is our Gitolite package working with Redis?  Even
> > > > > using the
> > > > > /usr/bin one?  Idem for SVN.
> > > > >
> > > > > Otherwise, I am favor to remove the 2 “problematic”
> > > > > references.   WDYT?
> > > >
> > > > Or change it to search the $PATH for the binary, so it would just
> > > > be
> > > > 'redis-server' or 'svnserve'
> > >
> > > Is our Gitolite package working with Redis?  If not, why try to
> > > fix. ;-)
> >
> > What is the status of this old bug [1]?  Is it actionable?  If yes,
> > what
> > is the action?  If no, let close it. :-)  WDYT?
>
> Seems like all we have to do is 'substitute*' a '/usr/bin/svnserve'
> into a '/gnu/store/...' (untested), so seems actionable to me.
> Alternatively, as Efraim wrote, let it search the $PATH (that might be
> useful if adding svnserve would increase the closure too much and it is
> an optional dependency in practice?).

I spent some time looking at gitolite and the service. As I understand
it, with the exception of svnserve, it searches $PATH for a number of
different binaries, including git-annex. I believe that this would only
work if git-annex (and potentially other packages) are installed
globally.

In addition, git (not git-minimal) and openssh are propagated inputs AND
wrapped. I haven't tested to see if wrapping only is enough.

I think the best choice is to:
A: Replace /usr/bin/svnserve with svnserve so it will just search $PATH,
like it does with the other helpers.
B: Adjust the service so that it automatically creates a variant (or
just a wrapped version) of the package which is wrapped with a list of
additional packages so that they can be in gitolite's path. If I were
deploying this to an arm device I wouldn't want it wrapped with
git-annex since it doesn't build, but would definitely want it for an
x86_64 machine.

I suppose we should try to find someone who is using the gitolite
service and see if they can be our test subject for wrapping the package
with optional addons.

--
Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmJBWnIACgkQQarn3Mo9
g1F36A//TNGr5K6tptHG5aR4e6/StBJM5qrUJb/ldVQZ8yr0LpHQZF+PgHsdOX3S
/thtlEwQuqVTnaBptF+9w8R35IowpDc1fwwQgy5zgKpFdH1r90ufFaNFm2YJ6Pep
dLxpRyr3hxc3bOk6/B7cEfyBbD2A56qh4pLcGUrSbmfOH1VYZDKWNonBz1zZd4PC
oyViPxFLWdXxblaSLo2CCxfKcsM0jmBV7VHgfOYaTrSvr+UStiKLf1Doib4Hfz2V
ZnPq2oTv+tTr4gI7dSF5eOu4UtK45KcFLwuPWDNBw3s9dDeGcZ8ly6snbPFRhevU
mAHo7rqkn1rpQLxAUYSZv35eZnpj2953TtekdCacHtrw+I9AhhBNwLhWJg3FF5By
KkG9jbLp1d64UdTDCrxCte5XacS3mailiW0Oh2wzr/R5W42CntrVGPK1PSduyCe1
EzbkdnGd3lLpmk6IDhTrjh6vGq07QdsyH+i9vfEsTboM2sbAUeT4IQagYAYL4Q0H
WFatKL4I00BUYI/cGfaxIO6Cc1566CI4YNkTCGf0Y6nOKMMjllZdOGXsQ3D/cm3T
nyBDsIZc2PPSkC63CN9KNz3XWFiRqw4i9pmLe0KLTOsL8i+goYkSYxkWlI8rdqad
HP75zsuJ8+uK/06SpwPRbodCr1CzGzMtCnXZjFhFAdL8K3zIF90=
=KBfr
-----END PGP SIGNATURE-----


T
T
Thompson, David wrote on 1 Sep 15:59 +0200
Re: [EXT] bug#25957: gitolite broken: created repositories keep references to /usr/bin for hooks
CAJ=RwfZ59KTZ=9z3j6tF9AcGteW+jmaDphqy_XD4uDXB_XbyQA@mail.gmail.com
Hi all,

Reviving this old thread.

On Mon, Mar 28, 2022 at 2:51 AM Efraim Flashner <efraim@flashner.co.il> wrote:
Toggle quote (20 lines)
> >
> > Seems like all we have to do is 'substitute*' a '/usr/bin/svnserve'
> > into a '/gnu/store/...' (untested), so seems actionable to me.
> > Alternatively, as Efraim wrote, let it search the $PATH (that might be
> > useful if adding svnserve would increase the closure too much and it is
> > an optional dependency in practice?).
>
> I spent some time looking at gitolite and the service. As I understand
> it, with the exception of svnserve, it searches $PATH for a number of
> different binaries, including git-annex. I believe that this would only
> work if git-annex (and potentially other packages) are installed
> globally.
>
> In addition, git (not git-minimal) and openssh are propagated inputs AND
> wrapped. I haven't tested to see if wrapping only is enough.
>
> I think the best choice is to:
> A: Replace /usr/bin/svnserve with svnserve so it will just search $PATH,
> like it does with the other helpers.

I see that you have done this. Thanks! We could also replace the
reference to /usr/sbin/redis-server in src/lib/Gitolite/Cache.pm.
That's the only other /usr reference I can find (that isn't in a
comment) in the output. I have the patch ready if that sounds good to
you.

Toggle quote (7 lines)
> B: Adjust the service so that it automatically creates a variant (or
> just a wrapped version) of the package which is wrapped with a list of
> additional packages so that they can be in gitolite's path. If I were
> deploying this to an arm device I wouldn't want it wrapped with
> git-annex since it doesn't build, but would definitely want it for an
> x86_64 machine.

The service configuration record could accept a list of addons like
'(git-annex cache svnserve), with a default of no addons '(), and
create a package that extends the gitolite package with the
appropriate propagated inputs. Does that sound like what you had in
mind? A more robust solution could modify the build to hardcode the
store paths needed for the add-ons but given that we already propagate
git and openssh I don't think it's necessary right now.

Toggle quote (4 lines)
> I suppose we should try to find someone who is using the gitolite
> service and see if they can be our test subject for wrapping the package
> with optional addons.

I use the gitolite service and can be the test subject. I don't
currently use any add-ons, but the redis one sounds easy enough to try
and hey maybe it's a good excuse to finally learn how to use
git-annex.

As a longer term thing, it would be cool to revisit propagating git
and openssh in this package. I punted on it back in 2015 for the
reason stated in the source comments but maybe there's a reasonable
and reliable way to directly embed the store paths now.

- Dave
E
E
Efraim Flashner wrote on 1 Sep 16:20 +0200
(name . Thompson, David)(address . dthompson2@worcester.edu)
YxC/kT5xhlVMbvEm@3900XT
On Thu, Sep 01, 2022 at 09:59:55AM -0400, Thompson, David wrote:
Toggle quote (31 lines)
> Hi all,
>
> Reviving this old thread.
>
> On Mon, Mar 28, 2022 at 2:51 AM Efraim Flashner <efraim@flashner.co.il> wrote:
> > >
> > > Seems like all we have to do is 'substitute*' a '/usr/bin/svnserve'
> > > into a '/gnu/store/...' (untested), so seems actionable to me.
> > > Alternatively, as Efraim wrote, let it search the $PATH (that might be
> > > useful if adding svnserve would increase the closure too much and it is
> > > an optional dependency in practice?).
> >
> > I spent some time looking at gitolite and the service. As I understand
> > it, with the exception of svnserve, it searches $PATH for a number of
> > different binaries, including git-annex. I believe that this would only
> > work if git-annex (and potentially other packages) are installed
> > globally.
> >
> > In addition, git (not git-minimal) and openssh are propagated inputs AND
> > wrapped. I haven't tested to see if wrapping only is enough.
> >
> > I think the best choice is to:
> > A: Replace /usr/bin/svnserve with svnserve so it will just search $PATH,
> > like it does with the other helpers.
>
> I see that you have done this. Thanks! We could also replace the
> reference to /usr/sbin/redis-server in src/lib/Gitolite/Cache.pm.
> That's the only other /usr reference I can find (that isn't in a
> comment) in the output. I have the patch ready if that sounds good to
> you.

Sounds good to me

Toggle quote (15 lines)
> > B: Adjust the service so that it automatically creates a variant (or
> > just a wrapped version) of the package which is wrapped with a list of
> > additional packages so that they can be in gitolite's path. If I were
> > deploying this to an arm device I wouldn't want it wrapped with
> > git-annex since it doesn't build, but would definitely want it for an
> > x86_64 machine.
>
> The service configuration record could accept a list of addons like
> '(git-annex cache svnserve), with a default of no addons '(), and
> create a package that extends the gitolite package with the
> appropriate propagated inputs. Does that sound like what you had in
> mind? A more robust solution could modify the build to hardcode the
> store paths needed for the add-ons but given that we already propagate
> git and openssh I don't think it's necessary right now.

Assuming this is deployed into some sort of container then propagated
inputs wouldn't help much, we'd need either the PATH for the container
to be extended to include those extra packages or to have gitolite
itself wrapped similar to icedove/wayland. Just extending the PATH in
the #:environment-variables would be enough I'd think.

Toggle quote (14 lines)
> > I suppose we should try to find someone who is using the gitolite
> > service and see if they can be our test subject for wrapping the package
> > with optional addons.
>
> I use the gitolite service and can be the test subject. I don't
> currently use any add-ons, but the redis one sounds easy enough to try
> and hey maybe it's a good excuse to finally learn how to use
> git-annex.
>
> As a longer term thing, it would be cool to revisit propagating git
> and openssh in this package. I punted on it back in 2015 for the
> reason stated in the source comments but maybe there's a reasonable
> and reliable way to directly embed the store paths now.

It's actually been forever since I looked at gitolite so I don't know
remember what those inputs were needed for, but it'd be great to improve
the service.

Toggle quote (2 lines)
> - Dave

Interestingly, I almost have a working ghc-8.6 for aarch64 after all
these years.

--
Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmMQv5EACgkQQarn3Mo9
g1E8nQ//Xj8Mzpi/6jL0+Ci+XmXAUOclrz/aNNsp2gzbRLzSCN76eW0d7ejVzVTj
F/kQdy3EkEwg1OXgEZhvlkWx3qUY5Tma840llz0h+IhTIRax/KCXfbT6Sk/m27G8
rD2XHSirW35YmJuQlHadtq1uzUDxWLCGxY9r32G1xp+laorujSjVMeHTsnAoD1gS
BpnieOvgD91DTk7zHJYtGCfavMRiuJOij/uS+9fs+oM8c17FcTaG4bndlxmy1F6+
5BudwiY/HjzVn+jvbLsY/vMpSmGxaexQyJKQVJS792wnFF5sNRC+QOTmR0UztU6m
mI9DCTPqHejR0i1jncnHKjUmI855avcEL8evNsjvjmF2NuWVb6YiwEtPuSpoCVn+
tLGnr4SceOgiRiXKLLOAzIA/JdcLmvkK6F15YhoBZSZEwsOAt5AVFXcOqiZ+FL6h
/gXmGOc5Wrc9SuX7Ssb7YnKRXbZMq2rzj2Uu0J4+hPx9d27nn/HRoYZRvsLQjSNk
+6W9R4DJc7clyu0dQ4qA/IynUud/UJ81GAVbqzT/Zcym7HQPMSM2hzLB28sluPag
w0/mUoMwLEznjW+n3+tYbH6J+obF70jep70RTO6ldM4dHC1D8udWVLo9PsfP1tRb
Eb1o967VkUg/rFWWL0ZexOip7lGs0s2LsSSKHyDirWFaRHvakCw=
=TD+i
-----END PGP SIGNATURE-----


T
T
Thompson, David wrote on 1 Sep 16:41 +0200
CAJ=Rwfb-3q3EMQTCdRV9Qz+_r5LPtN+7Hs-D-piD-mvQekamyg@mail.gmail.com
Hi Efraim,

On Thu, Sep 1, 2022 at 10:20 AM Efraim Flashner <efraim@flashner.co.il> wrote:
Toggle quote (35 lines)
>
> On Thu, Sep 01, 2022 at 09:59:55AM -0400, Thompson, David wrote:
> > Hi all,
> >
> > Reviving this old thread.
> >
> > On Mon, Mar 28, 2022 at 2:51 AM Efraim Flashner <efraim@flashner.co.il> wrote:
> > > >
> > > > Seems like all we have to do is 'substitute*' a '/usr/bin/svnserve'
> > > > into a '/gnu/store/...' (untested), so seems actionable to me.
> > > > Alternatively, as Efraim wrote, let it search the $PATH (that might be
> > > > useful if adding svnserve would increase the closure too much and it is
> > > > an optional dependency in practice?).
> > >
> > > I spent some time looking at gitolite and the service. As I understand
> > > it, with the exception of svnserve, it searches $PATH for a number of
> > > different binaries, including git-annex. I believe that this would only
> > > work if git-annex (and potentially other packages) are installed
> > > globally.
> > >
> > > In addition, git (not git-minimal) and openssh are propagated inputs AND
> > > wrapped. I haven't tested to see if wrapping only is enough.
> > >
> > > I think the best choice is to:
> > > A: Replace /usr/bin/svnserve with svnserve so it will just search $PATH,
> > > like it does with the other helpers.
> >
> > I see that you have done this. Thanks! We could also replace the
> > reference to /usr/sbin/redis-server in src/lib/Gitolite/Cache.pm.
> > That's the only other /usr reference I can find (that isn't in a
> > comment) in the output. I have the patch ready if that sounds good to
> > you.
>
> Sounds good to me

Thanks, pushed as commit c053dfa52dc778eb3d965f58a85c435ae7fab0dd.

Toggle quote (21 lines)
> > > B: Adjust the service so that it automatically creates a variant (or
> > > just a wrapped version) of the package which is wrapped with a list of
> > > additional packages so that they can be in gitolite's path. If I were
> > > deploying this to an arm device I wouldn't want it wrapped with
> > > git-annex since it doesn't build, but would definitely want it for an
> > > x86_64 machine.
> >
> > The service configuration record could accept a list of addons like
> > '(git-annex cache svnserve), with a default of no addons '(), and
> > create a package that extends the gitolite package with the
> > appropriate propagated inputs. Does that sound like what you had in
> > mind? A more robust solution could modify the build to hardcode the
> > store paths needed for the add-ons but given that we already propagate
> > git and openssh I don't think it's necessary right now.
>
> Assuming this is deployed into some sort of container then propagated
> inputs wouldn't help much, we'd need either the PATH for the container
> to be extended to include those extra packages or to have gitolite
> itself wrapped similar to icedove/wayland. Just extending the PATH in
> the #:environment-variables would be enough I'd think.

Hmm, I hadn't thought about the container use case. Your approach
sounds like the way to go. For what it's worth, I think the gitolite
service as-is would suffer the same issue in a containerized
environment because it relies upon the git and openssh propagated
inputs to do anything at all. With the gitolite service in my system,
/run/current-system/profile/bin has both git and ssh in it due to the
propagation. So it sounds like there's 2 steps needed: 1) Use a
wrapper like icedove/wayland for the base gitolite package so that git
and openssh no longer need propagation, and then 2) extend the
gitolite service to wrap up additional packages needed for the desired
extensions. Sound good?

Toggle quote (18 lines)
> > > I suppose we should try to find someone who is using the gitolite
> > > service and see if they can be our test subject for wrapping the package
> > > with optional addons.
> >
> > I use the gitolite service and can be the test subject. I don't
> > currently use any add-ons, but the redis one sounds easy enough to try
> > and hey maybe it's a good excuse to finally learn how to use
> > git-annex.
> >
> > As a longer term thing, it would be cool to revisit propagating git
> > and openssh in this package. I punted on it back in 2015 for the
> > reason stated in the source comments but maybe there's a reasonable
> > and reliable way to directly embed the store paths now.
>
> It's actually been forever since I looked at gitolite so I don't know
> remember what those inputs were needed for, but it'd be great to improve
> the service.

Are you referring to git and openssh or redis, svnserve, git-annex,
etc.? I'm no expert and I really don't like Perl, but I know gitolite
well enough to explain some of this stuff.

Toggle quote (3 lines)
> Interestingly, I almost have a working ghc-8.6 for aarch64 after all
> these years.

Some things move at a glacial pace, but eventually they get done.
Best of luck wrapping that up. :)

- Dave
Z
Z
zimoun wrote on 1 Sep 19:00 +0200
Re: bug#25957: [EXT] Re: [EXT] bug#25957: gitolite broken: created repositories keep references to /usr/bin for hooks
874jxruj0i.fsf@gmail.com
Hi,

On jeu., 01 sept. 2022 at 10:41, "Thompson, David" <dthompson2@worcester.edu> wrote:

Toggle quote (2 lines)
> Thanks, pushed as commit c053dfa52dc778eb3d965f58a85c435ae7fab0dd.

Cool! Thank you.


Cheers,
simon
E
E
Efraim Flashner wrote on 2 Sep 08:56 +0200
Re: [EXT] Re: [EXT] bug#25957: gitolite broken: created repositories keep references to /usr/bin for hooks
(name . Thompson, David)(address . dthompson2@worcester.edu)
YxGpMTiVE1x4oSAy@3900XT
On Thu, Sep 01, 2022 at 10:41:21AM -0400, Thompson, David wrote:
Toggle quote (101 lines)
> Hi Efraim,
>
> On Thu, Sep 1, 2022 at 10:20 AM Efraim Flashner <efraim@flashner.co.il> wrote:
> >
> > On Thu, Sep 01, 2022 at 09:59:55AM -0400, Thompson, David wrote:
> > > Hi all,
> > >
> > > Reviving this old thread.
> > >
> > > On Mon, Mar 28, 2022 at 2:51 AM Efraim Flashner <efraim@flashner.co.il> wrote:
> > > > >
> > > > > Seems like all we have to do is 'substitute*' a '/usr/bin/svnserve'
> > > > > into a '/gnu/store/...' (untested), so seems actionable to me.
> > > > > Alternatively, as Efraim wrote, let it search the $PATH (that might be
> > > > > useful if adding svnserve would increase the closure too much and it is
> > > > > an optional dependency in practice?).
> > > >
> > > > I spent some time looking at gitolite and the service. As I understand
> > > > it, with the exception of svnserve, it searches $PATH for a number of
> > > > different binaries, including git-annex. I believe that this would only
> > > > work if git-annex (and potentially other packages) are installed
> > > > globally.
> > > >
> > > > In addition, git (not git-minimal) and openssh are propagated inputs AND
> > > > wrapped. I haven't tested to see if wrapping only is enough.
> > > >
> > > > I think the best choice is to:
> > > > A: Replace /usr/bin/svnserve with svnserve so it will just search $PATH,
> > > > like it does with the other helpers.
> > >
> > > I see that you have done this. Thanks! We could also replace the
> > > reference to /usr/sbin/redis-server in src/lib/Gitolite/Cache.pm.
> > > That's the only other /usr reference I can find (that isn't in a
> > > comment) in the output. I have the patch ready if that sounds good to
> > > you.
> >
> > Sounds good to me
>
> Thanks, pushed as commit c053dfa52dc778eb3d965f58a85c435ae7fab0dd.
>
> > > > B: Adjust the service so that it automatically creates a variant (or
> > > > just a wrapped version) of the package which is wrapped with a list of
> > > > additional packages so that they can be in gitolite's path. If I were
> > > > deploying this to an arm device I wouldn't want it wrapped with
> > > > git-annex since it doesn't build, but would definitely want it for an
> > > > x86_64 machine.
> > >
> > > The service configuration record could accept a list of addons like
> > > '(git-annex cache svnserve), with a default of no addons '(), and
> > > create a package that extends the gitolite package with the
> > > appropriate propagated inputs. Does that sound like what you had in
> > > mind? A more robust solution could modify the build to hardcode the
> > > store paths needed for the add-ons but given that we already propagate
> > > git and openssh I don't think it's necessary right now.
> >
> > Assuming this is deployed into some sort of container then propagated
> > inputs wouldn't help much, we'd need either the PATH for the container
> > to be extended to include those extra packages or to have gitolite
> > itself wrapped similar to icedove/wayland. Just extending the PATH in
> > the #:environment-variables would be enough I'd think.
>
> Hmm, I hadn't thought about the container use case. Your approach
> sounds like the way to go. For what it's worth, I think the gitolite
> service as-is would suffer the same issue in a containerized
> environment because it relies upon the git and openssh propagated
> inputs to do anything at all. With the gitolite service in my system,
> /run/current-system/profile/bin has both git and ssh in it due to the
> propagation. So it sounds like there's 2 steps needed: 1) Use a
> wrapper like icedove/wayland for the base gitolite package so that git
> and openssh no longer need propagation, and then 2) extend the
> gitolite service to wrap up additional packages needed for the desired
> extensions. Sound good?
>
> > > > I suppose we should try to find someone who is using the gitolite
> > > > service and see if they can be our test subject for wrapping the package
> > > > with optional addons.
> > >
> > > I use the gitolite service and can be the test subject. I don't
> > > currently use any add-ons, but the redis one sounds easy enough to try
> > > and hey maybe it's a good excuse to finally learn how to use
> > > git-annex.
> > >
> > > As a longer term thing, it would be cool to revisit propagating git
> > > and openssh in this package. I punted on it back in 2015 for the
> > > reason stated in the source comments but maybe there's a reasonable
> > > and reliable way to directly embed the store paths now.
> >
> > It's actually been forever since I looked at gitolite so I don't know
> > remember what those inputs were needed for, but it'd be great to improve
> > the service.
>
> Are you referring to git and openssh or redis, svnserve, git-annex,
> etc.? I'm no expert and I really don't like Perl, but I know gitolite
> well enough to explain some of this stuff.
>
> > Interestingly, I almost have a working ghc-8.6 for aarch64 after all
> > these years.
>
> Some things move at a glacial pace, but eventually they get done.
> Best of luck wrapping that up. :)

I took a look at the gitolite service finally and I hadn't realized
there wasn't a running daemon to containerize. I assumed we could do
something like:

(start $~(make-forkexec-constructor/container
(list ...)
#:environment-variables
'("PATH=...")
#:mappings ...))

Given that's not the case then I'd need to look at gitolite itself to
see how it calls the other binaries it expects to be available, and if
wrapping it would be enough or if we would need to just propagate the
other packages for functionality.

--
Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmMRqS4ACgkQQarn3Mo9
g1GJmw//ZKP6b51L8skZwo84dNH4hRRRG4wee4pIPts+EtKaop6g0j2JVvTQTJ7k
0SuBySy6TUTokrNjLpiW0KgqSV5/UZLHqDV8W+an5hC0nxKj/7cux9Qaj1QcZXgq
x5DBlBE7y5XLGmxFtqz/BrdmfINAeuh7xlLxOg0YHxwISQ6XwRlC3X1OGliSTc7d
pCZC+zY0FloethSoFuSy+TWBO3yxm/aE5IevXSGFfjMreD45ma48C5AGwEH5je0C
ghytFcrZxolEelYJdRi62oFsVaQSYnUQKbssyh0E3KigOLDwuLnARmq5NzRyvt4W
N48ViseelBDSeR57Gs7gM+sY5AZQxBSjbesgqEVVGEklwnJTMqecDCP1KxQJI5nz
ywf8FOgQxx0+6Loa5opPumXJRl2QxPs++1aPXsMP+Km5KnDfl6ZjrhASA4JLOm4n
ZrPvHBHnFKj4HGbBdwK2gx6mbziIpFvmxrsYfdPr04YOYpfMJv/vA9OUC7DAiVmF
ncewfG4BAWlofkFKYOYZ9lYCKWZm2xKvYLFnQp5+s0LAMBoaENM5rGpM/7xF+mNJ
bE++fi1JtMK/h15iKJtGaqR9DYi5GZvNEf+bdRjT7u2kP2hbmQUbqSLFwO1nUw0G
xyjhJ0TgNWHp+rTOWMgPF5BuVm0JUvLq5CMDKbbVZvOc48UCWIs=
=7wb3
-----END PGP SIGNATURE-----


T
T
Thompson, David wrote on 2 Sep 13:11 +0200
Re: bug#25957: gitolite broken: created repositories keep references to /usr/bin for hooks
CAJ=Rwfaectmn3h=EW363NQArhsfPS17fWtffp-7md7nYYJ+H1A@mail.gmail.com
On Fri, Sep 2, 2022 at 3:00 AM Efraim Flashner <efraim@flashner.co.il> wrote:
Toggle quote (16 lines)
>
> I took a look at the gitolite service finally and I hadn't realized
> there wasn't a running daemon to containerize. I assumed we could do
> something like:
>
> (start $~(make-forkexec-constructor/container
> (list ...)
> #:environment-variables
> '("PATH=...")
> #:mappings ...))
>
> Given that's not the case then I'd need to look at gitolite itself to
> see how it calls the other binaries it expects to be available, and if
> wrapping it would be enough or if we would need to just propagate the
> other packages for functionality.

Gitolite simply expects tools like git to be on $PATH. It's a pretty
naive system, there's nothing like a configure script that is
determining the absolute file name of these tools and substituting
those names into the built files.

The executable is already wrapped so that coreutils, findutils, and
git are on $PATH, but notably not openssh:

(add-after 'install 'wrap-scripts
(lambda* (#:key inputs outputs #:allow-other-keys)
(let ((out (assoc-ref outputs "out"))
(coreutils (assoc-ref inputs "coreutils"))
(findutils (assoc-ref inputs "findutils"))
(git (assoc-ref inputs "git")))
(wrap-program (string-append out "/bin/gitolite")
`("PATH" ":" prefix
,(map (lambda (dir)
(string-append dir "/bin"))
(list out coreutils findutils git)))))))

However, git and openssh are still propagated inputs. I'm going to
move the propagated inputs to regular inputs, potentially add openssh
to the wrapper once I remind myself what gitolite does with those
tools, and test it all out on my server using the gitolite service.
If that all works, we have a good starting point for adding extension
support in the service.

- Dave
E
E
Efraim Flashner wrote on 2 Sep 14:41 +0200
(name . Thompson, David)(address . dthompson2@worcester.edu)
YxH6Bt7/uF+aMqeW@3900XT
On Fri, Sep 02, 2022 at 07:11:54AM -0400, Thompson, David wrote:
Toggle quote (44 lines)
> On Fri, Sep 2, 2022 at 3:00 AM Efraim Flashner <efraim@flashner.co.il> wrote:
> >
> > I took a look at the gitolite service finally and I hadn't realized
> > there wasn't a running daemon to containerize. I assumed we could do
> > something like:
> >
> > (start $~(make-forkexec-constructor/container
> > (list ...)
> > #:environment-variables
> > '("PATH=...")
> > #:mappings ...))
> >
> > Given that's not the case then I'd need to look at gitolite itself to
> > see how it calls the other binaries it expects to be available, and if
> > wrapping it would be enough or if we would need to just propagate the
> > other packages for functionality.
>
> Gitolite simply expects tools like git to be on $PATH. It's a pretty
> naive system, there's nothing like a configure script that is
> determining the absolute file name of these tools and substituting
> those names into the built files.
>
> The executable is already wrapped so that coreutils, findutils, and
> git are on $PATH, but notably not openssh:
>
> (add-after 'install 'wrap-scripts
> (lambda* (#:key inputs outputs #:allow-other-keys)
> (let ((out (assoc-ref outputs "out"))
> (coreutils (assoc-ref inputs "coreutils"))
> (findutils (assoc-ref inputs "findutils"))
> (git (assoc-ref inputs "git")))
> (wrap-program (string-append out "/bin/gitolite")
> `("PATH" ":" prefix
> ,(map (lambda (dir)
> (string-append dir "/bin"))
> (list out coreutils findutils git)))))))
>
> However, git and openssh are still propagated inputs. I'm going to
> move the propagated inputs to regular inputs, potentially add openssh
> to the wrapper once I remind myself what gitolite does with those
> tools, and test it all out on my server using the gitolite service.
> If that all works, we have a good starting point for adding extension
> support in the service.

I like it. Let us know how it goes.

--
Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmMR+gMACgkQQarn3Mo9
g1FfIBAAmVbRlh96YZnlmbAKijXgMIT8HXSTCgCqpuiUE3LH9HM+vh9pGHO+CSkd
W26HV5G+VCcpzLnSFAnUgpm4W2eMgZDt6IfTG7SzwpZs6z5+Oc9XRuxaqRdoBMiT
zK5ATgZoGkn4bK5yfVC4xTUTyrNJEsS7fZzNm7UnoU83G6GHv3SvoRICL5eWYgf5
CvLVmH5gRYtJSQGKZkh9iCLadGezDT2oyZXK58q0GrWb0M2esG+lVRw5xjLjXrsh
4vy+pACbM4iHg/XJsh6ObcUKDAtybsbaK9Os0uMKnOICC3s2giCLkZ8oU46+o5dT
Jga9j9ObWJlxiLhGzSzPBx2mEAolKjh1ofSe0f1I2/fBU+P+mjvLgKW7aFZIkdY0
ij6iJKGZ58SvbgX3cxptKkyzV8C9AgZUmFsmEOeKAgjxopjjPA3jG87t3Mivtteu
p1xWtimunzMFbNURB9/rMrK5Q9BqbB3cgTa4XJZPNMg7B44pzY0pnXt3F3IxMxea
dj5R/ceFSJclu9BSmZyRU7ZKMRrRnXshZd7bmxBqf+Qupq16qYvQhXFlMoxkOhdy
0bqswHRQ9YLgscXPlb2hmvNuceXEOm0dpHUa9V4aElh37zPO7fBIcZchfOnab/h3
7zVEfZZlIS5o2yunnuUmtt9TmeXx4bmp5cSGZ2Q9lIbQSYdPpyU=
=dCOk
-----END PGP SIGNATURE-----


T
T
Thompson, David wrote on 2 Sep 14:50 +0200
CAJ=RwfZKbTS5VAAorC8G4OVo=Rvs6PazqD1yTxOJ5-vWCc=Zcw@mail.gmail.com
On Fri, Sep 2, 2022 at 8:44 AM Efraim Flashner <efraim@flashner.co.il> wrote:
Toggle quote (48 lines)
>
> On Fri, Sep 02, 2022 at 07:11:54AM -0400, Thompson, David wrote:
> > On Fri, Sep 2, 2022 at 3:00 AM Efraim Flashner <efraim@flashner.co.il> wrote:
> > >
> > > I took a look at the gitolite service finally and I hadn't realized
> > > there wasn't a running daemon to containerize. I assumed we could do
> > > something like:
> > >
> > > (start $~(make-forkexec-constructor/container
> > > (list ...)
> > > #:environment-variables
> > > '("PATH=...")
> > > #:mappings ...))
> > >
> > > Given that's not the case then I'd need to look at gitolite itself to
> > > see how it calls the other binaries it expects to be available, and if
> > > wrapping it would be enough or if we would need to just propagate the
> > > other packages for functionality.
> >
> > Gitolite simply expects tools like git to be on $PATH. It's a pretty
> > naive system, there's nothing like a configure script that is
> > determining the absolute file name of these tools and substituting
> > those names into the built files.
> >
> > The executable is already wrapped so that coreutils, findutils, and
> > git are on $PATH, but notably not openssh:
> >
> > (add-after 'install 'wrap-scripts
> > (lambda* (#:key inputs outputs #:allow-other-keys)
> > (let ((out (assoc-ref outputs "out"))
> > (coreutils (assoc-ref inputs "coreutils"))
> > (findutils (assoc-ref inputs "findutils"))
> > (git (assoc-ref inputs "git")))
> > (wrap-program (string-append out "/bin/gitolite")
> > `("PATH" ":" prefix
> > ,(map (lambda (dir)
> > (string-append dir "/bin"))
> > (list out coreutils findutils git)))))))
> >
> > However, git and openssh are still propagated inputs. I'm going to
> > move the propagated inputs to regular inputs, potentially add openssh
> > to the wrapper once I remind myself what gitolite does with those
> > tools, and test it all out on my server using the gitolite service.
> > If that all works, we have a good starting point for adding extension
> > support in the service.
>
> I like it. Let us know how it goes.

The problem is that gitolite generates git hooks for the repositories
that it manages, and those hooks invoke git, so the only way those
scripts will be able to work (without input propagation) is to find a
way to inject the proper PATH or find a way to replace references to
things like 'git diff' with '/gnu/store/.../git diff'. I'm going to
keep exploring and report back when I have something to show.

- Dave
T
T
Thompson, David wrote on 2 Sep 21:58 +0200
Re: bug#25957: gitolite broken: created repositories keep references to /usr/bin for hooks
CAJ=RwfbPsTMdVPtUc0QeiPqYq3sFQPhGGvL+p2W7Dc0cicqf9w@mail.gmail.com
On Fri, Sep 2, 2022 at 8:50 AM Thompson, David <dthompson2@worcester.edu> wrote:
Toggle quote (57 lines)
>
> On Fri, Sep 2, 2022 at 8:44 AM Efraim Flashner <efraim@flashner.co.il> wrote:
> >
> > On Fri, Sep 02, 2022 at 07:11:54AM -0400, Thompson, David wrote:
> > > On Fri, Sep 2, 2022 at 3:00 AM Efraim Flashner <efraim@flashner.co.il> wrote:
> > > >
> > > > I took a look at the gitolite service finally and I hadn't realized
> > > > there wasn't a running daemon to containerize. I assumed we could do
> > > > something like:
> > > >
> > > > (start $~(make-forkexec-constructor/container
> > > > (list ...)
> > > > #:environment-variables
> > > > '("PATH=...")
> > > > #:mappings ...))
> > > >
> > > > Given that's not the case then I'd need to look at gitolite itself to
> > > > see how it calls the other binaries it expects to be available, and if
> > > > wrapping it would be enough or if we would need to just propagate the
> > > > other packages for functionality.
> > >
> > > Gitolite simply expects tools like git to be on $PATH. It's a pretty
> > > naive system, there's nothing like a configure script that is
> > > determining the absolute file name of these tools and substituting
> > > those names into the built files.
> > >
> > > The executable is already wrapped so that coreutils, findutils, and
> > > git are on $PATH, but notably not openssh:
> > >
> > > (add-after 'install 'wrap-scripts
> > > (lambda* (#:key inputs outputs #:allow-other-keys)
> > > (let ((out (assoc-ref outputs "out"))
> > > (coreutils (assoc-ref inputs "coreutils"))
> > > (findutils (assoc-ref inputs "findutils"))
> > > (git (assoc-ref inputs "git")))
> > > (wrap-program (string-append out "/bin/gitolite")
> > > `("PATH" ":" prefix
> > > ,(map (lambda (dir)
> > > (string-append dir "/bin"))
> > > (list out coreutils findutils git)))))))
> > >
> > > However, git and openssh are still propagated inputs. I'm going to
> > > move the propagated inputs to regular inputs, potentially add openssh
> > > to the wrapper once I remind myself what gitolite does with those
> > > tools, and test it all out on my server using the gitolite service.
> > > If that all works, we have a good starting point for adding extension
> > > support in the service.
> >
> > I like it. Let us know how it goes.
>
> The problem is that gitolite generates git hooks for the repositories
> that it manages, and those hooks invoke git, so the only way those
> scripts will be able to work (without input propagation) is to find a
> way to inject the proper PATH or find a way to replace references to
> things like 'git diff' with '/gnu/store/.../git diff'. I'm going to
> keep exploring and report back when I have something to show.

After several rounds of experimentation and breaking my git server a
few times, here's what I've found:

* Changing git and openssh to be regular inputs and wrapping both
gitolite and gitolite-shell with a $PATH that contains git works and
it's very little extra code.

* Trying to replace every invocation of a git command took a lot of
grepping and crafting of regexps to use for substitute* and I never
got to a point where the result wasn't buggy. In particular,
gitolite-shell never worked properly so I couldn't push to my repos.

So, I think the simple wrapper approach is the way to go. Patch
attached. I tested on my git server by making changes to my gitolite
configuration and pushing those changes to the special gitolite-admin
repo. This causes gitolite to refresh internal configuration using a
git hook, so I know that hooks can find the executables they need.
That plus the 'gitolite setup' invocation made by the service
activation script covers a fair amount of surface area, so I feel
comfortable committing it. What do you think?

Once this part is done, I'll turn my attention to the optional extensions.

- Dave
From 413f2d28aa8bea2274b74c2b574fb9f8bf9c16ba Mon Sep 17 00:00:00 2001
From: David Thompson <dthompson2@worcester.edu>
Date: Fri, 2 Sep 2022 14:33:01 -0400
Subject: [PATCH] gnu: gitolite: Wrap programs instead of using propagated
inputs.

* gnu/packages/version-control.scm (gitolite)[arguments]: Add git to wrapped
$PATH and additionally wrap gitolite-shell.
[inputs]: Add git and openssh.
[propagated-inputs]: Remove it.
---
gnu/packages/version-control.scm | 18 ++++++++----------
1 file changed, 8 insertions(+), 10 deletions(-)

Toggle diff (32 lines)
diff --git a/gnu/packages/version-control.scm b/gnu/packages/version-control.scm
index 15a9278fe8..1c775932c0 100644
--- a/gnu/packages/version-control.scm
+++ b/gnu/packages/version-control.scm
@@ -1573,17 +1573,15 @@ (define-public gitolite
                             (coreutils (assoc-ref inputs "coreutils"))
                             (findutils (assoc-ref inputs "findutils"))
                             (git (assoc-ref inputs "git")))
-                        (wrap-program (string-append out "/bin/gitolite")
-                          `("PATH" ":" prefix
-                            ,(map (lambda (dir)
-                                    (string-append dir "/bin"))
-                                  (list out coreutils findutils git))))))))))
+                        (for-each (lambda (file-name)
+                                    (wrap-program (string-append out file-name)
+                                      `("PATH" ":" prefix
+                                        ,(map (lambda (dir)
+                                                (string-append dir "/bin"))
+                                              (list out coreutils findutils git)))))
+                                  '("/bin/gitolite" "/bin/gitolite-shell"))))))))
     (inputs
-     (list bash-minimal perl coreutils findutils inetutils))
-    ;; git and openssh are propagated because trying to patch the source via
-    ;; regexp matching is too brittle and prone to false positives.
-    (propagated-inputs
-     (list git openssh))
+     (list bash-minimal git perl coreutils findutils inetutils openssh))
     (home-page "https://gitolite.com")
     (synopsis "Git access control layer")
     (description
-- 
2.37.2
E
E
Efraim Flashner wrote on 4 Sep 09:26 +0200
(name . Thompson, David)(address . dthompson2@worcester.edu)
YxRTDbn/aqfAvs8R@3900XT
On Fri, Sep 02, 2022 at 03:58:09PM -0400, Thompson, David wrote:
Toggle quote (81 lines)
> On Fri, Sep 2, 2022 at 8:50 AM Thompson, David <dthompson2@worcester.edu> wrote:
> >
> > On Fri, Sep 2, 2022 at 8:44 AM Efraim Flashner <efraim@flashner.co.il> wrote:
> > >
> > > On Fri, Sep 02, 2022 at 07:11:54AM -0400, Thompson, David wrote:
> > > > On Fri, Sep 2, 2022 at 3:00 AM Efraim Flashner <efraim@flashner.co.il> wrote:
> > > > >
> > > > > I took a look at the gitolite service finally and I hadn't realized
> > > > > there wasn't a running daemon to containerize. I assumed we could do
> > > > > something like:
> > > > >
> > > > > (start $~(make-forkexec-constructor/container
> > > > > (list ...)
> > > > > #:environment-variables
> > > > > '("PATH=...")
> > > > > #:mappings ...))
> > > > >
> > > > > Given that's not the case then I'd need to look at gitolite itself to
> > > > > see how it calls the other binaries it expects to be available, and if
> > > > > wrapping it would be enough or if we would need to just propagate the
> > > > > other packages for functionality.
> > > >
> > > > Gitolite simply expects tools like git to be on $PATH. It's a pretty
> > > > naive system, there's nothing like a configure script that is
> > > > determining the absolute file name of these tools and substituting
> > > > those names into the built files.
> > > >
> > > > The executable is already wrapped so that coreutils, findutils, and
> > > > git are on $PATH, but notably not openssh:
> > > >
> > > > (add-after 'install 'wrap-scripts
> > > > (lambda* (#:key inputs outputs #:allow-other-keys)
> > > > (let ((out (assoc-ref outputs "out"))
> > > > (coreutils (assoc-ref inputs "coreutils"))
> > > > (findutils (assoc-ref inputs "findutils"))
> > > > (git (assoc-ref inputs "git")))
> > > > (wrap-program (string-append out "/bin/gitolite")
> > > > `("PATH" ":" prefix
> > > > ,(map (lambda (dir)
> > > > (string-append dir "/bin"))
> > > > (list out coreutils findutils git)))))))
> > > >
> > > > However, git and openssh are still propagated inputs. I'm going to
> > > > move the propagated inputs to regular inputs, potentially add openssh
> > > > to the wrapper once I remind myself what gitolite does with those
> > > > tools, and test it all out on my server using the gitolite service.
> > > > If that all works, we have a good starting point for adding extension
> > > > support in the service.
> > >
> > > I like it. Let us know how it goes.
> >
> > The problem is that gitolite generates git hooks for the repositories
> > that it manages, and those hooks invoke git, so the only way those
> > scripts will be able to work (without input propagation) is to find a
> > way to inject the proper PATH or find a way to replace references to
> > things like 'git diff' with '/gnu/store/.../git diff'. I'm going to
> > keep exploring and report back when I have something to show.
>
> After several rounds of experimentation and breaking my git server a
> few times, here's what I've found:
>
> * Changing git and openssh to be regular inputs and wrapping both
> gitolite and gitolite-shell with a $PATH that contains git works and
> it's very little extra code.
>
> * Trying to replace every invocation of a git command took a lot of
> grepping and crafting of regexps to use for substitute* and I never
> got to a point where the result wasn't buggy. In particular,
> gitolite-shell never worked properly so I couldn't push to my repos.
>
> So, I think the simple wrapper approach is the way to go. Patch
> attached. I tested on my git server by making changes to my gitolite
> configuration and pushing those changes to the special gitolite-admin
> repo. This causes gitolite to refresh internal configuration using a
> git hook, so I know that hooks can find the executables they need.
> That plus the 'gitolite setup' invocation made by the service
> activation script covers a fair amount of surface area, so I feel
> comfortable committing it. What do you think?
>
> Once this part is done, I'll turn my attention to the optional extensions.

Overall it looks good to me. I was going to ask about inetutils and
openssh since they're not wrapping the binaries but I see their paths
are substituted in the 'patch-source phase.

LGTM!

Toggle quote (49 lines)
> From 413f2d28aa8bea2274b74c2b574fb9f8bf9c16ba Mon Sep 17 00:00:00 2001
> From: David Thompson <dthompson2@worcester.edu>
> Date: Fri, 2 Sep 2022 14:33:01 -0400
> Subject: [PATCH] gnu: gitolite: Wrap programs instead of using propagated
> inputs.
>
> * gnu/packages/version-control.scm (gitolite)[arguments]: Add git to wrapped
> $PATH and additionally wrap gitolite-shell.
> [inputs]: Add git and openssh.
> [propagated-inputs]: Remove it.
> ---
> gnu/packages/version-control.scm | 18 ++++++++----------
> 1 file changed, 8 insertions(+), 10 deletions(-)
>
> diff --git a/gnu/packages/version-control.scm b/gnu/packages/version-control.scm
> index 15a9278fe8..1c775932c0 100644
> --- a/gnu/packages/version-control.scm
> +++ b/gnu/packages/version-control.scm
> @@ -1573,17 +1573,15 @@ (define-public gitolite
> (coreutils (assoc-ref inputs "coreutils"))
> (findutils (assoc-ref inputs "findutils"))
> (git (assoc-ref inputs "git")))
> - (wrap-program (string-append out "/bin/gitolite")
> - `("PATH" ":" prefix
> - ,(map (lambda (dir)
> - (string-append dir "/bin"))
> - (list out coreutils findutils git))))))))))
> + (for-each (lambda (file-name)
> + (wrap-program (string-append out file-name)
> + `("PATH" ":" prefix
> + ,(map (lambda (dir)
> + (string-append dir "/bin"))
> + (list out coreutils findutils git)))))
> + '("/bin/gitolite" "/bin/gitolite-shell"))))))))
> (inputs
> - (list bash-minimal perl coreutils findutils inetutils))
> - ;; git and openssh are propagated because trying to patch the source via
> - ;; regexp matching is too brittle and prone to false positives.
> - (propagated-inputs
> - (list git openssh))
> + (list bash-minimal git perl coreutils findutils inetutils openssh))
> (home-page "https://gitolite.com")
> (synopsis "Git access control layer")
> (description
> --
> 2.37.2
>


--
Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmMUUwsACgkQQarn3Mo9
g1Ef8RAAuMXGFn56TZp8H76L63LFDVLuQZi/Q9MXuDefXeiENlZZtkM5kSQJjw0f
nPepnVBKNWiFyqMMTT+Dq+ag15oCZLebaYysMvUJGnTOraN1bi7vmLpqySuA5OQU
Da3EKXuLbFXrOUR8X13H0/R4ZQSPBIqY3NTtmFX5R9f1qGAadbACDibZch4yJ2i6
YafXWw4N2cd/3irSMOwVtkLUO/rH96m/kgpqxay8mPrfS4PvFICCaa+7w4kl23ga
pPjp0S+yjb7UUIRyUYaus0Gy/4YgGtzFVLgO7c15YNQk0JxQShdlJjwnHXv0ITGz
uK2//1SlvVgwnYSZAZAW8NQ6QzGgRMc2yOOtQn//m9xlYHi4mzZhI7Z9XfGonlbc
8DDXfDzsTdTVQRGtVrgMpdZ4A23iUDYlWnTy36HIgZ0+WEqgVZoO57Bqchfe9qCW
a1LPIEKGTkE0FknSU3f2YNjOQAVDieqWXyx4lVGMwNTiBYxZ0eAYyvK9IPfnK7PO
UXgaXW2GDzKYBiByUvIITiWF74MX0wvE2Q5Iil3yFjp7VduJYgJxP4VwK6hoBAGm
3UzrWdArDIJG7AUNnH2Zs+zp/lrsKB+4+Pa6sed33oEyg/tF6CMVFkqjtizV561v
IxA/DFGijQ0ns41Rp1B2uBy8tmMJOZ6Tf+Y9tPPzr6MkHy4E2xM=
=jLTU
-----END PGP SIGNATURE-----


T
T
Thompson, David wrote on 4 Sep 15:26 +0200
CAJ=RwfZtNPOAcw5_mK9fQUfgCd=A63MR9BB531K_99LyGWe6qQ@mail.gmail.com
Hi Efraim,

On Sun, Sep 4, 2022 at 3:26 AM Efraim Flashner <efraim@flashner.co.il> wrote:
Toggle quote (7 lines)
>
> Overall it looks good to me. I was going to ask about inetutils and
> openssh since they're not wrapping the binaries but I see their paths
> are substituted in the 'patch-source phase.
>
> LGTM!

Thanks! I made one minor tweak to sort the inputs list alphabetically
and pushed as commit 1aa46a7e29c5bd892219fe20fefb883d2103e29e.

I also pushed a follow-up commit
e4ccfcb22ad96e71ca4dfad95af5aa6229ed9869 that swaps out 'git' for
'git-minimal', saving about 75MiB in the package closure.

I think, technically speaking, this bug has been resolved. There are
no longer /usr/bin, /usr/sbin, etc. references in our gitolite
package, so extensions should work as long as the user adds the
relevant packages to their user or system profile. I will keep this
bug open for the moment, though, since I haven't gotten to the final
patch I said I would submit which will make those optional
dependencies easy to add via the gitolite service. Stay tuned!

- Dave
Z
Z
zimoun wrote on 5 Sep 10:16 +0200
871qsqtewp.fsf@gmail.com
Hi,

On dim., 04 sept. 2022 at 09:26, "Thompson, David" <dthompson2@worcester.edu> wrote:

Toggle quote (3 lines)
> Thanks! I made one minor tweak to sort the inputs list alphabetically
> and pushed as commit 1aa46a7e29c5bd892219fe20fefb883d2103e29e.

Cool!

Toggle quote (4 lines)
> I also pushed a follow-up commit
> e4ccfcb22ad96e71ca4dfad95af5aa6229ed9869 that swaps out 'git' for
> 'git-minimal', saving about 75MiB in the package closure.

Neat!

Toggle quote (8 lines)
> I think, technically speaking, this bug has been resolved. There are
> no longer /usr/bin, /usr/sbin, etc. references in our gitolite
> package, so extensions should work as long as the user adds the
> relevant packages to their user or system profile. I will keep this
> bug open for the moment, though, since I haven't gotten to the final
> patch I said I would submit which will make those optional
> dependencies easy to add via the gitolite service. Stay tuned!

Ok, thanks for almost closing this old bugs. :-)


Cheers,
simon
T
T
Thompson, David wrote on 6 Sep 15:50 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
CAJ=RwfYi-rngLpkxkqjq7FVY1anoRj69ExvVCpw6VX21RsrVvw@mail.gmail.com
On Mon, Sep 5, 2022 at 5:33 AM zimoun <zimon.toutoune@gmail.com> wrote:
Toggle quote (7 lines)
>
> > I also pushed a follow-up commit
> > e4ccfcb22ad96e71ca4dfad95af5aa6229ed9869 that swaps out 'git' for
> > 'git-minimal', saving about 75MiB in the package closure.
>
> Neat!

Unfortunately, it was so neat that it broke the system test for the
gitolite service so I had to revert it.

- Dave
T
T
Thompson, David wrote 3 hours ago
(name . zimoun)(address . zimon.toutoune@gmail.com)
CAJ=Rwfbp7TD43ywJe4rge1m5Bd42=Y8BzmPtQfkrQRWmA7UewQ@mail.gmail.com
Hello again Simon and Efraim,

On Mon, Sep 5, 2022 at 5:33 AM zimoun <zimon.toutoune@gmail.com> wrote:
Toggle quote (26 lines)
>
> Hi,
>
> On dim., 04 sept. 2022 at 09:26, "Thompson, David" <dthompson2@worcester.edu> wrote:
>
> > Thanks! I made one minor tweak to sort the inputs list alphabetically
> > and pushed as commit 1aa46a7e29c5bd892219fe20fefb883d2103e29e.
>
> Cool!
>
> > I also pushed a follow-up commit
> > e4ccfcb22ad96e71ca4dfad95af5aa6229ed9869 that swaps out 'git' for
> > 'git-minimal', saving about 75MiB in the package closure.
>
> Neat!
>
> > I think, technically speaking, this bug has been resolved. There are
> > no longer /usr/bin, /usr/sbin, etc. references in our gitolite
> > package, so extensions should work as long as the user adds the
> > relevant packages to their user or system profile. I will keep this
> > bug open for the moment, though, since I haven't gotten to the final
> > patch I said I would submit which will make those optional
> > dependencies easy to add via the gitolite service. Stay tuned!
>
> Ok, thanks for almost closing this old bugs. :-)

Some news: I have updated the gitolite package to use G-expressions.
The package builds and the gitolite system test passes so I pushed
that change to master a little while ago. That patch has made the
(hopefully) final step in this saga easier. The attached patch
introduces a 'make-gitolite' procedure that can be used to add
arbitrary packages to the wrappers for the gitolite and gitolite-shell
programs. The return value of this procedure can be used in the
gitolite service configuration to enable the desired optional features
like Redis or git-annex. The base package inputs are unchanged and
the gitolite system test still passes.

What do you think?

- Dave
From 3f3e2d002cb8c740081d58e83b6e89236d11f15f Mon Sep 17 00:00:00 2001
From: David Thompson <dthompson2@worcester.edu>
Date: Thu, 6 Oct 2022 08:45:48 -0400
Subject: [PATCH] gnu: version-control: Add make-gitolite procedure.

* gnu/packages/version-control.scm (make-gitolite): New procedure.
(gitolite): Use make-gitolite.
* doc/guix.texi (Gitolite service): Document how to use make-gitolite.
---
doc/guix.texi | 15 ++++++++++++++-
gnu/packages/version-control.scm | 15 +++++++++++----
2 files changed, 25 insertions(+), 5 deletions(-)

Toggle diff (77 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 523711bdf6..533b12d738 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -35285,7 +35285,20 @@ Data type representing the configuration for @code{gitolite-service-type}.
 
 @table @asis
 @item @code{package} (default: @var{gitolite})
-Gitolite package to use.
+Gitolite package to use.  There are optional Gitolite dependencies that
+are not included in the default package, such as Redis and git-annex.
+These features can be made available by using the @code{make-gitolite}
+procedure in the @code{(gnu packages version-control}) module to produce
+a variant of Gitolite with the desired additional dependencies.
+
+The following code returns a package in which the Redis and git-annex
+programs can be invoked by Gitolite's scripts:
+
+@example
+(use-modules (gnu packages databases)
+             (gnu packages haskell-apps))
+(make-gitolite (list redis git-annex))
+@end example
 
 @item @code{user} (default: @var{git})
 User to use for Gitolite.  This will be user that you use when accessing
diff --git a/gnu/packages/version-control.scm b/gnu/packages/version-control.scm
index bd0ad70ce8..68358cc1d1 100644
--- a/gnu/packages/version-control.scm
+++ b/gnu/packages/version-control.scm
@@ -137,7 +137,8 @@ (define-module (gnu packages version-control)
   #:use-module (gnu packages tls)
   #:use-module (gnu packages)
   #:use-module (ice-9 match)
-  #:use-module (srfi srfi-1))
+  #:use-module (srfi srfi-1)
+  #:export (make-gitolite))
 
 (define-public breezy
   (package
@@ -1482,7 +1483,9 @@ (define-public git-test-sequence
 also walk each side of a merge and test those changes individually.")
       (license (license:x11-style "file://LICENSE")))))
 
-(define-public gitolite
+(define* (make-gitolite #:optional (extra-inputs '()))
+  "Make a gitolite package object with EXTRA-INPUTS added to the binary
+wrappers, to be used for optional gitolite extensions."
   (package
     (name "gitolite")
     (version "3.6.12")
@@ -1574,10 +1577,12 @@ (define-public gitolite
                                          (list #$output
                                                #$coreutils
                                                #$findutils
-                                               #$git)))))
+                                               #$git
+                                               #$@extra-inputs)))))
                              '("/bin/gitolite" "/bin/gitolite-shell")))))))
     (inputs
-     (list bash-minimal coreutils findutils git inetutils openssh perl))
+     (append (list bash-minimal coreutils findutils git inetutils openssh perl)
+             extra-inputs))
     (home-page "https://gitolite.com")
     (synopsis "Git access control layer")
     (description
@@ -1585,6 +1590,8 @@ (define-public gitolite
 control to Git repositories.")
     (license license:gpl2)))
 
+(define-public gitolite (make-gitolite))
+
 (define-public gitile
   (package
     (name "gitile")
-- 
2.37.2
T
T
Thompson, David wrote 3 hours ago
(name . zimoun)(address . zimon.toutoune@gmail.com)
CAJ=RwfY+WL4VZ_U0W-FEmWL2ZtcnQbUkQqii7GF9Rtht-UJKrQ@mail.gmail.com
I made a small mistake in the last patch and forgot to include '(gnu
packages version-control)' in the 'use-modules' form in the example
within the manual. I have fixed that in my local copy.

- Dave
?