guix-daemon fails when invoking ‘guix authenticate’ on the Hurd

  • Done
  • quality assurance status badge
Details
3 participants
  • janneke
  • Ludovic Courtès
  • Ludovic Courtès
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 11 Sep 17:40 +0200
guix-daemon fails when invoking ‘guix authentic ate’ on the Hurd
(address . bug-guix@gnu.org)
87o74u5hcy.fsf@inria.fr
As reported by yelninei on IRC, Guix on the Hurd is in half-broken state
following the recent ‘core-updates’ merge.

Specifically, sending store items over SSH (and thus breaks offloading
too):

Toggle snippet (5 lines)
$ guix copy --to=localhost:10022 idutils
guix copy: sending 1 store item (1 MiB) to 'localhost'...
guix copy: error: unknown error while sending files over SSH

(Here localhost:10022 is a childhurd.)

Inside the childhurd, we get:

Toggle snippet (4 lines)
root@childhurd ~# tail -1 /var/log/guix-daemon.log
2024-09-09 21:09:03 unexpected build daemon error: stoi

Last time we got that error was in commit
21deb89e287b5821975544118bf137562a91d4e1: guix-daemon was running with
incorrect locale data, and thus ‘std::stoi’ would throw (‘std::stoi’ in
guix-daemon is used for the communication with the ‘guix authenticate’
process, see ‘readAuthenticateReply’).

When running guix-daemon by hand instead of via the Shepherd service,
with the same environment variables, everything works, unless we set one
of the LC_* variables.

Ludo’.
L
L
Ludovic Courtès wrote on 11 Sep 19:08 +0200
Re: bug#73181: guix-daemon fails when invoking ‘guix authenticate’ on the Hurd
(address . 73181@debbugs.gnu.org)
87a5ge5d94.fsf@gnu.org
Ludovic Courtès <ludovic.courtes@inria.fr> skribis:

Toggle quote (11 lines)
> Inside the childhurd, we get:
>
> root@childhurd ~# tail -1 /var/log/guix-daemon.log
> 2024-09-09 21:09:03 unexpected build daemon error: stoi
>
> Last time we got that error was in commit
> 21deb89e287b5821975544118bf137562a91d4e1: guix-daemon was running with
> incorrect locale data, and thus ‘std::stoi’ would throw (‘std::stoi’ in
> guix-daemon is used for the communication with the ‘guix authenticate’
> process, see ‘readAuthenticateReply’).

I wanted to ‘rpctrace’ to see what ‘guix authenticate’ and ‘guix-daemon’
are telling each other. The problem is that ‘rpctrace’ is kinda broken,
or at least it wrecks havoc when trying to follow forks or something.

But I found a trick that allowed me to trace just ‘guix authenticate’:

Toggle snippet (9 lines)
root@childhurd ~# cat intercept-guix.sh
#!/bin/sh
/usr/bin/env > /tmp/env.log
echo >> /tmp/env.log
echo "$@" >> /tmp/env.log
exec rpctrace -o /tmp/rpctrace.log -s 200 /gnu/store/cg64w7mv1v2r188rzcgksdva9cvj8vir-guix-1.4.0-24.9a2ddcc/bin/guix "$@"
root@childhurd ~# LC_ALL=fr_FR.utf8 GUIX=$PWD/intercept-guix.sh /gnu/store/cg64w7mv1v2r188rzcgksdva9cvj8vir-guix-1.4.0-24.9a2ddcc/bin/guix-daemon --build-users-group guixbuild --max-silent-time 3600 --timeout 86400 --log-compression gzip --discover=no --disable-chroot --disable-deduplication

(Here “LC_ALL=fr_FR.utf8” allows me to reproduce the stoi bug.)

We get this:

Toggle snippet (13 lines)
root@childhurd ~# tail /tmp/rpctrace.log
5<--35(pid166)->dir_lookup ("gnu/store/9ghq6s4mq5sff9cwqrmn26ivycn3p8ql-guile-3.0.9/lib/guile/3.0/ccache/system/foreign-library.go" 4194305 0) = 0 1 "" 67<--70(pid166)
67<--70(pid166)->io_seek_request (0 2) = 0 98141
67<--70(pid166)->io_map_request () = 0 69<--66(pid166) (null)
task30(pid166)-> 2089 (0 98141 0 1 69<--66(pid166) 0 1 1 7 1) = 0 61452288
task30(pid166)-> 3206 (pn{ 28}) = 0
task30(pid166)-> 3206 (pn{ 27}) = 0
task30(pid166)-> 2024 (61517824 18824 0 3) = 0
14<--31(pid166)->io_write_request ("GC Warning: Repeated allocation of very large block (appr. size 112 KiB):\n\tMay lead to memory leak and poor performance\n" -1) = 0 120
task30(pid166)-> 2012 (1 22) = 0 {0 25 195772416 15151104 0 0 0 0 1726072790 570000 0 0 0 0 0 0 0 0 1726072790 0 570000000 0}
task30(pid166)-> 2012 (3 12) = 0 {0 30000 0 20000 0 0 30000000 0 0 0 20000000 0}

That warning comes from libgc and there’s no such warning when LC_ALL is
left unset, which is why it works in that case.

The warning goes to stderr, but that’s what guix-daemon ends up reading,
hence the ‘stoi’ error. One can reproduce that by printing something on
(current-error-port) from ‘guix authenticate’ and running, say, “make
check TESTS=tests/store.scm”: a bunch of tests fail.

It’s not clear to me why libgc keeps printing that warning on i586-gnu
(I think it’s fine on i686-linux so there might be a portability issue),
and it’s not clear either why changing the locale triggers the warning
(it might have to do with the so-called GC “black lists”, since loading
a locale effectively changes the address space layout somewhat).

I see several possible things to do:

1. Disable GC warnings altogether in Guix by setting ‘GC_warn_proc’.

2. Make sure guix-daemon discards stderr from agents such as ‘guix
authenticate’.

3. Figure out why those “Repeated allocation” messages are so frequent
on i586-gnu and Do Something About It™.

Thoughts?

Ludo’.
J
J
janneke wrote on 31 Oct 20:14 +0100
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)(address . 73181@debbugs.gnu.org)
87h68snkzk.fsf@gnu.org
Ludovic Courtès writes:

Hi!

Toggle quote (13 lines)
> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>
>> Inside the childhurd, we get:
>>
>> root@childhurd ~# tail -1 /var/log/guix-daemon.log
>> 2024-09-09 21:09:03 unexpected build daemon error: stoi
>>
>> Last time we got that error was in commit
>> 21deb89e287b5821975544118bf137562a91d4e1: guix-daemon was running with
>> incorrect locale data, and thus ‘std::stoi’ would throw (‘std::stoi’ in
>> guix-daemon is used for the communication with the ‘guix authenticate’
>> process, see ‘readAuthenticateReply’).

[snip some great debugging, phew!]

Toggle quote (3 lines)
> That warning comes from libgc and there’s no such warning when LC_ALL is
> left unset, which is why it works in that case.

How about

0. Unsetting LC_ALL on the Hurd

would that work?

Toggle quote (12 lines)
> I see several possible things to do:
>
> 1. Disable GC warnings altogether in Guix by setting ‘GC_warn_proc’.
>
> 2. Make sure guix-daemon discards stderr from agents such as ‘guix
> authenticate’.
>
> 3. Figure out why those “Repeated allocation” messages are so frequent
> on i586-gnu and Do Something About It™.
>
> Thoughts?

If 0., which seems easiest, does not work then Obviously(?) 3. is nicer
but (much?) harder than 2., which is nicer than 1. However, 1. is much
nicer than our current situation of not having offloading to our
childhurds and not having up-to-date substitutes. How about doing
1. right now but only for the Hurd, until we get 2. or 3.?

Greeting,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 4 Nov 09:21 +0100
(address . janneke@gnu.org)(address . 73181@debbugs.gnu.org)
874j4n5s02.fsf@inria.fr
Hello,

<janneke@gnu.org> skribis:

Toggle quote (2 lines)
> Ludovic Courtès writes:

[...]

Toggle quote (9 lines)
>> That warning comes from libgc and there’s no such warning when LC_ALL is
>> left unset, which is why it works in that case.
>
> How about
>
> 0. Unsetting LC_ALL on the Hurd
>
> would that work?

I’m not sure, but if it does, it’s only “by chance”.

Toggle quote (18 lines)
>> I see several possible things to do:
>>
>> 1. Disable GC warnings altogether in Guix by setting ‘GC_warn_proc’.
>>
>> 2. Make sure guix-daemon discards stderr from agents such as ‘guix
>> authenticate’.
>>
>> 3. Figure out why those “Repeated allocation” messages are so frequent
>> on i586-gnu and Do Something About It™.
>>
>> Thoughts?
>
> If 0., which seems easiest, does not work then Obviously(?) 3. is nicer
> but (much?) harder than 2., which is nicer than 1. However, 1. is much
> nicer than our current situation of not having offloading to our
> childhurds and not having up-to-date substitutes. How about doing
> 1. right now but only for the Hurd, until we get 2. or 3.?

Yeah, let’s start with (1), maybe with the patch below (untested)?

(2) and (3) would be nice, but it goes with increasing difficulty.

Ludo’.
Toggle diff (20 lines)
diff --git a/gnu/packages/aux-files/guile-launcher.c b/gnu/packages/aux-files/guile-launcher.c
index ad0094bff5..6a59905a28 100644
--- a/gnu/packages/aux-files/guile-launcher.c
+++ b/gnu/packages/aux-files/guile-launcher.c
@@ -73,6 +73,15 @@ main (int argc, char **argv)
which is always preferable over the C locale. */
setlocale (LC_ALL, "en_US.utf8");
+#if defined __GNU__
+ /* XXX: On 32-bit GNU/Hurd (i586-gnu), libgc emits "Repeated allocation"
+ warnings that are annoying and interfere with communications between
+ 'guix-daemon' and 'guix authenticate':
+ <https://issues.guix.gnu.org/73181>. Silence them. */
+ static void no_warnings (char *message, GC_word arg) { };
+ GC_set_warn_proc (no_warnings);
+#endif
+
const char *str;
str = getenv ("GUILE_LOAD_PATH");
load_path = str != NULL ? strdup (str) : NULL;
J
J
janneke wrote on 5 Nov 12:26 +0100
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)(address . 73181@debbugs.gnu.org)
87ed3phqfm.fsf@gnu.org
Ludovic Courtès writes:

Hello,

Toggle quote (4 lines)
> <janneke@gnu.org> skribis:
>
>> Ludovic Courtès writes:

[..]

Toggle quote (10 lines)
>>> I see several possible things to do:
>>>
>>> 1. Disable GC warnings altogether in Guix by setting ‘GC_warn_proc’.
>>>
>>> 2. Make sure guix-daemon discards stderr from agents such as ‘guix
>>> authenticate’.
>>>
>>> 3. Figure out why those “Repeated allocation” messages are so frequent
>>> on i586-gnu and Do Something About It™.

[..]

Toggle quote (4 lines)
> Yeah, let’s start with (1), maybe with the patch below (untested)?
>
> (2) and (3) would be nice, but it goes with increasing difficulty.

So...find two patches attached. A tested version of (untested) which
was very helpful but did not compile. I should have tested with #if 1,
and use #:tests? #f with guix right away... ;)

Anyway, using this patch 0001 it seems that suppressing the warnings
works, I no longer get

Toggle snippet (3 lines)
"GC Warning: Repeated allocation of very large block (appr. size 112 KiB):\n\tMay lead to memory leak and poor performance\n"

but still get

Toggle snippet (3 lines)
unexpected build daemon error: stoi

and the copy (and offload) still fails. Then I tried resetting LC_ALL

Toggle quote (8 lines)
>> How about
>>
>> 0. Unsetting LC_ALL on the Hurd
>>
>> would that work?
>
> I’m not sure, but if it does, it’s only “by chance”.

(option 0.) anyway in a followup patch. Adding this patch 0003 (patch
0002 was just the guix package update), also the "stoi" warning is gone,
and offloading works.

So it seems that we have a workaround that "works by chance", WDYT?

Greetings,
Janneke
From 3d399e51104171ad328bea66ebdc1d6b0ac99685 Mon Sep 17 00:00:00 2001
Message-ID: <3d399e51104171ad328bea66ebdc1d6b0ac99685.1730803153.git.janneke@gnu.org>
From: Janneke Nieuwenhuizen <janneke@gnu.org>
Date: Mon, 4 Nov 2024 14:54:55 +0100
Subject: [PATCH 1/3] guile: Silence GC warnings on the Hurd.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=UTF-8

This should work around https://issues.guix.gnu.org/73181, resurrecting
offloading to the Hurd.

* gnu/packages/aux-files/guile-launcher.c (no_warnings)[__GNU__]: New
function.
(main)[__GNU__]: Use it to silence libgc warnings.

Co-authored-by: Ludovic Courtès <ludo@gnu.org>.
Change-Id: I8f30732d192ce46144da4a1a081813a104a5f376
---
gnu/packages/aux-files/guile-launcher.c | 19 ++++++++++++++++++-
1 file changed, 18 insertions(+), 1 deletion(-)

Toggle diff (49 lines)
diff --git a/gnu/packages/aux-files/guile-launcher.c b/gnu/packages/aux-files/guile-launcher.c
index ad0094bff5..bc7fa21b63 100644
--- a/gnu/packages/aux-files/guile-launcher.c
+++ b/gnu/packages/aux-files/guile-launcher.c
@@ -1,7 +1,8 @@
/* GNU Guix --- Functional package management for GNU
Copyright 1996-1997,2000-2001,2006,2008,2011,2013,2018,2020,2021
Free Software Foundation, Inc.
- Copyright (C) 2020 Ludovic Courtès <ludo@gnu.org>
+ Copyright (C) 2020, 2024 Ludovic Courtès <ludo@gnu.org>
+ Copyright (C) 2024 Janneke Nieuwenhuizen <janneke@gnu.org>
This file is part of GNU Guix.
@@ -28,6 +29,14 @@
#include <locale.h>
#include <libguile.h>
+#if defined __GNU__
+#include <gc.h>
+static void
+no_warnings (char *message, GC_word arg)
+{
+}
+#endif
+
/* Saved values of GUILE_LOAD_PATH and GUILE_LOAD_COMPILED_PATH. */
static const char *load_path, *load_compiled_path;
@@ -73,6 +82,14 @@ main (int argc, char **argv)
which is always preferable over the C locale. */
setlocale (LC_ALL, "en_US.utf8");
+#if defined __GNU__
+ /* XXX: On 32-bit GNU/Hurd (i586-gnu), libgc emits "Repeated allocation"
+ warnings that are annoying and interfere with communications between
+ 'guix-daemon' and 'guix authenticate':
+ <https://issues.guix.gnu.org/73181>. Silence them. */
+ GC_set_warn_proc (no_warnings);
+#endif
+
const char *str;
str = getenv ("GUILE_LOAD_PATH");
load_path = str != NULL ? strdup (str) : NULL;

base-commit: 20c7b8dd04e421a139a02438cf1ddfdfe544a446
--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com
From b62d59cf5cc9f968bedd8126f587bc8e14f7964c Mon Sep 17 00:00:00 2001
Message-ID: <b62d59cf5cc9f968bedd8126f587bc8e14f7964c.1730803153.git.janneke@gnu.org>
In-Reply-To: <3d399e51104171ad328bea66ebdc1d6b0ac99685.1730803153.git.janneke@gnu.org>
References: <3d399e51104171ad328bea66ebdc1d6b0ac99685.1730803153.git.janneke@gnu.org>
From: Janneke Nieuwenhuizen <janneke@gnu.org>
Date: Mon, 4 Nov 2024 17:38:46 +0100
Subject: [PATCH 3/3] squash! guile: Silence GC warnings on the Hurd.
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=UTF-8

Change-Id: Ia720221ed285d3e56938cfa37989d2c7c07b00e3
---
gnu/packages/aux-files/guile-launcher.c | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)

Toggle diff (37 lines)
diff --git a/gnu/packages/aux-files/guile-launcher.c b/gnu/packages/aux-files/guile-launcher.c
index bc7fa21b63..ee6da76cc4 100644
--- a/gnu/packages/aux-files/guile-launcher.c
+++ b/gnu/packages/aux-files/guile-launcher.c
@@ -75,19 +75,22 @@ inner_main (void *unused, int argc, char **argv)
int
main (int argc, char **argv)
{
- /* Try to install the current locale; remain silent if it fails. */
- if (setlocale (LC_ALL, "") == NULL)
- /* The 'guix pull'-provided 'guix' includes at least en_US.utf8 so use
- that. That gives us UTF-8 support for 'scm_to_locale_string', etc.,
- which is always preferable over the C locale. */
- setlocale (LC_ALL, "en_US.utf8");
-
#if defined __GNU__
+ /* Try to install the C locale; remain silent if it fails. */
+ setlocale (LC_ALL, "C");
+
/* XXX: On 32-bit GNU/Hurd (i586-gnu), libgc emits "Repeated allocation"
warnings that are annoying and interfere with communications between
'guix-daemon' and 'guix authenticate':
<https://issues.guix.gnu.org/73181>. Silence them. */
GC_set_warn_proc (no_warnings);
+#else
+ /* Try to install the current locale; remain silent if it fails. */
+ if (setlocale (LC_ALL, "") == NULL)
+ /* The 'guix pull'-provided 'guix' includes at least en_US.utf8 so use
+ that. That gives us UTF-8 support for 'scm_to_locale_string', etc.,
+ which is always preferable over the C locale. */
+ setlocale (LC_ALL, "en_US.utf8");
#endif
const char *str;
--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com
--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 5 Nov 15:03 +0100
(address . janneke@gnu.org)(address . 73181@debbugs.gnu.org)
877c9hzsjy.fsf@inria.fr
Hello!

<janneke@gnu.org> skribis:

Toggle quote (10 lines)
> Anyway, using this patch 0001 it seems that suppressing the warnings
> works, I no longer get
>
> "GC Warning: Repeated allocation of very large block (appr. size 112 KiB):\n\tMay lead to memory leak and poor performance\n"
>
>
> but still get
>
> unexpected build daemon error: stoi

Damnit. Could you check with rpctrace what the daemon receives?

I wonder if I misunderstood what the root cause is.

Toggle quote (21 lines)
> From 3d399e51104171ad328bea66ebdc1d6b0ac99685 Mon Sep 17 00:00:00 2001
> Message-ID: <3d399e51104171ad328bea66ebdc1d6b0ac99685.1730803153.git.janneke@gnu.org>
> From: Janneke Nieuwenhuizen <janneke@gnu.org>
> Date: Mon, 4 Nov 2024 14:54:55 +0100
> Subject: [PATCH 1/3] guile: Silence GC warnings on the Hurd.
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
> Content-Transfer-Encoding: 8bit
> Content-Type: text/plain; charset=UTF-8
>
> This should work around <https://issues.guix.gnu.org/73181>, resurrecting
> offloading to the Hurd.
>
> * gnu/packages/aux-files/guile-launcher.c (no_warnings)[__GNU__]: New
> function.
> (main)[__GNU__]: Use it to silence libgc warnings.
>
> Co-authored-by: Ludovic Courtès <ludo@gnu.org>.
> Change-Id: I8f30732d192ce46144da4a1a081813a104a5f376

LGTM.

Toggle quote (4 lines)
> #if defined __GNU__
> + /* Try to install the C locale; remain silent if it fails. */
> + setlocale (LC_ALL, "C");

This I’d rather avoid, unless we have a good understanding of why this
would help.

Thanks for working on it!

Ludo’.
J
J
janneke wrote on 5 Nov 16:41 +0100
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)(address . 73181@debbugs.gnu.org)
871pzphemc.fsf@gnu.org
Ludovic Courtès writes:

Hi!

Toggle quote (17 lines)
> <janneke@gnu.org> skribis:
>
>> Anyway, using this patch 0001 it seems that suppressing the warnings
>> works, I no longer get
>>
>> "GC Warning: Repeated allocation of very large block (appr. size 112
>> KiB):\n\tMay lead to memory leak and poor performance\n"
>>
>>
>> but still get
>>
>> unexpected build daemon error: stoi
>
> Damnit. Could you check with rpctrace what the daemon receives?
>
> I wonder if I misunderstood what the root cause is.

Yes :-( I captured a `guix offload test' run, see attached.

Toggle quote (14 lines)
>> From 3d399e51104171ad328bea66ebdc1d6b0ac99685 Mon Sep 17 00:00:00 2001
>> Message-ID: <3d399e51104171ad328bea66ebdc1d6b0ac99685.1730803153.git.janneke@gnu.org>
>> From: Janneke Nieuwenhuizen <janneke@gnu.org>
>> Date: Mon, 4 Nov 2024 14:54:55 +0100
>> Subject: [PATCH 1/3] guile: Silence GC warnings on the Hurd.
>> * gnu/packages/aux-files/guile-launcher.c (no_warnings)[__GNU__]: New
>> function.
>> (main)[__GNU__]: Use it to silence libgc warnings.
>>
>> Co-authored-by: Ludovic Courtès <ludo@gnu.org>.
>> Change-Id: I8f30732d192ce46144da4a1a081813a104a5f376
>
> LGTM.

Do we want to push this...as it doesn't really

Toggle quote (3 lines)
>> This should work around https://issues.guix.gnu.org/73181, resurrecting
>> offloading to the Hurd.

just yet.

Toggle quote (7 lines)
>> #if defined __GNU__
>> + /* Try to install the C locale; remain silent if it fails. */
>> + setlocale (LC_ALL, "C");
>
> This I’d rather avoid, unless we have a good understanding of why this
> would help.

Yes, that was clear to me; just wanted to have this data point :)

Toggle quote (2 lines)
> Thanks for working on it!

Yeat it's not much work, it just takes such a long wait each time.

Greetings,
Janneke
Attachment: rpctrace-1.4.0-27.3d399e5.log
--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 10 Nov 12:54 +0100
(address . janneke@gnu.org)(address . 73181@debbugs.gnu.org)
875xovwbgm.fsf@gnu.org
Hello,

<janneke@gnu.org> skribis:

Toggle quote (17 lines)
>>> Anyway, using this patch 0001 it seems that suppressing the warnings
>>> works, I no longer get
>>>
>>> "GC Warning: Repeated allocation of very large block (appr. size 112
>>> KiB):\n\tMay lead to memory leak and poor performance\n"
>>>
>>>
>>> but still get
>>>
>>> unexpected build daemon error: stoi
>>
>> Damnit. Could you check with rpctrace what the daemon receives?
>>
>> I wonder if I misunderstood what the root cause is.
>
> Yes :-( I captured a `guix offload test' run, see attached.

[...]

Toggle quote (3 lines)
> task31(pid198)-> 2058 (4) = 0 22<--9(pid198)
> 22<--9(pid198)->exec_startup_get_info () = 0 134517280 134512692 352 237568 16777216 0 "/gnu/store/7wgwfsbvq8m9zkz03d27ij53jciliz9n-guix-1.4.0-27.3d399e5/libexec/guix/guile\0\\0/run/current-system/profile/bin/guix\0authenticate\0" "SHELL=/gnu/store/dm5shwb20i38wqdkmyqvhqfi0hmq1lr1-bash-5.1.16/bin/bash\0XDG_CONFIG_DIRS=/root/.guix-profile/etc/xdg:/run/current-system/profile/etc/xdg\0PKG_CONFIG_PATH=/run/current-system/profile/lib/p" { 16<--25(pid198) 13<--27(pid198) 13<--27(pid198) 4<--32(pid198) 12<--33(pid198)} { 11<--34(pid198) 6<--35(pid198) 2<--36(pid198) 26<--38(pid198) (null) (null)} {18 0 0 0 0}

[...]

Toggle quote (13 lines)
> 26<--38(pid198)->proc_setmsgport_request ( 44<--48(pid-1)) = 0 (null)
> 26<--38(pid198)->proc_set_arg_locations_request (17014228 17014248) = 0
> task31(pid198)-> 3204 (1) = 0 pn{ 19}
> task31(pid198)-> 3215 (pn{ 19} 49) = 0
> task31(pid198)-> 3204 (1) = 0 pn{ 20}
> task31(pid198)-> 3210 (pn{ 20} 1) = 0
> 26<--38(pid198)->proc_handle_exceptions_request ( 49<--51(pid-1) 50<--52(pid-1) 5 {75 31 31 31 0 0 0 0 0 0 0 0 19291280 23 0 21548192 0}) = 0
> thread46(pid198)-> 2068 (3 51) = 0
> task31(pid198)-> 3206 (pn{ 19}) = 0
> task31(pid198)-> 2023 (17031168 20) = 0
> task31(pid198)-> 2023 (17027072 24) = 0
> 6<--35(pid198)->dir_lookup ("gnu/store/81ffz0prarfczr408ydnps31jf72s5ly-glibc-cross-i586-pc-gnu-2.39/share/locale/locale.alias" 4194305 0) = 0 1 "" 51<--47(pid198)

Does that ‘locale.alias’ file exists?

Did you try several LC_ALL=xxx values to see which one would work and
which one wouldn’t?

So after all, there may be two issues: the “Repeated allocation” thing,
and a locale issue.

Thanks,
Ludo’.
J
J
janneke wrote on 10 Nov 13:01 +0100
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 73181@debbugs.gnu.org)
87iksv1en9.fsf@gnu.org
Ludovic Courtès writes:

Hi,

Toggle quote (26 lines)
> <janneke@gnu.org> skribis:
>
>>>> Anyway, using this patch 0001 it seems that suppressing the warnings
>>>> works, I no longer get
>>>>
>>>> "GC Warning: Repeated allocation of very large block (appr. size 112
>>>> KiB):\n\tMay lead to memory leak and poor performance\n"
>>>>
>>>>
>>>> but still get
>>>>
>>>> unexpected build daemon error: stoi
>>>
>>> Damnit. Could you check with rpctrace what the daemon receives?
>>>
>>> I wonder if I misunderstood what the root cause is.
>>
>> Yes :-( I captured a `guix offload test' run, see attached.
>
> [...]
>
>> ("gnu/store/81ffz0prarfczr408ydnps31jf72s5ly-glibc-cross-i586-pc-gnu-2.39/share/locale/locale.alias"
>> 4194305 0) = 0 1 "" 51<--47(pid198)
>
> Does that ‘locale.alias’ file exists?

Yes

Toggle snippet (13 lines)
root@childhurd ~# tail /gnu/store/81ffz0prarfczr408ydnps31jf72s5ly-glibc-cross-i586-pc-gnu-2.39/share/locale/locale.alias
portuguese pt_PT.ISO-8859-1
romanian ro_RO.ISO-8859-2
russian ru_RU.ISO-8859-5
slovak sk_SK.ISO-8859-2
slovene sl_SI.ISO-8859-2
slovenian sl_SI.ISO-8859-2
spanish es_ES.ISO-8859-1
swedish sv_SE.ISO-8859-1
thai th_TH.TIS-620
turkish tr_TR.ISO-8859-9

Toggle quote (3 lines)
> Did you try several LC_ALL=xxx values to see which one would work and
> which one wouldn’t?

I tried fr_FR.UTF-8 and nl_NL.UTF-8, both fail. And then LC_ALL=C,
which works. Any other ideas?

Toggle quote (3 lines)
> So after all, there may be two issues: the “Repeated allocation” thing,
> and a locale issue.

Yes, it looks like the we can suppress the warnings though, which
hides the "Repeated allocation" issue...

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
J
J
janneke wrote on 18 Nov 09:41 +0100
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)(address . 73181@debbugs.gnu.org)
87jzd1hr33.fsf@gnu.org
Ludovic Courtès writes:

Hello,

Toggle quote (40 lines)
> <janneke@gnu.org> skribis:
>
>> Anyway, using this patch 0001 it seems that suppressing the warnings
>> works, I no longer get
>>
>> "GC Warning: Repeated allocation of very large block (appr. size 112
>> KiB):\n\tMay lead to memory leak and poor performance\n"
>>
>>
>> but still get
>>
>> unexpected build daemon error: stoi
>
> Damnit. Could you check with rpctrace what the daemon receives?
>
> I wonder if I misunderstood what the root cause is.
>
>> From 3d399e51104171ad328bea66ebdc1d6b0ac99685 Mon Sep 17 00:00:00 2001
>> Message-ID: <3d399e51104171ad328bea66ebdc1d6b0ac99685.1730803153.git.janneke@gnu.org>
>> From: Janneke Nieuwenhuizen <janneke@gnu.org>
>> Date: Mon, 4 Nov 2024 14:54:55 +0100
>> Subject: [PATCH 1/3] guile: Silence GC warnings on the Hurd.
>> MIME-Version: 1.0
>> Content-Type: text/plain; charset=UTF-8
>> Content-Transfer-Encoding: 8bit
>> Content-Transfer-Encoding: 8bit
>> Content-Type: text/plain; charset=UTF-8
>>
>> This should work around <https://issues.guix.gnu.org/73181>, resurrecting
>> offloading to the Hurd.
>>
>> * gnu/packages/aux-files/guile-launcher.c (no_warnings)[__GNU__]: New
>> function.
>> (main)[__GNU__]: Use it to silence libgc warnings.
>>
>> Co-authored-by: Ludovic Courtès <ludo@gnu.org>.
>> Change-Id: I8f30732d192ce46144da4a1a081813a104a5f376
>
> LGTM.

Okay, pushed to master as b0416b8503e38746717a2d1167f9d1410634981e.

I decided to add some more instrumentation

Toggle snippet (18 lines)
diff --git a/nix/libstore/local-store.cc b/nix/libstore/local-store.cc
index 0883a4bbce..f9f42074eb 100644
--- a/nix/libstore/local-store.cc
+++ b/nix/libstore/local-store.cc
@@ -1182,6 +1182,12 @@ static int readInteger(int fd)
}
}
+ if (str.empty()) {
+ printMsg(lvlError, format("nondigit: %1%") % str);
+ str = readLine(fd);
+ printMsg(lvlError, format("line: %1%") % str);
+ throw EndOfFile("unexpected non-digit reading an integer");
+ }
return stoi(str);
}

to possibly find out what the non-digit text is that we get, and build
with the above patch. However, after two initial offload failures and
wrong host key, manual chilhurd..

Toggle snippet (42 lines)
08:31:37 janneke@dundal:~/src/guix/hurd64-team
$ guix offload test
guix offload: testing 1 build machines defined in '/etc/guix/machines.scm'...
guix offload: Guix is usable on 'localhost' (test returned "/gnu/store/883yjkl46dxw9mzykykmbs0yzwyxm17z-test")
guix offload: 'localhost' is running GNU Guile 3.0.9
guix offload: sending 1 store item (0 MiB) to 'localhost'...
exporting path `/gnu/store/q8gsjh72lb4h2ga65g1rzj6f8nm9l7lg-export-test'
guix offload: error: unexpected non-digit reading an integer
[1] 08:32:10 janneke@dundal:~/src/guix/hurd64-team
$ guix offload test
guix offload: testing 1 build machines defined in '/etc/guix/machines.scm'...
guix offload: Guix is usable on 'localhost' (test returned "/gnu/store/883yjkl46dxw9mzykykmbs0yzwyxm17z-test")
guix offload: 'localhost' is running GNU Guile 3.0.9
guix offload: sending 1 store item (0 MiB) to 'localhost'...
exporting path `/gnu/store/5hfifv0ngkazyhbn0klbfq28fcxnlz3b-export-test'
guix offload: error: unexpected non-digit reading an integer
[1] 08:32:28 janneke@dundal:~/src/guix/hurd64-team
$ guix offload test
guix offload: testing 1 build machines defined in '/etc/guix/machines.scm'...
guix offload: Guix is usable on 'localhost' (test returned "/gnu/store/883yjkl46dxw9mzykykmbs0yzwyxm17z-test")
guix offload: 'localhost' is running GNU Guile 3.0.9
guix offload: sending 1 store item (0 MiB) to 'localhost'...
exporting path `/gnu/store/90517zpibi4bqsrbmd9ih64l8044ixnx-export-test'
guix offload: 'localhost' successfully imported '/gnu/store/90517zpibi4bqsrbmd9ih64l8044ixnx-export-test'
acquiring global GC lock `/var/guix/gc.lock'
acquiring read lock on `/var/guix/temproots/280'
acquiring write lock on `/var/guix/temproots/280'
downgrading to read lock on `/var/guix/temproots/280'
locking path `/gnu/store/r3d3r3xgf89axzp27krlj07rw6czlmcn-import-test'
lock acquired on `/gnu/store/r3d3r3xgf89axzp27krlj07rw6czlmcn-import-test.lock'
lock released on `/gnu/store/r3d3r3xgf89axzp27krlj07rw6czlmcn-import-test.lock'
retrieving 1 store item from 'localhost'...
guix offload: error: unauthorized public key: (public-key
(ecc
(curve Ed25519)
(q #842CFCACDF1E7F764A3E31621DA9AF70C2B3CA37222EC2C56C04228190F2EA48#)
)
)

[1] 08:33:34 janneke@dundal:~/src/guix/hurd64-team

...handle authorization..., I realized that the extra debugging is
printed on the host side, so I should also run the patched daemon there.
However, there was no more chance to do so because the offload test
started to work...

Toggle snippet (70 lines)
08:34:12 janneke@dundal:~/src/guix/hurd64-team
$ guix offload test
guix offload: testing 1 build machines defined in '/etc/guix/machines.scm'...
guix offload: Guix is usable on 'localhost' (test returned "/gnu/store/883yjkl46dxw9mzykykmbs0yzwyxm17z-test")
guix offload: 'localhost' is running GNU Guile 3.0.9
guix offload: sending 1 store item (0 MiB) to 'localhost'...
exporting path `/gnu/store/bvgz40q5x67iypa02qc7qb610f0vlb5d-export-test'
guix offload: 'localhost' successfully imported '/gnu/store/bvgz40q5x67iypa02qc7qb610f0vlb5d-export-test'
acquiring global GC lock `/var/guix/gc.lock'
acquiring read lock on `/var/guix/temproots/318'
acquiring write lock on `/var/guix/temproots/318'
downgrading to read lock on `/var/guix/temproots/318'
locking path `/gnu/store/zrqdfjvyyvyfz27678ia7jqidhf8557y-import-test'
lock acquired on `/gnu/store/zrqdfjvyyvyfz27678ia7jqidhf8557y-import-test.lock'
lock released on `/gnu/store/zrqdfjvyyvyfz27678ia7jqidhf8557y-import-test.lock'
retrieving 1 store item from 'localhost'...
guix offload: successfully imported '/gnu/store/zrqdfjvyyvyfz27678ia7jqidhf8557y-import-test' from 'localhost'
08:34:27 janneke@dundal:~/src/guix/hurd64-team
$ guix offload test
guix offload: testing 1 build machines defined in '/etc/guix/machines.scm'...
guix offload: Guix is usable on 'localhost' (test returned "/gnu/store/883yjkl46dxw9mzykykmbs0yzwyxm17z-test")
guix offload: 'localhost' is running GNU Guile 3.0.9
guix offload: sending 1 store item (0 MiB) to 'localhost'...
exporting path `/gnu/store/vcfgm1crwm5hfwmq2f04d67w5gix76yc-export-test'
guix offload: 'localhost' successfully imported '/gnu/store/vcfgm1crwm5hfwmq2f04d67w5gix76yc-export-test'
acquiring global GC lock `/var/guix/gc.lock'
acquiring read lock on `/var/guix/temproots/335'
acquiring write lock on `/var/guix/temproots/335'
downgrading to read lock on `/var/guix/temproots/335'
locking path `/gnu/store/g2gpgcqbaaj3gylrvskr7pdnq7w88b48-import-test'
lock acquired on `/gnu/store/g2gpgcqbaaj3gylrvskr7pdnq7w88b48-import-test.lock'
lock released on `/gnu/store/g2gpgcqbaaj3gylrvskr7pdnq7w88b48-import-test.lock'
retrieving 1 store item from 'localhost'...
guix offload: successfully imported '/gnu/store/g2gpgcqbaaj3gylrvskr7pdnq7w88b48-import-test' from 'localhost'
08:34:36 janneke@dundal:~/src/guix/hurd64-team
$ guix offload test
guix offload: testing 1 build machines defined in '/etc/guix/machines.scm'...
guix offload: Guix is usable on 'localhost' (test returned "/gnu/store/883yjkl46dxw9mzykykmbs0yzwyxm17z-test")
guix offload: 'localhost' is running GNU Guile 3.0.9
guix offload: sending 1 store item (0 MiB) to 'localhost'...
exporting path `/gnu/store/30np8abkfw4q70zkp779m2h0mhygzzkv-export-test'
guix offload: 'localhost' successfully imported '/gnu/store/30np8abkfw4q70zkp779m2h0mhygzzkv-export-test'
acquiring global GC lock `/var/guix/gc.lock'
acquiring read lock on `/var/guix/temproots/352'
acquiring write lock on `/var/guix/temproots/352'
downgrading to read lock on `/var/guix/temproots/352'
locking path `/gnu/store/381dqixhyg6dkm65cvp21qd19g49gp5h-import-test'
lock acquired on `/gnu/store/381dqixhyg6dkm65cvp21qd19g49gp5h-import-test.lock'
lock released on `/gnu/store/381dqixhyg6dkm65cvp21qd19g49gp5h-import-test.lock'
retrieving 1 store item from 'localhost'...
guix offload: successfully imported '/gnu/store/381dqixhyg6dkm65cvp21qd19g49gp5h-import-test' from 'localhost'
08:34:45 janneke@dundal:~/src/guix/hurd64-team
$ guix offload test
guix offload: testing 1 build machines defined in '/etc/guix/machines.scm'...
guix offload: Guix is usable on 'localhost' (test returned "/gnu/store/883yjkl46dxw9mzykykmbs0yzwyxm17z-test")
guix offload: 'localhost' is running GNU Guile 3.0.9
guix offload: sending 1 store item (0 MiB) to 'localhost'...
exporting path `/gnu/store/imb70z94d8050dl8nvwbxznfq4aq9jzh-export-test'
guix offload: 'localhost' successfully imported '/gnu/store/imb70z94d8050dl8nvwbxznfq4aq9jzh-export-test'
acquiring global GC lock `/var/guix/gc.lock'
acquiring read lock on `/var/guix/temproots/369'
acquiring write lock on `/var/guix/temproots/369'
downgrading to read lock on `/var/guix/temproots/369'
locking path `/gnu/store/prmbb24jl7shiqg56f9fips5izlai1l2-import-test'
lock acquired on `/gnu/store/prmbb24jl7shiqg56f9fips5izlai1l2-import-test.lock'
lock released on `/gnu/store/prmbb24jl7shiqg56f9fips5izlai1l2-import-test.lock'
retrieving 1 store item from 'localhost'...
guix offload: successfully imported '/gnu/store/prmbb24jl7shiqg56f9fips5izlai1l2-import-test' from 'localhost'

so, that's why I pushed the commit. When I tested, and later
re-asserted this patch didn't work, I only tried once or twice...
Guess I'm counting too strongly on deterministic behaviour. :)

Guix copy also works (two ways), yay!

Toggle snippet (16 lines)
$ guix copy --to=root@childhurd1 hello
guix copy: sending 0 store items (0 MiB) to 'localhost'...
08:38:55 janneke@dundal:~/src/guix/hurd64-team
$ guix copy --to=root@childhurd1 tar
guix copy: sending 1 store item (3 MiB) to 'localhost'...
/gnu/store/1w05qnl350b88g5wj1z5k2jywhhcxkyn-tar-1.34
08:58:54 janneke@dundal:~/src/guix/hurd-team
$ guix copy --from=root@childhurd1 /gnu/store/liwlc0500w5lq37p3skr8llqx436bbjr-profile.drv
retrieving 1 store item from 'localhost'...
/gnu/store/0mxnx8l4fgigvd7gakwdk6hc6im4wnai-disarchive-mirrors
/gnu/store/ajgpfirfcga0k48qvbf359b0ldgl2bic-mirrors
/gnu/store/wg1yp2vx8gb7qmcgyibqnwblahpp4bjg-content-addressed-mirrors
...
09:01:46 janneke@dundal:~/src/guix/hurd-team

Actually offloading shows the same behaviour as I observed using my
LC_ALL=C hack (with the 64bit childhurd):

Toggle snippet (23 lines)
$ ./pre-inst-env guix build --system=i586-gnu --expression='(@@ (gnu packages commencement) gnu-make-boot0)'
substitute: updating substitutes from 'http://192.168.178.20:8080'... 100.0%
substitute: updating substitutes from 'http://kluit.dezyne.org:8181'... 100.0%
substitute: updating substitutes from 'http://janneke.lilypond.org:8080'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
The following derivations will be built:
/gnu/store/9zznz8v0pbr7q4ix7syqhl6njrhw6xgf-glibc-stripped-2.39-i586-pc-gnu.tar.xz.drv
/gnu/store/zb9f8hnc1iyabi072251lc67d2syh8f9-glibc-bootstrap-0.drv
/gnu/store/xf4cnmaxkda92v9493lr43afq577nnx0-gcc-bootstrap-0.drv
/gnu/store/2469p0sxmw0m7d0g5xvdwgdzw95w947j-make-boot0-4.4.1.drv
process 11767 acquired build slot '/var/guix/offload/localhost:11022/0'
normalized load on machine 'localhost' is 1.00
waiting for locks or build slots...
process 11767 acquired build slot '/var/guix/offload/localhost:11022/0'
normalized load on machine 'localhost' is 1.00
process 11767 acquired build slot '/var/guix/offload/localhost:11022/0'
normalized load on machine 'localhost' is 1.00
process 11767 acquired build slot '/var/guix/offload/localhost:11022/0'
normalized load on machine 'localhost' is 1.00
...ad infinitum

...nothing happens. (Tried several times, let it run for 5min.) Maybe
that's another issue?

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
J
J
Janneke Nieuwenhuizen wrote on 6 Dec 19:49 +0100
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)(address . 73181@debbugs.gnu.org)
878qssbq9b.fsf@gnu.org
Ludovic Courtès writes:

Hello!

Toggle quote (17 lines)
> <janneke@gnu.org> skribis:
>
>> Anyway, using this patch 0001 it seems that suppressing the warnings
>> works, I no longer get
>>
>> "GC Warning: Repeated allocation of very large block (appr. size 112
>> KiB):\n\tMay lead to memory leak and poor performance\n"
>>
>>
>> but still get
>>
>> unexpected build daemon error: stoi
>
> Damnit. Could you check with rpctrace what the daemon receives?
>
> I wonder if I misunderstood what the root cause is.

TL;DR: Found it, attached is a patch to fix it.

Today I spent some time looking into this again, instrumented both
daemons with the attached patch for local-store.cc; a "guix copy
root@childhurd hello" gives:

host:
Toggle snippet (7 lines)
accepted connection from pid 21474, user janneke
00 nondigit: ` ' [32]
01 read until now: >>>0<<<
00 nondigit: `:' [58]
01 read until now: >>>430<<<

childhurd:
Toggle snippet (8 lines)
5 operations
00 nondigit: `G' [71]
01 read until now: >>><<<
02 nondigit: `G' [71]
03 nondigit: >>>GC Warning: Repeated allocation of very large block (appr. size 112 KiB):<<<
0 operations

...you already knew that from the rpctrace log.

So, the problem is that our patch doesrn't disable the warnings after all.
The guile-launcher has

guile-launcher.c:
Toggle snippet (12 lines)
#if defined __GNU__
/* XXX: On 32-bit GNU/Hurd (i586-gnu), libgc emits "Repeated allocation"
warnings that are annoying and interfere with communications between
'guix-daemon' and 'guix authenticate':
<https://issues.guix.gnu.org/73181>. Silence them. */
std::cerr << "silencing libgc warnings" << std::endl;
GC_set_warn_proc (no_warnings);
#endif
..
scm_boot_guile (argc, argv, inner_main, 0);

and then guile's gc.c just undoes that

init.c:
Toggle snippet (6 lines)
scm_i_init_guile (void *base)
{
..
scm_init_gc (); /* Requires hooks and `get_internal_run_time' */

gc.c:
Toggle snippet (7 lines)
void
scm_init_gc ()
{
..
GC_set_warn_proc (scm_gc_warn_proc);

Doh' So, attached is also a patch for Guix (that I made for Guile) that
fixed offloading again for me. Weirdly, I had to use
(overload-threshold #f) because my childhurd never falls below 1.0:

Toggle snippet (5 lines)
root@guixygnu ~# uptime
7:45:28 PM up 35 minutes, 0 users, load averages: 1.00, 1.01, 1.00


Toggle snippet (21 lines)
19:25:48 janneke@glimdal:~/src/guix/hurd-team
$ ./pre-inst-env guix build --system=i586-gnu -e '(@@ (gnu packages commencement) gnu-make-boot0)' --with-configure-flag=make-boot0=foo=bar
The following derivation will be built:
/gnu/store/cp1fy0ig5p8d72vnz1vybw2cilii5g88-make-boot0-4.4.1.drv
process 4454 acquired build slot '/var/guix/offload/localhost:11022/0'
normalized load on machine 'localhost' is 1.00
building /gnu/store/cp1fy0ig5p8d72vnz1vybw2cilii5g88-make-boot0-4.4.1.drv...
guix offload: sending 41 store items (259 MiB) to 'localhost'...
[..]
@ build-started /gnu/store/cp1fy0ig5p8d72vnz1vybw2cilii5g88-make-boot0-4.4.1.drv - i586-gnu /var/log/guix/drvs/cp//1fy0ig5p8d72vnz1vybw2cilii5g88-make-boot0-4.4.1.drv.gz 360
[..]
@ build-succeeded /gnu/store/cp1fy0ig5p8d72vnz1vybw2cilii5g88-make-boot0-4.4.1.drv -
retrieving 1 store item from 'localhost'...
importing file or directory '/gnu/store/j83d3mzcjw83lcbvyd8hrs6i6ymdqbmc-make-boot0-4.4.1'...
found valid signature for '/gnu/store/j83d3mzcjw83lcbvyd8hrs6i6ymdqbmc-make-boot0-4.4.1'
registering 1 items
done with offloaded '/gnu/store/cp1fy0ig5p8d72vnz1vybw2cilii5g88-make-boot0-4.4.1.drv'
successfully built /gnu/store/cp1fy0ig5p8d72vnz1vybw2cilii5g88-make-boot0-4.4.1.drv
/gnu/store/j83d3mzcjw83lcbvyd8hrs6i6ymdqbmc-make-boot0-4.4.1

WDYT?

Greetings,
Janneke
From 9695d17df1538460aad11500f56071500d9e4c80 Mon Sep 17 00:00:00 2001
Message-ID: <9695d17df1538460aad11500f56071500d9e4c80.1733510219.git.janneke@gnu.org>
From: Janneke Nieuwenhuizen <janneke@gnu.org>
Date: Mon, 18 Nov 2024 00:16:56 +0100
Subject: [PATCH] REMOVEME: stoi: add some debugging.

Change-Id: I114025c43cd2404a00dc65af7def06a312a31ca3
---
nix/libstore/local-store.cc | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)

Toggle diff (41 lines)
diff --git a/nix/libstore/local-store.cc b/nix/libstore/local-store.cc
index 0883a4bbce..e98bf04392 100644
--- a/nix/libstore/local-store.cc
+++ b/nix/libstore/local-store.cc
@@ -1164,9 +1164,9 @@ static std::shared_ptr<Agent> authenticationAgent()
static int readInteger(int fd)
{
string str;
+ char ch;
while (1) {
- char ch;
ssize_t rd = read(fd, &ch, 1);
if (rd == -1) {
if (errno != EINTR)
@@ -1177,11 +1177,20 @@ static int readInteger(int fd)
if (isdigit(ch)) {
str += ch;
} else {
- break;
+ std::cerr << "00 nondigit: `" << ch << "' [" << (int) ch << "]" << std::endl;
+ std::cerr << "01 read until now: >>>" << str << "<<<" << std::endl;
+ break;
}
}
}
+ if (str.empty()) {
+ std::cerr << "02 nondigit: `" << ch << "' [" << (int) ch << "]" << std::endl;
+ str += ch;
+ str += readLine(fd);
+ std::cerr << "03 nondigit: >>>" << str << "<<<" << std::endl;
+ throw EndOfFile("unexpected non-digit reading an integer");
+ }
return stoi(str);
}

base-commit: 889c396cc5fd948d2c874c2d4a51115f82591964
--
2.46.0
From c555134310728360cb3b12472e8d487421d57a7d Mon Sep 17 00:00:00 2001
Message-ID: <c555134310728360cb3b12472e8d487421d57a7d.1733509997.git.janneke@gnu.org>
From: Janneke Nieuwenhuizen <janneke@gnu.org>
Date: Fri, 6 Dec 2024 16:25:54 +0100
Subject: [PATCH] gnu: guile-3.0: Silence GC warnings on the Hurd.

* gnu/packages/patches/guile-hurd-silence-gc-warnings.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
* gnu/packages/guile.scm (guile-3.0): When building for the Hurd, use it in
new stage "patch-silence-gc-warnings".

Change-Id: I48f2641a162c3fab15655293e10c4aa2200d5843
---
gnu/local.mk | 1 +
gnu/packages/guile.scm | 11 ++++
.../guile-hurd-silence-gc-warnings.patch | 58 +++++++++++++++++++
3 files changed, 70 insertions(+)
create mode 100644 gnu/packages/patches/guile-hurd-silence-gc-warnings.patch

Toggle diff (102 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index a7bd32453b..d848ceddc4 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1520,6 +1520,7 @@ dist_patch_DATA = \
%D%/packages/patches/guile-fix-invalid-unicode-handling.patch \
%D%/packages/patches/guile-gdbm-ffi-support-gdbm-1.14.patch \
%D%/packages/patches/guile-hurd-posix-spawn.patch \
+ %D%/packages/patches/guile-hurd-silence-gc-warnings.patch \
%D%/packages/patches/guile-lzlib-hurd64.patch \
%D%/packages/patches/guile-present-coding.patch \
%D%/packages/patches/guile-rsvg-pkgconfig.patch \
diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
index 69dff9211f..41fbd895ed 100644
--- a/gnu/packages/guile.scm
+++ b/gnu/packages/guile.scm
@@ -413,6 +413,17 @@ (define-public guile-3.0
(search-patch "guile-hurd-posix-spawn.patch")))
(invoke "patch" "--force" "-p1" "-i" patch))))
#~())
+ #$@(if (target-hurd?)
+ #~((add-before 'build 'patch-silence-gc-warnings
+ (lambda _
+ ;; TODO: Move patch to 'source' on next rebuild
+ ;; cycle.
+ (define patch
+ #$(local-file
+ (search-patch
+ "guile-hurd-silence-gc-warnings.patch")))
+ (invoke "patch" "--force" "-p1" "-i" patch))))
+ #~())
#$@(if (system-hurd?)
#~((add-after 'unpack 'disable-popen.test-no-duplicate
;; This test hangs on the Hurd.
diff --git a/gnu/packages/patches/guile-hurd-silence-gc-warnings.patch b/gnu/packages/patches/guile-hurd-silence-gc-warnings.patch
new file mode 100644
index 0000000000..a006d93361
--- /dev/null
+++ b/gnu/packages/patches/guile-hurd-silence-gc-warnings.patch
@@ -0,0 +1,58 @@
+From 1256fb0925c5ff7c94249f53e0fb47d1ec280b3f Mon Sep 17 00:00:00 2001
+From: Janneke Nieuwenhuizen <janneke@gnu.org>
+Date: Fri, 6 Dec 2024 16:18:13 +0100
+Subject: [PATCH] Silence GC warnings on the Hurd.
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+On 32-bit GNU/Hurd (i586-gnu), libgc emits "Repeated allocation"
+warnings that are annoying and interfere with communications between
+'guix-daemon' and 'guix authenticate':
+<https://issues.guix.gnu.org/73181>.
+
+* libguile/gc.c (scm_gc_no_warnings)[__GNU__]: New procedure.
+(scm_init_gc)[__GNU__]: Use it to silence GC warnings.
+
+Co-authored-by: Ludovic Courtès <ludo@gnu.org>
+---
+ libguile/gc.c | 15 +++++++++++++++
+ 1 file changed, 15 insertions(+)
+
+diff --git a/libguile/gc.c b/libguile/gc.c
+index 3cbe43ec5..1ea4abcf8 100644
+--- a/libguile/gc.c
++++ b/libguile/gc.c
+@@ -84,6 +84,13 @@ int scm_debug_cells_gc_interval = 0;
+ garbage collection. */
+ static SCM scm_protects;
+
++#if defined __GNU__
++#include <gc.h>
++static void
++scm_gc_no_warnings (char *message, GC_word arg)
++{
++}
++#endif
+
+
+
+@@ -616,7 +623,15 @@ scm_init_gc ()
+ scm_c_hook_add (&scm_after_gc_c_hook, accumulate_gc_timer, NULL, 0);
+
+ GC_set_oom_fn (scm_oom_fn);
++#if __GNU__
++ /* XXX: On 32-bit GNU/Hurd (i586-gnu), libgc emits "Repeated allocation"
++ warnings that are annoying and interfere with communications between
++ 'guix-daemon' and 'guix authenticate':
++ <https://issues.guix.gnu.org/73181>. Silence them. */
++ GC_set_warn_proc (scm_gc_no_warnings);
++#else
+ GC_set_warn_proc (scm_gc_warn_proc);
++#endif
+ GC_set_start_callback (run_before_gc_c_hook);
+
+ #include "gc.x"
+--
+2.46.0
+

base-commit: 84fa76db424df8962ee21315ffd680c083edbf00
--
2.46.0
--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 10 Dec 09:07 +0100
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)(address . 73181@debbugs.gnu.org)
87plm0j700.fsf@inria.fr
Hi Janneke,

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (19 lines)
> So, the problem is that our patch doesrn't disable the warnings after all.
> The guile-launcher has
>
> guile-launcher.c:
>
> #if defined __GNU__
> /* XXX: On 32-bit GNU/Hurd (i586-gnu), libgc emits "Repeated allocation"
> warnings that are annoying and interfere with communications between
> 'guix-daemon' and 'guix authenticate':
> <https://issues.guix.gnu.org/73181>. Silence them. */
> std::cerr << "silencing libgc warnings" << std::endl;
> GC_set_warn_proc (no_warnings);
> #endif
> ..
> scm_boot_guile (argc, argv, inner_main, 0);
>
>
> and then guile's gc.c just undoes that

D’oh! Good catch!

Toggle quote (13 lines)
> From c555134310728360cb3b12472e8d487421d57a7d Mon Sep 17 00:00:00 2001
> Message-ID: <c555134310728360cb3b12472e8d487421d57a7d.1733509997.git.janneke@gnu.org>
> From: Janneke Nieuwenhuizen <janneke@gnu.org>
> Date: Fri, 6 Dec 2024 16:25:54 +0100
> Subject: [PATCH] gnu: guile-3.0: Silence GC warnings on the Hurd.
>
> * gnu/packages/patches/guile-hurd-silence-gc-warnings.patch: New file.
> * gnu/local.mk (dist_patch_DATA): Register it.
> * gnu/packages/guile.scm (guile-3.0): When building for the Hurd, use it in
> new stage "patch-silence-gc-warnings".
>
> Change-Id: I48f2641a162c3fab15655293e10c4aa2200d5843

You’ll hate me for suggesting that :-) but what if in ‘guile-launcher.c’
we moved the ‘GC_set_warn_proc’ call to ‘inner_main’? I believe in that
case it would have the desired effect, right?

That would avoid patching Guile wholesale.

Anyhow, kudos for the tricky and probably lengthy debugging session!

Thanks,
Ludo’.
J
J
Janneke Nieuwenhuizen wrote on 10 Dec 13:47 +0100
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)(address . 73181@debbugs.gnu.org)
8734iv3dsa.fsf@gnu.org
Ludovic Courtès writes:

Hey Ludo,
Toggle quote (24 lines)
>
> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> So, the problem is that our patch doesrn't disable the warnings after all.
>> The guile-launcher has
>>
>> guile-launcher.c:
>>
>> #if defined __GNU__
>> /* XXX: On 32-bit GNU/Hurd (i586-gnu), libgc emits "Repeated allocation"
>> warnings that are annoying and interfere with communications between
>> 'guix-daemon' and 'guix authenticate':
>> <https://issues.guix.gnu.org/73181>. Silence them. */
>> std::cerr << "silencing libgc warnings" << std::endl;
>> GC_set_warn_proc (no_warnings);
>> #endif
>> ..
>> scm_boot_guile (argc, argv, inner_main, 0);
>>
>>
>> and then guile's gc.c just undoes that
>
> D’oh! Good catch!

Yeah :-/

Toggle quote (2 lines)
> You’ll hate me for suggesting that :-)

You know better than that :)

Toggle quote (4 lines)
> but what if in ‘guile-launcher.c’
> we moved the ‘GC_set_warn_proc’ call to ‘inner_main’? I believe in that
> case it would have the desired effect, right?

Ah, I cannot believe I missed that -- I read "inner_main" and thought it
would call guile's inner main.

Toggle quote (4 lines)
> That would avoid patching Guile wholesale.
>
> Anyhow, kudos for the tricky and probably lengthy debugging session!

Yeah, so I'm proposing the attached patch -- tested and works for me --
well, apart from having to set overload-threshold. I asked on #hurd and
hopefully one of Damien's patch series


will take care of that; I'll try that some time soon.

Greetings,
Janneke
From 832ef198f7e423e0fdff7fe7c5558ff13d1f20c1 Mon Sep 17 00:00:00 2001
Message-ID: <832ef198f7e423e0fdff7fe7c5558ff13d1f20c1.1733834506.git.janneke@gnu.org>
From: Janneke Nieuwenhuizen <janneke@gnu.org>
Date: Tue, 10 Dec 2024 10:16:05 +0100
Subject: [PATCH] guile: Silence GC warnings on the Hurd, part II.

This is a follow-up to commit
b0416b8503e38746717a2d1167f9d1410634981e
guile: Silence GC warnings on the Hurd.

This should work around https://issues.guix.gnu.org/73181, resurrecting
offloading to the Hurd.

* gnu/packages/aux-files/guile-launcher.c: (main)[__GNU__]: Move code to
silence warnings to...
(inner_main)[__GNU__]: ...here.

Change-Id: I408201f7f4072c9a33837e6020ac63270b33702e
---
gnu/packages/aux-files/guile-launcher.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)

Toggle diff (38 lines)
diff --git a/gnu/packages/aux-files/guile-launcher.c b/gnu/packages/aux-files/guile-launcher.c
index bc7fa21b63..c205477f64 100644
--- a/gnu/packages/aux-files/guile-launcher.c
+++ b/gnu/packages/aux-files/guile-launcher.c
@@ -43,6 +43,14 @@ static const char *load_path, *load_compiled_path;
static void
inner_main (void *unused, int argc, char **argv)
{
+#if defined __GNU__
+ /* XXX: On 32-bit GNU/Hurd (i586-gnu), libgc emits "Repeated allocation"
+ warnings that are annoying and interfere with communications between
+ 'guix-daemon' and 'guix authenticate':
+ <https://issues.guix.gnu.org/73181>. Silence them. */
+ GC_set_warn_proc (no_warnings);
+#endif
+
if (load_path != NULL)
{
setenv ("GUILE_LOAD_PATH", load_path, 1);
@@ -82,14 +90,6 @@ main (int argc, char **argv)
which is always preferable over the C locale. */
setlocale (LC_ALL, "en_US.utf8");
-#if defined __GNU__
- /* XXX: On 32-bit GNU/Hurd (i586-gnu), libgc emits "Repeated allocation"
- warnings that are annoying and interfere with communications between
- 'guix-daemon' and 'guix authenticate':
- <https://issues.guix.gnu.org/73181>. Silence them. */
- GC_set_warn_proc (no_warnings);
-#endif
-
const char *str;
str = getenv ("GUILE_LOAD_PATH");
load_path = str != NULL ? strdup (str) : NULL;

base-commit: 93e1586116f39a30ba1fcb67bd839a43533dfaf4
--
2.46.0
--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 10 Dec 14:53 +0100
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)(address . 73181@debbugs.gnu.org)
87wmg7ir06.fsf@inria.fr
Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (7 lines)
> Yeah, so I'm proposing the attached patch -- tested and works for me --
> well, apart from having to set overload-threshold. I asked on #hurd and
> hopefully one of Damien's patch series
>
> <https://lists.gnu.org/archive/html/bug-hurd/2024-12/msg00100.html>
> <https://lists.gnu.org/archive/html/bug-hurd/2024-12/msg00114.html>

We could also set that field in ‘hurd-vm-guix-extension’, if that helps.

Toggle quote (19 lines)
> From 832ef198f7e423e0fdff7fe7c5558ff13d1f20c1 Mon Sep 17 00:00:00 2001
> Message-ID: <832ef198f7e423e0fdff7fe7c5558ff13d1f20c1.1733834506.git.janneke@gnu.org>
> From: Janneke Nieuwenhuizen <janneke@gnu.org>
> Date: Tue, 10 Dec 2024 10:16:05 +0100
> Subject: [PATCH] guile: Silence GC warnings on the Hurd, part II.
>
> This is a follow-up to commit
> b0416b8503e38746717a2d1167f9d1410634981e
> guile: Silence GC warnings on the Hurd.
>
> This should work around <https://issues.guix.gnu.org/73181>, resurrecting
> offloading to the Hurd.
>
> * gnu/packages/aux-files/guile-launcher.c: (main)[__GNU__]: Move code to
> silence warnings to...
> (inner_main)[__GNU__]: ...here.
>
> Change-Id: I408201f7f4072c9a33837e6020ac63270b33702e

LGTM! (We’ll have to update the ‘guix’ package, right?)

Thanks again!

Ludo’.
J
J
Janneke Nieuwenhuizen wrote on 10 Dec 15:02 +0100
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)(address . 73181@debbugs.gnu.org)
87ed2f1vqz.fsf@gnu.org
Ludovic Courtès writes:

Toggle quote (11 lines)
> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> Yeah, so I'm proposing the attached patch -- tested and works for me --
>> well, apart from having to set overload-threshold. I asked on #hurd and
>> hopefully one of Damien's patch series
>>
>> <https://lists.gnu.org/archive/html/bug-hurd/2024-12/msg00100.html>
>> <https://lists.gnu.org/archive/html/bug-hurd/2024-12/msg00114.html>
>
> We could also set that field in ‘hurd-vm-guix-extension’, if that helps.

Ah yes, let's do, that then, for now.

Toggle quote (21 lines)
>> From 832ef198f7e423e0fdff7fe7c5558ff13d1f20c1 Mon Sep 17 00:00:00 2001
>> Message-ID: <832ef198f7e423e0fdff7fe7c5558ff13d1f20c1.1733834506.git.janneke@gnu.org>
>> From: Janneke Nieuwenhuizen <janneke@gnu.org>
>> Date: Tue, 10 Dec 2024 10:16:05 +0100
>> Subject: [PATCH] guile: Silence GC warnings on the Hurd, part II.
>>
>> This is a follow-up to commit
>> b0416b8503e38746717a2d1167f9d1410634981e
>> guile: Silence GC warnings on the Hurd.
>>
>> This should work around <https://issues.guix.gnu.org/73181>, resurrecting
>> offloading to the Hurd.
>>
>> * gnu/packages/aux-files/guile-launcher.c: (main)[__GNU__]: Move code to
>> silence warnings to...
>> (inner_main)[__GNU__]: ...here.
>>
>> Change-Id: I408201f7f4072c9a33837e6020ac63270b33702e
>
> LGTM!

Pushed to master as 832ef198f7e423e0fdff7fe7c5558ff13d1f20c1.

Toggle quote (2 lines)
> (We’ll have to update the ‘guix’ package, right?)

Eh, yes!

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
J
J
Janneke Nieuwenhuizen wrote on 10 Dec 18:00 +0100
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)(address . 73181-done@debbugs.gnu.org)
87wmg7xykl.fsf@gnu.org
Ludovic Courtès writes:

Toggle quote (11 lines)
> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> Yeah, so I'm proposing the attached patch -- tested and works for me --
>> well, apart from having to set overload-threshold. I asked on #hurd and
>> hopefully one of Damien's patch series
>>
>> <https://lists.gnu.org/archive/html/bug-hurd/2024-12/msg00100.html>
>> <https://lists.gnu.org/archive/html/bug-hurd/2024-12/msg00114.html>
>
> We could also set that field in ‘hurd-vm-guix-extension’, if that helps.

Nice; done.

Toggle quote (21 lines)
>> From 832ef198f7e423e0fdff7fe7c5558ff13d1f20c1 Mon Sep 17 00:00:00 2001
>> Message-ID: <832ef198f7e423e0fdff7fe7c5558ff13d1f20c1.1733834506.git.janneke@gnu.org>
>> From: Janneke Nieuwenhuizen <janneke@gnu.org>
>> Date: Tue, 10 Dec 2024 10:16:05 +0100
>> Subject: [PATCH] guile: Silence GC warnings on the Hurd, part II.
>>
>> This is a follow-up to commit
>> b0416b8503e38746717a2d1167f9d1410634981e
>> guile: Silence GC warnings on the Hurd.
>>
>> This should work around <https://issues.guix.gnu.org/73181>, resurrecting
>> offloading to the Hurd.
>>
>> * gnu/packages/aux-files/guile-launcher.c: (main)[__GNU__]: Move code to
>> silence warnings to...
>> (inner_main)[__GNU__]: ...here.
>>
>> Change-Id: I408201f7f4072c9a33837e6020ac63270b33702e
>
> LGTM! (We’ll have to update the ‘guix’ package, right?)

Also done, pushed to master as 8966bd6c06df58316e01941577de1a7b1147ccbd.

With some luck, we have offloading again! With more luck, we can
configure 64bit offloading!

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
Closed
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 73181@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 73181
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch