[PATCH core-updates-frozen] gnu: diffutils: Fix signal processing.

OpenSubmitted by Thiago Jung Bauermann.
Details
3 participants
  • Thiago Jung Bauermann
  • Ludovic Courtès
  • Maxime Devos
Owner
unassigned
Severity
normal
T
T
Thiago Jung Bauermann wrote on 28 Aug 18:43 +0200
(address . guix-patches@gnu.org)(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)
20210828164357.8868-1-bauermann@kolabnow.com
diffutils has a race condition in its signal processing code which is easyto trigger on powerpc64le-linux. More often than not, it causes the‘colors’ test to fail and therefore the build of the package fails as well.
Add the patch proposed in Debian bug 922552 which fixes the problem.
* gnu/packages/patches/diffutils-fix-signal-processing.patch: New file.* gnu/local.mk (dist_patch_DATA): Add it.* gnu/packages/base.scm (diffutils)[source]: Use it.---
Hello,
This fixes the build of diffutils on powerpc64le-linux, which currentlyfails more often than not. The patch I’m adding here isn’t beingshipped by Debian and hasn’t been seen by upstream yet. I just broughtit to their attention here:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34519#11
I’m not familiar with the diffutils code base, but FWIW I analysedthe patch and it looks very reasonable to me. To be honest I’m notsure if it completely fixes the race condition or just makes it muchless likely to happen, but in any case I can’t hit the race conditionanymore.
In addition, since all it does is add a new call to the function whichchecks and processes any pending signal, I don’t think it can causeany harm.
Finally, this patch is based on top of the one which updates diffutilsto version 3.8:
https://issues.guix.gnu.org/50233
The fix works equally well in version 3.7 so if you think it’s notworth updating diffutils I can rebase this patch on top of current‘core-updates-frozen’.
gnu/local.mk | 1 + gnu/packages/base.scm | 3 +- .../diffutils-fix-signal-processing.patch | 60 +++++++++++++++++++ 3 files changed, 63 insertions(+), 1 deletion(-) create mode 100644 gnu/packages/patches/diffutils-fix-signal-processing.patch
Toggle diff (92 lines)diff --git a/gnu/local.mk b/gnu/local.mkindex 11b002b66e72..bc385eecc592 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -962,6 +962,7 @@ dist_patch_DATA = \ %D%/packages/patches/desmume-gcc6-fixes.patch \ %D%/packages/patches/desmume-gcc7-fixes.patch \ %D%/packages/patches/dfu-programmer-fix-libusb.patch \+ %D%/packages/patches/diffutils-fix-signal-processing.patch \ %D%/packages/patches/diffutils-gets-undeclared.patch \ %D%/packages/patches/disarchive-cross-compilation.patch \ %D%/packages/patches/dkimproxy-add-ipv6-support.patch \diff --git a/gnu/packages/base.scm b/gnu/packages/base.scmindex 2c648953ae39..0e3b346b93a0 100644--- a/gnu/packages/base.scm+++ b/gnu/packages/base.scm@@ -272,7 +272,8 @@ differences.") version ".tar.xz")) (sha256 (base32- "1v4g8gi0lgakqa7iix8s4fq7lq6l92vw3rjd9wfd2rhjng8xggd6"))))+ "1v4g8gi0lgakqa7iix8s4fq7lq6l92vw3rjd9wfd2rhjng8xggd6"))+ (patches (search-patches "diffutils-fix-signal-processing.patch")))) (build-system gnu-build-system) (native-inputs (list perl)) (synopsis "Comparing and merging files")diff --git a/gnu/packages/patches/diffutils-fix-signal-processing.patch b/gnu/packages/patches/diffutils-fix-signal-processing.patchnew file mode 100644index 000000000000..24130bd4c37a--- /dev/null+++ b/gnu/packages/patches/diffutils-fix-signal-processing.patch@@ -0,0 +1,60 @@+Author: Frédéric Bonnard <frediz@debian.org>++Obtained from:++https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=922552#19++and slightly adapted to apply on v3.8.++Fixes bug reported upstream at:++https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34519++diff --git a/src/diff.c b/src/diff.c+index 9938daa0c8fd..2bc443f1ca70 100644+--- a/src/diff.c++++ b/src/diff.c+@@ -1453,6 +1453,8 @@ compare_files (struct comparison const *parent,+ }+ }+ ++ final_process_signals ();+++ /* Now the comparison has been done, if no error prevented it,+ and STATUS is the value this function will return. */+ +diff --git a/src/diff.h b/src/diff.h+index 27362c010fd2..28c89b0797ef 100644+--- a/src/diff.h++++ b/src/diff.h+@@ -390,6 +390,7 @@ extern enum changes analyze_hunk (struct change *, lin *, lin *, lin *, lin *);+ extern void begin_output (void);+ extern void debug_script (struct change *);+ extern void fatal (char const *) __attribute__((noreturn));++extern void final_process_signals (void);+ extern void finish_output (void);+ extern void message (char const *, char const *, char const *);+ extern void message5 (char const *, char const *, char const *,+diff --git a/src/util.c b/src/util.c+index 4348757e1507..8954197f33fc 100644+--- a/src/util.c++++ b/src/util.c+@@ -237,6 +237,18 @@ process_signals (void)+ }+ }+ ++/* Process remaining signals once before exit */++void++final_process_signals (void)++{++ static int last = 1;++++ if (last) {++ process_signals ();++ last = 0;++ }++}+++ static void+ install_signal_handlers (void)+ {
M
M
Maxime Devos wrote on 30 Aug 13:43 +0200
d7f74cb51997e491e98564df10c1b4c9144ef9de.camel@telenet.be
Thiago Jung Bauermann via Guix-patches via schreef op za 28-08-2021 om 13:43 [-0300]:
Toggle quote (6 lines)> [...]> diffutils has a race condition in its signal processing code which is easy> to trigger on powerpc64le-linux. More often than not, it causes the> ‘colors’ test to fail and therefore the build of the package fails as well.> [...]
Adding this patch makes sense to me too, though I don't have a powerpc64le-linuxto test this on.
Toggle quote (2 lines)> Add the patch proposed in Debian bug 922552 which fixes the problem.
Do you think upstream will have its own working patch soonish?(See https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36488#41, for peoplewho aren't Thiago Jung Bauermann.) If so, it might make sense to wait a littlefor an ‘official’ patch.
Toggle quote (134 lines)> * gnu/packages/patches/diffutils-fix-signal-processing.patch: New file.> * gnu/local.mk (dist_patch_DATA): Add it.> * gnu/packages/base.scm (diffutils)[source]: Use it.> ---> > Hello,> > This fixes the build of diffutils on powerpc64le-linux, which currently> fails more often than not. The patch I’m adding here isn’t being> shipped by Debian and hasn’t been seen by upstream yet. I just brought> it to their attention here:> > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34519#11> > I’m not familiar with the diffutils code base, but FWIW I analysed> the patch and it looks very reasonable to me. To be honest I’m not> sure if it completely fixes the race condition or just makes it much> less likely to happen, but in any case I can’t hit the race condition> anymore.> > In addition, since all it does is add a new call to the function which> checks and processes any pending signal, I don’t think it can cause> any harm.> > Finally, this patch is based on top of the one which updates diffutils> to version 3.8:> > https://issues.guix.gnu.org/50233> > The fix works equally well in version 3.7 so if you think it’s not> worth updating diffutils I can rebase this patch on top of current> ‘core-updates-frozen’.> > gnu/local.mk | 1 +> gnu/packages/base.scm | 3 +-> .../diffutils-fix-signal-processing.patch | 60 +++++++++++++++++++> 3 files changed, 63 insertions(+), 1 deletion(-)> create mode 100644 gnu/packages/patches/diffutils-fix-signal-processing.patch> > diff --git a/gnu/local.mk b/gnu/local.mk> index 11b002b66e72..bc385eecc592 100644> --- a/gnu/local.mk> +++ b/gnu/local.mk> @@ -962,6 +962,7 @@ dist_patch_DATA = \> %D%/packages/patches/desmume-gcc6-fixes.patch \> %D%/packages/patches/desmume-gcc7-fixes.patch \> %D%/packages/patches/dfu-programmer-fix-libusb.patch \> + %D%/packages/patches/diffutils-fix-signal-processing.patch \> %D%/packages/patches/diffutils-gets-undeclared.patch \> %D%/packages/patches/disarchive-cross-compilation.patch \> %D%/packages/patches/dkimproxy-add-ipv6-support.patch \> diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm> index 2c648953ae39..0e3b346b93a0 100644> --- a/gnu/packages/base.scm> +++ b/gnu/packages/base.scm> @@ -272,7 +272,8 @@ differences.")> version ".tar.xz"))> (sha256> (base32> - "1v4g8gi0lgakqa7iix8s4fq7lq6l92vw3rjd9wfd2rhjng8xggd6"))))> + "1v4g8gi0lgakqa7iix8s4fq7lq6l92vw3rjd9wfd2rhjng8xggd6"))> + (patches (search-patches "diffutils-fix-signal-processing.patch"))))> (build-system gnu-build-system)> (native-inputs (list perl))> (synopsis "Comparing and merging files")> diff --git a/gnu/packages/patches/diffutils-fix-signal-processing.patch b/gnu/packages/patches/diffutils-fix-signal-processing.patch> new file mode 100644> index 000000000000..24130bd4c37a> --- /dev/null> +++ b/gnu/packages/patches/diffutils-fix-signal-processing.patch> @@ -0,0 +1,60 @@> +Author: Frédéric Bonnard <frediz@debian.org>> +> +Obtained from:> +> +https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=922552#19> +> +and slightly adapted to apply on v3.8.> +> +Fixes bug reported upstream at:> +> +https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34519> +> +diff --git a/src/diff.c b/src/diff.c> +index 9938daa0c8fd..2bc443f1ca70 100644> +--- a/src/diff.c> ++++ b/src/diff.c> +@@ -1453,6 +1453,8 @@ compare_files (struct comparison const *parent,> + }> + }> + > ++ final_process_signals ();> ++> + /* Now the comparison has been done, if no error prevented it,> + and STATUS is the value this function will return. */> + > +diff --git a/src/diff.h b/src/diff.h> +index 27362c010fd2..28c89b0797ef 100644> +--- a/src/diff.h> ++++ b/src/diff.h> +@@ -390,6 +390,7 @@ extern enum changes analyze_hunk (struct change *, lin *, lin *, lin *, lin *);> + extern void begin_output (void);> + extern void debug_script (struct change *);> + extern void fatal (char const *) __attribute__((noreturn));> ++extern void final_process_signals (void);> + extern void finish_output (void);> + extern void message (char const *, char const *, char const *);> + extern void message5 (char const *, char const *, char const *,> +diff --git a/src/util.c b/src/util.c> +index 4348757e1507..8954197f33fc 100644> +--- a/src/util.c> ++++ b/src/util.c> +@@ -237,6 +237,18 @@ process_signals (void)> + }> + }> + > ++/* Process remaining signals once before exit */> ++void> ++final_process_signals (void)> ++{> ++ static int last = 1;> ++> ++ if (last) {> ++ process_signals ();> ++ last = 0;> ++ }> ++}> ++> + static void> + install_signal_handlers (void)> + {> > >
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYSzEbhccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7rUlAQCbcxjoa5V6n4oZnEv6nbnZT+9Pb8qf7Nbj7wDRExUt+QEAliiATW9dvq36BWlPAGy+AWNTSUxutctcsAvnQLFH4Ao==W8ld-----END PGP SIGNATURE-----

T
T
Thiago Jung Bauermann wrote on 30 Aug 15:39 +0200
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 50239@debbugs.gnu.org)
3088529.eW9a4NE73H@popigai
Hello Maxime,
Thank you for your review!
Em segunda-feira, 30 de agosto de 2021, às 08:43:42 -03, Maxime Devos escreveu:
Toggle quote (1 lines)> Thiago Jung Bauermann via Guix-patches via schreef op za 28-08-2021 om
13:43 [-0300]:
Toggle quote (9 lines)> > [...]> > diffutils has a race condition in its signal processing code which is> > easy to trigger on powerpc64le-linux. More often than not, it causes> > the ‘colors’ test to fail and therefore the build of the package fails> > as well. [...]> > Adding this patch makes sense to me too, though I don't have a> powerpc64le-linux to test this on.
Nice, thanks!
Toggle quote (4 lines)> > Add the patch proposed in Debian bug 922552 which fixes the problem.> > Do you think upstream will have its own working patch soonish?
I think so. Paul Eggert was very quick in his response and is already working on a fix.
Toggle quote (3 lines)> (See https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36488#41, for people> who aren't Thiago Jung Bauermann.)
Yeah, the discussion moved unexpectedly to a different bug report. Thanks for tracking it down and posting the new link.
Toggle quote (2 lines)> If so, it might make sense to wait a little for an ‘official’ patch.
Yes, I agree.
-- Thanks,Thiago
L
L
Ludovic Courtès wrote on 6 Sep 13:55 +0200
Re: bug#50239: [PATCH core-updates-frozen] gnu: diffutils: Fix signal processing.
(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)
877dft9auv.fsf@gnu.org
Hi,
Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:
Toggle quote (7 lines)> This fixes the build of diffutils on powerpc64le-linux, which currently> fails more often than not. The patch I’m adding here isn’t being> shipped by Debian and hasn’t been seen by upstream yet. I just brought> it to their attention here:>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34519#11
Thanks for working on this!
Toggle quote (19 lines)> I’m not familiar with the diffutils code base, but FWIW I analysed> the patch and it looks very reasonable to me. To be honest I’m not> sure if it completely fixes the race condition or just makes it much> less likely to happen, but in any case I can’t hit the race condition> anymore.>> In addition, since all it does is add a new call to the function which> checks and processes any pending signal, I don’t think it can cause> any harm.>> Finally, this patch is based on top of the one which updates diffutils> to version 3.8:>> https://issues.guix.gnu.org/50233>> The fix works equally well in version 3.7 so if you think it’s not> worth updating diffutils I can rebase this patch on top of current> ‘core-updates-frozen’.
Normally we won’t update diffutils on ‘core-updates-frozen’.
Thus, could you adjust this patch so that (1) it applies on‘core-updates-frozen’, and (2) it doesn’t change derivations on otherplatforms (thus, the patch needs to be applied from a build phase)?Bonus point if there’s an upstream patch to use, as Maxime suggested.
TIA,Ludo’.
T
T
Thiago Jung Bauermann wrote on 6 Sep 15:58 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)
20395088.hJciOmStQJ@popigai
Hi Ludo,
Em segunda-feira, 6 de setembro de 2021, às 08:55:52 -03, Ludovic Courtès escreveu:
Toggle quote (12 lines)> Hi,> > Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:> > This fixes the build of diffutils on powerpc64le-linux, which currently> > fails more often than not. The patch I’m adding here isn’t being> > shipped by Debian and hasn’t been seen by upstream yet. I just brought> > it to their attention here:> > > > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34519#11> > Thanks for working on this!
You’re welcome. Thank you for reviewing the patches!
Toggle quote (21 lines)> > I’m not familiar with the diffutils code base, but FWIW I analysed> > the patch and it looks very reasonable to me. To be honest I’m not> > sure if it completely fixes the race condition or just makes it much> > less likely to happen, but in any case I can’t hit the race condition> > anymore.> > > > In addition, since all it does is add a new call to the function which> > checks and processes any pending signal, I don’t think it can cause> > any harm.> > > > Finally, this patch is based on top of the one which updates diffutils> > to version 3.8:> > > > https://issues.guix.gnu.org/50233> > > > The fix works equally well in version 3.7 so if you think it’s not> > worth updating diffutils I can rebase this patch on top of current> > ‘core-updates-frozen’.> > Normally we won’t update diffutils on ‘core-updates-frozen’.
Yes, it makes sense.
Toggle quote (4 lines)> Thus, could you adjust this patch so that (1) it applies on> ‘core-updates-frozen’, and (2) it doesn’t change derivations on other> platforms (thus, the patch needs to be applied from a build phase)?
Sure, no problem. I’ll do that. Thanks for the tip about having to apply the patch from a build phase.
Toggle quote (2 lines)> Bonus point if there’s an upstream patch to use, as Maxime suggested.
There isn’t one for now. We have three alternatives if the situation doesn’t change by the time ‘core-updates-frozen’ gets merged:
1. Apply this patch.
2. Disable the ‘colors’ test. This is what Debian is doing. It’s a bit unfortunate in that we’re disabling a test that just did its job of finding a bug, but on the other hand the bug is minor (the terminal is left in a wrong state if ‘diff’ is interrupted in the middle of printing colored output) and easy to work around (just run ‘reset’ if that happens).
3. Do nothing. If the diffutils build fails on the CI, restart it until it succeeds. :-)
That is my personal order of preference, but option 1 carries a slight risk in that we would be shipping a patch that isn’t “battle tested”, so the other options are very reasonable too.
-- Thanks,Thiago
T
T
Thiago Jung Bauermann wrote on 9 Sep 16:41 +0200
[PATCH core-updates-frozen v2] gnu: diffutils: Fix signal processing.
(address . 50239@debbugs.gnu.org)(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)
20210909144114.5850-1-bauermann@kolabnow.com
diffutils has a race condition in its signal processing code which is easyto trigger on powerpc64le-linux. More often than not, it causes the‘colors’ test to fail and therefore the build of the package fails as well.
Add the patch proposed in Debian bug 922552 which fixes the problem.
* gnu/packages/patches/diffutils-fix-signal-processing.patch: New file.* gnu/local.mk (dist_patch_DATA): Add it.* gnu/packages/base.scm (diffutils)[source]: Use it.---
Hello,
Here is the same patch rebased on top of ‘core-updates-frozen’. I didn’tyet do it in a way that preserves derivations on other platforms becausethere will be a batch of world-rebuilding changes¹, so depending on howsoon the branch is going to be merged into master, it could be a goodopportunity to land this fix now.
So far, there hasn’t been news from upstream on a fix for the problem,but it’s been only a bit over a week since they last proposed a patchso it’s too early to be worried about anything.
I am perfectly fine with skipping this world-rebuilding window and sendinga v3 which preserves derivations on other platforms as you suggested. Oreven not fixing this bug on ’core-updates-frozen’ if the upstream fixdoesn’t arrive by then. As I mentioned earlier, we can just hit the “tryagain” button until the CI produces a successful build. :-)
I’m just sending this version to have the option on the table.
Thanks!
¹ https://lists.nongnu.org/archive/html/guix-devel/2021-09/msg00097.html
gnu/local.mk | 1 + gnu/packages/base.scm | 3 +- .../diffutils-fix-signal-processing.patch | 58 +++++++++++++++++++ 3 files changed, 61 insertions(+), 1 deletion(-) create mode 100644 gnu/packages/patches/diffutils-fix-signal-processing.patch
Toggle diff (90 lines)diff --git a/gnu/local.mk b/gnu/local.mkindex 07e6787642e9..d37b1b577a04 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -967,6 +967,7 @@ dist_patch_DATA = \ %D%/packages/patches/desmume-gcc6-fixes.patch \ %D%/packages/patches/desmume-gcc7-fixes.patch \ %D%/packages/patches/dfu-programmer-fix-libusb.patch \+ %D%/packages/patches/diffutils-fix-signal-processing.patch \ %D%/packages/patches/diffutils-gets-undeclared.patch \ %D%/packages/patches/disarchive-cross-compilation.patch \ %D%/packages/patches/dkimproxy-add-ipv6-support.patch \diff --git a/gnu/packages/base.scm b/gnu/packages/base.scmindex 70eb135a1e41..bfe769fbc4ed 100644--- a/gnu/packages/base.scm+++ b/gnu/packages/base.scm@@ -274,7 +274,8 @@ differences.") (sha256 (base32 "09isrg0isjinv8c535nxsi1s86wfdfzml80dbw41dj9x3hiad9xk"))- (patches (search-patches "coreutils-gnulib-tests.patch"))))+ (patches (search-patches "coreutils-gnulib-tests.patch"+ "diffutils-fix-signal-processing.patch")))) (build-system gnu-build-system) (native-inputs (list perl)) (synopsis "Comparing and merging files")diff --git a/gnu/packages/patches/diffutils-fix-signal-processing.patch b/gnu/packages/patches/diffutils-fix-signal-processing.patchnew file mode 100644index 000000000000..134dd3f718bd--- /dev/null+++ b/gnu/packages/patches/diffutils-fix-signal-processing.patch@@ -0,0 +1,58 @@+Author: Frédéric Bonnard <frediz@debian.org>++Obtained from:++https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=922552#19++Fixes bug reported upstream at:++https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34519++diff --git a/src/diff.c b/src/diff.c+index e2eb32437353..b574e8282dc9 100644+--- a/src/diff.c++++ b/src/diff.c+@@ -1451,6 +1451,8 @@ compare_files (struct comparison const *parent,+ }+ }+ ++ final_process_signals ();+++ /* Now the comparison has been done, if no error prevented it,+ and STATUS is the value this function will return. */+ +diff --git a/src/diff.h b/src/diff.h+index 03daaa4a0530..e177fe600a25 100644+--- a/src/diff.h++++ b/src/diff.h+@@ -390,6 +390,7 @@ extern enum changes analyze_hunk (struct change *, lin *, lin *, lin *, lin *);+ extern void begin_output (void);+ extern void debug_script (struct change *);+ extern void fatal (char const *) __attribute__((noreturn));++extern void final_process_signals (void);+ extern void finish_output (void);+ extern void message (char const *, char const *, char const *);+ extern void message5 (char const *, char const *, char const *,+diff --git a/src/util.c b/src/util.c+index 4f4d9bb285eb..56d292de2927 100644+--- a/src/util.c++++ b/src/util.c+@@ -237,6 +237,18 @@ process_signals (void)+ }+ }+ ++/* Process remaining signals once before exit */++void++final_process_signals (void)++{++ static int last = 1;++++ if (last) {++ process_signals ();++ last = 0;++ }++}+++ static void+ install_signal_handlers (void)+ {
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send email to 50239@debbugs.gnu.org