diffutils always referrers to the native coreutils

  • Open
  • quality assurance status badge
Details
3 participants
  • Jan Nieuwenhuizen
  • Ludovic Courtès
  • Mathieu Othacehe
Owner
unassigned
Submitted by
Mathieu Othacehe
Severity
normal
M
M
Mathieu Othacehe wrote on 28 Aug 2020 09:36
(address . bug-guix@gnu.org)
87eenrkyrz.fsf@gnu.org
Hello,

The "diff" executable from diffutils stores a reference to the native,
canonical "pr" from coreutils. This is bad for two reasons:

* It makes the Guix System closure bigger by dragging the canonical
coreutils.

* The cross-compiled diffutils drags the native coreutils to its
closure, see:

Toggle snippet (13 lines)
mathieu@cervin:~$ guix size /gnu/store/5bj91pfnm3z6qbpbl0hp07w12arzk93k-diffutils-3.7
store item total self
/gnu/store/j48jp74s1j2mrh7nckg9asdyjv1hvi21-glibc-cross-aarch64-linux-gnu-2.31 158.9 71.4 34.8%
/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31 38.4 36.7 17.9%
/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib 71.0 32.6 15.9%
/gnu/store/xw23yv6fv0cb6n0sbc552l1mkc15yq3n-gcc-cross-aarch64-linux-gnu-7.5.0-lib 186.8 27.9 13.6%
/gnu/store/57xj5gcy1jbl9ai2lnrqnpr0dald9i65-coreutils-8.32 88.0 17.0 8.3%
/gnu/store/sl883skd8gbzcj4fl44w7dv1a5yaa3k4-gcc-cross-sans-libc-aarch64-linux-gnu-7.5.0-lib 80.8 9.8 4.8%
/gnu/store/66m8j38495zdcy2iv55m5d1hj3pxayff-linux-libre-headers-cross-aarch64-linux-gnu-5.4.20 5.1 5.1 2.5%
/gnu/store/bcjcd97xvh0qkvq1maqj6qab88xb30dv-bash-static-5.0.16 1.6 1.6 0.8%
/gnu/store/mmhimfwmmidf09jw1plw3aw1g1zn2nkh-bash-static-5.0.16 1.6 1.6 0.8%

The native version of glibc, gcc-lib, coreutils and bash-static are part
of the closure of the cross-compiled coreutils.

I hoped to fix it with the naive patch attached, but it fails for
obscure reasons while building acl:

Toggle snippet (10 lines)
FAIL: test/malformed-restore
============================

[4] $ cp "/tmp/guix-build-acl-2.2.53.drv-0/acl-2.2.53/test/malformed-restore-double-owner.acl" tmp.acl -- ok
[5] $ sed -i "s/USER/30001/g" tmp.acl -- failed
ERROR: ld.so: object '/tmp/guix-build-acl-2.2.53.drv-0/acl-2.2.53/.libs/libtestlookup.so' from LD_PRELOAD cannot be preloaded: ignored. != ~
[6] $ sed -i "s/GROUP/30000/g" tmp.acl -- failed
ERROR: ld.so: object '/tmp/guix-build-acl-2.2.53.drv-0/acl-2.2.53/.libs/libtestlookup.so' from LD_PRELOAD cannot be preloaded: ignored. != ~

Any idea, what's going on?

Thanks,

Mathieu
From 59ff84c360e25ac754cc17f285d9bbf077a5e6c3 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe@gnu.org>
Date: Fri, 28 Aug 2020 09:32:17 +0200
Subject: [PATCH] gnu: diffutils: Do not refer to canonical coreutils.

* gnu/packages/base.scm (diffutils)[inputs]: Add "coreutils",
[arguments]: and use it by passing PR_PROGRAM.
---
gnu/packages/base.scm | 6 ++++++
1 file changed, 6 insertions(+)

Toggle diff (20 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index f22e4ab9cc..99f8aefb67 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -269,7 +269,13 @@ differences.")
(base32
"09isrg0isjinv8c535nxsi1s86wfdfzml80dbw41dj9x3hiad9xk"))))
(build-system gnu-build-system)
+ (arguments
+ '(#:configure-flags
+ (list (string-append "PR_PROGRAM="
+ (assoc-ref %build-inputs "coreutils*")
+ "/bin/pr"))))
(native-inputs `(("perl" ,perl)))
+ (inputs `(("coreutils*" ,coreutils)))
(synopsis "Comparing and merging files")
(description
"GNU Diffutils is a package containing tools for finding the
--
2.24.0
L
L
Ludovic Courtès wrote on 28 Aug 2020 16:27
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 43081@debbugs.gnu.org)
87ft86ongg.fsf@gnu.org
Hi!

Mathieu Othacehe <othacehe@gnu.org> skribis:

Toggle quote (9 lines)
> The "diff" executable from diffutils stores a reference to the native,
> canonical "pr" from coreutils. This is bad for two reasons:
>
> * It makes the Guix System closure bigger by dragging the canonical
> coreutils.
>
> * The cross-compiled diffutils drags the native coreutils to its
> closure, see:

[...]

Toggle quote (12 lines)
> I hoped to fix it with the naive patch attached, but it fails for
> obscure reasons while building acl:
>
> FAIL: test/malformed-restore
> ============================
>
> [4] $ cp "/tmp/guix-build-acl-2.2.53.drv-0/acl-2.2.53/test/malformed-restore-double-owner.acl" tmp.acl -- ok
> [5] $ sed -i "s/USER/30001/g" tmp.acl -- failed
> ERROR: ld.so: object '/tmp/guix-build-acl-2.2.53.drv-0/acl-2.2.53/.libs/libtestlookup.so' from LD_PRELOAD cannot be preloaded: ignored. != ~
> [6] $ sed -i "s/GROUP/30000/g" tmp.acl -- failed
> ERROR: ld.so: object '/tmp/guix-build-acl-2.2.53.drv-0/acl-2.2.53/.libs/libtestlookup.so' from LD_PRELOAD cannot be preloaded: ignored. != ~

Could it be that there are two different libcs involved?

Do you know which acl is fixing (commencement.scm vs. acl.scm)?

Thanks,
Ludo’.
M
M
Mathieu Othacehe wrote on 28 Aug 2020 16:51
(name . Ludovic Courtès)(address . ludo@gnu.org)
871rjq2589.fsf@gnu.org
Hey Ludo!

Toggle quote (2 lines)
> Could it be that there are two different libcs involved?

I can only spot one in environment-variables file.

Toggle quote (2 lines)
> Do you know which acl is fixing (commencement.scm vs. acl.scm)?

Yes it's the acl that is rooted in the bootstrap toolchain. Here's the
full build log attached. I'm also cc-ing janneke which has some
experience in that matter :).

Thanks for having a look,

Mathieu
Attachment: log
L
L
Ludovic Courtès wrote on 31 Aug 2020 14:00
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87k0xfdnys.fsf@gnu.org
Hi,

Mathieu Othacehe <othacehe@gnu.org> skribis:

Toggle quote (6 lines)
>> Do you know which acl is fixing (commencement.scm vs. acl.scm)?
>
> Yes it's the acl that is rooted in the bootstrap toolchain. Here's the
> full build log attached. I'm also cc-ing janneke which has some
> experience in that matter :).

Yes, now that you mention it, I remember janneke and I looking at a
similar LD_PRELOAD issue in Marrakesh last December. :-)

Toggle quote (3 lines)
> [24] $ setfacl -m u:bin:rx /test -- failed
> ERROR: ld.so: object '/tmp/guix-build-acl-2.2.53.drv-0/acl-2.2.53/.libs/libtestlookup.so' from LD_PRELOAD cannot be preloaded: ignored. != ~

Does that ring a bell, janneke?

Ludo’.
J
J
Jan Nieuwenhuizen wrote on 31 Aug 2020 14:38
(name . Ludovic Courtès)(address . ludo@gnu.org)
878sdvj8i6.fsf@gnu.org
Ludovic Courtès writes:

Toggle quote (18 lines)
> Hi,
>
> Mathieu Othacehe <othacehe@gnu.org> skribis:
>
>>> Do you know which acl is fixing (commencement.scm vs. acl.scm)?
>>
>> Yes it's the acl that is rooted in the bootstrap toolchain. Here's the
>> full build log attached. I'm also cc-ing janneke which has some
>> experience in that matter :).
>
> Yes, now that you mention it, I remember janneke and I looking at a
> similar LD_PRELOAD issue in Marrakesh last December. :-)
>
>> [24] $ setfacl -m u:bin:rx /test -- failed
>> ERROR: ld.so: object '/tmp/guix-build-acl-2.2.53.drv-0/acl-2.2.53/.libs/libtestlookup.so' from LD_PRELOAD cannot be preloaded: ignored. != ~
>
> Does that ring a bell, janneke?

Oh my...I do remember we looked into this or something very similar.

Is it doable to run the test suite using the canonical PR again( maybe
setenv "PR_PROGRAM" just before the test phase or some substituting)?

Janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
?