Go retains a reference to GCC

OpenSubmitted by Ludovic Courtès.
Details
2 participants
  • Sarah Morgensen
  • Ludovic Courtès
Owner
unassigned
Severity
normal
L
L
Ludovic Courtès wrote on 2 Feb 2020 23:42
(address . bug-Guix@gnu.org)
87v9oo391q.fsf@gnu.org
Hello!
It seems that Go unduly retains a reference to GCC:
Toggle snippet (30 lines)$ guix size gostore item total self/gnu/store/g4rrz9rnr8ssbnj3gx3dmsxv4glll8nf-go-1.12.15 646.3 355.9 55.1%/gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0 177.4 107.2 16.6%/gnu/store/ziinjmbnq004866mwjrczsk12wf35qb8-perl-5.30.0 148.2 57.1 8.8%/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29 37.4 35.8 5.5%/gnu/store/64xzk1kwk30n7arqgmxs2qy7wzcnc8i3-gcc-7.4.0-lib 70.1 32.8 5.1%/gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib 70.0 32.6 5.0%/gnu/store/9kzrrccpzl6i1sfwb0drb00gi2gwk0x0-coreutils-8.31 90.0 16.5 2.6%/gnu/store/dvs3acxwfnwgc7yma6h3y937ri2li47y-gmp-6.1.2 72.6 2.6 0.4%/gnu/store/9mmsilz9avdl49i6a6nj5mzfyim8ihv2-tzdata-2019c 2.0 2.0 0.3%/gnu/store/cp72ncw4prnsga65n3pzll07hpsg524f-bash-static-5.0.7 1.6 1.6 0.2%/gnu/store/29jhbbg1hf557x8j53f9sxd9imlmf02a-bash-minimal-5.0.7 38.4 1.0 0.2%/gnu/store/nffbgghxyvrj29lcgxs5fpmi3sx9zzql-acl-2.2.53 70.7 0.5 0.1%/gnu/store/in1738m2zvhgpz78n2yqa972sdzc42ss-attr-2.4.48 70.3 0.3 0.0%/gnu/store/y127kw0vmw46zcrvfz0i5s2rynq7ddjv-zlib-1.2.11 37.6 0.2 0.0%/gnu/store/waw5ci4lazbf2a1x9v6gw1v274nk0gny-libcap-2.27 70.2 0.2 0.0%/gnu/store/hzk6d08yiih0alc2raciah570plkjxzh-net-base-5.3 0.0 0.0 0.0%total: 646.3 MiB$ guix describeGeneracio 126 Jan 27 2020 23:59:19 (nuna) guix 93f4511 repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: 93f4511eb0c9b33f5083c2a04f4148e0a494059c$ grep -r x3jx25cd3q363mr7nbgzrhrv1vza6cf7 /gnu/store/g4rrz9rnr8ssbnj3gx3dmsxv4glll8nf-go-1.12.15Duuma dosiero /gnu/store/g4rrz9rnr8ssbnj3gx3dmsxv4glll8nf-go-1.12.15/bin/go kongruasDuuma dosiero /gnu/store/g4rrz9rnr8ssbnj3gx3dmsxv4glll8nf-go-1.12.15/pkg/tool/linux_amd64/cgo kongruas
Ludo’.
S
S
Sarah Morgensen wrote on 4 Jul 07:30 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)
86k0m6abn2.fsf@mgsn.dev
Hello all,
(I have CC'd Tobias since we briefly discussed this on #guix recently.)
Ludovic Courtès <ludo@gnu.org> writes:
Toggle quote (10 lines)> Hello!>> It seems that Go unduly retains a reference to GCC:>> $ guix size go> store item total self> /gnu/store/g4rrz9rnr8ssbnj3gx3dmsxv4glll8nf-go-1.12.15 646.3 355.9 55.1%> /gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0 177.4 107.2 16.6%> [...]
This is still the case:
$ guix size gostore item total self/gnu/store/vvly7zgn981brb37v8y8a7f9psx7c6ay-go-1.16.5 570.0 371.5 65.2%/gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1y17lk-gcc-7.5.0 178.5 107.3 18.8%/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31 38.4 36.7 6.4%/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib 71.0 32.6 5.7%/gnu/store/57xj5gcy1jbl9ai2lnrqnpr0dald9i65-coreutils-8.32 88.0 17.0 3.0%/gnu/store/kl68v5mclwp511xgpsl2h1s9gmsdxpzh-tzdata-2021a 1.9 1.9 0.3%/gnu/store/mmhimfwmmidf09jw1plw3aw1g1zn2nkh-bash-static-5.0.16 1.6 1.6 0.3%/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16 39.4 1.0 0.2%/gnu/store/g2s5jfkfd4k973wb58476b1bbv9zpm6m-zlib-1.2.11 38.6 0.2 0.0%/gnu/store/zfbbn61ij7w0bl4wbrwi87x5ghqx968c-net-base-5.3 0.0 0.0 0.0%total: 570.0 MiB
But... is the "baking-in" of (a particular version of) GCC a bad thing?I am not sure either way.
Go invokes gcc when compiling with cgo, and cgo (or at least the usageof standard libraries which use cgo) is getting fairly common. If we donot provide a default gcc with Go, a plain "go build" will produce anerror if it encounters something which uses cgo and it can't find gcc:
$ go build# runtime/cgocgo: exec gcc: exec: "gcc": executable file not found in $PATH
The user would then have to either install a gcc-toolchain, or figureout that they must set CGO_ENABLED=0. From this perspective, it is moreconvenient to have GCC baked-in for the average user, who likely has noreason to want a different CC anyway.
On the other hand, currently GCC is baked-in to Go in two ways:
* CC is set to /gnu/store/...-gcc-7.5.0/bin/gcc* Go is patched so that it adds /gnu/store/...-gcc-7.5.0-lib/lib as arunpath when linking with cgo files
This means that even if the user provides a different CC, thegcc-7.5.0-lib dir will also be in the runpath. I do not know if, or howmuch, this would conflict with other gcc-lib runpaths.
I have experimented with a couple ways of removing the gcc-7.5.0 reference:
1. Simply set CC=gcc. This works to remove gcc-7.5.0 from references,but we still get a gcc-7.5.0-lib runpath. We can't remove this runpathcompletely, as anything using cgo-enabled parts of the standard libraryrequire it, and Go does not save the library location anywhere.
2. Make Go require external linking for anything using cgo, which wouldremove the need to patch internal linking at all. Some platforms do notsupport internally linking cgo at all, so Go should have no troublehandling this. It does break some tests which expect to be able tointernally link, but I have not yet found any actual packages it breaks.
My instinct says that removing the reference, and doing so via #2, isthe way to go, but I am just a newcomer to Guix. WDYT?
--Sarah
S
S
Sarah Morgensen wrote on 6 Jul 22:17 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)
86k0m3jiyc.fsf@mgsn.dev
Hello,
A quick addendum...
Sarah Morgensen <iskarian@mgsn.dev> writes:
Toggle quote (5 lines)> This means that even if the user provides a different CC, the> gcc-7.5.0-lib dir will also be in the runpath. I do not know if, or how> much, this would conflict with other gcc-lib runpaths.>
I have just seen the consequences of this: the binary is unable to loadsymbols from the newer library. More info athttps://issues.guix.gnu.org/36823.
Toggle quote (8 lines)> I have experimented with a couple ways of removing the gcc-7.5.0 reference:>> 1. Simply set CC=gcc. This works to remove gcc-7.5.0 from references,> but we still get a gcc-7.5.0-lib runpath. We can't remove this runpath> completely, as anything using cgo-enabled parts of the standard library> require it, and Go does not save the library location anywhere.>
--Sarah
L
L
Ludovic Courtès wrote on 7 Jul 23:49 +0200
(name . Sarah Morgensen)(address . iskarian@mgsn.dev)
87k0m1okv5.fsf@gnu.org
Hi,
Sarah Morgensen <iskarian@mgsn.dev> skribis:
Toggle quote (40 lines)> Ludovic Courtès <ludo@gnu.org> writes:>>> Hello!>>>> It seems that Go unduly retains a reference to GCC:>>>> $ guix size go>> store item total self>> /gnu/store/g4rrz9rnr8ssbnj3gx3dmsxv4glll8nf-go-1.12.15 646.3 355.9 55.1%>> /gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0 177.4 107.2 16.6%>> [...]>> This is still the case:>> $ guix size go> store item total self> /gnu/store/vvly7zgn981brb37v8y8a7f9psx7c6ay-go-1.16.5 570.0 371.5 65.2%> /gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1y17lk-gcc-7.5.0 178.5 107.3 18.8%> /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31 38.4 36.7 6.4%> /gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib 71.0 32.6 5.7%> /gnu/store/57xj5gcy1jbl9ai2lnrqnpr0dald9i65-coreutils-8.32 88.0 17.0 3.0%> /gnu/store/kl68v5mclwp511xgpsl2h1s9gmsdxpzh-tzdata-2021a 1.9 1.9 0.3%> /gnu/store/mmhimfwmmidf09jw1plw3aw1g1zn2nkh-bash-static-5.0.16 1.6 1.6 0.3%> /gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16 39.4 1.0 0.2%> /gnu/store/g2s5jfkfd4k973wb58476b1bbv9zpm6m-zlib-1.2.11 38.6 0.2 0.0%> /gnu/store/zfbbn61ij7w0bl4wbrwi87x5ghqx968c-net-base-5.3 0.0 0.0 0.0%> total: 570.0 MiB>> But... is the "baking-in" of (a particular version of) GCC a bad thing?> I am not sure either way.>> Go invokes gcc when compiling with cgo, and cgo (or at least the usage> of standard libraries which use cgo) is getting fairly common. If we do> not provide a default gcc with Go, a plain "go build" will produce an> error if it encounters something which uses cgo and it can't find gcc:>> $ go build> # runtime/cgo> cgo: exec gcc: exec: "gcc": executable file not found in $PATH
Ah, I didn’t know about cgo (a helper for C bindings, right?).
I think it’s a case where “dynamic composition” (i.e., looking for gccin $PATH at run time) is preferable, because there are lots ofsituations where gcc is not needed at all.
[...]
Toggle quote (7 lines)> I have experimented with a couple ways of removing the gcc-7.5.0 reference:>> 1. Simply set CC=gcc. This works to remove gcc-7.5.0 from references,> but we still get a gcc-7.5.0-lib runpath. We can't remove this runpath> completely, as anything using cgo-enabled parts of the standard library> require it, and Go does not save the library location anywhere.
Sounds good to me. (gcc-7.5.0-lib is always in the RUNPATH ofexecutables, we don’t have to worry about this one.)
Toggle quote (6 lines)> 2. Make Go require external linking for anything using cgo, which would> remove the need to patch internal linking at all. Some platforms do not> support internally linking cgo at all, so Go should have no trouble> handling this. It does break some tests which expect to be able to> internally link, but I have not yet found any actual packages it breaks.
What do you mean by “external linking” and “internal linking” in thiscontext? (I know very little about Go.)
Thanks!
Ludo’.
S
S
Sarah Morgensen wrote on 8 Jul 03:54 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)
86bl7dk1t6.fsf@mgsn.dev
Hi,
Ludovic Courtès <ludo@gnu.org> writes:
Toggle quote (11 lines)>> Go invokes gcc when compiling with cgo, and cgo (or at least the usage>> of standard libraries which use cgo) is getting fairly common. If we do>> not provide a default gcc with Go, a plain "go build" will produce an>> error if it encounters something which uses cgo and it can't find gcc:>>>> $ go build>> # runtime/cgo>> cgo: exec gcc: exec: "gcc": executable file not found in $PATH>> Ah, I didn’t know about cgo (a helper for C bindings, right?).
Yes, cgo allows you to compile Go programs which interface with C, aswell as straight .c files.
Toggle quote (17 lines)>> I think it’s a case where “dynamic composition” (i.e., looking for gcc> in $PATH at run time) is preferable, because there are lots of> situations where gcc is not needed at all.>> [...]>>> I have experimented with a couple ways of removing the gcc-7.5.0 reference:>>>> 1. Simply set CC=gcc. This works to remove gcc-7.5.0 from references,>> but we still get a gcc-7.5.0-lib runpath. We can't remove this runpath>> completely, as anything using cgo-enabled parts of the standard library>> require it, and Go does not save the library location anywhere.>> Sounds good to me. (gcc-7.5.0-lib is always in the RUNPATH of> executables, we don’t have to worry about this one.)
I recently discovered that there is actually an issue with thisparticular approach. If the user uses a newer gcc-toolchain, thealways-added gcc-7.5.0-lib shadows the newer libraries and newer symbolsare unavailable. See https://issues.guix.gnu.org/36823.
Toggle quote (10 lines)>>> 2. Make Go require external linking for anything using cgo, which would>> remove the need to patch internal linking at all. Some platforms do not>> support internally linking cgo at all, so Go should have no trouble>> handling this. It does break some tests which expect to be able to>> internally link, but I have not yet found any actual packages it breaks.>> What do you mean by “external linking” and “internal linking” in this> context? (I know very little about Go.)
"external linking" => Go invokes gcc to link object files together"internal linking" => Go does the linking itself
--Sarah
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send email to 39400@debbugs.gnu.org