Golang programs keeping references [gnu: go: Update default to 1.11.]

DoneSubmitted by Leo Famulari.
Details
One participant
  • Leo Famulari
Owner
unassigned
Severity
normal
Merged with
L
L
Leo Famulari wrote on 5 Dec 2018 03:16
(address . bug-guix@gnu.org)
20181205021601.GA22628@jasmine.lan
----- Forwarded message from Leo Famulari <leo@famulari.name> -----
Date: Mon, 12 Nov 2018 12:29:25 -0500From: Leo Famulari <leo@famulari.name>To: guix-devel@gnu.orgCc: Pierre Neidhardt <ambrevar@gmail.com>Subject: Golang programs keeping references [gnu: go: Update default to 1.11.]User-Agent: Mutt/1.10.1 (2018-07-13)
On Mon, Nov 12, 2018 at 04:32:46AM -0500, Pierre Neidhardt wrote:
Toggle quote (8 lines)> commit 9a65a052016572b61e3c4247fcdf9e0478656f71> Author: Pierre Neidhardt <mail@ambrevar.xyz>> Date: Sun Nov 11 22:02:18 2018 +0100> > gnu: go: Update default to 1.11.> > * gnu/packages/golang.scm (go): Update default to 1.11.
I noticed that since this change, Go programs (that is, command-lineexecutables) keep references to their inputs:
$ guix gc --references $(./pre-inst-env guix build --no-grafts kurly)/gnu/store/2b2md66fbzyspsmd5dj6zkj9hilac40r-tzdata-2018e/gnu/store/4iwksvq53rlzphfp3xvp63ihlw226c0n-go-github-com-aki237-nscjar-0.0.0-0.e2df936/gnu/store/5rxdjbk8h0bh1hbaan8y8ib13va2bcmw-net-base-5.3/gnu/store/ahvdlp6y44qj6kx63rmx1sq8r61x3zc2-go-github-com-alsm-ioprogress-0.0.0-0.063c372/gnu/store/f8yps0l8p371jgzh6cki0z5n2kgfjiwy-go-github-com-urfave-cli-1.19.1-0.934abfb/gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/gnu/store/pp0bakrbyv9xmp1kyv2114l19s11b74z-gcc-6.4.0-lib
Previously, they did not:
$ guix gc --references $(guix build --no-grafts kurly)/gnu/store/2b2md66fbzyspsmd5dj6zkj9hilac40r-tzdata-2018e/gnu/store/5rxdjbk8h0bh1hbaan8y8ib13va2bcmw-net-base-5.3/gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/gnu/store/pp0bakrbyv9xmp1kyv2114l19s11b74z-gcc-6.4.0-lib
Is this expected? I thought that Go programs were always staticallylinked.


----- End forwarded message -----
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlwHNOEACgkQJkb6MLrKfwimChAA1l4ncYXHAmRY+9JWzUt3aVHGnJKZB3Lyl076uwM/sjpGgglTp10JFX8HWGi7H/eqxV5DlQkW47T4MZvjDkG6FqFhbV/mGThc2spfEMGIULNvTZY8D7DqyYboaKte8vh3PCBwQHmkHZgFLBPGkMs3Mn7CLBKGatQ3XnaWeYyfaApG/ZuDRYiTREqXplojmwhk/bkTqGPzDi4Spy8M7HcyB6ZeuvALbA+hhJzLg3IRyHw49/hRQqQyAxmbpGL8digJPz7eV0RNXIrrPOqS09iOzpjV6LDL4qzaGuEtbrkeH+8aue/lo8n/ky9N4NzVwZQh9Pbf59jC7xMdZh6fPCra9v6X3lWdsJo7wV7rONBTioUmQ+Ccsbcc7rYC06z5FeJ1gTI/SW7w9VNxoRP0WNYk4a+nO5k8qD6xmk1XFWfnFdm9R4yzFv/c5HaymzkdyRODrPQmM/swdAF1u7Fw3cxPMMWkRT4K0TgJJem1EJWqrkJEIQHP+xNjmps8Ikp8GFzXZ7CZbwQ7nVQrmT4aTr7qe1OAENWS1gjILjiVpj3t4L2YPUpRHyDb4g3BsHHEDcjjpPjyY0X/3XlxH8S3i0CL8zONcGiitJHERPLXiKoBMxxA4sMDlnRXiXjfTx/xgJwZZEbhWV43IILU2019HAhhWq8P5h3wkR1/jdqB0l3CQ4U==lXa+-----END PGP SIGNATURE-----

L
L
Leo Famulari wrote on 15 Dec 2018 20:54
[mail@ambrevar.xyz: Re: Golang programs keeping references [gnu: go: Update default to 1.11.]]
(address . 33620@debbugs.gnu.org)
20181215195457.GD9685@jasmine.lan
----- Forwarded message from Pierre Neidhardt <mail@ambrevar.xyz> -----
Date: Sun, 09 Dec 2018 19:46:03 +0100From: Pierre Neidhardt <mail@ambrevar.xyz>To: Leo Famulari <leo@famulari.name>Cc: guix-devel@gnu.orgSubject: Re: Golang programs keeping references [gnu: go: Update default to 1.11.]User-agent: mu4e 1.0; emacs 26.1
I've investigated the possible solutions to avoid including the paths into thebinaries.
I've found this:
https://github.com/golang/go/issues/16860
It's still unresolved and only planned for Go 1.13.
In the meantime, I've played with the -trimpath option to see if I could getsomething out of it.
I've added this to Go's (build) function:
Toggle snippet (4 lines) "-asmflags=all=-trimpath=/gnu/store" "-gcflags=all=-trimpath=/gnu/store"
The resulting binary is indeed trimmed, but that's not enough: it seems thatGuix detects the remaining part of the path as a store item and includes it inthe list of requisites. Is this really how Guix works? It does not need thefull path?
Go supports only one call to -trimpath, so we can't even set this to the Goinputs.
Regarding Boyer-Moore over the binary, we would have to apply the changes forall recursive Go libraries. Now is there a reliable way to detect what's a Golibrary and what is not? We don't want to patch non-Go libraries, right?(Let's not forget that there is CGo...)
If we can't think of a way to detect a Go library, Boyer-Moore does not seem tobe a solution either. And we might have to wait for Go 1.13...
-- Pierre Neidhardthttps://ambrevar.xyz/


----- End forwarded message -----
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlwVXBEACgkQJkb6MLrKfwhBYw/9HRo55iLh+r5+AlGJBUksSqBofjodfx2z2miyfj0IVJZCoP0uREe8+Ki4ocvS/E57ncMv7jhERJUrNnoxoBaEtYb2Cugpmk7Ncw/9jqJqZTlRPa3Z98mc+QGtcw6vdvJ6BtpvCscK/MtLEATV0VMfLMJwqEjB6wMsvI1gdkJ3xJlkXlhqHzJgFBAtmMJeKse1Mx8I5WC3bvCoOBMlxEQT5aFB2o0tQwDm05ao4jE0y/vmSwYtXKmBTQwH7mXoyZ1Z/gdL5ENv5YKkmWCYLN3fjEomhijFRzFEMWINoknIvII8kpTEExe9P8NABDyN3VmCwTv7RvbDojpjXx/MW8kljp4cjk2rYHm8pjXYp+JhXzSnpaMMTrEmd27kEf/8PNpyJWvK+/fi+3z2eMox4rehsqCVtRslggA9rKAIoke2V/m82quWy5GMoIfiE2E5WPdyDK5G9cGFOCr7u6ETSFhr6fqoe6+9T/cCR3o4seh6uF0slx2SyWwD4lFXutb84IivKD+57nN76Hg7qS8I31HqT0JzJbumWGmZuREk9UiqLaEpuH7I0nJYiJ+txA6lwbeUzWt2WjHtC8mg4qYl5lPJs4rX4jzbUglWMV0k5QgZ2lq0LxrMOmT9n3yMRuhdilEhnbRKmPQtLiHnNW74Dq+wagj8z/d9edE0Y9LDM6rAaW0==cXb9-----END PGP SIGNATURE-----

L
L
Leo Famulari wrote on 16 Dec 2018 22:26
(no subject)
(address . control@debbugs.gnu.org)
20181216212623.GA32482@jasmine.lan
merge 33741 33620
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlwWwv8ACgkQJkb6MLrKfwhztg//fsLVRKJJ1OHzOA9i0aMZJII8pm7Xy1ZWrzmhxUpDq083xjkHEltEIYj+grdNKEwjfbyVGiH+4fomRMdqZSCXJJrmdVL1bvUEJYNmURD9vRi2g4ofn/gQ0svVEBwpH96FZs2clVgZ9XRbNJcI9RuUmcMOolNIde9n+CmrdKlKD5BK9YoW0EocqXY2DHOjfUlYtKXMFDXzzFH5tQ54A7n/ytK1zfRrNTccxdb4WjPLcJKB4jzvCf9Y85spT6/qBi9PcoG7PiLwqmEeqSnlxy8dCoeepU0gIACdCgLbtjOjMGn4SkcdCrMrxuekZTBrPAM88ymIoMfntOG7JXwqCQySaP/bm56DFOs8ThtIlGZkKFBOaGX2fgNQSoGO2ZIM+lDd5J1OFHBDnKmOEAchdHGAIVdPgBlOWYgxlfb5CiZCQzpV7V059wnvI2AhvDI/UXt3hnAZAsd0MICtA4jNYt6avSAdxeM1ZUkcCp2QR+LcP1Fb55tF+GhN4tmfbvyc9YxRjSnlSetGQjfTSJBSMMQKhnUKUvTTsR0ukPSZKaUKJSq4g90OxdyuCfuq0iAZKjMj8liEmt4aScH6Di94nuHv1/UKgIWVQ5mD8ez4TrbboUbyukTFLI8kc7N1a5HiAzeuCqhgJwxLuM3bNq2bW01SxCuDT2HzYsStP+yw9jy7Dlo==KMK0-----END PGP SIGNATURE-----

L
L
Leo Famulari wrote on 14 Mar 2019 20:44
Re: [mail@ambrevar.xyz: Re: Golang programs keeping references [gnu: go: Update default to 1.11.]]
(address . 33620-done@debbugs.gnu.org)
20190314194415.GB17982@jasmine.lan
The issue of Go packages keeping references to all their Go-language isinputs is resolved with commit e3900a4d64e4bf6f426809d6bff058e5a2ae9bc8.
This commit basically avoids bringing the store paths into the buildenvironment at all by symlinking them into a union directory in theTMPDIR. This is a bit of a hack but it's actually more idiomatic in Goto have all the inputs in a single directory ("workspace" in Go). Theprevious technique of passing a list of directories in the GOPATHvariable is officially supported but is definitely a 2nd-class techniquein practice.
----- Forwarded message from Pierre Neidhardt <mail@ambrevar.xyz> -----
Toggle quote (12 lines)> I've added this to Go's (build) function:> > --8<---------------cut here---------------start------------->8---> "-asmflags=all=-trimpath=/gnu/store"> "-gcflags=all=-trimpath=/gnu/store"> --8<---------------cut here---------------end--------------->8---> > The resulting binary is indeed trimmed, but that's not enough: it seems that> Guix detects the remaining part of the path as a store item and includes it in> the list of requisites. Is this really how Guix works? It does not need the> full path?
Not having read the reference scanner code carefully, I expect that itignores the '/gnu/store/' since this path is actually configurable.
Toggle quote (5 lines)> Regarding Boyer-Moore over the binary, we would have to apply the changes for> all recursive Go libraries. Now is there a reliable way to detect what's a Go> library and what is not? We don't want to patch non-Go libraries, right?> (Let's not forget that there is CGo...)
In (guix build-system go), I'd like to construct of list of inputs thatuse the go-build-system and pass this list to the biuld side. This wouldbe useful for other things, but could also be used to detect whichinputs are Go libraries.
Toggle quote (3 lines)> If we can't think of a way to detect a Go library, Boyer-Moore does not seem to> be a solution either. And we might have to wait for Go 1.13...
Waiting for upstream is always the easiest way :)
But it would still be nice to have Boyer-Moore available in (guix buildutils) for whenever we want to use it. Or even in Guile itself...
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlyKrw8ACgkQJkb6MLrKfwjqmRAAzOUIUa6m2/jL2hawh24O+ais4tPAj4WfpdilVBgkXMKbKs8nzkXUocElcahgMCZQuV8EQPTYYHrMPLToZ2UIbIMinu8LSrfBAI2IWmghH7BxlotufQcj6nF846UpM2gnf6TSMvuzSCZPqrV4JwSpUaO+hPyRDKg6mahMc01kYoIhrCRc9pJoALnc9DfxsX9GAGtrDQvCz7Q7zGhahQsmJpX9+MTTCla6cVLGFEAsIZESTmnl7GTH8jdu2R2Fnshu/auJM1qYBME9jWRRXwtCajW39YWbWnn/7+Lupo43qxvk/vz34fzjPPIP8z1gxa+sp/GQNOveeM3jdilyGxMGGmUWbEAiNXgB7cYO+cSMyQcdU9lLYAwh7TeQgLozcujqTufR/cwsuErt48ZUVO67ybWOsYRKm37/SuFUz9PsBtptftDFhPEhFYePuMp1n7FkJNtmvi1eKYn2/rs/cjUplGHL5heyFFxB5nB5vXJ3bvnsj9GzXS+ARa0mCtns8MEhC5nZrJTGhoetzbBBjaQLOkINmb58MRmuRg3NQ84I67N6E85urPf8WSUQYjcvP7ScPnEmJ+6NRpxjQA8llN1Puo1brLMCqt349uoHIFOzncGwLivzZiqrEfpR0j+u6k9e5Eh0lamuuYLBpJXHErSeesrhWt58yLhAYP4JskHNXWI==4/wz-----END PGP SIGNATURE-----

Closed
?
Your comment

This issue is archived.

To comment on this conversation send email to 33620@debbugs.gnu.org