Golang programs keeping references [gnu: go: Update default to 1.11.]

Done

Details

One participant

Leo Famulari

Owner: unassigned

Submitted by: Leo Famulari

Severity: normal

Merged with

33741

Leo Famulari wrote on 5 Dec 2018 03:16

Recipients:(address . bug-guix@gnu.org)

Message-ID:20181205021601.GA22628@jasmine.lan

----- Forwarded message from Leo Famulari <leo@famulari.name> -----

To: guix-devel@gnu.org

Cc: Pierre Neidhardt <ambrevar@gmail.com>

User-Agent: Mutt/1.10.1 (2018-07-13)

On Mon, Nov 12, 2018 at 04:32:46AM -0500, Pierre Neidhardt wrote:

Toggle quote (8 lines)

> commit 9a65a052016572b61e3c4247fcdf9e0478656f71

> Author: Pierre Neidhardt <mail@ambrevar.xyz>

> Date: Sun Nov 11 22:02:18 2018 +0100

> gnu: go: Update default to 1.11.

> * gnu/packages/golang.scm (go): Update default to 1.11.

I noticed that since this change, Go programs (that is, command-line

executables) keep references to their inputs:

$ guix gc --references $(./pre-inst-env guix build --no-grafts kurly)

/gnu/store/2b2md66fbzyspsmd5dj6zkj9hilac40r-tzdata-2018e

/gnu/store/4iwksvq53rlzphfp3xvp63ihlw226c0n-go-github-com-aki237-nscjar-0.0.0-0.e2df936

/gnu/store/5rxdjbk8h0bh1hbaan8y8ib13va2bcmw-net-base-5.3

/gnu/store/ahvdlp6y44qj6kx63rmx1sq8r61x3zc2-go-github-com-alsm-ioprogress-0.0.0-0.063c372

/gnu/store/f8yps0l8p371jgzh6cki0z5n2kgfjiwy-go-github-com-urfave-cli-1.19.1-0.934abfb

/gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27

/gnu/store/pp0bakrbyv9xmp1kyv2114l19s11b74z-gcc-6.4.0-lib

Previously, they did not:

$ guix gc --references $(guix build --no-grafts kurly)

/gnu/store/2b2md66fbzyspsmd5dj6zkj9hilac40r-tzdata-2018e

/gnu/store/5rxdjbk8h0bh1hbaan8y8ib13va2bcmw-net-base-5.3

/gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27

/gnu/store/pp0bakrbyv9xmp1kyv2114l19s11b74z-gcc-6.4.0-lib

Is this expected? I thought that Go programs were always statically

linked.

----- End forwarded message -----

Leo Famulari wrote on 15 Dec 2018 20:54

[mail@ambrevar.xyz: Re: Golang programs keeping references [gnu: go: Update default to 1.11.]]

Recipients:(address . 33620@debbugs.gnu.org)

Message-ID:20181215195457.GD9685@jasmine.lan

----- Forwarded message from Pierre Neidhardt <mail@ambrevar.xyz> -----

To: Leo Famulari <leo@famulari.name>

Cc: guix-devel@gnu.org

User-agent: mu4e 1.0; emacs 26.1

I've investigated the possible solutions to avoid including the paths into the

binaries.

I've found this:

https://github.com/golang/go/issues/16860

It's still unresolved and only planned for Go 1.13.

In the meantime, I've played with the -trimpath option to see if I could get

something out of it.

I've added this to Go's (build) function:

Toggle snippet (4 lines)

"-asmflags=all=-trimpath=/gnu/store"

"-gcflags=all=-trimpath=/gnu/store"

The resulting binary is indeed trimmed, but that's not enough: it seems that

Guix detects the remaining part of the path as a store item and includes it in

the list of requisites. Is this really how Guix works? It does not need the

full path?

Go supports only one call to -trimpath, so we can't even set this to the Go

inputs.

Regarding Boyer-Moore over the binary, we would have to apply the changes for

all recursive Go libraries. Now is there a reliable way to detect what's a Go

library and what is not? We don't want to patch non-Go libraries, right?

(Let's not forget that there is CGo...)

If we can't think of a way to detect a Go library, Boyer-Moore does not seem to

be a solution either. And we might have to wait for Go 1.13...

Pierre Neidhardt

https://ambrevar.xyz/

----- End forwarded message -----

Leo Famulari wrote on 16 Dec 2018 22:26

(no subject)

Recipients:(address . control@debbugs.gnu.org)

Message-ID:20181216212623.GA32482@jasmine.lan

merge 33741 33620

Leo Famulari wrote on 14 Mar 2019 20:44

Re: [mail@ambrevar.xyz: Re: Golang programs keeping references [gnu: go: Update default to 1.11.]]

Recipients:(address . 33620-done@debbugs.gnu.org)

Message-ID:20190314194415.GB17982@jasmine.lan

The issue of Go packages keeping references to all their Go-language is
inputs is resolved with commit e3900a4d64e4bf6f426809d6bff058e5a2ae9bc8.

This commit basically avoids bringing the store paths into the build
environment at all by symlinking them into a union directory in the
TMPDIR. This is a bit of a hack but it's actually more idiomatic in Go
to have all the inputs in a single directory ("workspace" in Go). The
previous technique of passing a list of directories in the GOPATH
variable is officially supported but is definitely a 2nd-class technique
in practice.

----- Forwarded message from Pierre Neidhardt <mail@ambrevar.xyz> -----

Toggle quote (12 lines)> I've added this to Go's (build) function:
> 
> --8<---------------cut here---------------start------------->8---
>               "-asmflags=all=-trimpath=/gnu/store"
>               "-gcflags=all=-trimpath=/gnu/store"
> --8<---------------cut here---------------end--------------->8---
> 
> The resulting binary is indeed trimmed, but that's not enough: it seems that
> Guix detects the remaining part of the path as a store item and includes it in
> the list of requisites.  Is this really how Guix works?  It does not need the
> full path?

Not having read the reference scanner code carefully, I expect that it

ignores the '/gnu/store/' since this path is actually configurable.

Toggle quote (5 lines)

> Regarding Boyer-Moore over the binary, we would have to apply the changes for

> all recursive Go libraries. Now is there a reliable way to detect what's a Go

> library and what is not? We don't want to patch non-Go libraries, right?

> (Let's not forget that there is CGo...)

In (guix build-system go), I'd like to construct of list of inputs that

use the go-build-system and pass this list to the biuld side. This would

be useful for other things, but could also be used to detect which

inputs are Go libraries.

Toggle quote (3 lines)

> If we can't think of a way to detect a Go library, Boyer-Moore does not seem to

> be a solution either. And we might have to wait for Go 1.13...

Waiting for upstream is always the easiest way :)

But it would still be nice to have Boyer-Moore available in (guix build

utils) for whenever we want to use it. Or even in Guile itself...

Closed

Your comment

This issue is archived.

To comment on this conversation send an email to 33620@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 33620

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date