go incremental builds broken

Hello,

It seems the guix's go package is broken when the go tool is used for

incremental builds. Any attempt to use 'install' or 'build -i' results

in an attempt to write to the store. A one-liner:

guix environment --ad-hoc go -- bash -c 'export t=$(mktemp -d); cd

$t && export GOPATH=$(pwd) GOBIN=$(pwd)/bin && go install cmd/go; cd

&& rm -rf $t'

Another command reports that (seemingly) the entire stdlib is marked as stale:

guix environment --ad-hoc go -- bash -c 'export t=$(mktemp -d); cd

$t && export GOPATH=$(pwd) GOBIN=$(pwd)/bin && go list -f '\''{{join

.Deps "\n"}}'\'' cmd/go | xargs -n1 go list -f '\''{{if

.Stale}}{{.ImportPath}}: {{.StaleReason}}{{end}}'\''; cd && rm -rf $t'

The function for determining staleness is here (after the giant

comment explaining the reasoning):

I don't see anything wrong with the package definition, but could be

missing something. My only hunch at this point is that something might

be modifying src/runtime/internal/sys/zversion.go, as that entire file

is included in the build ID computation.

Thanks,

After talking on IRC, it seems like grafts are the problem. Adding

`--no-grafts` to the above commands has them succeed.

The GOROOT (store location, in this case) gets encoded in the

generated source by the 'dist' tool, so changing that is going to

cause issues.

Is there a way to disable grafts on a per-package basis, in the definition?

On Thu, Feb 16, 2017 at 10:06 AM, GNU bug Tracking System

<help-debbugs@gnu.org> wrote:

On Fri, Feb 24, 2017 at 02:50:10PM -0500, Hank Donnay wrote:

You can disable them on the command-line for any tool that builds

things, using the '--no-grafts' option, which is a "Common build option"

[0]. For example, `guix environment --ad-hoc go --no-grafts`.

If you do this, you should understand that the ungrafted Go package will

contain some serious security vulnerabilities, since we only use grafts

to fix vulnerabilities in core packages. [1]

We should figure out why this issue manifests when go is grafted, and

fix the root cause.

[0]

[1]

Hello,

Hank Donnay <hdonnay@gmail.com> skribis:

This method relies on the build ID to, which is defined like this (info

"(ld) Options"):

`--build-id'

`--build-id=STYLE'

Request the creation of a `.note.gnu.build-id' ELF note section or

a `.buildid' COFF section. The contents of the note are unique

bits identifying this linked file. STYLE can be `uuid' to use 128

random bits, `sha1' to use a 160-bit SHA1 hash on the normative

parts of the output contents, `md5' to use a 128-bit MD5 hash on

the normative parts of the output contents, or `0xHEXSTRING' to

use a chosen bit string specified as an even number of hexadecimal

digits (`-' and `:' characters between digit pairs are ignored).

If STYLE is omitted, `sha1' is used.

The `md5' and `sha1' styles produces an identifier that is always

the same in an identical output file, but will be unique among all

nonidentical output files. It is not intended to be compared as a

checksum for the file's contents. A linked file may be changed

later by other tools, but the build ID bit string identifying the

original linked file does not change.

Passing `none' for STYLE disables the setting from any

`--build-id' options earlier on the command line.

I suppose Go uses one of md5 or sha1, which is a good thing since it

allows for reproducible builds.

However, grafting breaks this, similarly to https://bugs.gnu.org/19973

since they change file contents without recomputing the build ID.

Having Go use --build-id=uuid would work around the problem, but it

would also prevent bit-reproducible builds.

Perhaps our grafting code will have to handle .note.gnu.build-id

specially.

Thoughts?

Thanks for reporting the issue,

Ludo’.

Hi,

On Sat, 25 Feb 2017 at 10:58, Leo Famulari <leo@famulari.name> wrote:

I suspect closing this bug had been forgotten. If yes, feel free to

close it. Otherwise, I do not understand what is the next actionable

step.

All the best,

simon

On Fri, Dec 18, 2020 at 09:00:14PM +0100, zimoun wrote:

I don't think this bug has been fixed, although I have not checked.

Currently, Go incremental builds and library re-use is not functional in

Guix, unless I missed a new development.

Hi Leo,

On Fri, 18 Dec 2020 at 16:31, Leo Famulari <leo@famulari.name> wrote:

Thanks. Could you provide a Go example for reproducing? Well, a

reproducer to track what could be wrong on the grafting mechanism for

Go. If I understand correctly.

All the best,

simon

On Mon, Jan 11, 2021 at 01:46:48PM +0100, zimoun wrote:

I actually spent a bit of time on this today/yesterday. I modified the

go-build-system to copy the pkg and src folders from build to build

(instead of just the src folders) and the pkg folders were ignored.

Upstream documentation isn't clear about when the pkg folder is used and

when it is ignored.

One thing I did try was to set the pkgdir to (string-append (getenv

"GOROOT") "/pkg"), but then it didn't find the pkg folder from go:out. I

didn't try copying go:out/pkg to GOROOT/pkg to see if that would fix it;

I assumed that even if it did work we wouldn't want to propagate all of

go with the go packages.

--

Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר

GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351

Confidentiality cannot be guaranteed on emails sent or received unencrypted

Hi,

On Mon, 11 Jan 2021 at 22:01, Efraim Flashner <efraim@flashner.co.il> wrote:

What is the status of this bug? Since I am not using Go at all, it is

hard for me to have a reproducer. Is it possible to provide a

reproducer? Thanks.

Cheers,

simon

Hi,

On Fri, 18 Dec 2020 at 16:31, Leo Famulari <leo@famulari.name> wrote:

What is the status of this bug? Since I am not using Go at all, it is

hard for me to have a reproducer. Is it possible to provide a

reproducer? Thanks.

Cheers,

simon

PS: I asked the same thing 13 weeks ago. Therefore, without any answer

and giving this bug is really old not so much commented, I propose to

close it.

tags 25752 + moreinfo

quit

Hi,

On Mon, 11 Jan 2021 at 22:01, Efraim Flashner <efraim@flashner.co.il> wrote:

What is the status of this old bug [1]? It is marked ’moreinfo’ because

it misses a reproducer to be able to act on. Therefore, I am going to

close it.

Cheers,

simon