git-annex is not Reproducible

> Dear,

>

> Using Guix 04a459a, the package ’git-annex’ is not reproducible:

>

> guix build git-annex

> guix build git-annex --no-grafts --check -K

>

> return:

>

> --8<---------------cut here---------------start------------->8---

> guix build: error: derivation `/gnu/store/70qkv5hi5yq20m1h9jfm17m5lsc235mk-git-annex-8.20200908.drv' may not be deterministic: output `/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908' differs from ‘/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check’

> --8<---------------cut here---------------end--------------->8---

>

> then:

>

> --8<---------------cut here---------------start------------->8---

> $ diff -r --no-dereference /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908{,-check}

> Binary files /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908/bin/git-annex and /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check/bin/git-annex differ

> --8<---------------cut here---------------end--------------->8---

>

> Is it expected? I have not tried ’diffoscope’ yet to investigate.

> On 10/07/20 12:35:51 +0200, zimoun wrote:

> > Dear,

> >

> > Using Guix 04a459a, the package ’git-annex’ is not reproducible:

> >

> > guix build git-annex

> > guix build git-annex --no-grafts --check -K

> >

> > return:

> >

> > --8<---------------cut here---------------start------------->8---

> > guix build: error: derivation `/gnu/store/70qkv5hi5yq20m1h9jfm17m5lsc235mk-git-annex-8.20200908.drv' may not be deterministic: output `/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908' differs from ‘/gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check’

> > --8<---------------cut here---------------end--------------->8---

> >

> > then:

> >

> > --8<---------------cut here---------------start------------->8---

> > $ diff -r --no-dereference /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908{,-check}

> > Binary files /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908/bin/git-annex and /gnu/store/cbrwrlwx6kg7l5q6ifl7q08icm17704g-git-annex-8.20200908-check/bin/git-annex differ

> > --8<---------------cut here---------------end--------------->8---

> >

> > Is it expected? I have not tried ’diffoscope’ yet to investigate.

>

> Thanks for noticing. I see the same on my end (and also with the

> git-annex update [1] that I submitted before I saw this message).

>

> I suspect this is a relatively new issue, as I've run `build --check'

> successfully on git-annex in the past. The git-annex definition prior

> to the one above was introduced with e13fefbcff (gnu: git-annex: Create

> fake static output, 2020-08-17). On that commit, the check also failed

> for me:

>

> $ guix time-machine --commit=e13fefbcffa50144d9301d67b61928ac3c964fb3 -- build --no-grafts --check git-annex

> guix build: error: derivation `/gnu/store/5illriaaijqrhl17aymzgiw6wzaq666s-git-annex-8.20200810.drv' may not be deterministic: output `/gnu/store/32h0xbqdwkm22naqba10fk9124cdgw9q-git-annex-8.20200810' differs

>

> However, going back one more to 35e24fab73 (gnu: git-annex: Update to

> 8.20200810, 2020-08-11), the check was successful:

>

> $ guix time-machine --commit=35e24fab73bc450c6e0d3b9a5bbdfa88d2c425f4 -- build --no-grafts --check git-annex

> [...]

> /gnu/store/m79ri0g2s3d8pk52ysjkdvallk2zqbwb-git-annex-8.20200810

>

>

> [1]: bug#43854: <20201007204946.22878-1-kyle@kyleam.com>

>

>> $ guix time-machine --commit=e13fefbcffa50144d9301d67b61928ac3c964fb3 --

>> build --no-grafts --check git-annex

>> guix build: error: derivation

>> `/gnu/store/5illriaaijqrhl17aymzgiw6wzaq666s-git-annex-8.20200810.drv' may

>> not be deterministic: output

>> `/gnu/store/32h0xbqdwkm22naqba10fk9124cdgw9q-git-annex-8.20200810' differs

>>

>> However, going back one more to 35e24fab73 (gnu: git-annex: Update to

>> 8.20200810, 2020-08-11), the check was successful:

>>

>> $ guix time-machine --commit=35e24fab73bc450c6e0d3b9a5bbdfa88d2c425f4 --

>> build --no-grafts --check git-annex

>> [...]

>> /gnu/store/m79ri0g2s3d8pk52ysjkdvallk2zqbwb-git-annex-8.20200810

> In between those two commits is also the change to the haskell build

> system update, where we use the dynamic libraries. The best bet is to

> make sure the dependencies are built reproducibly.

> Dear,

>

> On Thu, 08 Oct 2020 at 09:17, Efraim Flashner <efraim@flashner.co.il> wrote:

>

>>> $ guix time-machine --commit=e13fefbcffa50144d9301d67b61928ac3c964fb3 --

>>> build --no-grafts --check git-annex

>>> guix build: error: derivation

>>> `/gnu/store/5illriaaijqrhl17aymzgiw6wzaq666s-git-annex-8.20200810.drv' may

>>> not be deterministic: output

>>> `/gnu/store/32h0xbqdwkm22naqba10fk9124cdgw9q-git-annex-8.20200810' differs

>>>

>>> However, going back one more to 35e24fab73 (gnu: git-annex: Update to

>>> 8.20200810, 2020-08-11), the check was successful:

>>>

>>> $ guix time-machine --commit=35e24fab73bc450c6e0d3b9a5bbdfa88d2c425f4 --

>>> build --no-grafts --check git-annex

>>> [...]

>>> /gnu/store/m79ri0g2s3d8pk52ysjkdvallk2zqbwb-git-annex-8.20200810

>

>> In between those two commits is also the change to the haskell build

>> system update, where we use the dynamic libraries. The best bet is to

>> make sure the dependencies are built reproducibly.

>

> Yep, it is what “git bisect” says.

>

> --8<---------------cut here---------------start------------->8---

> 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2 is the first bad commit

> commit 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2

> Author: Ricardo Wurmus <rekado@elephly.net>

> Date: Tue Jun 16 22:25:48 2020 +0200

>

> build-system/haskell: Support parallel builds.

>

> * guix/build-system/haskell.scm (haskell-build): Add keyword

> PARALLEL-BUILD?

> and pass it on to the builder.

> * guix/build/haskell-build-system.scm (build): Accept keyword

> PARALLEL-BUILD?

> and pass the number of parallel jobs to GHC.

>

> guix/build-system/haskell.scm | 2 ++

> guix/build/haskell-build-system.scm | 7 +++++--

> 2 files changed, 7 insertions(+), 2 deletions(-)

> bisect run success

> --8<---------------cut here---------------end--------------->8---

> > --8<---------------cut here---------------start------------->8---

> > 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2 is the first bad commit

> > commit 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2

> > Author: Ricardo Wurmus <rekado@elephly.net>

> > Date: Tue Jun 16 22:25:48 2020 +0200

> >

> > build-system/haskell: Support parallel builds.

> >

> > * guix/build-system/haskell.scm (haskell-build): Add keyword

> > PARALLEL-BUILD?

> > and pass it on to the builder.

> > * guix/build/haskell-build-system.scm (build): Accept keyword

> > PARALLEL-BUILD?

> > and pass the number of parallel jobs to GHC.

> >

> > guix/build-system/haskell.scm | 2 ++

> > guix/build/haskell-build-system.scm | 7 +++++--

> > 2 files changed, 7 insertions(+), 2 deletions(-)

> > bisect run success

> > --8<---------------cut here---------------end--------------->8---

>

> Shall we revert it? Previously the build system also supported parallel

> builds, but didn’t seem to pass the right arguments to actually make it

> happen.

> On Sat, 10 Oct 2020 at 11:34, Ricardo Wurmus <rekado@elephly.net> wrote:

>

>> > --8<---------------cut here---------------start------------->8---

>> > 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2 is the first bad commit

>> > commit 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2

>> > Author: Ricardo Wurmus <rekado@elephly.net>

>> > Date: Tue Jun 16 22:25:48 2020 +0200

>> >

>> > build-system/haskell: Support parallel builds.

>> >

>> > * guix/build-system/haskell.scm (haskell-build): Add keyword

>> > PARALLEL-BUILD?

>> > and pass it on to the builder.

>> > * guix/build/haskell-build-system.scm (build): Accept keyword

>> > PARALLEL-BUILD?

>> > and pass the number of parallel jobs to GHC.

>> >

>> > guix/build-system/haskell.scm | 2 ++

>> > guix/build/haskell-build-system.scm | 7 +++++--

>> > 2 files changed, 7 insertions(+), 2 deletions(-)

>> > bisect run success

>> > --8<---------------cut here---------------end--------------->8---

>>

>> Shall we revert it? Previously the build system also supported parallel

>> builds, but didn’t seem to pass the right arguments to actually make it

>> happen.

>

> What do the Haskellers think about that?

> Does the parallel build save a lot of time and CPU? If yes, maybe we

> could to provide a transformation for the expert, something like

> "haskell-build-system-with-parellel-build" which tweaks

> "PARALLEL-BUILD?", similarly to the recent "no tests". WDYT?

> > Does the parallel build save a lot of time and CPU? If yes, maybe we

> > could to provide a transformation for the expert, something like

> > "haskell-build-system-with-parellel-build" which tweaks

> > "PARALLEL-BUILD?", similarly to the recent "no tests". WDYT?

>

> We shouldn’t compromise reproducibility for parallel builds. Ideally we

> would figure out what exactly causes the differences and fix that

> instead of disabling parallel builds, but if that turns out to be too

> difficult I think we should just revert this until we have a good fix.

>

> Perhaps something can be done by fixing the order of files somewhere.

> On Mon, 12 Oct 2020 at 12:34, Ricardo Wurmus <rekado@elephly.net> wrote:

>

>> > Does the parallel build save a lot of time and CPU? If yes, maybe we

>> > could to provide a transformation for the expert, something like

>> > "haskell-build-system-with-parellel-build" which tweaks

>> > "PARALLEL-BUILD?", similarly to the recent "no tests". WDYT?

>>

>> We shouldn’t compromise reproducibility for parallel builds. Ideally we

>> would figure out what exactly causes the differences and fix that

>> instead of disabling parallel builds, but if that turns out to be too

>> difficult I think we should just revert this until we have a good fix.

>>

>> Perhaps something can be done by fixing the order of files somewhere.

>

> My proposal is for the expert and not at the CLI level. Something

> like "(funky-name ghc-foo)" which returns a new package with

> PARALLEL-BUILD? turned to #t. And the default should be #f. WDYT?

> You can always build with ‘--cores=1’, which will ensure both the build

> and test will run sequentially.

> Fixes https://bugs.gnu.org/43843.

>

> PARALLEL-BUILD? introduced by commit 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2

> leads to unreproducibility. Instead of reverting, default is set to #false

> which allows user to locally build with parallelism.

>

> * guix/build-system/haskell.scm (haskell-build): Turn off PARALLEL-BUILD? by

> default.