Guitarix builds non-deterministically

  • Done
  • quality assurance status badge
Details
3 participants
  • Ludovic Courtès
  • Maxim Cournoyer
  • zimoun
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 1 Nov 2015 11:30
(address . bug-guix@gnu.org)
87k2q257w0.fsf@gnu.org
Guitarix 0.33.0, as of Guix commit 3c3e697, builds
non-deterministically:

Toggle snippet (11 lines)
$ ./pre-inst-env guix challenge guitarix
updating list of substitutes from 'http://hydra.gnu.org'... 100.0%
/gnu/store/6ksnwcqn92z2nf6qw5js7njnfqlifgpb-guitarix-0.33.0 contents differ:
local hash: 1rh7qxmylsbsaah59h7sclqqxcz0lwsixlc0krkzwhx8gfhlyam6
http://hydra.gnu.org/nar/6ksnwcqn92z2nf6qw5js7njnfqlifgpb-guitarix-0.33.0: 1cq4k1wdqibwraqk0wkjj6n5hgs9v9zcvwr2wfgxvgxnf5l1rfhf
$ wget -q -O - http://hydra.gnu.org/nar/6ksnwcqn92z2nf6qw5js7njnfqlifgpb-guitarix-0.33.0 | bunzip2 | guix archive -x t
$ LC_ALL=C diff -r /gnu/store/6ksnwcqn92z2nf6qw5js7njnfqlifgpb-guitarix-0.33.0 t
$ LC_ALL=C diff -r --no-dereference /gnu/store/6ksnwcqn92z2nf6qw5js7njnfqlifgpb-guitarix-0.33.0 t
File /gnu/store/6ksnwcqn92z2nf6qw5js7njnfqlifgpb-guitarix-0.33.0/lib/libgxw.so is a symbolic link while file t/lib/libgxw.so is a regular file

It appears to create libgxw.so either as a symlink or as a regular file
in a non-deterministic fashion.

It would be nice to see why this happens, and whether this affects all
Waf-based packages.

Ludo’.
Z
Z
zimoun wrote on 12 Nov 2019 21:44
Bug #21803 Hunting: status?
CAJ3okZ35r-9X-CwM4OBP8bSkuf4zdbKWkWB8hCpy0R0idjrT9Q@mail.gmail.com
Hi Ludo,

You reported this bug more than 4 years ago about Guitarix 0.33 and
now it is gone and replaced by Guitarix 0.38.

From my test with Guix d258d9c7d222e6b64531c14293f41bd8d62ea4f7,
"guix challenge guitarix" and "guix build --rounds=3" do not report
issues about reproducibility.

And from my knowledge, the waf-based packages are not affected.

Do you agree to close this bug since it is not relevant anymore?

Thanks in advance for any comments.


All the best,
simon
L
L
Ludovic Courtès wrote on 13 Nov 2019 14:42
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 21803@debbugs.gnu.org)
87sgmropu2.fsf@gnu.org
Hello,

zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (8 lines)
> From my test with Guix d258d9c7d222e6b64531c14293f41bd8d62ea4f7,
> "guix challenge guitarix" and "guix build --rounds=3" do not report
> issues about reproducibility.
>
> And from my knowledge, the waf-based packages are not affected.
>
> Do you agree to close this bug since it is not relevant anymore?

If you’ve checked that a local build gives the same result several times
in a row (make sure it actually rebuilt things; “guix build --rounds=3
foo” does nothing if “foo” is already in the store), then you can
definitely close it!

Thanks,
Ludo’.
Z
Z
zimoun wrote on 13 Nov 2019 19:14
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 21803@debbugs.gnu.org)
CAJ3okZ0E3z38CBhUZJo0hkVwxfJVw=8kS6qPLNKYwRRFuMSe5Q@mail.gmail.com
On Wed, 13 Nov 2019 at 14:42, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (15 lines)
> zimoun <zimon.toutoune@gmail.com> skribis:
>
> > From my test with Guix d258d9c7d222e6b64531c14293f41bd8d62ea4f7,
> > "guix challenge guitarix" and "guix build --rounds=3" do not report
> > issues about reproducibility.
> >
> > And from my knowledge, the waf-based packages are not affected.
> >
> > Do you agree to close this bug since it is not relevant anymore?
>
> If you’ve checked that a local build gives the same result several times
> in a row (make sure it actually rebuilt things; “guix build --rounds=3
> foo” does nothing if “foo” is already in the store), then you can
> definitely close it!

I think I did but how do you do that cleanly?



I am testing with "brute force" method: "guix gc -C" then proceed. I
notice unexpected behaviour; not sure it is the right place to report.


1. The first issue is that the two following commands do not populate
the same way.

(Because my machine is not very powerful, before building I populate
the store with the dependencies from substitutes.)

Toggle snippet (3 lines)
$ guix environment guitarix -- echo Done

Toggle snippet (3 lines)
$ guix build `guix show guitarix | recsel -R dependencies`

The main issue is about `gettext-minimal`.

(Below, I pinpoint with star (*) which is not common between the both
commands and with sharp (#) which appears twice in the same list.)


2. The second issue is the inconsistent outputs:

Toggle snippet (8 lines)
$ guix build guitarix --no-substitutes --dry-run
The following derivations would be built:
/gnu/store/ikdd9740fifdcqwmf170gmlrlkirwa8j-guitarix-0.38.1.drv
/gnu/store/ismr6xqwsi165phwjx9kbcrmr9lsz61r-module-import.drv
/gnu/store/zsigy6yfllikzxmnlii6ivxczlv8k5h3-guitarix2-0.38.1.tar.xz.drv
/gnu/store/zya9lhiqqr24lbgkp510v1z9s4qdcqid-module-import-compiled.drv

Toggle snippet (19 lines)
$ guix build guitarix --no-substitutes
building /gnu/store/4fqc5dydkc4svkl1zjyz5ymnyycfakx8-module-import.drv...
successfully built /gnu/store/4fqc5dydkc4svkl1zjyz5ymnyycfakx8-module-import.drv
building /gnu/store/s659hxn6zh7havik6bghip7mslarxfcx-ghostscript-9.27.tar.xz.drv...

Starting download of
/gnu/store/gz3sh75g8rwvqqhmj7z1wbdrn36bwk1g-ghostscript-9.27.tar.xz
From https://github.com/ArtifexSoftware/ghostpdl-downloads/releases/download/gs927/ghostscript-9.27.tar.xz...
following redirection to
`https://github-production-release-asset-2e65be.s3.amazonaws.com/50461376/70f10a80-56c2-11e9-8208-d05d335afc94?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20191113%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191113T173022Z&X-Amz-Expires=300&X-Amz-Signature=6498073580142d43071eba62060a7379b0dae94f148fc191b1a8ec53f284107a&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dghostscript-9.27.tar.xz&response-content-type=application%2Foctet-stream'...
downloading from
https://github.com/ArtifexSoftware/ghostpdl-downloads/releases/download/gs927/ghostscript-9.27.tar.xz...
ghostscript-9.27.tar.xz 31.6MiB

4.9MiB/s 00:02 [###### ] 34.0%
^C


Why `ghostscript` is downloaded and not reported; neither by the two
populating commands?
Do I miss the obvious?

So I add it to the store with "guix build ghostscript" then I run
again "guix build guitarix --no-substitutes" and again pieces are
missing:

Toggle snippet (10 lines)
[...]
downloading from
http://downloads.sourceforge.net/project/cunit/CUnit/2.1-3/CUnit-2.1-3.tar.bz2
[...]
downloading from https://c-ares.haxx.se/download/c-ares-1.15.0.tar.gz
[...]
downloading from http://www.digip.org/jansson/releases/jansson-2.12.tar.bz2


Then other pieces are missing. I suppose it is related to BAG and
depth of the dependencies.


Therefore, how is it possible to check the reproducibility of a
package without compiling the World? For example, the test suite of
Git kills my desktop machine. :-)



Thank you in advance.
simon



--

Toggle snippet (77 lines)
$ guix environment guitarix --dry-run -- echo Done
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
The following derivations would be built:
/gnu/store/9p90w999xdqyhv6k1x5wp0sq312l10ri-profile.drv
/gnu/store/qxhk96270rgn61nv7znbmgj5d4pamzgw-config.scm.drv
116.4 MB would be downloaded:
* /gnu/store/zjnz5gg04zfyzn5gi66lay8sgv6i4rj8-module-import-compiled
* /gnu/store/05kyg8pg8zzbrn366imllhxavdcwqgsk-mkfontdir-1.0.7
* /gnu/store/0dsc5kh6qfwshfaq01iqrfpdhlaici8n-libfontenc-1.1.4
* /gnu/store/r1p07fn372rcxzvfzqwm44w16n8gcqfc-mkfontscale-1.2.1
* /gnu/store/sclspjcznk50s894irfk2wzn6nfnxa2g-guile-gdbm-ffi-20120209.fa1d5b6
* /gnu/store/s2hqjjp057l2k6ix3yaa7cc2dpwxpfm7-module-import
* /gnu/store/pfqvay49nk9cja05sqw5cwm7fn3w4fn3-module-import-compiled
* /gnu/store/zqyr73kc60f2dyj2ykl3ijk86kias0yc-module-import-compiled
* /gnu/store/nsrym6zn3yb4390fazx1gq8rg0m8dhkd-llvm-8.0.0
* /gnu/store/vf5lkc27z5rcbvaw6rkdmqisyg923m5r-mesa-19.1.4
* /gnu/store/kjlnb30snlawpb4fb8qwhbzhcpbxlckk-libepoxy-1.5.3
* /gnu/store/4ppvmpyir7qwmhzkfxnqlb27j85rjqyz-gtk+-3.24.10-bin
* /gnu/store/wwwnp8025yb7k6qbv28hi1l6qy6j7d9z-module-import-compiled
* /gnu/store/iaqmwj2290z5nnrk69bss6r0d9lpr8cs-python2-2.7.16
/gnu/store/fzjnhsfgkqcsqjzan2dxkqgw6fl5kniy-lv2-1.16.0
/gnu/store/p8yabbkcywr8a5pcy3dg6w0mjaw6ddlq-zita-resampler-1.3.0
* /gnu/store/7zlxdamykwrd1vjp3kxv54qyyv7ya6jr-util-macros-1.19.2
/gnu/store/ksjvb9m0ky0g7yv27v70l44h8nj6v1qy-gtkmm-2.24.5
/gnu/store/pxwz526473fynfhwnljs0wb5vd2qcg4c-atkmm-2.28.0
/gnu/store/l8nphg0idd8pfddyad8f92lx8d1hc053-python-wrapper-3.7.4
/gnu/store/ms1q76ikx7f78y7i8crg42lyv88xfvfv-ghostscript-with-cups-9.27
/gnu/store/xc98lvvcbaabn0v1md34hxbdn5ivg72g-cups-filters-1.25.1
/gnu/store/lv9xckn9in3cbswaprv5j9xgsa8wghpr-cups-2.2.11
/gnu/store/vb4g1m42k156gwiwq62hd1db92ndvnch-gtk+-2.24.32
* /gnu/store/b824dq3bccq0bhjli3li0fzi11lg1bh3-xorgproto-2019.1
* /gnu/store/arzn28zwj8bqv2qiid7ybx3aad49c3pd-libpthread-stubs-0.4
/gnu/store/j1ldfckx81mm6fydlhzw03cv7hsr58ya-faust-0.9.90
/gnu/store/h10dlkcf019047d8jmjkw2w0h96zad2d-gperf-3.1
/gnu/store/zc4shgdr8pw9z5rcv9657p37wyha4nqa-fftwf-3.3.8
/gnu/store/dgkwjigfbadm0jn2y9z6d29lq0i3wj97-zita-convolver-3.1.0
/gnu/store/rm9q1vqhg333nwpywsbf017fj88dlgsf-eigen-3.3.5
/gnu/store/3vgaq9ga221mjx21sll3lxqxdgclgws4-flac-1.3.3
/gnu/store/y8wkxd3pmp29k1j0knmg9hnjnrfnm9iy-libsndfile-1.0.28
/gnu/store/f5g4av3mwn7zr81yqr1gn9hpb5d2c4m4-boost-1.70.0
/gnu/store/wrlkvgl0lz7b03gwqf27ql6pjkmj5v6r-bdb-6.2.32
/gnu/store/q1mz5mid5y4y3z12g5ify10ci7h72dnq-jack-0.125.0
* /gnu/store/5686hrxkbsm6ycf1ks9nja3mjjxjpl9a-desktop-file-utils-0.23
/gnu/store/a9rsi05xscg0bq6q0rbhcv5586zvf3li-cairomm-1.12.2
/gnu/store/s0djjrx5x8c12sa1p0wl10crvl7rzs0v-libsigc++-2.10.2
/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4
* /gnu/store/fpj5mspa7jmksixqpnzbvzs3q2vbqq31-glib-2.60.6-bin
/gnu/store/b8pr2k0i2zd07zmb7kpffmcimqi337if-glib-2.60.6
/gnu/store/94s92fpkk14y514qwf4f4hnc54qz0zfj-glibmm-2.60.0
/gnu/store/dhnzrhs9vf40p5v09817rbcd3ks07slc-pangomm-2.42.0
* /gnu/store/20ilkjz5kd98zdm6rsk6zdw5p9nh0hq7-perl-xml-parser-2.44
/gnu/store/qw4p5qwd1f1kcwspm455njd4ny7v9gww-sratom-0.6.2
/gnu/store/r0jrq3jvvxrbx7sbqhs168yxaan9311v-sord-0.16.2
/gnu/store/77h391w9aynwwb5j83yakav5m6qi7bbg-lilv-0.24.4
/gnu/store/vfcbn39fwng8d35gvic51f235fxvj7y1-intltool-0.51.0
* /gnu/store/k3m2kz55qiklkyihnnvhxhv0ylsyyaqc-module-import-compiled
/gnu/store/6zasp7vh5jww0naybhva026z1967scr7-ladspa-1.13
/gnu/store/v557q2wd91sm5vj3lrwjzajafblklr6w-libxslt-1.1.33
/gnu/store/k2m4q2av9hw73hw2jx6qrxqdyh855398-openssl-1.1.1c
/gnu/store/f8aljw2qhv3d1br9czn8v5afbgfdrxkg-cyrus-sasl-2.1.27
/gnu/store/bjxd9jzc560d6i3i35f5yy5mljk0ib6m-openldap-2.4.47
/gnu/store/4m8dlhrzis07787xznx73ang35c3lly1-curl-7.65.3
/gnu/store/dgv36cg3x3pi2v00arwlfcvq3p7id4h5-raptor2-2.0.15
/gnu/store/x5k749kbwbmbd1sn0j6ybpcc9450fba2-lrdf-0.6.1
/gnu/store/pvz6zmy4lwaicjk07999gbsaim0p4iai-serd-0.30.0
The following profile hooks would be built:
/gnu/store/5jc3lsfm9mj8smp3a4g4aqkq05zrjsq5-xdg-desktop-database.drv
/gnu/store/6csqwjlmnsmpcgcdrxjm841d54xf7pcl-xdg-mime-database.drv
/gnu/store/6k5057zfsxqf3kn5iqf27zz91jl7zbmh-info-dir.drv
/gnu/store/chzwnq3kl6gc36sh2krlbfvxfga13cq5-gtk-icon-themes.drv
/gnu/store/i22j0197hy2byv23f884dd0j6y7r93hp-manual-database.drv
/gnu/store/immlg222fg9wpnjb04j6czwp8gviw7as-gtk-im-modules.drv
/gnu/store/j7m81fkl3qacc1cnzifqiy0a4hzb8i6s-fonts-dir.drv
/gnu/store/j96f9a5m2d7wi8pdvj6d5jvldbzjc3k8-glib-schemas.drv
/gnu/store/mf3abgwykhp243r8cbfjwl1w7x6d7dkm-ca-certificate-bundle.drv

Toggle snippet (49 lines)
$ guix build `guix show guitarix | recsel -R dependencies` --dry-run
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
76.1 MB would be downloaded:
/gnu/store/f5g4av3mwn7zr81yqr1gn9hpb5d2c4m4-boost-1.70.0
* /gnu/store/33f8qhxa69dmd43yqdx3wq1b2hqjddgb-curl-7.65.3-doc
# /gnu/store/4m8dlhrzis07787xznx73ang35c3lly1-curl-7.65.3
/gnu/store/rm9q1vqhg333nwpywsbf017fj88dlgsf-eigen-3.3.5
/gnu/store/j1ldfckx81mm6fydlhzw03cv7hsr58ya-faust-0.9.90
* /gnu/store/3pfj84hcpw0xfxh8briill4c5mnk51ha-gettext-minimal-0.20.1-doc
* /gnu/store/ypwxvcnrsdn0snllv944ckylwx3p1m79-gettext-minimal-0.20.1
/gnu/store/h10dlkcf019047d8jmjkw2w0h96zad2d-gperf-3.1
* /gnu/store/vmwm54y790r3ipbyd1l8qzxhzw0byv7d-gtk+-2.24.32-doc
# /gnu/store/vb4g1m42k156gwiwq62hd1db92ndvnch-gtk+-2.24.32
/gnu/store/pxwz526473fynfhwnljs0wb5vd2qcg4c-atkmm-2.28.0
/gnu/store/ms1q76ikx7f78y7i8crg42lyv88xfvfv-ghostscript-with-cups-9.27
/gnu/store/xc98lvvcbaabn0v1md34hxbdn5ivg72g-cups-filters-1.25.1
/gnu/store/lv9xckn9in3cbswaprv5j9xgsa8wghpr-cups-2.2.11
# /gnu/store/vb4g1m42k156gwiwq62hd1db92ndvnch-gtk+-2.24.32
/gnu/store/a9rsi05xscg0bq6q0rbhcv5586zvf3li-cairomm-1.12.2
/gnu/store/s0djjrx5x8c12sa1p0wl10crvl7rzs0v-libsigc++-2.10.2
/gnu/store/fpj5mspa7jmksixqpnzbvzs3q2vbqq31-glib-2.60.6-bin
/gnu/store/b8pr2k0i2zd07zmb7kpffmcimqi337if-glib-2.60.6
/gnu/store/94s92fpkk14y514qwf4f4hnc54qz0zfj-glibmm-2.60.0
/gnu/store/dhnzrhs9vf40p5v09817rbcd3ks07slc-pangomm-2.42.0
/gnu/store/ksjvb9m0ky0g7yv27v70l44h8nj6v1qy-gtkmm-2.24.5
/gnu/store/vfcbn39fwng8d35gvic51f235fxvj7y1-intltool-0.51.0
/gnu/store/wrlkvgl0lz7b03gwqf27ql6pjkmj5v6r-bdb-6.2.32
/gnu/store/q1mz5mid5y4y3z12g5ify10ci7h72dnq-jack-0.125.0
/gnu/store/6zasp7vh5jww0naybhva026z1967scr7-ladspa-1.13
/gnu/store/3vgaq9ga221mjx21sll3lxqxdgclgws4-flac-1.3.3
/gnu/store/y8wkxd3pmp29k1j0knmg9hnjnrfnm9iy-libsndfile-1.0.28
/gnu/store/qw4p5qwd1f1kcwspm455njd4ny7v9gww-sratom-0.6.2
/gnu/store/r0jrq3jvvxrbx7sbqhs168yxaan9311v-sord-0.16.2
/gnu/store/pvz6zmy4lwaicjk07999gbsaim0p4iai-serd-0.30.0
/gnu/store/77h391w9aynwwb5j83yakav5m6qi7bbg-lilv-0.24.4
/gnu/store/v557q2wd91sm5vj3lrwjzajafblklr6w-libxslt-1.1.33
/gnu/store/f8aljw2qhv3d1br9czn8v5afbgfdrxkg-cyrus-sasl-2.1.27
/gnu/store/bjxd9jzc560d6i3i35f5yy5mljk0ib6m-openldap-2.4.47
# /gnu/store/4m8dlhrzis07787xznx73ang35c3lly1-curl-7.65.3
/gnu/store/dgv36cg3x3pi2v00arwlfcvq3p7id4h5-raptor2-2.0.15
/gnu/store/x5k749kbwbmbd1sn0j6ybpcc9450fba2-lrdf-0.6.1
/gnu/store/k2m4q2av9hw73hw2jx6qrxqdyh855398-openssl-1.1.1c
/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4
/gnu/store/l8nphg0idd8pfddyad8f92lx8d1hc053-python-wrapper-3.7.4
/gnu/store/fzjnhsfgkqcsqjzan2dxkqgw6fl5kniy-lv2-1.16.0
/gnu/store/zc4shgdr8pw9z5rcv9657p37wyha4nqa-fftwf-3.3.8
/gnu/store/dgkwjigfbadm0jn2y9z6d29lq0i3wj97-zita-convolver-3.1.0
/gnu/store/p8yabbkcywr8a5pcy3dg6w0mjaw6ddlq-zita-resampler-1.3.0
L
L
Ludovic Courtès wrote on 14 Nov 2019 22:09
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 21803@debbugs.gnu.org)
87v9rmmah9.fsf@gnu.org
Hello,

zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (24 lines)
> On Wed, 13 Nov 2019 at 14:42, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> zimoun <zimon.toutoune@gmail.com> skribis:
>>
>> > From my test with Guix d258d9c7d222e6b64531c14293f41bd8d62ea4f7,
>> > "guix challenge guitarix" and "guix build --rounds=3" do not report
>> > issues about reproducibility.
>> >
>> > And from my knowledge, the waf-based packages are not affected.
>> >
>> > Do you agree to close this bug since it is not relevant anymore?
>>
>> If you’ve checked that a local build gives the same result several times
>> in a row (make sure it actually rebuilt things; “guix build --rounds=3
>> foo” does nothing if “foo” is already in the store), then you can
>> definitely close it!
>
> I think I did but how do you do that cleanly?
>
>
>
> I am testing with "brute force" method: "guix gc -C" then proceed. I
> notice unexpected behaviour; not sure it is the right place to report.

I would probably do “guix build guitarix” (get the substitute) and then
“guix build guitarix --check --no-grafts -K”, possibly several times*,
which builds guitarix alone from source.

(*) ‘--rounds’ is ignored when combined with ‘--check’, go figure…

Toggle quote (8 lines)
> 1. The first issue is that the two following commands do not populate
> the same way.
>
> (Because my machine is not very powerful, before building I populate
> the store with the dependencies from substitutes.)
>
> $ guix environment guitarix -- echo Done

That should work.

Toggle quote (2 lines)
> $ guix build `guix show guitarix | recsel -R dependencies`

That gives a different result because ‘dependencies’ does not show
implicit inputs.

Toggle quote (3 lines)
> Why `ghostscript` is downloaded and not reported; neither by the two
> populating commands?

This is because of grafts, which lead to a poor UX as things are…

Perhaps there are genuine issues described in the rest of your message
but we’d need to isolate them first. :-)

Thanks,
Ludo’.
Z
Z
zimoun wrote on 15 Nov 2019 12:53
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 21803@debbugs.gnu.org)
CAJ3okZ3kmFhNAsg-EWe2973gWoA=FTQMgjWr-Kt4_TdbV5LN2g@mail.gmail.com
Hi Ludo,

Thank you for the explanations.


On Thu, 14 Nov 2019 at 22:09, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (4 lines)
> I would probably do “guix build guitarix” (get the substitute) and then
> “guix build guitarix --check --no-grafts -K”, possibly several times*,
> which builds guitarix alone from source.

I confirm what you reported:

Toggle snippet (9 lines)
$ LC_ALL=C diff -r --no-dereference
/gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1
/gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1-check
File /gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1/lib/libgxw.so
is a symbolic link while file
/gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1-check/lib/libgxw.so
is a regular file

As you said, it needs more investigation... :-)


Toggle quote (3 lines)
> Perhaps there are genuine issues described in the rest of your message
> but we’d need to isolate them first. :-)

Yes, first things first. :-)
And I will open discussion elsewhere because it is not related to this bug.


Thanks,
simon
Z
Z
zimoun wrote on 15 Nov 2019 16:35
other waf non reproducible: mpv and ardour
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 21803@debbugs.gnu.org)
CAJ3okZ3veC+BuAnaGevHU=Gj+xQbb86M-+v3KhMp561Q7VAnvw@mail.gmail.com
Hi,

I do not know if it is related but these 2 packages using the waf
build system are not reproducible too.

- mpv
- ardour

Toggle snippet (12 lines)
$ diff -r --no-dereference
/gnu/store/gb935w89qyyi4ljyaz40brai2z9wg0fr-mpv-0.30.0
/gnu/store/gb935w89qyyi4ljyaz40brai2z9wg0fr-mpv-0.30.0-check
Binary files /gnu/store/gb935w89qyyi4ljyaz40brai2z9wg0fr-mpv-0.30.0/bin/mpv
and /gnu/store/gb935w89qyyi4ljyaz40brai2z9wg0fr-mpv-0.30.0-check/bin/mpv
differ
Binary files /gnu/store/gb935w89qyyi4ljyaz40brai2z9wg0fr-mpv-0.30.0/lib/libmpv.so.1.106.0
and /gnu/store/gb935w89qyyi4ljyaz40brai2z9wg0fr-mpv-0.30.0-check/lib/libmpv.so.1.106.0
differ


Toggle snippet (30 lines)
$ diff -r --no-dereference
/gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12
/gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12-check
Binary files /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12/lib/ardour5/ardour-5.12.0
and /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12-check/lib/ardour5/ardour-5.12.0
differ
Binary files /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12/lib/ardour5/engines/libclearlooks.so
and /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12-check/lib/ardour5/engines/libclearlooks.so
differ
Binary files /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12/lib/ardour5/libardour.so.3.0.0
and /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12-check/lib/ardour5/libardour.so.3.0.0
differ
Binary files /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12/lib/ardour5/libevoral.so.0.0.0
and /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12-check/lib/ardour5/libevoral.so.0.0.0
differ
Binary files /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12/lib/ardour5/LV2/a-comp.lv2/a-comp.so
and /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12-check/lib/ardour5/LV2/a-comp.lv2/a-comp.so
differ
Binary files /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12/lib/ardour5/LV2/a-eq.lv2/a-eq.so
and /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12-check/lib/ardour5/LV2/a-eq.lv2/a-eq.so
differ
Binary files /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12/lib/ardour5/LV2/a-fluidsynth.lv2/a-fluidsynth.so
and /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12-check/lib/ardour5/LV2/a-fluidsynth.lv2/a-fluidsynth.so
differ
Binary files /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12/lib/ardour5/LV2/reasonablesynth.lv2/reasonablesynth.so
and /gnu/store/7l5isavp1db8q5mv3zq7bmk29l6ld9qp-ardour-5.12-check/lib/ardour5/LV2/reasonablesynth.lv2/reasonablesynth.so
differ


All the best,
simon
Z
Z
zimoun wrote on 15 Nov 2019 20:46
guitarix non-reproducible hard to reproduce
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 21803@debbugs.gnu.org)
CAJ3okZ0PROt-bvhcK6g3McBvTO858di0mY_nMdxw+E2wC=xQ5g@mail.gmail.com
Dear,

Resume:
The command "guix build guitarix" downloads from ci.guix.gnu.org.
Then "guix build guitarix --no-grafts -K --check" rebuilds locally.
The two differ of one symlink.

Toggle snippet (10 lines)
diff -r --no-dereference
/gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1
/gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1-check
File /gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1/lib/libgxw.so
is a symbolic link while file
/gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1-check/lib/libgxw.so
is a regular file


Inspect:

After more than 10 attempts to reproduce the behaviour where the plain
binary is replaced by the symlink, it never happens locally, so I
conclude that the bug is really hard to track. Kind of bad luck. :-)


My intuition is: the non-determinism comes from the WAF configuration
files (./waf or wscript or wafadmin/).

The process conditionally enters in functions such as "do_install" or
"symlink_as", so maybe something is hidden by Python module "os" or
similar. Moreover, note that wscript:l.990 "add_group()" is used after
"add_subdirs('libgxw/gxw')".


Basically, the output of "--check" is:

Toggle snippet (15 lines)
[...]
* installing build/default/libgxw/gxw/libgxw.so as
/gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1/lib/libgxw.so.0.1
* symlink /gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1/lib/libgxw.so.0
(-> libgxw.so.0.1)
* symlink /gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1/lib/libgxw.so
(-> libgxw.so.0.1)
* installing build/default/libgxw/gxw/libgxw.so as
/gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1/lib/libgxw.so
* installing build/default/libgxwmm/gxwmm/libgxwmm.so as
/gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1/lib/libgxwmm.so.0.1
[...]


Here the step (3rd *) "symlink
/gnu/store/...-guitarix-0.38.1/lib/libgxw.so (-> libgxw.so.0.1)" is
overridden by the next one "installing
build/default/libgxw/gxw/libgxw.so as
/gnu/store/...-guitarix-0.38.1/lib/libgxw.so".


Replaying with "guix environment guitarix" then from
"/tmp/guix-build-guitarix-0.38.1.drv-0", the classical "./waf
configure --prefix=install" and "./waf build" "./waf install". In this
case, one symlink is not done and the other one is overriden by the
next step.

Toggle snippet (13 lines)
[...]
* installing build/default/libgxw/gxw/libgxw.so as
/tmp/guix-build-guitarix-0.38.1.drv-0/guitarix-0.38.1/install/lib/libgxw.so.0.1
* symlink /tmp/guix-build-guitarix-0.38.1.drv-0/guitarix-0.38.1/install/lib/libgxw.so
(-> libgxw.so.0.1)
* installing build/default/libgxw/gxw/libgxw.so as
/tmp/guix-build-guitarix-0.38.1.drv-0/guitarix-0.38.1/install/lib/libgxw.so
* installing build/default/libgxwmm/gxwmm/libgxwmm.so as
/tmp/guix-build-guitarix-0.38.1.drv-0/guitarix-0.38.1/install/lib/libgxwmm.so.0.1
[...]


Interestingly, the 3 last-last evaluations by Cuirass is of the kind "--check".


Therefore, if we were comparing on November 3rd, we would conclude
differently. ;-)


However, the last evaluation is creating the symlink without overriding it.


Toggle snippet (14 lines)
[...]
* installing build/default/libgxw/gxw/libgxw.so as
/gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1/lib/libgxw.so.0.1
* symlink /gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1/lib/libgxw.so.0
(-> libgxw.so.0.1)
* symlink /gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1/lib/libgxw.so
(-> libgxw.so.0.1)
* installing build/default/libgxwmm/gxwmm/libgxwmm.so as
/gnu/store/16g7l26rpwhza7fm5jm3lz1ycavky9yl-guitarix-0.38.1/lib/libgxwmm.so.0.1
[...]



Well, even this bug is annoying speaking about reproducibility --
because it it not the *exact* same installation -- it is mitigated by
the fact that the same binary is symlinked or not. I will revisit this
bug when WAF will fully use Python 3.


All the best,
simon
L
L
Ludovic Courtès wrote on 16 Nov 2019 17:12
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 21803@debbugs.gnu.org)
87y2wfx0kd.fsf@gnu.org
Hello!

zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (8 lines)
> My intuition is: the non-determinism comes from the WAF configuration
> files (./waf or wscript or wafadmin/).
>
> The process conditionally enters in functions such as "do_install" or
> "symlink_as", so maybe something is hidden by Python module "os" or
> similar. Moreover, note that wscript:l.990 "add_group()" is used after
> "add_subdirs('libgxw/gxw')".

My intuition :-) is that waf traverses files using directly
opendir/readdir, which returns files in an order that’s file
system-dependent. That, in turn, leads it to make .so a symlink or not
in a non-deterministic fashion.

So I would suggest looking for uses of ‘readdir’ (or anything equivalent
in Python).

It would also be worth checking what others involved in the Reproducible
Builds effort have done (Debian, openSuSE, etc.).

Thanks,
Ludo’.
Z
Z
zimoun wrote on 18 Nov 2019 19:18
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 21803@debbugs.gnu.org)
CAJ3okZ0m73bor5q-zH-Mqr_xLvjJRr5Sy_j8SHpKsQV7hYqtZw@mail.gmail.com
Hi Ludo,

On Sat, 16 Nov 2019 at 17:12, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (15 lines)
> zimoun <zimon.toutoune@gmail.com> skribis:
>
> > My intuition is: the non-determinism comes from the WAF configuration
> > files (./waf or wscript or wafadmin/).
> >
> > The process conditionally enters in functions such as "do_install" or
> > "symlink_as", so maybe something is hidden by Python module "os" or
> > similar. Moreover, note that wscript:l.990 "add_group()" is used after
> > "add_subdirs('libgxw/gxw')".
>
> My intuition :-) is that waf traverses files using directly
> opendir/readdir, which returns files in an order that’s file
> system-dependent. That, in turn, leads it to make .so a symlink or not
> in a non-deterministic fashion.

Yes, it should come from the function 'os.listdir' (readdir). The
documentation [1] says: "The list is in arbitrary order."



The function 'os.listdir' is called some times ;-)

Toggle snippet (23 lines)
$ egrep -r "os\.listdir\("
wafadmin3/py3kfixes.py: for x in os.listdir(os.path.join(dir,y)):
wafadmin3/Scripting.py: lst=os.listdir(cwd)
wafadmin3/Scripting.py: if WSCRIPT_FILE in os.listdir(calldir):
wafadmin3/Scripting.py: dirlst=os.listdir(cwd)
wafadmin3/Scripting.py: names=os.listdir(src)
wafadmin3/Scripting.py: lst=os.listdir('.')
wafadmin3/Tools/qt4.py: lst=os.listdir('/usr/local/Trolltech/')
wafadmin3/Tools/javaw.py: lst=os.listdir(path)
wafadmin3/Utils.py: return os.listdir(s)
tools/check_rpc: for f in os.listdir(basedir):
src/gx_head/builder/make:for f in os.listdir(__dir__):
wafadmin/py3kfixes.py: for x in os.listdir(os.path.join(dir,y)):
wafadmin/Scripting.py: lst=os.listdir(cwd)
wafadmin/Scripting.py: if WSCRIPT_FILE in os.listdir(calldir):
wafadmin/Scripting.py: dirlst=os.listdir(cwd)
wafadmin/Scripting.py: names=os.listdir(src)
wafadmin/Scripting.py: lst=os.listdir('.')
wafadmin/Tools/qt4.py: lst=os.listdir('/usr/local/Trolltech/')
wafadmin/Tools/javaw.py: lst=os.listdir(path)
wafadmin/Utils.py: return os.listdir(s)

And do not talk about the function 'walk' which internally use 'listdir'. :-)


I have no idea how to address this issue.


Toggle quote (3 lines)
> It would also be worth checking what others involved in the Reproducible
> Builds effort have done (Debian, openSuSE, etc.).

Lot of sun for Debian [2] ;-)
I mean from what I understand, they do not find any reproducibility
issue and they apply only this patch [3].



Do not know about openSuSE.


Cheers,
simon
L
L
Ludovic Courtès wrote on 18 Nov 2019 21:29
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 21803@debbugs.gnu.org)
875zjhgc7j.fsf@gnu.org
Hello,

zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (2 lines)
> On Sat, 16 Nov 2019 at 17:12, Ludovic Courtès <ludo@gnu.org> wrote:

[...]

Toggle quote (13 lines)
>> My intuition :-) is that waf traverses files using directly
>> opendir/readdir, which returns files in an order that’s file
>> system-dependent. That, in turn, leads it to make .so a symlink or not
>> in a non-deterministic fashion.
>
> Yes, it should come from the function 'os.listdir' (readdir). The
> documentation [1] says: "The list is in arbitrary order."
>
> [1] https://docs.python.org/2/library/os.html#os.listdir
>
>
> The function 'os.listdir' is called some times ;-)

Hmm.

BTW, did you try comparing both build logs, in particular the lines
corresponding to the creation of the offending .so file? That might
help narrow the search space.

Also, we can play with disorderfs to perhaps reproduce the problem
locally and be in more favorable debugging conditions.

Toggle quote (7 lines)
>> It would also be worth checking what others involved in the Reproducible
>> Builds effort have done (Debian, openSuSE, etc.).
>
> Lot of sun for Debian [2] ;-)
> I mean from what I understand, they do not find any reproducibility
> issue and they apply only this patch [3].

Heh, OK.

Thanks,
Ludo’.
Z
Z
zimoun wrote on 11 Sep 2020 15:57
Re: bug#21803: Guitarix builds non-deterministically
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 21803@debbugs.gnu.org)
877dt0h0we.fsf@gmail.com
Dear,

On Sun, 01 Nov 2015 at 11:30, ludo@gnu.org (Ludovic Courtès) wrote:
Toggle quote (16 lines)
> Guitarix 0.33.0, as of Guix commit 3c3e697, builds
> non-deterministically:
>
> $ ./pre-inst-env guix challenge guitarix
> updating list of substitutes from 'http://hydra.gnu.org'... 100.0%
> /gnu/store/6ksnwcqn92z2nf6qw5js7njnfqlifgpb-guitarix-0.33.0 contents differ:
> local hash: 1rh7qxmylsbsaah59h7sclqqxcz0lwsixlc0krkzwhx8gfhlyam6
> http://hydra.gnu.org/nar/6ksnwcqn92z2nf6qw5js7njnfqlifgpb-guitarix-0.33.0: 1cq4k1wdqibwraqk0wkjj6n5hgs9v9zcvwr2wfgxvgxnf5l1rfhf
> $ wget -q -O - http://hydra.gnu.org/nar/6ksnwcqn92z2nf6qw5js7njnfqlifgpb-guitarix-0.33.0 | bunzip2 | guix archive -x t
> $ LC_ALL=C diff -r /gnu/store/6ksnwcqn92z2nf6qw5js7njnfqlifgpb-guitarix-0.33.0 t
> $ LC_ALL=C diff -r --no-dereference /gnu/store/6ksnwcqn92z2nf6qw5js7njnfqlifgpb-guitarix-0.33.0 t
> File /gnu/store/6ksnwcqn92z2nf6qw5js7njnfqlifgpb-guitarix-0.33.0/lib/libgxw.so is a symbolic link while file t/lib/libgxw.so is a regular file
>
> It appears to create libgxw.so either as a symlink or as a regular file
> in a non-deterministic fashion.

Guitarix has been updated to 0.41.0 by commit
bf592ef506e1db0340dc11faa0514fe80793e6d6.

I have tried "guix build guitarix --no-grafts --check -K" at least 10
times without noticing an unreproducible behaviour.

Could someone confirm this and then close this almost 5 years old bug?


Toggle quote (3 lines)
> It would be nice to see why this happens, and whether this affects all
> Waf-based packages.

I propose to open another bug report to track the waf-based packages issues.


All the best,
simon
M
M
Maxim Cournoyer wrote on 11 Sep 2020 18:48
(name . zimoun)(address . zimon.toutoune@gmail.com)
87wo106yyw.fsf@gmail.com
Hello,

[...]

Toggle quote (8 lines)
> Guitarix has been updated to 0.41.0 by commit
> bf592ef506e1db0340dc11faa0514fe80793e6d6.
>
> I have tried "guix build guitarix --no-grafts --check -K" at least 10
> times without noticing an unreproducible behaviour.
>
> Could someone confirm this and then close this almost 5 years old bug?

Confirmed!

Toggle quote (5 lines)
>> It would be nice to see why this happens, and whether this affects all
>> Waf-based packages.
>
> I propose to open another bug report to track the waf-based packages issues.

This sounds reasonable, given this is 5 years old and perhaps not
relevant anymore.

Closing, thank you!

Maxim
Closed
?