build hash inconsistency

Hi all,

tl;dr I run the following command on two aarch64-linux machines and get

two different hashes for the 'qutebrowser' package:

guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qutebrowser --dry-run

Both machines use only the main guix repository, and guix describe gives

the same output (except generation number and date, which is fine).

Coming from aarch64, building is incredibly expensive. If the build

hash doesn't match, then (I believe) there is no hope that my machine

will find the packages on a substitute server. To get around this

issue, I built my guix home once, guix copy'd the store items, and

manually added a symlink in

/var/guix/profiles/per-user/USER/guix-home-N-link to point to the

foreign guix home build. I couldn't find this issue elsewhere in the

issues, but my "hashes don't match" problem is pretty vague.

Is this an expected problem? Is this a novel problem? Am I

misunderstanding guix time-machine (which seems like it should produce

an identical store item)?

-Zacchae

Some more context might be useful:

Device A (which successfully built qutebrowser over a couple days)

~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qutebrowser --dry-run

/gnu/store/i9ir7a26gv1ii98b4bzgvxp1sx0akind-qutebrowser-2.5.4

Device B (trying to avoid building qutebrowser)

~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qutebrowser --dry-run

The following derivations would be built:

/gnu/store/6n9aq7l5x26xfgrbvws7gvscbzvq5gqn-qutebrowser-2.5.4.drv

/gnu/store/05wj7wf7bdlkm1ar58kpakvp52drrz7p-qtwebengine-5.15.10.drv

/gnu/store/7nmv55qqw62jwzrd650vm434s3wpi7i3-python-pyqtwebengine-5.15.9.drv

I have noticed in the past that sometimes a package gets built twice

with two hashes, so I went back and checked if device A had built the

6n9..5gqn qutebrowser store item reported by B in addition to the

i9i..ind one A was reporting. There was no 6n9..5gqn qutebrowser build

on device A. I don't believe I have never run guix gc on either device.

-Zacchae

Those hashes are not comparable: i9ir..nd (A) is the hash of the built

store item and 6n9aq..qn (B) is the hash of the derivation that builds

the store item.

But I do think it is weird if the derivation is not present on the

machine that build qutebrowser. Would you mind sending the output of

`guix time-machine ... -- build --derivations ...` from both machines?

If they return a different hash you could maybe also check the returned

`/gnu/store/...-qutebrowser-2.5.4.drv` files to see what is the

difference between them.

- Saku

Saku Laesvuori <saku@laesvuori.fi> writes:

Ah, rookie mistake :|

The derivations returned by guix build are still different:

Device A:

~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qutebrowser --derivations --dry-run

/gnu/store/vyk5zfr2bjh4cilw4zk3d5xkjiar99h0-qutebrowser-2.5.4.drv

Device B:

~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qutebrowser --derivations --dry-run

The following derivations would be built:

/gnu/store/6n9aq7l5x26xfgrbvws7gvscbzvq5gqn-qutebrowser-2.5.4.drv

/gnu/store/05wj7wf7bdlkm1ar58kpakvp52drrz7p-qtwebengine-5.15.10.drv

/gnu/store/7nmv55qqw62jwzrd650vm434s3wpi7i3-python-pyqtwebengine-5.15.9.drv

I wanted to build the .drv for qutebrowser as you suggested, but trying

to do so caused qtwebengine to actually begin to build (not an option).

I decided to try and build the .drv for qtwebengine on both devices and

got even more interesting differences (see below). Main things I

noticed: qtwebengine-everywhere-src has different hashes but they are

both found on substitute servers, the qtwebengine .drv file also has

different hashes, oh no that's a lot of thing my device is trying to

build.

I get errors if I try to build for armhf-linux, so I don't think I

installed the wrong version of Guix. Could it be a difference in

guix-daemon version? I have Guix on foreign distro, but I modified the

guix-daemon service to point to /var/guix/../root/../guix daemon instead

of /usr/bin/guix-daemon. Those root profile versions could be quite

different between the two devices (though in theory that shouldn't

matter).

-Zacchae

Device A:

~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qtwebengine --derivations --dry-run

substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%

substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%

The following derivation would be built:

/gnu/store/gnrk76mlrv3ipm2k3lpmy1533mn9dqc3-qtwebengine-6.5.2.drv

470.1 MB would be downloaded:

/gnu/store/yw7r6bzl0yya0s0h9glz06vy2j88f14d-python-soupsieve-2.2.1

/gnu/store/i42j0d1797cw6jh3mnrdfgwz2mvsgnv7-clang-14.0.6

/gnu/store/2ghh4x5j4hsanan9rpka80lxdh1kxnrq-lld-wrapper-0

/gnu/store/9ylcnhf8rcgwd3jazghz72jcq67kb7hs-lld-14.0.6

/gnu/store/zsa0xdc4pqjils8j8gn6s0cgcf22j48c-ld.lld-wrapper-0

/gnu/store/bjd1gyvag1fjwbm4q8fdkx4ci348khwn-lld-as-ld-wrapper-14.0.6

/gnu/store/b3j2dabjfk3z8vrcdn04dxdr4m8834hs-openh264-2.3.0

/gnu/store/ymjvjq5cyi8cclsrkqmmih6jsmxidjwp-cmake-minimal-3.24.2

/gnu/store/66anfjz8bv5a5rf9ii9fd66dzga9dlv6-jsoncpp-1.9.5

/gnu/store/b2bkchln7na32bk5mfd1gl896ac2ng9d-clang-runtime-14.0.6

/gnu/store/na1xngg46bwpxh9jl74hnsw5gp5q8mbp-python-beautifulsoup4-4.11.1

/gnu/store/i5hhylp02w3qqf2xjyws9kbii93jjqcs-libxkbfile-1.1.0

/gnu/store/cy356bl5cj9sd915pwzvx1l2njvvh80l-qtmultimedia-6.5.2

/gnu/store/w3ja0fcg7x2g8pr0l5ggzpbjbs1sxzw2-python-html5lib-1.1

/gnu/store/8c5al6nn6ihqm6vv71fszv0k3pm0zy4b-llvm-14.0.6

/gnu/store/i74mgypyv2j7i6p28x5z5r1l7mn3q11d-python-chardet-5.1.0

/gnu/store/x0j572gnp5az71lcxs58274z11g27rsg-qtwebchannel-6.5.2

/gnu/store/yqppj5kr0fzph6wvbhxlq6v6ynpa5xsq-protobuf-3.21.9

/gnu/store/0cwx6qvc0jl5amb6lcanninis5xwsm49-md4c-0.4.8

/gnu/store/a51gd25m22p623c3mwpbsc0bah30r534-qtbase-6.5.2

/gnu/store/xylrvhrdv72x62vi94f73m7wdq8361r6-qtlanguageserver-6.5.2

/gnu/store/6m5andsmqhxk7jrk1f0fmsa5012jmiw9-qtdeclarative-6.5.2

/gnu/store/qi88j5lvxg2cmc391frvvlz91m4jv3ij-qtwebengine-everywhere-src-6.5.2.tar.xz

/gnu/store/733pxaaxhdqs95a7qp7ydh831a77lyzp-ninja-1.11.1

/gnu/store/qjby2hz1d2154jx06755x8wxyfi5ycmd-python-webencodings-0.5.1

/gnu/store/1ik19yadbhc959nbk42gb548139vypp3-node-18.19.0

Device B:

~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build --derivations qtwebengine --dry-run

substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%

substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%

The following derivations would be built:

/gnu/store/qgk0k5312p9bfikhgi87644rdb9h71g5-qtwebengine-6.5.2.drv

/gnu/store/3kr5g43593v5x42nhz2396hb8a4sp7k8-qtwebchannel-6.5.2.drv

/gnu/store/1waf6s55grvdx686nn4fg5mfsxz0567g-qtwebsockets-6.5.2.drv

/gnu/store/lffwivm3lgk8p5sgifa49wx3j29srlc7-qtdeclarative-6.5.2.drv

/gnu/store/082dd9z9d8n5s09x4m6k4d1ckxx4yzj2-qtshadertools-6.5.2.drv

/gnu/store/zi5iqbjqi7m4a6rd7d1ynvp6k4cklbh6-qtbase-6.5.2.drv

/gnu/store/k8s6y1rmg7qz5k9vrbh34p0qp19zraqn-md4c-0.4.8.drv

/gnu/store/nj9pps7kcr4y4xdv46l84n8vy07y63yy-qtbase-everywhere-src-6.5.2.tar.xz.drv

/gnu/store/1x4swi9dmq7wvh58pdqn4qjyqvzf6z11-qtdeclarative-everywhere-src-6.5.2.tar.xz.drv

/gnu/store/56q3r3nn2cm91r14pn3lwwwrmhx6i89p-qtimageformats-6.5.2.drv

/gnu/store/bw3zc4phc52wiah8mwwmb5d4533c3rdi-qtimageformats-everywhere-src-6.5.2.tar.xz.drv

/gnu/store/bdb9kkf7w2c41imdchx7b3f3sqv8i2dy-qtsvg-6.5.2.drv

/gnu/store/y85bhd7s2jj5klrwvj5hnjj9gjiwry8q-qtlanguageserver-6.5.2.drv

/gnu/store/3lfwgdq5qvwd6cywj0bq2japs7clv5qx-qtwebengine-everywhere-src-6.5.2.tar.xz.drv

/gnu/store/a5vpw1yl0xc1r49r22wfzdj40s85pzab-lld-as-ld-wrapper-14.0.6.drv

/gnu/store/l4j3wl0dvdw4f1xffk4vcax48anig5yp-lld-wrapper-0.drv

/gnu/store/rigww18q26c102h93kj1yqklh8nr6fmw-lld-14.0.6.drv

/gnu/store/akgv8z3df74x4438l7amhy461nsq0kws-llvm-14.0.6.drv

/gnu/store/iqvgng8pcmkxn9c8z8552d5wkhg25hzg-llvm-project-14.0.6-checkout.drv

/gnu/store/z1isl8dl5xhc9c6hqi1zhm4pj07lgiai-ld.lld-wrapper-0.drv

/gnu/store/g1g308r6rs5frcx6qs22m8aivhjssj2c-clang-runtime-14.0.6.drv

/gnu/store/ixhdpsvchd147ka9iizqi1gba8bgc8ws-qtmultimedia-6.5.2.drv

/gnu/store/6qpjcxxzj1zz8hpwbd7h37ax9s7lghqy-qtmultimedia-everywhere-src-6.5.2.tar.xz.drv

/gnu/store/pypzr5m6wqj867fjyp3166p9as71bn61-gst-plugins-good-1.22.2.drv

/gnu/store/f9zys916nr1kz2brx4mp3p1wxaz8lp2i-libsoup-3.1.4.drv

/gnu/store/syv3mj26j0ggvw46c8mhxv7qqrhsqn59-gst-libav-1.22.2.drv

/gnu/store/k7ga8s1ds7dmwvpb031p8s5kahvm4nz8-clang-14.0.6.drv

/gnu/store/rnj3bamx0wzdyfr9smi33cx1yw9gl62f-node-18.19.0.drv

/gnu/store/h20fm9fy8xiybpw5xisrqq1w4k6jpv7z-llhttp-6.0.11.drv

/gnu/store/91ws1b9ckzrj0hvbgx36ripgdf0hh1j7-node-llparse-7.1.0.drv

/gnu/store/9za67qcddkchhl0m6xrpfh25hi4nr9pn-module-import-compiled.drv

/gnu/store/s5h43lrgqfdh46s819arhxy64xdalxpv-node-llparse-7.1.0-checkout.drv

/gnu/store/svw3w50if1mq5cv93zcwyw9hp5z5c8j2-node-debug-4.3.0.drv

/gnu/store/8zlx7n5cnyy50yvwmj9pl0xkzgx7ip3k-node-ms-2.1.2.drv

/gnu/store/z4l1iy97z9kpbl991vhk0yn7akya3pm4-node-llparse-frontend-3.0.0.drv

/gnu/store/fp155qzisf9xvppnvxj51wazkilm6mpn-node-llparse-builder-1.5.2.drv

/gnu/store/06rkh3k2qh776ipqxm8bzvi15y88y3zs-node-binary-search-1.3.6.drv

/gnu/store/850nb2yf2z4lkdrfa3x21s5z040zxghm-node-llparse-builder-1.5.2-checkout.drv

/gnu/store/iwzzpkigabb4lxgxkil439aabxbfrjpp-node-llparse-frontend-3.0.0-checkout.drv

/gnu/store/i6a3c6k2g9dvhpzkjha352f866y9kpkv-llhttp-6.0.11-checkout.drv

/gnu/store/qp1zjmb7wrnc5jdydlc47ijl25ypi0w1-node-semver-7.2.1.drv

/gnu/store/j6hf21a1ckssx5g98bl10qgpwslj3ha7-node-v18.19.0.tar.xz.drv

422.4 MB would be downloaded:

/gnu/store/qixglkg4928gixdiwrfb1raykab66j4r-ruby-3.1.4

/gnu/store/1dv7a5a3l5ixpqmlb4jpys0jj8w2qchr-qtwebengine-everywhere-src-6.5.2.tar.xz

/gnu/store/il59vx2siqx7bzqm0y3pahqk79w7p1s3-xorg-server-21.1.10

/gnu/store/gagpiy55s58nrphxcyq1pkzffjywf8wy-xvfb-run-21.1.7-1

/gnu/store/gi04nc9q1z9bm5kxy11ghc5yh5skg0m1-qtwebsockets-everywhere-src-6.5.2.tar.xz

Hi Zacchaeus,

Can you try the same, but this time with the --no-grafts option? That

could be a source of issues.

Best,

--

Josselin Poiret

Hi Josselin,

Alas, the problem persists ~.~

Device A:

~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qtwebengine --derivations --system=aarch64-linux --no-grafts --dry-run

/gnu/store/gnrk76mlrv3ipm2k3lpmy1533mn9dqc3-qtwebengine-6.5.2.drv

Device B:

~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qtwebengine --derivations --system=aarch64-linux --no-grafts --dry-run

/gnu/store/qpb6d1qqx357rkydk0xv6ail6b9dcqs6-qtwebengine-6.5.2.drv

The fact that Device A finds substitutes for nearly everything while

Device B doesn't makes me think that B is "wrong". Device A is a 3GiB

RAM Librem 5 (made in China), and Device B is a 4GiB RAM Liberty Phone

(made in USA). (Maybe US backdoor bots need to up their stealth game?)

You can find info on how I set up guix at

https://zacchae.us/guix-usd.org,specifically the section "Setting up

guix home"

-Zacchae

PS - I see your response on issues.guix.gnu.org, but didn't get an

email. I thought submitting bugs automatically subscribed me?

Hi all,

I just tried the previous command on Device C, an x86_64-linux Guix System:

~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qtwebengine --derivations --system=aarch64-linux --no-grafts --dry-run

<lots of non-error prints>

/gnu/store/gnrk76mlrv3ipm2k3lpmy1533mn9dqc3-qtwebengine-6.5.2.drv

As suspected, the hash matches that of device A.

I'm tempted to reflash Device B, wipe and reencrypt the SD (which has

/gnu/store), and reinstall Guix, but I also want to get to the bottom of

this. So long as this thread doesn't get stale, I'm happy to try

whatever suggestions people have.

-Zacchae