build hash inconsistency

Open

Details

3 participants

Josselin Poiret
Saku Laesvuori
Zacchaeus Scheffer

Owner: unassigned

Submitted by: Zacchaeus Scheffer

Severity: normal

Zacchaeus Scheffer wrote on 30 Jan 01:06 +0100

Recipients:(address . bug-guix@gnu.org)

Message-ID:87bk938z2u.fsf@gmail.com

Hi all,

tl;dr I run the following command on two aarch64-linux machines and get

two different hashes for the 'qutebrowser' package:

guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qutebrowser --dry-run

Both machines use only the main guix repository, and guix describe gives

the same output (except generation number and date, which is fine).

Coming from aarch64, building is incredibly expensive. If the build

hash doesn't match, then (I believe) there is no hope that my machine

will find the packages on a substitute server. To get around this

issue, I built my guix home once, guix copy'd the store items, and

manually added a symlink in

/var/guix/profiles/per-user/USER/guix-home-N-link to point to the

foreign guix home build. I couldn't find this issue elsewhere in the

issues, but my "hashes don't match" problem is pretty vague.

Is this an expected problem? Is this a novel problem? Am I

misunderstanding guix time-machine (which seems like it should produce

an identical store item)?

-Zacchae

Zacchaeus Scheffer wrote on 30 Jan 01:38 +0100

Recipients:(address . 68811@debbugs.gnu.org)

Message-ID:877cjr8xkv.fsf_-_@gmail.com

Some more context might be useful:

Device A (which successfully built qutebrowser over a couple days)

~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qutebrowser --dry-run

/gnu/store/i9ir7a26gv1ii98b4bzgvxp1sx0akind-qutebrowser-2.5.4

Device B (trying to avoid building qutebrowser)

~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qutebrowser --dry-run

The following derivations would be built:

/gnu/store/6n9aq7l5x26xfgrbvws7gvscbzvq5gqn-qutebrowser-2.5.4.drv

/gnu/store/05wj7wf7bdlkm1ar58kpakvp52drrz7p-qtwebengine-5.15.10.drv

/gnu/store/7nmv55qqw62jwzrd650vm434s3wpi7i3-python-pyqtwebengine-5.15.9.drv

I have noticed in the past that sometimes a package gets built twice

with two hashes, so I went back and checked if device A had built the

6n9..5gqn qutebrowser store item reported by B in addition to the

i9i..ind one A was reporting. There was no 6n9..5gqn qutebrowser build

on device A. I don't believe I have never run guix gc on either device.

-Zacchae

Saku Laesvuori wrote on 30 Jan 08:06 +0100

Re: bug#68811: build hash inconsistency

Recipients:(name . Zacchaeus Scheffer)(address . zaccysc@gmail.com)(address . 68811@debbugs.gnu.org)

Message-ID:iubdi4grl2vsaoahdpdyiyiufeisrhrdgr46rhaswjnkc7pktt@pohwj3tphrm5

Toggle quote (13 lines)> Some more context might be useful:
> 
> Device A (which successfully built qutebrowser over a couple days)
> ~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qutebrowser --dry-run
> /gnu/store/i9ir7a26gv1ii98b4bzgvxp1sx0akind-qutebrowser-2.5.4
> 
> Device B (trying to avoid building qutebrowser)
> ~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qutebrowser --dry-run
> The following derivations would be built:
>   /gnu/store/6n9aq7l5x26xfgrbvws7gvscbzvq5gqn-qutebrowser-2.5.4.drv
>   /gnu/store/05wj7wf7bdlkm1ar58kpakvp52drrz7p-qtwebengine-5.15.10.drv
>   /gnu/store/7nmv55qqw62jwzrd650vm434s3wpi7i3-python-pyqtwebengine-5.15.9.drv

Those hashes are not comparable: i9ir..nd (A) is the hash of the built

store item and 6n9aq..qn (B) is the hash of the derivation that builds

the store item.

Toggle quote (6 lines)

> I have noticed in the past that sometimes a package gets built twice

> with two hashes, so I went back and checked if device A had built the

> 6n9..5gqn qutebrowser store item reported by B in addition to the

> i9i..ind one A was reporting. There was no 6n9..5gqn qutebrowser build

> on device A. I don't believe I have never run guix gc on either device.

But I do think it is weird if the derivation is not present on the
machine that build qutebrowser. Would you mind sending the output of
`guix time-machine ... -- build --derivations ...` from both machines?
If they return a different hash you could maybe also check the returned
`/gnu/store/...-qutebrowser-2.5.4.drv` files to see what is the
difference between them.

- Saku

Zacchaeus Scheffer wrote on 30 Jan 19:37 +0100

Recipients:(name . Saku Laesvuori)(address . saku@laesvuori.fi)(address . 68811@debbugs.gnu.org)

Message-ID:87le867jm0.fsf@gmail.com

Saku Laesvuori <saku@laesvuori.fi> writes:

Toggle quote (4 lines)

> Those hashes are not comparable: i9ir..nd (A) is the hash of the built

> store item and 6n9aq..qn (B) is the hash of the derivation that builds

> the store item.

Ah, rookie mistake :|

Toggle quote (7 lines)

> But I do think it is weird if the derivation is not present on the

> machine that build qutebrowser. Would you mind sending the output of

> `guix time-machine ... -- build --derivations ...` from both machines?

> If they return a different hash you could maybe also check the returned

> `/gnu/store/...-qutebrowser-2.5.4.drv` files to see what is the

> difference between them.

The derivations returned by guix build are still different:

Device A:
~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qutebrowser --derivations --dry-run
/gnu/store/vyk5zfr2bjh4cilw4zk3d5xkjiar99h0-qutebrowser-2.5.4.drv

Device B:
~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qutebrowser --derivations --dry-run
The following derivations would be built:
  /gnu/store/6n9aq7l5x26xfgrbvws7gvscbzvq5gqn-qutebrowser-2.5.4.drv
  /gnu/store/05wj7wf7bdlkm1ar58kpakvp52drrz7p-qtwebengine-5.15.10.drv
  /gnu/store/7nmv55qqw62jwzrd650vm434s3wpi7i3-python-pyqtwebengine-5.15.9.drv

I wanted to build the .drv for qutebrowser as you suggested, but trying
to do so caused qtwebengine to actually begin to build (not an option).
I decided to try and build the .drv for qtwebengine on both devices and
got even more interesting differences (see below).  Main things I
noticed: qtwebengine-everywhere-src has different hashes but they are
both found on substitute servers, the qtwebengine .drv file also has
different hashes, oh no that's a lot of thing my device is trying to
build.

I get errors if I try to build for armhf-linux, so I don't think I
installed the wrong version of Guix.  Could it be a difference in
guix-daemon version?  I have Guix on foreign distro, but I modified the
guix-daemon service to point to /var/guix/../root/../guix daemon instead
of /usr/bin/guix-daemon.  Those root profile versions could be quite
different between the two devices (though in theory that shouldn't
matter).


-Zacchae


Device A:
~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qtwebengine --derivations --dry-run
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
The following derivation would be built:
  /gnu/store/gnrk76mlrv3ipm2k3lpmy1533mn9dqc3-qtwebengine-6.5.2.drv
470.1 MB would be downloaded:
  /gnu/store/yw7r6bzl0yya0s0h9glz06vy2j88f14d-python-soupsieve-2.2.1
  /gnu/store/i42j0d1797cw6jh3mnrdfgwz2mvsgnv7-clang-14.0.6
  /gnu/store/2ghh4x5j4hsanan9rpka80lxdh1kxnrq-lld-wrapper-0
  /gnu/store/9ylcnhf8rcgwd3jazghz72jcq67kb7hs-lld-14.0.6
  /gnu/store/zsa0xdc4pqjils8j8gn6s0cgcf22j48c-ld.lld-wrapper-0
  /gnu/store/bjd1gyvag1fjwbm4q8fdkx4ci348khwn-lld-as-ld-wrapper-14.0.6
  /gnu/store/b3j2dabjfk3z8vrcdn04dxdr4m8834hs-openh264-2.3.0
  /gnu/store/ymjvjq5cyi8cclsrkqmmih6jsmxidjwp-cmake-minimal-3.24.2
  /gnu/store/66anfjz8bv5a5rf9ii9fd66dzga9dlv6-jsoncpp-1.9.5
  /gnu/store/b2bkchln7na32bk5mfd1gl896ac2ng9d-clang-runtime-14.0.6
  /gnu/store/na1xngg46bwpxh9jl74hnsw5gp5q8mbp-python-beautifulsoup4-4.11.1
  /gnu/store/i5hhylp02w3qqf2xjyws9kbii93jjqcs-libxkbfile-1.1.0
  /gnu/store/cy356bl5cj9sd915pwzvx1l2njvvh80l-qtmultimedia-6.5.2
  /gnu/store/w3ja0fcg7x2g8pr0l5ggzpbjbs1sxzw2-python-html5lib-1.1
  /gnu/store/8c5al6nn6ihqm6vv71fszv0k3pm0zy4b-llvm-14.0.6
  /gnu/store/i74mgypyv2j7i6p28x5z5r1l7mn3q11d-python-chardet-5.1.0
  /gnu/store/x0j572gnp5az71lcxs58274z11g27rsg-qtwebchannel-6.5.2
  /gnu/store/yqppj5kr0fzph6wvbhxlq6v6ynpa5xsq-protobuf-3.21.9
  /gnu/store/0cwx6qvc0jl5amb6lcanninis5xwsm49-md4c-0.4.8
  /gnu/store/a51gd25m22p623c3mwpbsc0bah30r534-qtbase-6.5.2
  /gnu/store/xylrvhrdv72x62vi94f73m7wdq8361r6-qtlanguageserver-6.5.2
  /gnu/store/6m5andsmqhxk7jrk1f0fmsa5012jmiw9-qtdeclarative-6.5.2
  /gnu/store/qi88j5lvxg2cmc391frvvlz91m4jv3ij-qtwebengine-everywhere-src-6.5.2.tar.xz
  /gnu/store/733pxaaxhdqs95a7qp7ydh831a77lyzp-ninja-1.11.1
  /gnu/store/qjby2hz1d2154jx06755x8wxyfi5ycmd-python-webencodings-0.5.1
  /gnu/store/1ik19yadbhc959nbk42gb548139vypp3-node-18.19.0

Device B:
~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build --derivations qtwebengine --dry-run
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
The following derivations would be built:
  /gnu/store/qgk0k5312p9bfikhgi87644rdb9h71g5-qtwebengine-6.5.2.drv
  /gnu/store/3kr5g43593v5x42nhz2396hb8a4sp7k8-qtwebchannel-6.5.2.drv
  /gnu/store/1waf6s55grvdx686nn4fg5mfsxz0567g-qtwebsockets-6.5.2.drv
  /gnu/store/lffwivm3lgk8p5sgifa49wx3j29srlc7-qtdeclarative-6.5.2.drv
  /gnu/store/082dd9z9d8n5s09x4m6k4d1ckxx4yzj2-qtshadertools-6.5.2.drv
  /gnu/store/zi5iqbjqi7m4a6rd7d1ynvp6k4cklbh6-qtbase-6.5.2.drv
  /gnu/store/k8s6y1rmg7qz5k9vrbh34p0qp19zraqn-md4c-0.4.8.drv
  /gnu/store/nj9pps7kcr4y4xdv46l84n8vy07y63yy-qtbase-everywhere-src-6.5.2.tar.xz.drv
  /gnu/store/1x4swi9dmq7wvh58pdqn4qjyqvzf6z11-qtdeclarative-everywhere-src-6.5.2.tar.xz.drv
  /gnu/store/56q3r3nn2cm91r14pn3lwwwrmhx6i89p-qtimageformats-6.5.2.drv
  /gnu/store/bw3zc4phc52wiah8mwwmb5d4533c3rdi-qtimageformats-everywhere-src-6.5.2.tar.xz.drv
  /gnu/store/bdb9kkf7w2c41imdchx7b3f3sqv8i2dy-qtsvg-6.5.2.drv
  /gnu/store/y85bhd7s2jj5klrwvj5hnjj9gjiwry8q-qtlanguageserver-6.5.2.drv
  /gnu/store/3lfwgdq5qvwd6cywj0bq2japs7clv5qx-qtwebengine-everywhere-src-6.5.2.tar.xz.drv
  /gnu/store/a5vpw1yl0xc1r49r22wfzdj40s85pzab-lld-as-ld-wrapper-14.0.6.drv
  /gnu/store/l4j3wl0dvdw4f1xffk4vcax48anig5yp-lld-wrapper-0.drv
  /gnu/store/rigww18q26c102h93kj1yqklh8nr6fmw-lld-14.0.6.drv
  /gnu/store/akgv8z3df74x4438l7amhy461nsq0kws-llvm-14.0.6.drv
  /gnu/store/iqvgng8pcmkxn9c8z8552d5wkhg25hzg-llvm-project-14.0.6-checkout.drv
  /gnu/store/z1isl8dl5xhc9c6hqi1zhm4pj07lgiai-ld.lld-wrapper-0.drv
  /gnu/store/g1g308r6rs5frcx6qs22m8aivhjssj2c-clang-runtime-14.0.6.drv
  /gnu/store/ixhdpsvchd147ka9iizqi1gba8bgc8ws-qtmultimedia-6.5.2.drv
  /gnu/store/6qpjcxxzj1zz8hpwbd7h37ax9s7lghqy-qtmultimedia-everywhere-src-6.5.2.tar.xz.drv
  /gnu/store/pypzr5m6wqj867fjyp3166p9as71bn61-gst-plugins-good-1.22.2.drv
  /gnu/store/f9zys916nr1kz2brx4mp3p1wxaz8lp2i-libsoup-3.1.4.drv
  /gnu/store/syv3mj26j0ggvw46c8mhxv7qqrhsqn59-gst-libav-1.22.2.drv
  /gnu/store/k7ga8s1ds7dmwvpb031p8s5kahvm4nz8-clang-14.0.6.drv
  /gnu/store/rnj3bamx0wzdyfr9smi33cx1yw9gl62f-node-18.19.0.drv
  /gnu/store/h20fm9fy8xiybpw5xisrqq1w4k6jpv7z-llhttp-6.0.11.drv
  /gnu/store/91ws1b9ckzrj0hvbgx36ripgdf0hh1j7-node-llparse-7.1.0.drv
  /gnu/store/9za67qcddkchhl0m6xrpfh25hi4nr9pn-module-import-compiled.drv
  /gnu/store/s5h43lrgqfdh46s819arhxy64xdalxpv-node-llparse-7.1.0-checkout.drv
  /gnu/store/svw3w50if1mq5cv93zcwyw9hp5z5c8j2-node-debug-4.3.0.drv
  /gnu/store/8zlx7n5cnyy50yvwmj9pl0xkzgx7ip3k-node-ms-2.1.2.drv
  /gnu/store/z4l1iy97z9kpbl991vhk0yn7akya3pm4-node-llparse-frontend-3.0.0.drv
  /gnu/store/fp155qzisf9xvppnvxj51wazkilm6mpn-node-llparse-builder-1.5.2.drv
  /gnu/store/06rkh3k2qh776ipqxm8bzvi15y88y3zs-node-binary-search-1.3.6.drv
  /gnu/store/850nb2yf2z4lkdrfa3x21s5z040zxghm-node-llparse-builder-1.5.2-checkout.drv
  /gnu/store/iwzzpkigabb4lxgxkil439aabxbfrjpp-node-llparse-frontend-3.0.0-checkout.drv
  /gnu/store/i6a3c6k2g9dvhpzkjha352f866y9kpkv-llhttp-6.0.11-checkout.drv
  /gnu/store/qp1zjmb7wrnc5jdydlc47ijl25ypi0w1-node-semver-7.2.1.drv
  /gnu/store/j6hf21a1ckssx5g98bl10qgpwslj3ha7-node-v18.19.0.tar.xz.drv
422.4 MB would be downloaded:
  /gnu/store/qixglkg4928gixdiwrfb1raykab66j4r-ruby-3.1.4
  /gnu/store/1dv7a5a3l5ixpqmlb4jpys0jj8w2qchr-qtwebengine-everywhere-src-6.5.2.tar.xz
  /gnu/store/il59vx2siqx7bzqm0y3pahqk79w7p1s3-xorg-server-21.1.10
  /gnu/store/gagpiy55s58nrphxcyq1pkzffjywf8wy-xvfb-run-21.1.7-1
  /gnu/store/gi04nc9q1z9bm5kxy11ghc5yh5skg0m1-qtwebsockets-everywhere-src-6.5.2.tar.xz

Josselin Poiret wrote on 30 Jan 22:59 +0100

Recipients:(address . 68811@debbugs.gnu.org)

Message-ID:87v87azdnc.fsf@jpoiret.xyz

Hi Zacchaeus,

Can you try the same, but this time with the --no-grafts option? That

could be a source of issues.

Best,

Josselin Poiret

Zacchaeus Scheffer wrote on 31 Jan 00:25 +0100

Recipients:(address . 68811@debbugs.gnu.org)

Message-ID:87eddy76ay.fsf@gmail.com

Hi Josselin,

Alas, the problem persists ~.~

Device A:

~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qtwebengine --derivations --system=aarch64-linux --no-grafts --dry-run

/gnu/store/gnrk76mlrv3ipm2k3lpmy1533mn9dqc3-qtwebengine-6.5.2.drv

Device B:

~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qtwebengine --derivations --system=aarch64-linux --no-grafts --dry-run

/gnu/store/qpb6d1qqx357rkydk0xv6ail6b9dcqs6-qtwebengine-6.5.2.drv

The fact that Device A finds substitutes for nearly everything while

Device B doesn't makes me think that B is "wrong". Device A is a 3GiB

RAM Librem 5 (made in China), and Device B is a 4GiB RAM Liberty Phone

(made in USA). (Maybe US backdoor bots need to up their stealth game?)

You can find info on how I set up guix at

https://zacchae.us/guix-usd.org,specifically the section "Setting up

guix home"

-Zacchae

PS - I see your response on issues.guix.gnu.org, but didn't get an

email. I thought submitting bugs automatically subscribed me?

Zacchaeus Scheffer wrote on 31 Jan 19:34 +0100

Recipients:(address . 68811@debbugs.gnu.org)

Message-ID:875xz98i92.fsf@gmail.com

Hi all,

I just tried the previous command on Device C, an x86_64-linux Guix System:

~ $ guix time-machine --commit=deeb7d1f53d7ddfa977b3eadd760312bbd0a2509 -- build qtwebengine --derivations --system=aarch64-linux --no-grafts --dry-run

/gnu/store/gnrk76mlrv3ipm2k3lpmy1533mn9dqc3-qtwebengine-6.5.2.drv

As suspected, the hash matches that of device A.

I'm tempted to reflash Device B, wipe and reencrypt the SD (which has

/gnu/store), and reinstall Guix, but I also want to get to the bottom of

this. So long as this thread doesn't get stale, I'm happy to try

whatever suggestions people have.

-Zacchae

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 68811@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 68811

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date