guix time-machine fails when a tarball was modified in-place

Jan Nieuwenhuizen wrote on 12 Feb 2020 14:40

Recipients:(address . bug-guix@gnu.org)

Message-ID:87y2t7j54n.fsf@gnu.org

Hi,
Trying to travel back to Sun Apr 7 22:07:14 2019 +0200 (commit56e95d54d209c2428f970d65d9b27ae4168449ad) to re-create mcrl2-minimal bydoing

Toggle snippet (3 lines)

guix time-machine --commit=56e95d54d209c2428f970d65d9b27ae4168449ad -- environment --ad-hoc mcrl2-minimal

fails with

Toggle snippet (13 lines)building /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv...downloading from https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-2.4.0.tar.bz2...|offloading build of /gnu/store/6fgg1irkcvqyb4f9f8n0nzi5gknyqhfn-gcc-mesboot1-4.7.4.drv to 'kluit.dezyne.org'- 'build' phasesha256 hash mismatch for /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2:  expected hash: 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch  actual hash:   0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5lhash mismatch for store item '/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2'build of /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv failedView build log at '/var/log/guix/drvs/cj/im33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv.bz2'.cannot build derivation `/gnu/store/p6gfcdacjcqf2br0zwsyzx1chfvg9gxi-harfbuzz-2.4.0.drv': 1 dependencies couldn't be builtkilling process 5083

The recipe for harfbuzz has a sha256 that used to be valid in April, buthasn't been valid anymore since May, as this fix

Toggle snippet (12 lines)

commit a8bb8fccd82a10a46f127b2235675b4f6cbaaf98Author: Marius Bakke <mbakke@fastmail.com>Date: Sat May 4 18:01:12 2019 +0200
gnu: harfbuzz: Update source hash. The previous tarball was modified in-place; see <https://github.com/harfbuzz/harfbuzz/issues/1641>. * gnu/packages/gtk.scm (harfbuzz)[source](sha256): Update.

shows. Thoughts?
Greetings,janneke

-- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.orgFreelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com

zimoun wrote on 12 Feb 2020 15:55

Recipients:(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)(address . 39575@debbugs.gnu.org)

Message-ID:CAJ3okZ0TL01imwQ-m50EY5O0tTsOiqittvY2TCgPu=Priz9JoA@mail.gmail.com

Hi,

On Wed, 12 Feb 2020 at 14:44, Jan Nieuwenhuizen <janneke@gnu.org> wrote:

Toggle quote (8 lines)

> Trying to travel back to Sun Apr 7 22:07:14 2019 +0200 (commit> 56e95d54d209c2428f970d65d9b27ae4168449ad) to re-create mcrl2-minimal by> doing>> --8<---------------cut here---------------start------------->8---> guix time-machine --commit=56e95d54d209c2428f970d65d9b27ae4168449ad -- environment --ad-hoc mcrl2-minimal> --8<---------------cut here---------------end--------------->8---

Even the simple:

Toggle snippet (3 lines)

guix time-machine --commit=56e95d54d209c2428f970d65d9b27ae4168449ad -- help

Toggle quote (16 lines)> fails with>> --8<---------------cut here---------------start------------->8---> building /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv...> downloading from https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-2.4.0.tar.bz2...> |offloading build of /gnu/store/6fgg1irkcvqyb4f9f8n0nzi5gknyqhfn-gcc-mesboot1-4.7.4.drv to 'kluit.dezyne.org'> - 'build' phasesha256 hash mismatch for /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2:>   expected hash: 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch>   actual hash:   0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l> hash mismatch for store item '/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2'> build of /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv failed> View build log at '/var/log/guix/drvs/cj/im33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv.bz2'.> cannot build derivation `/gnu/store/p6gfcdacjcqf2br0zwsyzx1chfvg9gxi-harfbuzz-2.4.0.drv': 1 dependencies couldn't be built> killing process 5083> --8<---------------cut here---------------end--------------->8---

same for e.g., this commit:

Toggle snippet (4 lines)

guix time-machine --commit=ae528aaf19f3828d3d7d204b15570800e1bbf100 -- help

Toggle quote (18 lines)> The recipe for harfbuzz has a sha256 that used to be valid in April, but> hasn't been valid anymore since May, as this fix>> --8<---------------cut here---------------start------------->8---> commit a8bb8fccd82a10a46f127b2235675b4f6cbaaf98> Author: Marius Bakke <mbakke@fastmail.com>> Date:   Sat May 4 18:01:12 2019 +0200>>     gnu: harfbuzz: Update source hash.>>     The previous tarball was modified in-place; see>     <https://github.com/harfbuzz/harfbuzz/issues/1641>.>>     * gnu/packages/gtk.scm (harfbuzz)[source](sha256): Update.> --8<---------------cut here---------------end--------------->8--->> shows.  Thoughts?

Therefore, all the commits between the introduction of harfbuzz withthe old sha256 (commit 2da9b81837fd1e6f08a10952784d3358be982855) andthe commit updating to the new sha256 should be broken.Roughly speaking, all the commits between April, 7th and the May, 4th;i.e., 1100+ commits, isn't it?

Well, this ask an interesting question: how Guix can fallback whenupstream is doing wrong?

Considering this 'harbuzz' issue, is it possible to rebuild the oldtarball and push it to SoftWare Heritage? Then when a sha mismatchhappens, fallback and try to fetch it from SWH?

WDYT?
Cheers,simon

Ludovic Courtès wrote on 13 Feb 2020 22:34

Recipients:(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)

Message-ID:87eeuy2mua.fsf@gnu.org

Hi,
Jan Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (8 lines)

> building /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv...> downloading from https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-2.4.0.tar.bz2...> |offloading build of /gnu/store/6fgg1irkcvqyb4f9f8n0nzi5gknyqhfn-gcc-mesboot1-4.7.4.drv to 'kluit.dezyne.org'> - 'build' phasesha256 hash mismatch for /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2:> expected hash: 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch> actual hash: 0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l> hash mismatch for store item '/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2'

The problem here is really that we fall back to content-addressedmirrors instead of using them directly:
https://issues.guix.gnu.org/issue/28659
The file itself is still available on our machines though, and you canget it with:
guix download -o harfbuzz-2.4.0.tar.bz2 \ https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l
guix download file://$PWD/harfbuzz-2.4.0.tar.bz2
After that, re-running ‘guix time-machine’ should work.
Using ci.guix.gnu.org for substitutes should have the same effect.
HTH,Ludo’.

zimoun wrote on 14 Feb 2020 02:05

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:CAJ3okZ26EvDxWeDQDzamC7WDhJiLJa7p1rch1zeP1ye8+skSLg@mail.gmail.com

Hi Ludo,
On Thu, 13 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (5 lines)

> The problem here is really that we fall back to content-addressed> mirrors instead of using them directly:>> https://issues.guix.gnu.org/issue/28659

Thank you for the pointer. Good to see that the problem is almost addressed.I will try to understand the discussion and see what is the status ofthe proposed patch.

Toggle quote (3 lines)

> The file itself is still available on our machines though, and you can> get it with:

It is an half cooked solution because the Guix project cannot archiveall; for example in term of store resources.
The content-addressed mirror should be SWH, IMHO.
Well, once sources.json will be up, it should be almost done for the future.But we still need to push all the correct sources that are onci.guix.gnu.org; at least all the url based source of package.
Do you have suggestion for a plan?

Toggle quote (7 lines)

> guix download -o harfbuzz-2.4.0.tar.bz2 \> https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l>> guix download file://$PWD/harfbuzz-2.4.0.tar.bz2>> After that, re-running ‘guix time-machine’ should work.

Thank you. This should fix the harfbuzz mismatch issue. Cool! :-)

Toggle quote (2 lines)

> Using ci.guix.gnu.org for substitutes should have the same effect.

Hum? I thought that I used ci.guix.gnu.org as substitutes... soI need to check.

Cheers,simon

Giovanni Biscuolo wrote on 14 Feb 2020 11:03

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 39575@debbugs.gnu.org)

Message-ID:87y2t5a3ks.fsf@roquette.mug.biscuolo.net

Hello Ludo'
Ludovic Courtès <ludo@gnu.org> writes:
[...]

Toggle quote (5 lines)

> The problem here is really that we fall back to content-addressed> mirrors instead of using them directly:>> https://issues.guix.gnu.org/issue/28659

Given the natute (AFAIU) of this issue is the very same of the bug youmention, shouldn't this bug be merged (forcemerged?) whith #28659?
If you agree and have no time, I can try doing this

Toggle quote (10 lines)> The file itself is still available on our machines though, and you can> get it with:>>   guix download -o harfbuzz-2.4.0.tar.bz2 \>   https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l>>   guix download file://$PWD/harfbuzz-2.4.0.tar.bz2>> After that, re-running ‘guix time-machine’ should work.

Does make it sense to make this workaround more general and add this asa Cookbook or blog entry? If the patch you propose in #28659 solve anyissue and is ready to be merget of course it's not worth the effort
[...]
Thanks! Gio'
-- Giovanni Biscuolo
Xelera IT Infrastructures

zimoun wrote on 14 Feb 2020 11:47

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:CAJ3okZ0TsUwD29L1=kmBaDdfJbB+GzUqUB8ohbETyXskm+Lz4A@mail.gmail.com

Hi Ludo,
On Thu, 13 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (19 lines)>> Hi,>> Jan Nieuwenhuizen <janneke@gnu.org> skribis:>> > building /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv...> > downloading from https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-2.4.0.tar.bz2...> > |offloading build of /gnu/store/6fgg1irkcvqyb4f9f8n0nzi5gknyqhfn-gcc-mesboot1-4.7.4.drv to 'kluit.dezyne.org'> > - 'build' phasesha256 hash mismatch for /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2:> >   expected hash: 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch> >   actual hash:   0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l> > hash mismatch for store item '/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2'>> The file itself is still available on our machines though, and you can> get it with:>>   guix download -o harfbuzz-2.4.0.tar.bz2 \>   https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l

Maybe I miss a point, but the file we need is the old one, not the newone, i.e., the one with the expected hash1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch. And I should dowrong but ci.guix.gnu.org does not have this file -- otherwise it willfind it because of substitutes mechanism.

Toggle snippet (16 lines)

$ guix download -o /tmp/harfbuzz-old.tar.bz2 \ https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
Starting download of /tmp/harfbuzz-old.tar.bz2From https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch...download failed"https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch"404 "Not Found"failed to download "/tmp/harfbuzz-old.tar.bz2" from"https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch"guix download: error: open-file: No such file or directory:"/tmp/harfbuzz-old.tar.bz2"

Cheers,simon

zimoun wrote on 14 Feb 2020 11:56

Recipients:(name . Giovanni Biscuolo)(address . g@xelera.eu)

Message-ID:CAJ3okZ1X_AHLCOJ3z3fd91Kz_S1aOVvSnge2VKkeHrk90sAiNA@mail.gmail.com

Hi Giovanni,
On Fri, 14 Feb 2020 at 11:07, Giovanni Biscuolo <g@xelera.eu> wrote:

Toggle quote (10 lines)

> Ludovic Courtès <ludo@gnu.org> writes:
> > The problem here is really that we fall back to content-addressed> > mirrors instead of using them directly:> >> > https://issues.guix.gnu.org/issue/28659>> Given the natute (AFAIU) of this issue is the very same of the bug you> mention, shouldn't this bug be merged (forcemerged?) whith #28659?

AFAIU, there is 2 issues: 1. how to do with the particular case of harfbuzz -- bug#39575 2. how to solve the general case of unreliable sources -- bug#28659
So instead of merging the bugs (case 2.), I would like to solve 1.,mark it as done and report to bug#28659. It will ease to followbecause the thread in bug#28659 is already heavy. :-)

All the best,simon

Ludovic Courtès wrote on 14 Feb 2020 13:26

Recipients:(name . zimoun)(address . zimon.toutoune@gmail.com)

Message-ID:87tv3ticcx.fsf@gnu.org

Hi,
zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (24 lines)> On Thu, 13 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:>>>> Hi,>>>> Jan Nieuwenhuizen <janneke@gnu.org> skribis:>>>> > building /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv...>> > downloading from https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-2.4.0.tar.bz2...>> > |offloading build of /gnu/store/6fgg1irkcvqyb4f9f8n0nzi5gknyqhfn-gcc-mesboot1-4.7.4.drv to 'kluit.dezyne.org'>> > - 'build' phasesha256 hash mismatch for /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2:>> >   expected hash: 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch>> >   actual hash:   0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l>> > hash mismatch for store item '/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2'>>>> The file itself is still available on our machines though, and you can>> get it with:>>>>   guix download -o harfbuzz-2.4.0.tar.bz2 \>>   https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l>> Maybe I miss a point, but the file we need is the old one, not the new> one, i.e., the one with the expected hash> 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch.

Oops, my bad.

Toggle quote (6 lines)

> And I should do wrong but ci.guix.gnu.org does not have this file --> otherwise it will find it because of substitutes mechanism.>> $ guix download -o /tmp/harfbuzz-old.tar.bz2 \> https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch

I checked on a bunch of machines and couldn’t find it.
Everyone, please check whether you have/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2 andso share!
Ludo’.

Tobias Geerinckx-Rice wrote on 14 Feb 2020 13:45

Recipients:(name . zimoun)(address . zimon.toutoune@gmail.com)

Message-ID:878sl55odt.fsf@nckx

zimoun 写道：

Toggle quote (6 lines)

> Maybe I miss a point, but the file we need is the old one, not > the new> one, i.e., the one with the expected hash> 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch. And I > should do

~ λ guix download https://www.tobias.gr/guix/harfbuzz-2.4.0.tar.bz2Starting download of /tmp/guix-file.JSWxOlFrom https://www.tobias.gr/guix/harfbuzz-2.4.0.tar.bz2... ...4.0.tar.bz2  17.1MiB  6.5MiB/s 00:03 [##################] 100.0%/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz21mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
Enjoy,
T G-R

Ludovic Courtès wrote on 14 Feb 2020 14:14

Recipients:(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)

Message-ID:877e0pia4l.fsf@gnu.org

Hi,
Tobias Geerinckx-Rice <me@tobias.gr> skribis:

Toggle quote (9 lines)

> zimoun 写道：>> Maybe I miss a point, but the file we need is the old one, not the>> new>> one, i.e., the one with the expected hash>> 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch. And I should>> do>> ~ λ guix download https://www.tobias.gr/guix/harfbuzz-2.4.0.tar.bz2

Thanks, you saved us!
Now we have it here:
https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
I’ve also registered a GC root.
Anyway, everything will be so much better when SWH archives tarballs!
Ludo’.

Jan Nieuwenhuizen wrote on 14 Feb 2020 14:24

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87r1yxe1z7.fsf@gnu.org

Ludovic Courtès writes:

Toggle quote (42 lines)> Hi,>> zimoun <zimon.toutoune@gmail.com> skribis:>>> On Thu, 13 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:>>>>>> Hi,>>>>>> Jan Nieuwenhuizen <janneke@gnu.org> skribis:>>>>>> > building /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv...>>> > downloading from https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-2.4.0.tar.bz2...>>> > |offloading build of /gnu/store/6fgg1irkcvqyb4f9f8n0nzi5gknyqhfn-gcc-mesboot1-4.7.4.drv to 'kluit.dezyne.org'>>> > - 'build' phasesha256 hash mismatch for /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2:>>> >   expected hash: 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch>>> >   actual hash:   0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l>>> > hash mismatch for store item '/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2'>>>>>> The file itself is still available on our machines though, and you can>>> get it with:>>>>>>   guix download -o harfbuzz-2.4.0.tar.bz2 \>>>   https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l>>>> Maybe I miss a point, but the file we need is the old one, not the new>> one, i.e., the one with the expected hash>> 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch.>> Oops, my bad.>>> And I should do wrong but ci.guix.gnu.org does not have this file -->> otherwise it will find it because of substitutes mechanism.>>>>  $ guix download -o /tmp/harfbuzz-old.tar.bz2 \>>  https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch>> I checked on a bunch of machines and couldn’t find it.>> Everyone, please check whether you have> /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2 and> so share!

What about
https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2
(The strange thing being here, that snapshot.debian.org does not providea copy of the the in-place rewritten upstream tarball, either on2019-05-06 or later.)
So, this now becomes the recipe
wget -O harfbuzz-2.4.0.tar.bz2 https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2 guix download $PWD/harfbuzz-2.4.0.tar.bz2 guix time-machine --commit=56e95d54d209c2428f970d65d9b27ae4168449ad --no-offload -- help
that i'm trying now, and for now it looks fine (lots of stuff to build,i'll report success or failure when it's done).
It seems, however, that for offload builds to work the guix downloadneeds to be repeated on the offload build farm machines too?
janneke
-- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.orgFreelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com

Jan Nieuwenhuizen wrote on 14 Feb 2020 14:45

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87pnehe0zk.fsf@gnu.org

Ludovic Courtès writes:

Toggle quote (15 lines)> Jan Nieuwenhuizen <janneke@gnu.org> skribis:>>> building /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv...>> downloading from https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-2.4.0.tar.bz2...>> |offloading build of /gnu/store/6fgg1irkcvqyb4f9f8n0nzi5gknyqhfn-gcc-mesboot1-4.7.4.drv to 'kluit.dezyne.org'>> - 'build' phasesha256 hash mismatch for /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2:>>   expected hash: 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch>>   actual hash:   0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l>> hash mismatch for store item '/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2'>> The problem here is really that we fall back to content-addressed> mirrors instead of using them directly:>>   https://issues.guix.gnu.org/issue/28659

Wait, what happened here; you finally proposed a patch two years ago andnothing happened/we all forgot to follow up?
I cannot determine if possibly we were hoping to "wait" for the guilebuild daemon?
janneke
-- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.orgFreelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com

Ludovic Courtès wrote on 14 Feb 2020 14:51

Recipients:(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)

Message-ID:87tv3tgtv2.fsf@gnu.org

Hi,
Jan Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (4 lines)

> What about>> https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2

Good idea.

Toggle quote (9 lines)

> So, this now becomes the recipe>> wget -O harfbuzz-2.4.0.tar.bz2 https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2> guix download $PWD/harfbuzz-2.4.0.tar.bz2> guix time-machine --commit=56e95d54d209c2428f970d65d9b27ae4168449ad --no-offload -- help>> that i'm trying now, and for now it looks fine (lots of stuff to build,> i'll report success or failure when it's done).

OK!

Toggle quote (3 lines)

> It seems, however, that for offload builds to work the guix download> needs to be repeated on the offload build farm machines too?

No, I don’t think so, because the head node copies all the inputs tobuild machines before it actually offloads the build.
Ludo’.

Ludovic Courtès wrote on 14 Feb 2020 22:34

Recipients:(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)

Message-ID:878sl47t0q.fsf@gnu.org

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (2 lines)

> Ludovic Courtès writes:

[...]

Toggle quote (8 lines)

>> The problem here is really that we fall back to content-addressed>> mirrors instead of using them directly:>>>> https://issues.guix.gnu.org/issue/28659>> Wait, what happened here; you finally proposed a patch two years ago and> nothing happened/we all forgot to follow up?

I think we forgot, indeed.
One thing I don’t quite like about the patch is the fact that ‘guixsubstitutes’ connects to the daemon in ‘content-addressed-item?’.
Also, one could argue that we’d steer users towards downloading from ourserver, which could be a privacy concern (probably not a strong argumentsince one can easily change the substitute URLs.)
Thoughts?
Ludo’.

zimoun wrote on 15 Feb 2020 16:32

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:CAJ3okZ00U062A0aa8ppxTQA+7z=bo9p03NyrUL6Wk2o4gjq6Cw@mail.gmail.com

Hi,
On Fri, 14 Feb 2020 at 14:51, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (8 lines)

> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
> > What about> >> > https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2>> Good idea.

Cool!But how do you determine the "date", i.e., this reference '20190406T212022Z' ?
Could it be automated?

Cheers,simon

zimoun wrote on 15 Feb 2020 16:43

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:CAJ3okZ0-zPcs+pC4tQEymD-On-aN_-hgKRkRzBJzusdbtdYdAg@mail.gmail.com

Hi,
On Fri, 14 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (4 lines)

> Also, one could argue that we’d steer users towards downloading from our> server, which could be a privacy concern (probably not a strong argument> since one can easily change the substitute URLs.)

I am not following the privacy concern.What do you mean?
Cheers,simon

zimoun wrote on 15 Feb 2020 16:51

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:CAJ3okZ1Nj+fu8VnF=pgghXfj1vO4Mu+uyXtLLfF05qHh-kDQhQ@mail.gmail.com

Hi,
On Fri, 14 Feb 2020 at 14:14, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (6 lines)

> Tobias Geerinckx-Rice <me@tobias.gr> skribis:
> > ~ λ guix download https://www.tobias.gr/guix/harfbuzz-2.4.0.tar.bz2>> Thanks, you saved us!

Thank you! :-)

Toggle quote (2 lines)

> Anyway, everything will be so much better when SWH archives tarballs!

The future will be better. :-)Even some details need to be discussed: frequency of source.jsongeneration, frequency of the SWH crawler ingest it, etc.
How could all the archives living in ci.guix.gnu.org be sent to SWH?Because IMHO the of Berlin (or other) is to archive the world but tobuild it. :-)

Cheers,simon

Tobias Geerinckx-Rice wrote on 15 Feb 2020 21:01

Recipients:(address . 39575@debbugs.gnu.org)

Message-ID:875zg78vsk.fsf@nckx

Jan, Simon,
Janneke 写道：

Toggle quote (2 lines)

> https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2

This is a wonderful resource! Thank you, Janneke (and Debian)!
zimoun 写道：

Toggle quote (4 lines)

> Cool!> But how do you determine the "date", i.e., this reference > '20190406T212022Z' ?

You'd take the timestamp immediately preceding your desired (Guix) commit's date, or something like that. The fact that git commit dates aren't linear shouldn't hurt here.

Toggle quote (2 lines)

> Could it be automated?

Not without parsing HTML to get the valid timestamps: https://snapshot.debian.org/archive/debian/?year=2020&month=2.
Also, this doesn't seem to be a supported service yet[0]:
“This is an implementation for a possible snapshot.debian.org service. It's not yet finished, it's more a prototype/proof of concept to show and learn what we want and can provide. So far it seems to actually work.”
Still really cool,
T G-R
[0]: https://salsa.debian.org/snapshot-team/snapshot

Bengt Richter wrote on 16 Feb 2020 00:57

Recipients:(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)

Message-ID:20200215235716.GA10983@LionPure

On +2020-02-15 21:01:36 +0100, Tobias Geerinckx-Rice via Bug reports for GNU Guix wrote:

Toggle quote (22 lines)> Jan, Simon,> > Janneke 写道：> > https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2> > This is a wonderful resource!  Thank you, Janneke (and Debian)!> > zimoun 写道：> > Cool!> > But how do you determine the "date", i.e., this reference> > '20190406T212022Z' ?> > You'd take the timestamp immediately preceding your desired (Guix) commit's> date, or something like that.  The fact that git commit dates aren't linear> shouldn't hurt here.> > > Could it be automated?> > Not without parsing HTML to get the valid timestamps:> <https://snapshot.debian.org/archive/debian/?year=2020&month=2>.>

You may not need to parse the html fully if the part you need isisolatable into delimited scopes that you can successively narrow.
For example, I while back I wanted a command I could type to getthe url of the latest linux kernel at kernel.org:
stable-kernel.scm -h

Toggle snippet (7 lines)Usage: stable-kernel-scm [ -h ]       -h for this message          (without args):       go to https://www.kernel.org/ to wget page,       extract URL of latest stable release tarball       and write that URL to stdout.

(oops, I see I din't use $0 in the usage text -- should be .scm, not -scm)
I offer it below [1], with the thought that you could probablymodify (not to mention improve :-) it to get the timestamps you want.Especially if you could get them to make the narrow context unique enoughthat it's delimiters can delimit it in one shot.
The page at kernel.org is apparently stable enough that this still works,but YMMV until the snapshot page is similarly stable. (You could askthem to make it easy :)

Toggle quote (13 lines)> Also, this doesn't seem to be a supported service yet[0]:> >  “This is an implementation for a possible snapshot.debian.org  service.>   It's not yet finished, it's more a prototype/proof of concept   to show>   and learn what we want and can provide.  So far it seems to   actually> work.”> > Still really cool,> > T G-R> > [0]: https://salsa.debian.org/snapshot-team/snapshot

HTH or is useful some way.-- Regards,Bengt Richter
[1]

Toggle snippet (78 lines)#!/usr/bin/bashexec guile -e main -s "$0" "$@"!#;;;; stable-kernel.scm;;;; goes to https://www.kernel.org/ to wget page, then;;;; extracts name of latest stable release tarball to stdout
;;;;(define (usage)  (format (current-error-port)	  (string-join	   '(	    "Usage: stable-kernel-scm [ -h ]"	    "       -h for this message"	    "          (without args):"            "       go to https://www.kernel.org/ to wget page,"	    "       extract URL of latest stable release tarball"	    "       and write that URL to stdout."	    "")	   "\n")))
(use-modules (ice-9 format))(use-modules (ice-9 rdelim))(use-modules (ice-9 popen))(use-modules (ice-9 textual-ports))(use-modules (ice-9 and-let-star))(use-modules (ice-9 regex))
(define (extract-delimited str s-beg s-end)  (and-let*   ((ix-beg (string-contains str s-beg))    (ix-post-beg (+ ix-beg (string-length s-beg)))    (ix-end   (string-contains str s-end ix-post-beg)))   (substring str ix-post-beg  ix-end)))
(define kernel-url "https://www.kernel.org/")
(define (get-kern-name)  (let*((cmd-kern (string-append "wget -q -O - " kernel-url))	(p-inp (open-input-pipe cmd-kern))	(wgot-pinp-str (get-string-all p-inp))	(extracted-table-releases	 (extract-delimited wgot-pinp-str			    "<table id=\"releases\">"			    "</table>"))	(extracted-stable-tarball-anchor	 (extract-delimited extracted-table-releases			    "<td>stable:</td>"			    ">tarball<"))	(extracted-stable-href	 (extract-delimited extracted-stable-tarball-anchor			    "<a href=\""			    "\"")))   (begin    extracted-stable-href)))
(define (main args)  (begin    (set! args (cdr args)) ;; always dump callee arg    (if (not (pair? args))	(set! args '("-do-default") ))
    ;; simple opts...    (cond     ((string-prefix? "-h" (car args))      (begin (usage)	     (exit)))          ((string-prefix? "-do-default" (car args))      (let ((kern-name (get-kern-name)))	(display kern-name)(newline)))
     (#t      (begin	(format (current-error-port) "Error: bad opt: '~a'\n" (car args))	(usage)	(exit))))))

Ludovic Courtès wrote on 16 Feb 2020 11:59

Recipients:(name . zimoun)(address . zimon.toutoune@gmail.com)

Message-ID:87k14m3iiy.fsf@gnu.org

Hi!
zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (9 lines)

> On Fri, 14 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:>>> Also, one could argue that we’d steer users towards downloading from our>> server, which could be a privacy concern (probably not a strong argument>> since one can easily change the substitute URLs.)>> I am not following the privacy concern.> What do you mean?

I mean that by default, someone who’s disabled substitutes (presumablyout of security or privacy concerns) would find themself downloadingsource code from ci.guix.gnu.org instead of various upstream sites.
Ludo’.

zimoun wrote on 17 Feb 2020 09:47

Recipients:(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)

Message-ID:CAJ3okZ3NPiLPYt6ESVxxCeybPd4j-4SfhP47KZ4OwTbkA=+PNg@mail.gmail.com

Hi,
On Sat, 15 Feb 2020 at 21:01, Tobias Geerinckx-Rice <me@tobias.gr> wrote:

Toggle quote (14 lines)> Janneke 写道：> > https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2>> This is a wonderful resource!  Thank you, Janneke (and Debian)!>> zimoun 写道：> > Cool!> > But how do you determine the "date", i.e., this reference> > '20190406T212022Z' ?>> You'd take the timestamp immediately preceding your desired (Guix)> commit's date, or something like that.  The fact that git commit> dates aren't linear shouldn't hurt here.

You assume that Debian packs packages as fast as Guix, I mean on thesame schedule which is a strong assumption IMHO.For example, if it was the contrary and the "new" release of harfbuzz2.4.0 were missing, then would Debian be helpful?

Toggle quote (11 lines)> Also, this doesn't seem to be a supported service yet[0]:>>   “This is an implementation for a possible snapshot.debian.org>   service.>    It's not yet finished, it's more a prototype/proof of concept>    to show>    and learn what we want and can provide.  So far it seems to>    actually work.”>> Still really cool,

Yes, still cool! :-)

Thanks,simon

zimoun wrote on 17 Feb 2020 11:18

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:CAJ3okZ08ibXTBqsZMwnuEVdhpyXgHVp6+rNGXB02gsHVqwu53A@mail.gmail.com

Hi Ludo,
On Sun, 16 Feb 2020 at 11:59, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (14 lines)

> zimoun <zimon.toutoune@gmail.com> skribis:> > On Fri, 14 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:
> >> Also, one could argue that we’d steer users towards downloading from our> >> server, which could be a privacy concern (probably not a strong argument> >> since one can easily change the substitute URLs.)> >> > I am not following the privacy concern.> > What do you mean?>> I mean that by default, someone who’s disabled substitutes (presumably> out of security or privacy concerns) would find themself downloading> source code from ci.guix.gnu.org instead of various upstream sites.

I do not see the difference between mirroring and traveling back intime with missing upstream sources.And because it is content-addressed, it seems even more secure thandownloading from a upstream URL, IMHO.If one trusts Guix, then an attacker needs to corrupt in the same timethe Guix history and Berlin (and/or any other farm).If one does not trust Guix, why does they use the recipe coming fromGuix? To be precise, this person has to check all the recipes of allthe dependencies.
Well, I do not see a security concern because we are talking aboutserving the sources.It is another story when the substitutes serve the results of thebuild (binaries); because one does not have any strong guarantee thatthe substitute serves the expected binaries.
By privacy concern, do you mean that Guix could collect who downloadswhat; in a central fashion? Which is not the case when one downloadsfrom several distributed upstream sources. Right?Well, I am not convinced because the case of missing upstream sourceis rare. And it is easy to protect against such collecting dataprocess.In paranoid mode, traveling back in time is becoming difficult becauseof the reliability of the sources; I mean if the sources werereliable, SWH would not exist. ;-) The solution should be an IPFS /GNUnet / full distributed archive... which is not ready... yet! :-)

Well, maybe for the TODO list of the time-machine: add an option toallow substitutes *only* for the sources (substitutes meaningci.guix.gnu.org and/or SWH). If this option does not exist yet. ;-)

Cheers,simon

Efraim Flashner wrote on 17 Feb 2020 14:26

Recipients:(name . zimoun)(address . zimon.toutoune@gmail.com)

Message-ID:20200217132604.GJ1968@E5400

On Mon, Feb 17, 2020 at 09:47:41AM +0100, zimoun wrote:

Toggle quote (25 lines)> Hi,> > On Sat, 15 Feb 2020 at 21:01, Tobias Geerinckx-Rice <me@tobias.gr> wrote:> > > Janneke 写道：> > > https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2> >> > This is a wonderful resource!  Thank you, Janneke (and Debian)!> >> > zimoun 写道：> > > Cool!> > > But how do you determine the "date", i.e., this reference> > > '20190406T212022Z' ?> >> > You'd take the timestamp immediately preceding your desired (Guix)> > commit's date, or something like that.  The fact that git commit> > dates aren't linear shouldn't hurt here.> > You assume that Debian packs packages as fast as Guix, I mean on the> same schedule which is a strong assumption IMHO.> For example, if it was the contrary and the "new" release of harfbuzz> 2.4.0 were missing, then would Debian be helpful?> > 

We could first trymirror://debian/pool/main/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2
and then scrape https://snapshot.debian.org/package/harfbuzz/for2.4.0-1 and then parse the website for harfbuzz_2.4.0.orig.tar.bz2. Orfor just 'orig.tar'

Toggle quote (20 lines)> > Also, this doesn't seem to be a supported service yet[0]:> >> >   “This is an implementation for a possible snapshot.debian.org> >   service.> >    It's not yet finished, it's more a prototype/proof of concept> >    to show> >    and learn what we want and can provide.  So far it seems to> >    actually work.”> >> > Still really cool,> > Yes, still cool! :-)> > > Thanks,> simon> > > 

-- Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנרGPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351Confidentiality cannot be guaranteed on emails sent or received unencrypted

Ludovic Courtès wrote on 17 Feb 2020 15:40

Recipients:(name . zimoun)(address . zimon.toutoune@gmail.com)

Message-ID:87pned6zw2.fsf@gnu.org

Hi,
zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (15 lines)> On Sun, 16 Feb 2020 at 11:59, Ludovic Courtès <ludo@gnu.org> wrote:>> zimoun <zimon.toutoune@gmail.com> skribis:>> > On Fri, 14 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:>>> >> Also, one could argue that we’d steer users towards downloading from our>> >> server, which could be a privacy concern (probably not a strong argument>> >> since one can easily change the substitute URLs.)>> >>> > I am not following the privacy concern.>> > What do you mean?>>>> I mean that by default, someone who’s disabled substitutes (presumably>> out of security or privacy concerns) would find themself downloading>> source code from ci.guix.gnu.org instead of various upstream sites.

[...]

Toggle quote (4 lines)

> By privacy concern, do you mean that Guix could collect who downloads> what; in a central fashion? Which is not the case when one downloads> from several distributed upstream sources. Right?

Exactly. But like I wrote above, I don’t think it’s a strong argument.
What remains is the issue with ‘content-addressed-item?’, then.
Ludo’.

Tobias Geerinckx-Rice wrote on 17 Feb 2020 16:02

Recipients:(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 39575@debbugs.gnu.org)

Message-ID:877e0l6yu4.fsf@nckx

zimoun 写道：

Toggle quote (2 lines)

> You assume that Debian packs packages as fast as Guix

Indeed I do! :-D
Efraim's solution sounds reasonable.
Kind regards,
T G-R

zimoun wrote on 17 Feb 2020 16:04

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:CAJ3okZ0BrbfqmrsmfJykwqPA8PQFtSUB3JBdNaN1=npZNa36Eg@mail.gmail.com

On Mon, 17 Feb 2020 at 15:40, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (2 lines)

> Exactly. But like I wrote above, I don’t think it’s a strong argument.

I agree and the big picture depends on the audience.Scientific communities would be fine with centralized archives such asSWH. And only centralized archives IMHO can provide a reliable "longterm" support which is the point for that communities. (Quote becausenot clearly defined what it is. :-))Other communities would prefer distributed archive such as IPFS orGNUnet but 1. it still needs some work and 2. the "long term" is notguarantee by nature, IMHO. But it is probably not an issue for thatcommunities.

Toggle quote (2 lines)

> What remains is the issue with ‘content-addressed-item?’, then.

I agree.The bridge with SWH is in good shape, IMHO.And the pending IPFS patch would deserve more love. :-) Maybe soon...

Cheers,simon

zimoun wrote on 17 Feb 2020 19:32

Recipients:(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)

Message-ID:CAJ3okZ1JUeXW1ExMrrE96yxr1kiTz+Zj1bjdkfZX=4896nThJg@mail.gmail.com

HI Jan,
On Fri, 14 Feb 2020 at 14:24, Jan Nieuwenhuizen <janneke@gnu.org> wrote:
This command

Toggle quote (3 lines)

> >> $ guix download -o /tmp/harfbuzz-old.tar.bz2 \> >> https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch

now works.

However, this command
$ guix time-machine \ --commit=56e95d54d209c2428f970d65d9b27ae4168449ad -- help
still fails for me with the message:

Toggle snippet (18 lines)[...]building /gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv...- 'check' phasebuilder for`/gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv' failedwith exit code 1build of /gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv failedView build log at'/var/log/guix/drvs/gg/lbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv.bz2'.cannot build derivation`/gnu/store/yqpxm07zm0mirrdvl2c4qvf8biyzg468-guix-56e95d54d.drv': 1dependencies couldn't be builtcannot build derivation`/gnu/store/7z7p0m7abi246gzigw8as2q3w33k1n31-profile.drv': 1dependencies couldn't be builtguix time-machine: error: build of`/gnu/store/7z7p0m7abi246gzigw8as2q3w33k1n31-profile.drv' failed

The log /var/log/guix/drvs/gg/lbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv.bz2is not meaningful for me... but I can report it here.

Toggle quote (3 lines)

> that i'm trying now, and for now it looks fine (lots of stuff to build,> i'll report success or failure when it's done).

Well, is it a success or a failure for you?

Cheers,simon

Jan Nieuwenhuizen wrote on 19 Feb 2020 12:58

Recipients:(name . zimoun)(address . zimon.toutoune@gmail.com)

Message-ID:871rqq23hy.fsf@gnu.org

zimoun writes:
Hi Simon,

Toggle quote (43 lines)> On Fri, 14 Feb 2020 at 14:24, Jan Nieuwenhuizen <janneke@gnu.org> wrote:>> This command>>> >>  $ guix download -o /tmp/harfbuzz-old.tar.bz2 \>> >>  https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch>> now works.>>> However, this command>> $ guix time-machine \>      --commit=56e95d54d209c2428f970d65d9b27ae4168449ad -- help>> still fails for me with the message:>> [...]> building /gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv...> - 'check' phasebuilder for> `/gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv' failed> with exit code 1> build of /gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv failed> View build log at> '/var/log/guix/drvs/gg/lbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv.bz2'.> cannot build derivation> `/gnu/store/yqpxm07zm0mirrdvl2c4qvf8biyzg468-guix-56e95d54d.drv': 1> dependencies couldn't be built> cannot build derivation> `/gnu/store/7z7p0m7abi246gzigw8as2q3w33k1n31-profile.drv': 1> dependencies couldn't be built> guix time-machine: error: build of> `/gnu/store/7z7p0m7abi246gzigw8as2q3w33k1n31-profile.drv' failed>> The log /var/log/guix/drvs/gg/lbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv.bz2> is not meaningful for me... but I can report it here.>>>> that i'm trying now, and for now it looks fine (lots of stuff to build,>> i'll report success or failure when it's done).>> Well, is it a success or a failure for you?

For me, pythohn-minimal fails to build
build-started /gnu/store/s0lw23myd3hvpw28sffkhz8b30x1hcz0-python-minimal-3.7.3.drv - x86_64-linux /var/log/guix/drvs/s0//lw23myd3hvpw28sffkhz8b30x1hcz0-python-minimal-3.7.3.drv.bz2 20827
====================================================================== FAIL: test_register_chain (test.test_faulthandler.FaultHandlerTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/test/test_faulthandler.py", line 724, in test_register_chain self.check_register(chain=True) File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/test/test_faulthandler.py", line 702, in check_register self.assertEqual(exitcode, 0) AssertionError: -11 != 0
----------------------------------------------------------------------
Ran 42 tests in 18.782s
FAILED (failures=1, skipped=4) 1 test failed again: test_faulthandler
== Tests result: FAILURE then FAILURE ==
382 tests OK.
1 test failed: test_faulthandler
Not sure what to do here. Could this be a (harmless) coincident?
Greetingsjanneke
-- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.orgFreelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com

zimoun wrote on 21 Feb 2020 16:58

Recipients:(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)

Message-ID:CAJ3okZ3fHdsgBvw96np1BcdmMrdGe_h9dhqLom0G2Wx8DPjJvQ@mail.gmail.com

Hi,
Some follow ups.

For me, now, the command

Toggle quote (3 lines)

> > $ guix time-machine \> > --commit=56e95d54d209c2428f970d65d9b27ae4168449ad -- help

does not fail anymore at the Guile step:

Toggle snippet (5 lines)

building /gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv... - 'check' phasebuilder for `/gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv' failed

because it was about "FAIL: test-out-of-memory".

Well, now, I am failing at the Python 3.7.3 step too:

Toggle snippet (4 lines)

/gnu/store/s0lw23myd3hvpw28sffkhz8b30x1hcz0-python-minimal-3.7.3.drv failed

However, the python error seems about TLS:

Toggle snippet (25 lines)test.test_asyncio.test_windows_utils (unittest.loader.ModuleSkipped)... test test_asyncio failedskipped 'Windows only'
======================================================================ERROR: test_start_tls_server_1(test.test_asyncio.test_sslproto.SelectorStartTLSTests)----------------------------------------------------------------------Traceback (most recent call last):  File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/test/test_asyncio/test_sslproto.py",line 507, in test_start_tls_server_1    self.loop.run_until_complete(run_main())  File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/asyncio/base_events.py",line 584, in run_until_complete    return future.result()  File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/test/test_asyncio/test_sslproto.py",line 502, in run_main    loop=self.loop, timeout=self.TIMEOUT)  File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/asyncio/tasks.py",line 423, in wait_for    raise futures.TimeoutError()concurrent.futures._base.TimeoutError

Toggle quote (2 lines)

> Not sure what to do here. Could this be a (harmless) coincident?

Me neither.

Cheers,simon

Ludovic Courtès wrote on 21 Feb 2020 21:48

Recipients:(name . zimoun)(address . zimon.toutoune@gmail.com)

Message-ID:875zfz3bwa.fsf@gnu.org

Hi,
zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (31 lines)> Well, now, I am failing at the Python 3.7.3 step too:>>   /gnu/store/s0lw23myd3hvpw28sffkhz8b30x1hcz0-python-minimal-3.7.3.drv failed>>>> However, the python error seems about TLS:>> test.test_asyncio.test_windows_utils (unittest.loader.ModuleSkipped)> ... test test_asyncio failed> skipped 'Windows only'>> ======================================================================> ERROR: test_start_tls_server_1> (test.test_asyncio.test_sslproto.SelectorStartTLSTests)> ----------------------------------------------------------------------> Traceback (most recent call last):>   File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/test/test_asyncio/test_sslproto.py",> line 507, in test_start_tls_server_1>     self.loop.run_until_complete(run_main())>   File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/asyncio/base_events.py",> line 584, in run_until_complete>     return future.result()>   File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/test/test_asyncio/test_sslproto.py",> line 502, in run_main>     loop=self.loop, timeout=self.TIMEOUT)>   File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/asyncio/tasks.py",> line 423, in wait_for>     raise futures.TimeoutError()> concurrent.futures._base.TimeoutError

It seems to be timing-sensitive, is it deterministic?
One lesson here is that we should keep substitutes for a longer amountof time—we actually have a lot of storage space on berlin so we shouldinvestigate what happened. (The NixOS folks currently keep substitutesforever but they found it’s starting to be expensive…)
Thanks,Ludo’.

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send email to 39575@debbugs.gnu.org

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date