'guix substitute' creates files with incorrect names when not running in a UTF-8 locale

DoneSubmitted by Brett Gilio.
Details
3 participants
  • Brett Gilio
  • Ludovic Courtès
  • maxim.cournoyer
Owner
unassigned
Severity
important
B
B
Brett Gilio wrote on 3 Dec 2018 20:58
Invalid hash for NSS-Certs
(address . bug-guix@gnu.org)
87pnui8jrq.fsf@posteo.net
Generation 10 Dec 03 2018 11:42:41 (current) guix 4f03aa2 repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: 4f03aa23e805bd653de774e1d74ed2f50826899b
downloading from https://mirror.hydra.gnu.org/guix/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39... nss-certs-3.39 145KiB 417KiB/s 00:00 [##################] 100.0%
sha256 hash mismatch for /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39: expected hash: 101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla actual hash: 08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9ssubstitution of /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 failed
L
L
Ludovic Courtès wrote on 5 Dec 2018 12:47
(name . Brett Gilio)(address . brettg@posteo.net)(address . 33603@debbugs.gnu.org)
87mupkyz2p.fsf@gnu.org
Hi Brett,
Brett Gilio <brettg@posteo.net> skribis:
Toggle quote (14 lines)> Generation 10 Dec 03 2018 11:42:41 (current)> guix 4f03aa2> repository URL: https://git.savannah.gnu.org/git/guix.git> branch: master> commit: 4f03aa23e805bd653de774e1d74ed2f50826899b>> downloading from https://mirror.hydra.gnu.org/guix/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39...> nss-certs-3.39 145KiB 417KiB/s 00:00 [##################] 100.0%>> sha256 hash mismatch for /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39:> expected hash: 101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla> actual hash: 08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s> substitution of /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 failed
The problem seems to be gone because I’m seeing the right hash here:
Toggle snippet (6 lines)$ wget -q -O - https://mirror.hydra.gnu.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix hash -101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla$ wget -q -O - https://mirror.hydra.gnu.org/xbj4fhad0lnz0ziflwi90gyqbls8ains.narinfo |grep HashNarHash: sha256:101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla
Could you try again?
However berlin.guixsd.org is publishing the one with hash08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s, and the differenceis an encoding bug:
Toggle snippet (7 lines)$ diff -ru /tmp/nss-certs.{hydra,berlin}Only in /tmp/nss-certs.hydra/etc/ssl/certs: AC_Raíz_Certicámara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pemOnly in /tmp/nss-certs.berlin/etc/ssl/certs: AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pemOnly in /tmp/nss-certs.hydra/etc/ssl/certs: NetLock_Arany_=Class_Gold=_Főtanúsítvány:2.6.73.65.44.228.0.16.pemOnly in /tmp/nss-certs.berlin/etc/ssl/certs: NetLock_Arany_=Class_Gold=_F?tan?s?tv?ny:2.6.73.65.44.228.0.16.pem
(Probably related to https://issues.guix.info/issue/32942.)
Ludo’.
M
M
maxim.cournoyer wrote on 8 Jan 2019 17:24
(name . Ludovic Courtès)(address . ludo@gnu.org)
874lajnmlp.fsf@kwak.i-did-not-set--mail-host-address--so-tickle-me
ludo@gnu.org (Ludovic Courtès) writes:
Toggle quote (25 lines)> Hi Brett,>> Brett Gilio <brettg@posteo.net> skribis:>>> Generation 10 Dec 03 2018 11:42:41 (current)>> guix 4f03aa2>> repository URL: https://git.savannah.gnu.org/git/guix.git>> branch: master>> commit: 4f03aa23e805bd653de774e1d74ed2f50826899b>>>> downloading from https://mirror.hydra.gnu.org/guix/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39...>> nss-certs-3.39 145KiB 417KiB/s 00:00 [##################] 100.0%>>>> sha256 hash mismatch for /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39:>> expected hash: 101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla>> actual hash: 08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s>> substitution of /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 failed>> The problem seems to be gone because I’m seeing the right hash here:>> $ wget -q -O - https://mirror.hydra.gnu.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix hash -> 101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla> $ wget -q -O - https://mirror.hydra.gnu.org/xbj4fhad0lnz0ziflwi90gyqbls8ains.narinfo |grep Hash> NarHash: sha256:101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla
I got the failure while trying to reconfigure:
Toggle snippet (10 lines)downloading from https://mirror.hydra.gnu.org/guix/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39... nss-certs-3.39 145KiB 608KiB/s 00:00 [##################] 100.0%
sha256 hash mismatch for /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39: expected hash: 101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla actual hash: 08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9ssubstitution of /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 failed
Maxim
L
L
Ludovic Courtès wrote on 11 Jan 2019 09:19
(address . maxim.cournoyer@gmail.com)
87d0p3a9oi.fsf@gnu.org
maxim.cournoyer@gmail.com skribis:
Toggle quote (38 lines)> ludo@gnu.org (Ludovic Courtès) writes:>>> Hi Brett,>>>> Brett Gilio <brettg@posteo.net> skribis:>>>>> Generation 10 Dec 03 2018 11:42:41 (current)>>> guix 4f03aa2>>> repository URL: https://git.savannah.gnu.org/git/guix.git>>> branch: master>>> commit: 4f03aa23e805bd653de774e1d74ed2f50826899b>>>>>> downloading from https://mirror.hydra.gnu.org/guix/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39...>>> nss-certs-3.39 145KiB 417KiB/s 00:00 [##################] 100.0%>>>>>> sha256 hash mismatch for /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39:>>> expected hash: 101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla>>> actual hash: 08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s>>> substitution of /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 failed>>>> The problem seems to be gone because I’m seeing the right hash here:>>>> $ wget -q -O - https://mirror.hydra.gnu.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix hash ->> 101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla>> $ wget -q -O - https://mirror.hydra.gnu.org/xbj4fhad0lnz0ziflwi90gyqbls8ains.narinfo |grep Hash>> NarHash: sha256:101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla>> I got the failure while trying to reconfigure:>> downloading from https://mirror.hydra.gnu.org/guix/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39...> nss-certs-3.39 145KiB 608KiB/s 00:00 [##################] 100.0%>> sha256 hash mismatch for /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39:> expected hash: 101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla> actual hash: 08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s> substitution of> /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 failed
The wget commands above still give me the correct result, with hash101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla.
Are you running Guix on a foreign distro? If so, could it be thatguix-daemon is effectively running in the C locale?
Thanks,Ludo’.
M
M
Maxim Cournoyer wrote on 14 Jan 2019 03:33
(name . Ludovic Courtès)(address . ludo@gnu.org)
87muo4vuhg.fsf@gmail.com
Hello!
Ludovic Courtès <ludo@gnu.org> writes:[...]
Toggle quote (6 lines)> The wget commands above still give me the correct result, with hash> 101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla.>> Are you running Guix on a foreign distro? If so, could it be that> guix-daemon is effectively running in the C locale?
This is a good guess, and we've seen this very issue before. I am usingGuixSD. I had to use --fallback to work around it.
I've digged a little bit:
Toggle snippet (20 lines)$ wget -q -O - https://mirror.hydra.gnu.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 | gunzip | guix archive -x /tmp/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39_from-hydra
$ guix hash -r /tmp/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39_from-hydra101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla
$ guix build nss-certs/gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39
$ guix hash -r /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.3908ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s
$ diff -r /tmp/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39_from-hydra /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39Only in /tmp/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39_from-hydra/etc/ssl/certs: AC_Raíz_Certicámara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pemOnly in /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39/etc/ssl/certs: AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pemOnly in /tmp/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39_from-hydra/etc/ssl/certs: NetLock_Arany_=Class_Gold=_Főtanúsítvány:2.6.73.65.44.228.0.16.pemOnly in/gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39/etc/ssl/certs:NetLock_Arany_=Class_Gold=_F?tan?s?tv?ny:2.6.73.65.44.228.0.16.pem
It's a rather old install (late 2016 -- but kept up-to-date, of course:-)) so there might be remnants from the past? How could I verify inwhich locale the guix-daemon is running?
Thanks!
Maxim
L
L
Ludovic Courtès wrote on 14 Jan 2019 09:48
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
878szny69n.fsf@gnu.org
Hi!
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
Toggle quote (36 lines)> Ludovic Courtès <ludo@gnu.org> writes:> [...]>> The wget commands above still give me the correct result, with hash>> 101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla.>>>> Are you running Guix on a foreign distro? If so, could it be that>> guix-daemon is effectively running in the C locale?>> This is a good guess, and we've seen this very issue before. I am using> GuixSD. I had to use --fallback to work around it.>> I've digged a little bit:>> $ wget -q -O - https://mirror.hydra.gnu.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 | gunzip | guix archive -x /tmp/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39_from-hydra>> $ guix hash -r /tmp/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39_from-hydra> 101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla>> $ guix build nss-certs> /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39>> $ guix hash -r /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39> 08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s>> $ diff -r /tmp/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39_from-hydra /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39> Only in /tmp/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39_from-hydra/etc/ssl/certs: AC_Raíz_Certicámara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem> Only in /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39/etc/ssl/certs: AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem> Only in /tmp/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39_from-hydra/etc/ssl/certs: NetLock_Arany_=Class_Gold=_Főtanúsítvány:2.6.73.65.44.228.0.16.pem> Only in> /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39/etc/ssl/certs:> NetLock_Arany_=Class_Gold=_F?tan?s?tv?ny:2.6.73.65.44.228.0.16.pem>> It's a rather old install (late 2016 -- but kept up-to-date, of course> :-)) so there might be remnants from the past? How could I verify in> which locale the guix-daemon is running?
You could check /proc/$(pidof guix-daemon)/environ for variables like‘LC_ALL’. And of course, you can see if ‘guix substitute’ emits theinfamous “can’t install locale” message. :-)
Regardless, I think ‘guix substitute’ should ideally belocale-insensitive, or it should error out rather than produce fileswith the wrong names.
HTH,Ludo’.
L
L
Ludovic Courtès wrote on 14 Jan 2019 09:49
control message for bug #33603
(address . control@debbugs.gnu.org)
877ef7y67k.fsf@gnu.org
retitle 33603 'guix substitute' creates files with incorrect names when not running in a UTF-8 locale
L
L
Ludovic Courtès wrote on 14 Jan 2019 09:49
(address . control@debbugs.gnu.org)
875zury67g.fsf@gnu.org
severity 33603 important
M
M
Maxim Cournoyer wrote on 15 Jan 2019 06:03
Re: bug#33603: Invalid hash for NSS-Certs
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 33603@debbugs.gnu.org)
87imyqqzpk.fsf@gmail.com
Hello!
Ludovic Courtès <ludo@gnu.org> writes:
Toggle quote (2 lines)> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
[...]
Toggle quote (12 lines)>> It's a rather old install (late 2016 -- but kept up-to-date, of course>> :-)) so there might be remnants from the past? How could I verify in>> which locale the guix-daemon is running?>> You could check /proc/$(pidof guix-daemon)/environ for variables like> ‘LC_ALL’. And of course, you can see if ‘guix substitute’ emits the> infamous “can’t install locale” message. :-)>> Regardless, I think ‘guix substitute’ should ideally be> locale-insensitive, or it should error out rather than produce files> with the wrong names.
The only environment variable(s?) defined for the guix-daemon process onthat machine is:
Toggle snippet (7 lines)$ pidof guix-daemon270
sudo cat /proc/270/environGUIX_LOCPATH=/gnu/store/94k5w17z54w25lgp90czdqfv9m4hwzhq-glibc-utf8-locales-2.28/lib/localeLC_ALL=en_US.utf8
I'm not familiar with this systemfs structure, but shouldn't there be anewline before the LC_ALL=en_US.utf8 variable assignment?
It's the same on a 2nd GuixSD machine.
Toggle snippet (5 lines)$ sudo guix substitute --help# Usage: guix substitute [OPTION]......
No infamous locale error here.
Not sure what happened here :-/
Maxim
L
L
Ludovic Courtès wrote on 15 Jan 2019 13:57
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 33603@debbugs.gnu.org)
87d0oy13jg.fsf@gnu.org
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
Toggle quote (29 lines)> Hello!>> Ludovic Courtès <ludo@gnu.org> writes:>>> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:>> [...]>>>> It's a rather old install (late 2016 -- but kept up-to-date, of course>>> :-)) so there might be remnants from the past? How could I verify in>>> which locale the guix-daemon is running?>>>> You could check /proc/$(pidof guix-daemon)/environ for variables like>> ‘LC_ALL’. And of course, you can see if ‘guix substitute’ emits the>> infamous “can’t install locale” message. :-)>>>> Regardless, I think ‘guix substitute’ should ideally be>> locale-insensitive, or it should error out rather than produce files>> with the wrong names.>> The only environment variable(s?) defined for the guix-daemon process on> that machine is:>> $ pidof guix-daemon> 270>> sudo cat /proc/270/environ> GUIX_LOCPATH=/gnu/store/94k5w17z54w25lgp90czdqfv9m4hwzhq-glibc-utf8-locales-2.28/lib/localeLC_ALL=en_US.utf8
This is perfect (see commit 7e4bc215098f334bc2a11737f2665dd4992fc2da,which gave you this, fixing the issue we’re talking about on GuixSD.)
So I don’t think this machine has any problem. Perhaps nss-certs wasinstalled before the fix above?
Toggle quote (3 lines)> I'm not familiar with this systemfs structure, but shouldn't there be a> newline before the LC_ALL=en_US.utf8 variable assignment?
No, there are actually newlines, try:
cat /proc/270/environ | xargs -0 echo
HTH,Ludo’.
M
M
Maxim Cournoyer wrote on 18 Jan 2019 05:59
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 33603@debbugs.gnu.org)
87muny36jp.fsf@gmail.com
Hi Ludovic!
Ludovic Courtès <ludo@gnu.org> writes:
Toggle quote (2 lines)> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
[...]
Toggle quote (15 lines)>> The only environment variable(s?) defined for the guix-daemon process on>> that machine is:>>>> $ pidof guix-daemon>> 270>>>> sudo cat /proc/270/environ>> GUIX_LOCPATH=/gnu/store/94k5w17z54w25lgp90czdqfv9m4hwzhq-glibc-utf8-locales-2.28/lib/localeLC_ALL=en_US.utf8>> This is perfect (see commit 7e4bc215098f334bc2a11737f2665dd4992fc2da,> which gave you this, fixing the issue we’re talking about on GuixSD.)>> So I don’t think this machine has any problem. Perhaps nss-certs was> installed before the fix above?
Yes, that is likely the cause! I think I was using a system generationfrom November to cope with some network instabilities I had at thetime. These have been resolved since :-).
Toggle quote (7 lines)>> I'm not familiar with this systemfs structure, but shouldn't there be a>> newline before the LC_ALL=en_US.utf8 variable assignment?>> No, there are actually newlines, try:>> cat /proc/270/environ | xargs -0 echo
Indeed. Thanks for continuously helping me to refine my knowledge ^^.
Shall we close this ticket, or did you want to keep it until we makeguix substitute fail when the locale of the daemon is not set to a UTF-8based one?
Thank you!
Maxim
L
L
Ludovic Courtès wrote on 18 Jan 2019 17:53
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 33603-done@debbugs.gnu.org)
878szhsy90.fsf@gnu.org
Hi Maxim,
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
Toggle quote (4 lines)> Shall we close this ticket, or did you want to keep it until we make> guix substitute fail when the locale of the daemon is not set to a UTF-8> based one?
Commit 9fe3f11398e858f1d06120bd046cab506efc86dc does that.Done!
Ludo’.
Closed
M
M
Maxim Cournoyer wrote on 19 Jan 2019 04:32
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 33603-done@debbugs.gnu.org)
874la5pbk9.fsf@gmail.com
Ludovic Courtès <ludo@gnu.org> writes:
Toggle quote (13 lines)> Hi Maxim,>> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:>>> Shall we close this ticket, or did you want to keep it until we make>> guix substitute fail when the locale of the daemon is not set to a UTF-8>> based one?>> Commit 9fe3f11398e858f1d06120bd046cab506efc86dc does that.> Done!>> Ludo’.
That was quick! Well done! :-)
Maxim
Closed
?
Your comment

This issue is archived.

To comment on this conversation send email to 33603@debbugs.gnu.org