From debbugs-submit-bounces@debbugs.gnu.org Mon May 29 16:15:14 2017 Received: (at 26948) by debbugs.gnu.org; 29 May 2017 20:15:14 +0000 Received: from localhost ([127.0.0.1]:43814 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dFR4M-0006kG-6M for submit@debbugs.gnu.org; Mon, 29 May 2017 16:15:14 -0400 Received: from mail-pf0-f181.google.com ([209.85.192.181]:34238) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dFR4K-0006dN-Gj for 26948@debbugs.gnu.org; Mon, 29 May 2017 16:15:13 -0400 Received: by mail-pf0-f181.google.com with SMTP id 9so52669260pfj.1 for <26948@debbugs.gnu.org>; Mon, 29 May 2017 13:15:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=jp85q8oIMkSZFXKyLmP8M6F0LS5LaU/d5lCdPx9lBkI=; b=cI9Pjf+i9s2mPrn8WubqtCc2MJ/FJYEDkOy3XymardLJaLIc4z4YhKq/Cp+Wo9wucp caa2SB94KCiA3ZtSxssusvk02bH2tKeAKUO54c9J2oC8UJREcggVO+36lbhEvbNqDX+q MyaK+r4J5oeWtClIST5zxXezm0i3zJrLs/07/5wlyxUtOWoVsEHf1t0Asod5eAs7ATLh HCq4E6uzF4wURvMhjJ87NoZqFxciAi6isd0foOOK/mh7q+eqDQzSfuf3bwKLOJHChJmL P1bHqS76Ztju1RAxKln1S+Vt5K8dbXWP0bt5U40leQdtxvaM8prdQqfi1+lKd9YDRdWN 5yuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=jp85q8oIMkSZFXKyLmP8M6F0LS5LaU/d5lCdPx9lBkI=; b=c3vtKnwu8+lAuvQneKl9D7Vz+gs7DB2W+KKsmVI99fLsUk8UvZ7a1FOxFyjbJTD5/3 KCu/TgaEcND6NSvZsBuO/0os/W4y4/UuurbXFk7oHRBEYFNrqQ6IdYFV0+P9xg8yq7ta A5BIyF2u8qFEs2xk92aEQ4Sr4U9M6aLmfFHyKaqNswzJk7A3OWnipP8vCkeL+qd9hMpJ XPIEzvdBiJTwFN07Sq74BYVBuVQ1q0ITEC7ndiNOwcC1RpxmCRpfScj6VCG34Xe53CwT O9NQvE/+43krTFwVmq6T6fOjTjy6GGMmGRvQKYdez3yICy7z04dNWncpowCKhIf7Vaj1 EC/A== X-Gm-Message-State: AODbwcBLo/8ix2vbBkHSQX/E6kXauFVSVLmBBVjksjmThLIUuNgPQtuv vLakMSZxWEOOM0LMZZA= X-Received: by 10.101.86.67 with SMTP id m3mr20878186pgs.137.1496088906256; Mon, 29 May 2017 13:15:06 -0700 (PDT) Received: from apteryx ([2601:647:4a01:bacd:c2f8:daff:fe5d:2f2f]) by smtp.gmail.com with ESMTPSA id o29sm18211766pgc.27.2017.05.29.13.15.05 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 29 May 2017 13:15:05 -0700 (PDT) From: Maxim Cournoyer To: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: =?utf-8?B?4oCYd3JpdGUtZmlsZeKAmQ==?= output should not be locale-dependent References: <8737c51e6r.fsf@gmail.com> <87shk3y74g.fsf@gnu.org> <8737btieie.fsf@gmail.com> <87vaoovvvz.fsf@gnu.org> <87o9ucu1t3.fsf@gmail.com> <87mv9wc9gp.fsf_-_@gnu.org> Date: Mon, 29 May 2017 13:15:04 -0700 In-Reply-To: <87mv9wc9gp.fsf_-_@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\?\= \=\?utf-8\?Q\?\=22's\?\= message of "Mon, 29 May 2017 11:12:54 +0200") Message-ID: <87h903s9mf.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 26948 Cc: 26948@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) ludo@gnu.org (Ludovic Court=C3=A8s) writes: > Maxim Cournoyer skribis: > >> ludo@gnu.org (Ludovic Court=C3=A8s) writes: > > [...] > >>> Strangely that file name has question marks instead of the non-ASCII >>> characters on my GuixSD system: >>> >>> $ ls -l /etc/ssl/certs/*Certi*mara* >>> lrwxrwxrwx 8 root root 162 Jan 1 1970 >>> '/etc/ssl/certs/AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.2= 27.87.240.105.140.203.236.12.pem' >>> -> >>> '/gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2/etc/ssl/c= erts/AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.1= 40.203.236.12.pem' >> >> Hmm. That is strange. It seems like you also have a locale problem, but >> that it is handled in a way that doesn't break nss-certs? > > AFAICS the file is really called that way, with question marks: > > scheme@(guile-user)> (stat "/gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-n= ss-certs-3.30.2/etc/ssl/certs/AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.12= 3.224.21.227.87.240.105.140.203.236.12.pem") > $2 =3D #(64768 4719936 33060 8 0 0 0 2444 1496043280 1 1492867575 4096 8 = regular 292 130744281 0 1492867575) > > > And: > > $ wget -O - https://mirror.hydra.gnu.org/guix/nar/gzip/3ql0vilc0zv6ra42gh= i04787vrg6bb71-nss-certs-3.30.2 |gunzip -c | guix archive -x t > --2017-05-29 10:55:36-- https://mirror.hydra.gnu.org/guix/nar/gzip/3ql0v= ilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2 > Ni solvigas mirror.hydra.gnu.org (mirror.hydra.gnu.org)... 131.159.14.26,= 2001:4ca0:2001:10:225:90ff:fedb:c720 > Konektado al mirror.hydra.gnu.org (mirror.hydra.gnu.org)|131.159.14.26|:4= 43... konektita. > HTTP peto sendita, ni atendas respondon... 200 OK > Grando: 171969 (168K) [application/octet-stream] > Ni konservas al: 'STDOUT' > > - 100%[=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D>] 167.94K --.-KB/s en 0.08s=20=20=20 > > 2017-05-29 10:55:37 (2.02 MB/s) - skribita al =C4=89efeligujo [171969/171= 969] > > $ find t -name AC_Ra\* > t/etc/ssl/certs/AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.227= .87.240.105.140.203.236.12.pem > $ locale > LANG=3Den_US.utf8 > LC_CTYPE=3D"en_US.utf8" > LC_NUMERIC=3D"en_US.utf8" > LC_TIME=3D"en_US.utf8" > LC_COLLATE=3D"en_US.utf8" > LC_MONETARY=3D"en_US.utf8" > LC_MESSAGES=3D"en_US.utf8" > LC_PAPER=3Dfr_FR.utf8 > LC_NAME=3D"en_US.utf8" > LC_ADDRESS=3D"en_US.utf8" > LC_TELEPHONE=3D"en_US.utf8" > LC_MEASUREMENT=3D"en_US.utf8" > LC_IDENTIFICATION=3D"en_US.utf8" > LC_ALL=3D > --8<---------------cut here---------------start------------->8--- $ find /etc/ssl/certs -name AC_Ra\* /etc/ssl/certs/AC_Ra=C3=ADz_Certic=C3=A1mara_S.A.:2.15.7.126.82.147.123.224= .21.227.87.240.105.140.203.236.12.pem $ locale LANG=3Den_US.UTF-8 LC_CTYPE=3D"en_US.UTF-8" LC_NUMERIC=3D"en_US.UTF-8" LC_TIME=3D"en_US.UTF-8" LC_COLLATE=3D"en_US.UTF-8" LC_MONETARY=3D"en_US.UTF-8" LC_MESSAGES=3D"en_US.UTF-8" LC_PAPER=3D"en_US.UTF-8" LC_NAME=3D"en_US.UTF-8" LC_ADDRESS=3D"en_US.UTF-8" LC_TELEPHONE=3D"en_US.UTF-8" LC_MEASUREMENT=3D"en_US.UTF-8" LC_IDENTIFICATION=3D"en_US.UTF-8" LC_ALL=3D --8<---------------cut here---------------end--------------->8--- The file name appears normally here (in xterm). I'm not sure why it's different on your side, since we are both using UTF-8 locales. It does still look strange when seen from strace though, but I guess this is peculiarity of strace: open("/etc/ssl/certs/AC_Ra\303\255z_Certic\303\241mara_S.A.:2.15.7.126.82.1= 47.123.224.21.227.87.240.105.140.203.236.12.p", O_RDONLY) =3D -1 ENOENT (No= such file or directory) > But wait! =E2=80=9Cguix build nss-certs --check -K=E2=80=9D fails, and t= he diff is: > > $ LANGUAGE=3D diff -ur /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-ce= rts-3.30.2{,-check} > Only in /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2-chec= k/etc/ssl/certs: AC_Ra=C3=ADz_Certic=C3=A1mara_S.A.:2.15.7.126.82.147.123.2= 24.21.227.87.240.105.140.203.236.12.pem > Only in /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2/etc/= ssl/certs: AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240= .105.140.203.236.12.pem > diff -ur /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2/etc= /ssl/certs/ae8153b9.0 /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs= -3.30.2-check/etc/ssl/certs/ae8153b9.0 > --- /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2/etc/ssl/= certs/ae8153b9.0 1970-01-01 01:00:01.000000000 +0100 > +++ /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2-check/et= c/ssl/certs/ae8153b9.0 1970-01-01 01:00:01.000000000 +0100 > @@ -3,10 +3,10 @@ > # distrust=3D > # openssl-trust=3DcodeSigning emailProtection serverAuth > -----BEGIN CERTIFICATE----- > -MIIHyTCCBbGgAwIBAgIBATANBgkqhkiG9w0BAQUFADB9MQswCQYDVQQGEwJJTDEW > +MIIHhzCCBW+gAwIBAgIBLTANBgkqhkiG9w0BAQsFADB9MQswCQYDVQQGEwJJTDEW Can this be explained by locale alone? That is troubling. > MBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMiU2VjdXJlIERpZ2l0YWwg > Q2VydGlmaWNhdGUgU2lnbmluZzEpMCcGA1UEAxMgU3RhcnRDb20gQ2VydGlmaWNh > -dGlvbiBBdXRob3JpdHkwHhcNMDYwOTE3MTk0NjM2WhcNMzYwOTE3MTk0NjM2WjB9 > +dGlvbiBBdXRob3JpdHkwHhcNMDYwOTE3MTk0NjM3WhcNMzYwOTE3MTk0NjM2WjB9 ^ ??? > MQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMi > > [...] > > +O3NJo2pXh5Tl1njFmUNj403gdy3hZZlyaQQaRwnmDwFWJPsfvw55qVguucQJAX6V > +um0ABj6y6koQOdjQK/W/7HW/lwLFCRsI3FU34oH7N4RDYiDK51ZLZer+bMEkkySh > +NOsF/5oirpt9P/FlUQqmMGqz9IgcgA38corog14=3D > -----END CERTIFICATE----- > Only in /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2-chec= k/etc/ssl/certs: Certinomis_-_Autorit=C3=A9_Racine:2.1.1.pem > Only in /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2/etc/= ssl/certs: Certinomis_-_Autorit?_Racine:2.1.1.pem > Only in /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2-chec= k/etc/ssl/certs: NetLock_Arany_=3DClass_Gold=3D_F=C5=91tan=C3=BAs=C3=ADtv= =C3=A1ny:2.6.73.65.44.228.0.16.pem > Only in /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2/etc/= ssl/certs: NetLock_Arany_=3DClass_Gold=3D_F?tan?s?tv?ny:2.6.73.65.44.228.0.= 16.pem > Only in /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2/etc/= ssl/certs: T?B?TAK_UEKAE_K?k_Sertifika_Hizmet_Sa?lay?c?s?_-_S?r?m_3:2.1.17.= pem > Only in /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2/etc/= ssl/certs: T?RKTRUST_Elektronik_Sertifika_Hizmet_Sa?lay?c?s?_H5:2.7.0.142.2= 3.254.36.32.129.pem > Only in /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2-chec= k/etc/ssl/certs: T=C3=9CB=C4=B0TAK_UEKAE_K=C3=B6k_Sertifika_Hizmet_Sa=C4=9F= lay=C4=B1c=C4=B1s=C4=B1_-_S=C3=BCr=C3=BCm_3:2.1.17.pem > Only in /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2-chec= k/etc/ssl/certs: T=C3=9CRKTRUST_Elektronik_Sertifika_Hizmet_Sa=C4=9Flay=C4= =B1c=C4=B1s=C4=B1_H5:2.7.0.142.23.254.36.32.129.pem > > See? (The difference in the first certificate is weird too=E2=80=A6) > > There are two ways to create nars. One is via the =E2=80=98export-paths= =E2=80=99 RPC > (implemented in the daemon in C++), which does not interpret file names > and thus leaves them untouched. The other one is via =E2=80=98write-file= =E2=80=99 from > (guix serialization), which is written in Scheme and thus converts file > names from locale encoding (specifically, =E2=80=98scandir=E2=80=99 does = that.) > > =E2=80=98guix publish=E2=80=99 uses the latter, so =E2=80=98guix publish= =E2=80=99 is sensitive to locale > settings, which is pretty bad. > > Guile currently does not allow us to specify whether/how file names > should be decoded, but possible solutions have been discussed for 2.2. > > In the meantime, solutions are: > > 1. To run =E2=80=98guix publish=E2=80=99 in a UTF-8 locale, which appar= ently was not > the case. I'm surprised by that. Wouldn't a utf8 locale be the default? > > 2. Add to (guix build syscalls) a separate locale-independent > =E2=80=98scandir=E2=80=99 implementation and use that. If the general solution is to fix it in Guile, the workaround proposed in 1. seems preferable. Maxim