Store references in 8-byte chunks in compiled code

Done

Details

4 participants

Török Edwin
Leo Famulari
Ludovic Courtès
Mark H Weaver

Owner: unassigned

Submitted by: Mark H Weaver

Severity: serious

Mark H Weaver wrote on 16 Oct 2016 05:49

fontconfig keeps obfuscated reference to itself, not grafted

Recipients:(address . bug-guix@gnu.org)

Message-ID:87mvi5lzqu.fsf@netris.org

After running "guix gc", I've found that my fonts are broken.

Gnome-terminal, Xfce4-terminal, and xterm are no longer able to find my

preferred "Dejavu Sans Mono" font. IceCat also uses a different

selection of fonts. Of the programs I've tried, only Emacs is able to

find them.

The failing programs output the following message to stderr:

Fontconfig error: Cannot load default config file

Strace reveals:

access("/gnu/store/b484nvn9nnr3ddclpz2fma9yxmimg2jj-fontconfig-2.11.94/etc/fonts/fonts.conf", R_OK) = -1 ENOENT (No such file or directory)

write(2, "Fontconfig error: ", 18Fontconfig error: ) = 18

write(2, "Cannot load default config file", 31Cannot load default config file) = 31

On my system, which includes a graft for 'bash' (not yet pushed to

Savannah), 'fontconfig' is grafted. After running 'guix gc', the

original ungrafted 'fontconfig' was removed, but libfontconfig is trying

to access it above.

It turns out there's an obfuscated self-reference to fontconfig's store

directory. Here's an excerpt of the output of "hexdump -C

libfontconfig.so.1.9.0":

0000cca0 00 48 b9 2f 67 6e 75 2f 73 74 6f c6 40 48 00 45 |.H./gnu/sto.@H.E|

0000ccb0 31 e4 48 89 08 48 b9 72 65 2f 62 34 38 34 6e 48 |1.H..H.re/b484nH|

0000ccc0 89 48 08 48 b9 76 6e 39 6e 6e 72 33 64 48 89 48 |.H.H.vn9nnr3dH.H|

0000ccd0 10 48 b9 64 63 6c 70 7a 32 66 6d 48 89 48 18 48 |.H.dclpz2fmH.H.H|

0000cce0 b9 61 39 79 78 6d 69 6d 67 48 89 48 20 48 b9 32 |.a9yxmimgH.H H.2|

0000ccf0 6a 6a 2d 66 6f 6e 74 48 89 48 28 48 b9 63 6f 6e |jj-fontH.H(H.con|

0000cd00 66 69 67 2d 32 48 89 48 30 48 b9 2e 31 31 2e 39 |fig-2H.H0H..11.9|

0000cd10 34 2f 65 48 89 48 38 48 b9 74 63 2f 66 6f 6e 74 |4/eH.H8H.tc/font|

0000cd20 73 48 89 48 40 48 8b 04 24 48 8b 18 48 89 c5 48 |sH.H@H..$H..H..H|

To be continued...

Mark

Mark H Weaver wrote on 16 Oct 2016 06:26

Recipients:(address . 24703@debbugs.gnu.org)

Message-ID:87inssncln.fsf@netris.org

Mark H Weaver <mhw@netris.org> writes:

Toggle quote (14 lines)> It turns out there's an obfuscated self-reference to fontconfig's store
> directory.  Here's an excerpt of the output of "hexdump -C
> libfontconfig.so.1.9.0":
>
> 0000cca0  00 48 b9 2f 67 6e 75 2f  73 74 6f c6 40 48 00 45  |.H./gnu/sto.@H.E|
> 0000ccb0  31 e4 48 89 08 48 b9 72  65 2f 62 34 38 34 6e 48  |1.H..H.re/b484nH|
> 0000ccc0  89 48 08 48 b9 76 6e 39  6e 6e 72 33 64 48 89 48  |.H.H.vn9nnr3dH.H|
> 0000ccd0  10 48 b9 64 63 6c 70 7a  32 66 6d 48 89 48 18 48  |.H.dclpz2fmH.H.H|
> 0000cce0  b9 61 39 79 78 6d 69 6d  67 48 89 48 20 48 b9 32  |.a9yxmimgH.H H.2|
> 0000ccf0  6a 6a 2d 66 6f 6e 74 48  89 48 28 48 b9 63 6f 6e  |jj-fontH.H(H.con|
> 0000cd00  66 69 67 2d 32 48 89 48  30 48 b9 2e 31 31 2e 39  |fig-2H.H0H..11.9|
> 0000cd10  34 2f 65 48 89 48 38 48  b9 74 63 2f 66 6f 6e 74  |4/eH.H8H.tc/font|
> 0000cd20  73 48 89 48 40 48 8b 04  24 48 8b 18 48 89 c5 48  |sH.H@H..$H..H..H|

It turns out that this is part of the compiled x86_64 code for

'FcConfigFilename' in src/fccfg.c, which copies a compile-time string

constant, 8 bytes at a time, into a buffer:

Toggle snippet (40 lines)$ objdump -d libfontconfig.so.1.9.0 | grep -B1 -A35 '48 b9 2f 67 6e 75 2f'
    cc9b:	0f 84 3f 01 00 00    	je     cde0 <FcConfigFilename+0x2d0>
    cca1:	48 b9 2f 67 6e 75 2f 	movabs $0x6f74732f756e672f,%rcx
    cca8:	73 74 6f 
    ccab:	c6 40 48 00          	movb   $0x0,0x48(%rax)
    ccaf:	45 31 e4             	xor    %r12d,%r12d
    ccb2:	48 89 08             	mov    %rcx,(%rax)
    ccb5:	48 b9 72 65 2f 62 34 	movabs $0x6e343834622f6572,%rcx
    ccbc:	38 34 6e 
    ccbf:	48 89 48 08          	mov    %rcx,0x8(%rax)
    ccc3:	48 b9 76 6e 39 6e 6e 	movabs $0x6433726e6e396e76,%rcx
    ccca:	72 33 64 
    cccd:	48 89 48 10          	mov    %rcx,0x10(%rax)
    ccd1:	48 b9 64 63 6c 70 7a 	movabs $0x6d66327a706c6364,%rcx
    ccd8:	32 66 6d 
    ccdb:	48 89 48 18          	mov    %rcx,0x18(%rax)
    ccdf:	48 b9 61 39 79 78 6d 	movabs $0x676d696d78793961,%rcx
    cce6:	69 6d 67 
    cce9:	48 89 48 20          	mov    %rcx,0x20(%rax)
    cced:	48 b9 32 6a 6a 2d 66 	movabs $0x746e6f662d6a6a32,%rcx
    ccf4:	6f 6e 74 
    ccf7:	48 89 48 28          	mov    %rcx,0x28(%rax)
    ccfb:	48 b9 63 6f 6e 66 69 	movabs $0x322d6769666e6f63,%rcx
    cd02:	67 2d 32 
    cd05:	48 89 48 30          	mov    %rcx,0x30(%rax)
    cd09:	48 b9 2e 31 31 2e 39 	movabs $0x652f34392e31312e,%rcx
    cd10:	34 2f 65 
    cd13:	48 89 48 38          	mov    %rcx,0x38(%rax)
    cd17:	48 b9 74 63 2f 66 6f 	movabs $0x73746e6f662f6374,%rcx
    cd1e:	6e 74 73 
    cd21:	48 89 48 40          	mov    %rcx,0x40(%rax)
    cd25:	48 8b 04 24          	mov    (%rsp),%rax
    cd29:	48 8b 18             	mov    (%rax),%rbx
    cd2c:	48 89 c5             	mov    %rax,%rbp
    cd2f:	48 85 db             	test   %rbx,%rbx
    cd32:	48 89 df             	mov    %rbx,%rdi
    cd35:	75 16                	jne    cd4d <FcConfigFilename+0x23d>
    cd37:	eb 44                	jmp    cd7d <FcConfigFilename+0x26d>

So far, I've not been able to find any evidence of the fontconfig code
doing anything strange here.  I strongly suspect that GCC is generating
this code, most likely due to an inlinable string/memory copy function
where the source is a string literal.

Obviously, this could be a serious problem for Guix (and Nix), since it
suggests that we may not be able to continue with our simplistic
assumption that references to the store in compiled code will be easy to
find and replace.

      Mark

Mark H Weaver wrote on 16 Oct 2016 06:30

(no subject)

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87eg3gncfq.fsf@netris.org

retitle 24703 Store references in 8-byte chunks in compiled code

thanks

Mark H Weaver wrote on 16 Oct 2016 06:35

Recipients:(address . control@debbugs.gnu.org)

Message-ID:877f98nc6q.fsf@netris.org

severity 24703 important

thanks

Mark H Weaver wrote on 16 Oct 2016 07:00

Re: bug#24703: fontconfig keeps obfuscated reference to itself, not grafted

Recipients:(address . 24703@debbugs.gnu.org)

Message-ID:8737jwnb1c.fsf@netris.org

Mark H Weaver <mhw@netris.org> writes:

Toggle quote (64 lines)> Mark H Weaver <mhw@netris.org> writes:
>
>> It turns out there's an obfuscated self-reference to fontconfig's store
>> directory.  Here's an excerpt of the output of "hexdump -C
>> libfontconfig.so.1.9.0":
>>
>> 0000cca0  00 48 b9 2f 67 6e 75 2f  73 74 6f c6 40 48 00 45  |.H./gnu/sto.@H.E|
>> 0000ccb0  31 e4 48 89 08 48 b9 72  65 2f 62 34 38 34 6e 48  |1.H..H.re/b484nH|
>> 0000ccc0  89 48 08 48 b9 76 6e 39  6e 6e 72 33 64 48 89 48  |.H.H.vn9nnr3dH.H|
>> 0000ccd0  10 48 b9 64 63 6c 70 7a  32 66 6d 48 89 48 18 48  |.H.dclpz2fmH.H.H|
>> 0000cce0  b9 61 39 79 78 6d 69 6d  67 48 89 48 20 48 b9 32  |.a9yxmimgH.H H.2|
>> 0000ccf0  6a 6a 2d 66 6f 6e 74 48  89 48 28 48 b9 63 6f 6e  |jj-fontH.H(H.con|
>> 0000cd00  66 69 67 2d 32 48 89 48  30 48 b9 2e 31 31 2e 39  |fig-2H.H0H..11.9|
>> 0000cd10  34 2f 65 48 89 48 38 48  b9 74 63 2f 66 6f 6e 74  |4/eH.H8H.tc/font|
>> 0000cd20  73 48 89 48 40 48 8b 04  24 48 8b 18 48 89 c5 48  |sH.H@H..$H..H..H|
>
> It turns out that this is part of the compiled x86_64 code for
> 'FcConfigFilename' in src/fccfg.c, which copies a compile-time string
> constant, 8 bytes at a time, into a buffer:
>
> $ objdump -d libfontconfig.so.1.9.0 | grep -B1 -A35 '48 b9 2f 67 6e 75 2f'
>     cc9b:	0f 84 3f 01 00 00    	je     cde0 <FcConfigFilename+0x2d0>
>     cca1:	48 b9 2f 67 6e 75 2f 	movabs $0x6f74732f756e672f,%rcx
>     cca8:	73 74 6f 
>     ccab:	c6 40 48 00          	movb   $0x0,0x48(%rax)
>     ccaf:	45 31 e4             	xor    %r12d,%r12d
>     ccb2:	48 89 08             	mov    %rcx,(%rax)
>     ccb5:	48 b9 72 65 2f 62 34 	movabs $0x6e343834622f6572,%rcx
>     ccbc:	38 34 6e 
>     ccbf:	48 89 48 08          	mov    %rcx,0x8(%rax)
>     ccc3:	48 b9 76 6e 39 6e 6e 	movabs $0x6433726e6e396e76,%rcx
>     ccca:	72 33 64 
>     cccd:	48 89 48 10          	mov    %rcx,0x10(%rax)
>     ccd1:	48 b9 64 63 6c 70 7a 	movabs $0x6d66327a706c6364,%rcx
>     ccd8:	32 66 6d 
>     ccdb:	48 89 48 18          	mov    %rcx,0x18(%rax)
>     ccdf:	48 b9 61 39 79 78 6d 	movabs $0x676d696d78793961,%rcx
>     cce6:	69 6d 67 
>     cce9:	48 89 48 20          	mov    %rcx,0x20(%rax)
>     cced:	48 b9 32 6a 6a 2d 66 	movabs $0x746e6f662d6a6a32,%rcx
>     ccf4:	6f 6e 74 
>     ccf7:	48 89 48 28          	mov    %rcx,0x28(%rax)
>     ccfb:	48 b9 63 6f 6e 66 69 	movabs $0x322d6769666e6f63,%rcx
>     cd02:	67 2d 32 
>     cd05:	48 89 48 30          	mov    %rcx,0x30(%rax)
>     cd09:	48 b9 2e 31 31 2e 39 	movabs $0x652f34392e31312e,%rcx
>     cd10:	34 2f 65 
>     cd13:	48 89 48 38          	mov    %rcx,0x38(%rax)
>     cd17:	48 b9 74 63 2f 66 6f 	movabs $0x73746e6f662f6374,%rcx
>     cd1e:	6e 74 73 
>     cd21:	48 89 48 40          	mov    %rcx,0x40(%rax)
>     cd25:	48 8b 04 24          	mov    (%rsp),%rax
>     cd29:	48 8b 18             	mov    (%rax),%rbx
>     cd2c:	48 89 c5             	mov    %rax,%rbp
>     cd2f:	48 85 db             	test   %rbx,%rbx
>     cd32:	48 89 df             	mov    %rbx,%rdi
>     cd35:	75 16                	jne    cd4d <FcConfigFilename+0x23d>
>     cd37:	eb 44                	jmp    cd7d <FcConfigFilename+0x26d>
>
> So far, I've not been able to find any evidence of the fontconfig code
> doing anything strange here.  I strongly suspect that GCC is generating
> this code, most likely due to an inlinable string/memory copy function
> where the source is a string literal.

I've confirmed this. After building this package manually, "objdump -d

--source src/.libs/fccfg.o" reveals that the corresponding source code

is:

dir = (FcChar8 *) FONTCONFIG_PATH;

path[i] = malloc (strlen ((char *) dir) + 1);

if (!path[i])

goto bail1;

strcpy ((char *) path[i], (const char *) dir);

It is part of 'FcConfigGetPath', inlined into 'FcConfigFilename', in

src/fccfg.c. -DFONTCONFIG_PATH='"$(BASECONFIGDIR)"' is one of the flags

passed to GCC, via AM_CPPFLAGS in src/Makefile.am.

Toggle quote (5 lines)

> Obviously, this could be a serious problem for Guix (and Nix), since it

> suggests that we may not be able to continue with our simplistic

> assumption that references to the store in compiled code will be easy to

> find and replace.

Mark

Mark H Weaver wrote on 16 Oct 2016 07:10

(no subject)

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87y41olvz1.fsf@netris.org

severity 24703 serious

thanks

Mark H Weaver wrote on 16 Oct 2016 08:24

Re: Store references in 8-byte chunks in compiled code

Recipients:(address . 24703@debbugs.gnu.org)

Message-ID:87r37gstf6.fsf_-_@netris.org

Mark H Weaver <mhw@netris.org> writes:

Toggle quote (85 lines)> Mark H Weaver <mhw@netris.org> writes:
>
>> Mark H Weaver <mhw@netris.org> writes:
>>
>>> It turns out there's an obfuscated self-reference to fontconfig's store
>>> directory.  Here's an excerpt of the output of "hexdump -C
>>> libfontconfig.so.1.9.0":
>>>
>>> 0000cca0  00 48 b9 2f 67 6e 75 2f  73 74 6f c6 40 48 00 45  |.H./gnu/sto.@H.E|
>>> 0000ccb0  31 e4 48 89 08 48 b9 72  65 2f 62 34 38 34 6e 48  |1.H..H.re/b484nH|
>>> 0000ccc0  89 48 08 48 b9 76 6e 39  6e 6e 72 33 64 48 89 48  |.H.H.vn9nnr3dH.H|
>>> 0000ccd0  10 48 b9 64 63 6c 70 7a  32 66 6d 48 89 48 18 48  |.H.dclpz2fmH.H.H|
>>> 0000cce0  b9 61 39 79 78 6d 69 6d  67 48 89 48 20 48 b9 32  |.a9yxmimgH.H H.2|
>>> 0000ccf0  6a 6a 2d 66 6f 6e 74 48  89 48 28 48 b9 63 6f 6e  |jj-fontH.H(H.con|
>>> 0000cd00  66 69 67 2d 32 48 89 48  30 48 b9 2e 31 31 2e 39  |fig-2H.H0H..11.9|
>>> 0000cd10  34 2f 65 48 89 48 38 48  b9 74 63 2f 66 6f 6e 74  |4/eH.H8H.tc/font|
>>> 0000cd20  73 48 89 48 40 48 8b 04  24 48 8b 18 48 89 c5 48  |sH.H@H..$H..H..H|
>>
>> It turns out that this is part of the compiled x86_64 code for
>> 'FcConfigFilename' in src/fccfg.c, which copies a compile-time string
>> constant, 8 bytes at a time, into a buffer:
>>
>> $ objdump -d libfontconfig.so.1.9.0 | grep -B1 -A35 '48 b9 2f 67 6e 75 2f'
>>     cc9b:	0f 84 3f 01 00 00    	je     cde0 <FcConfigFilename+0x2d0>
>>     cca1:	48 b9 2f 67 6e 75 2f 	movabs $0x6f74732f756e672f,%rcx
>>     cca8:	73 74 6f 
>>     ccab:	c6 40 48 00          	movb   $0x0,0x48(%rax)
>>     ccaf:	45 31 e4             	xor    %r12d,%r12d
>>     ccb2:	48 89 08             	mov    %rcx,(%rax)
>>     ccb5:	48 b9 72 65 2f 62 34 	movabs $0x6e343834622f6572,%rcx
>>     ccbc:	38 34 6e 
>>     ccbf:	48 89 48 08          	mov    %rcx,0x8(%rax)
>>     ccc3:	48 b9 76 6e 39 6e 6e 	movabs $0x6433726e6e396e76,%rcx
>>     ccca:	72 33 64 
>>     cccd:	48 89 48 10          	mov    %rcx,0x10(%rax)
>>     ccd1:	48 b9 64 63 6c 70 7a 	movabs $0x6d66327a706c6364,%rcx
>>     ccd8:	32 66 6d 
>>     ccdb:	48 89 48 18          	mov    %rcx,0x18(%rax)
>>     ccdf:	48 b9 61 39 79 78 6d 	movabs $0x676d696d78793961,%rcx
>>     cce6:	69 6d 67 
>>     cce9:	48 89 48 20          	mov    %rcx,0x20(%rax)
>>     cced:	48 b9 32 6a 6a 2d 66 	movabs $0x746e6f662d6a6a32,%rcx
>>     ccf4:	6f 6e 74 
>>     ccf7:	48 89 48 28          	mov    %rcx,0x28(%rax)
>>     ccfb:	48 b9 63 6f 6e 66 69 	movabs $0x322d6769666e6f63,%rcx
>>     cd02:	67 2d 32 
>>     cd05:	48 89 48 30          	mov    %rcx,0x30(%rax)
>>     cd09:	48 b9 2e 31 31 2e 39 	movabs $0x652f34392e31312e,%rcx
>>     cd10:	34 2f 65 
>>     cd13:	48 89 48 38          	mov    %rcx,0x38(%rax)
>>     cd17:	48 b9 74 63 2f 66 6f 	movabs $0x73746e6f662f6374,%rcx
>>     cd1e:	6e 74 73 
>>     cd21:	48 89 48 40          	mov    %rcx,0x40(%rax)
>>     cd25:	48 8b 04 24          	mov    (%rsp),%rax
>>     cd29:	48 8b 18             	mov    (%rax),%rbx
>>     cd2c:	48 89 c5             	mov    %rax,%rbp
>>     cd2f:	48 85 db             	test   %rbx,%rbx
>>     cd32:	48 89 df             	mov    %rbx,%rdi
>>     cd35:	75 16                	jne    cd4d <FcConfigFilename+0x23d>
>>     cd37:	eb 44                	jmp    cd7d <FcConfigFilename+0x26d>
>>
>> So far, I've not been able to find any evidence of the fontconfig code
>> doing anything strange here.  I strongly suspect that GCC is generating
>> this code, most likely due to an inlinable string/memory copy function
>> where the source is a string literal.
>
> I've confirmed this.  After building this package manually, "objdump -d
> --source src/.libs/fccfg.o" reveals that the corresponding source code
> is:
>
>     dir = (FcChar8 *) FONTCONFIG_PATH;
>     path[i] = malloc (strlen ((char *) dir) + 1);
>     if (!path[i])
> 	goto bail1;
>     strcpy ((char *) path[i], (const char *) dir);
>
> It is part of 'FcConfigGetPath', inlined into 'FcConfigFilename', in
> src/fccfg.c.  -DFONTCONFIG_PATH='"$(BASECONFIGDIR)"' is one of the flags
> passed to GCC, via AM_CPPFLAGS in src/Makefile.am.
>
>> Obviously, this could be a serious problem for Guix (and Nix), since it
>> suggests that we may not be able to continue with our simplistic
>> assumption that references to the store in compiled code will be easy to
>> find and replace.

To get an idea of how widespread this problem is, I searched for
occurrences of /gnu/sto[^r] on my GNOME/Xfce desktop system:

$ LC_ALL=C grep '/gnu/sto[^r]' --recursive /gnu/store/
[.../gnu/store/.links hits removed...]
Binary file /gnu/store/0010wvgs40kdq8chzsh403qm7la9jxq7-bash-static-4.3.42/bin/bash matches
Binary file /gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/sbin/sln matches
Binary file /gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/lib/libc-2.23.so matches
Binary file /gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/lib/libc.a matches
Binary file /gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/bin/locale matches
Binary file /gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/bin/localedef matches
Binary file /gnu/store/4fs8dg5jhf99xl3ikl093dn1va4wlv79-fontconfig-2.11.94/lib/libfontconfig.so.1.9.0 matches
Binary file /gnu/store/b484nvn9nnr3ddclpz2fma9yxmimg2jj-fontconfig-2.11.94/lib/libfontconfig.so.1.9.0 matches
Binary file /gnu/store/80m11l8km7bwi9iljfyr82hmscaq4xk2-unixodbc-2.3.4/lib/libodbcinst.so.2.0.0 matches
Binary file /gnu/store/80m11l8km7bwi9iljfyr82hmscaq4xk2-unixodbc-2.3.4/lib/libodbc.so.2.0.0 matches
Binary file /gnu/store/ld16jy012l3jpkj6azynzmldxn28cspw-ncurses-6.0/lib/libncursesw.so.6.0 matches
Binary file /gnu/store/ld16jy012l3jpkj6azynzmldxn28cspw-ncurses-6.0/lib/libncursesw.a matches
Binary file /gnu/store/lxv20br9ic8abvqd7fipszbs4mg8hkjv-ncurses-6.0/lib/libncursesw.so.6.0 matches
Binary file /gnu/store/lxv20br9ic8abvqd7fipszbs4mg8hkjv-ncurses-6.0/lib/libncursesw.a matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/gcc matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-gcc-4.9.3 matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-c++ matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-gcc matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/g++ matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-g++ matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/cpp matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/c++ matches
Binary file /gnu/store/sxa3b6l28ckcnyg3g7f4brvl2rdpncy1-gnuplot-5.0.4-1/bin/gnuplot matches
Binary file /gnu/store/2jvvwqz16hj2c5ws0xa46x98fvl9m90m-fontconfig-2.11.94/lib/libfontconfig.so.1.9.0 matches
Binary file /gnu/store/bxy1hwriqzdw6xk7bl28qfsw4s1s5xdq-bash-static-4.3.48/bin/bash matches
Binary file /gnu/store/3chnjjkazbv0fqbshrwahq7c3zfg42s8-ncurses-6.0/lib/libncursesw.so.6.0 matches
Binary file /gnu/store/3chnjjkazbv0fqbshrwahq7c3zfg42s8-ncurses-6.0/lib/libncursesw.a matches
Binary file /gnu/store/my7f7fq2ca5rqq4wyyrg20cw2bjrj2l4-ncurses-6.0/lib/libncursesw.so.6.0 matches
Binary file /gnu/store/my7f7fq2ca5rqq4wyyrg20cw2bjrj2l4-ncurses-6.0/lib/libncursesw.a matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/gcc matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-gcc-4.9.3 matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-c++ matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-gcc matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/g++ matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-g++ matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/cpp matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/c++ matches
Binary file /gnu/store/fjsaprcdmdn39pk39jrhbby1jl5i8rp5-xorg-server-1.18.1/bin/Xorg matches
Binary file /gnu/store/mz301kb7wqvyl9kxil4bpn8ng99ikgqy-glibc-2.23/sbin/sln matches
Binary file /gnu/store/mz301kb7wqvyl9kxil4bpn8ng99ikgqy-glibc-2.23/lib/libc-2.23.so matches
Binary file /gnu/store/mz301kb7wqvyl9kxil4bpn8ng99ikgqy-glibc-2.23/lib/libc.a matches
Binary file /gnu/store/mz301kb7wqvyl9kxil4bpn8ng99ikgqy-glibc-2.23/bin/locale matches
Binary file /gnu/store/mz301kb7wqvyl9kxil4bpn8ng99ikgqy-glibc-2.23/bin/localedef matches
Binary file /gnu/store/slhwk75g7d8bywpq2hifs7g9fxr6jx3d-bash-static-4.3.48/bin/bash matches
Binary file /gnu/store/qjsmp5s85qrba18fxf319m5lv7f8awf8-graphviz-2.38.0/bin/lefty matches
Binary file /gnu/store/6ddfp4pwlrxc5jdgf17ddh4m5wi1cldy-xorg-server-1.18.1/bin/Xorg matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/gcc matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-gcc-4.9.3 matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-c++ matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-gcc matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/g++ matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-g++ matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/cpp matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/c++ matches
Binary file /gnu/store/xl19qrfzga52vrvp4ncccwjlnrjqwj95-ncurses-6.0/lib/libncursesw.so.6.0 matches
Binary file /gnu/store/xl19qrfzga52vrvp4ncccwjlnrjqwj95-ncurses-6.0/lib/libncursesw.a matches
Binary file /gnu/store/kh3awka9xslyp52dldb3gma47rr0kp2x-xorg-server-1.18.1/bin/Xorg matches
Binary file /gnu/store/121596cgx25s8zcl3yznyh2vh1f842ni-babl-0.1.18/lib/libbabl-0.1.so.0.117.1 matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-gcc-6.2.0 matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/gcc matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-c++ matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-gcc matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/g++ matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-g++ matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/cpp matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/c++ matches
Binary file /gnu/store/j4q20kwzd5g1d3gv419692k66ghfzymz-gnuplot-5.0.4-1/bin/gnuplot matches
Binary file /gnu/store/mdc84lh0mfzw9n404cnzi9l1l8qr7a4r-gnuplot-5.0.4-1/bin/gnuplot matches
Binary file /gnu/store/8mj5yd1z936j64sdpx3hbqi3qkdif0c4-alsa-lib-1.0.27.1/lib/libasound.so.2.0.0 matches
Binary file /gnu/store/ninvaqcyhm6s11yp97m17h2i1q3aj24s-recode-3.7.0.201402/lib/librecode.a matches
Binary file /gnu/store/ninvaqcyhm6s11yp97m17h2i1q3aj24s-recode-3.7.0.201402/lib/librecode.so.0.0.0 matches
Binary file /gnu/store/kjybzn7az86n1qzxcm8zdz2gaypp4az6-fontconfig-2.11.94/lib/libfontconfig.so.1.9.0 matches
Binary file /gnu/store/cl2vwkvmk60s7vpamivpclzgyfxlb7wx-graphviz-2.38.0/bin/lefty matches
Binary file /gnu/store/qc8qg71k1b7gizqxa785c6ls71i8qk6d-units-2.13/bin/units matches
Binary file /gnu/store/sj8ygx2yz58hn1142yjjsb34sql4b9xr-unixodbc-2.3.4/lib/libodbcinst.so.2.0.0 matches
Binary file /gnu/store/sj8ygx2yz58hn1142yjjsb34sql4b9xr-unixodbc-2.3.4/lib/libodbc.so.2.0.0 matches
Binary file /gnu/store/571c58j8f06x8svykg4n5s0ip36kna5c-util-linux-2.27/lib/libuuid.so.1.3.0 matches
Binary file /gnu/store/1i3xmm18dw9kq6wi46f6sj9nxy9pckjl-alsa-lib-1.0.27.1/lib/libasound.so.2.0.0 matches
Binary file /gnu/store/x1kh4kkifc3f4gnlwvfgk53fm2zhfhmm-gnuplot-5.0.4-1/bin/gnuplot matches
Binary file /gnu/store/fnw55giyr1gnqyyw5yx4hf96mlrkp603-gnuplot-5.0.4-1/bin/gnuplot matches
Binary file /gnu/store/mj727bz0l8afmn97l8gsc5wh30jnql8s-babl-0.1.18/lib/libbabl-0.1.so.0.117.1 matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-gcc-6.2.0 matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/gcc matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-c++ matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-gcc matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/g++ matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-g++ matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/cpp matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/c++ matches
Binary file /gnu/store/lqybibdq19q6ypm29p9p9s9ns0s66n0b-recode-3.7.0.201402/lib/librecode.a matches
Binary file /gnu/store/lqybibdq19q6ypm29p9p9s9ns0s66n0b-recode-3.7.0.201402/lib/librecode.so.0.0.0 matches
Binary file /gnu/store/pxdl28wbikj2jy0jjldmii245gcsh5fq-fontconfig-2.11.94/lib/libfontconfig.so.1.9.0 matches
Binary file /gnu/store/lhxgw5ynrccx9vvk9jx2k9pdsgq0yxm3-units-2.13/bin/units matches
Binary file /gnu/store/cji1mwp7n6zhdnpg73gighf3fvrh2gdi-ncurses-6.0/lib/libncursesw.so.6.0 matches
Binary file /gnu/store/cji1mwp7n6zhdnpg73gighf3fvrh2gdi-ncurses-6.0/lib/libncursesw.a matches
Binary file /gnu/store/1rcsqn8h2xwmgdra3zr33xv73d44wf1s-fontconfig-2.11.94/lib/libfontconfig.so.1.9.0 matches
Binary file /gnu/store/iiakyhsw889rvv3ghk6jl3mfdrs1wfan-fontconfig-2.11.94/lib/libfontconfig.so.1.9.0 matches
Binary file /gnu/store/zb3kh929v8yhkyg3sn8405zj4x49aza4-gnuplot-5.0.4-1/bin/gnuplot matches
Binary file /gnu/store/mkq7h60l1kx4hjclsrd0nbz8v4mnx4lv-gnuplot-5.0.4-1/bin/gnuplot matches
Binary file /gnu/store/4vhgx5xkbx0x7gnph9fq3c581rnj5ynq-libgphoto2-2.5.2/lib/libgphoto2/2.5.2/konica.so matches
Binary file /gnu/store/r6kwxzban57ghxsy8p8dqjvym6vnb2nz-graphviz-2.38.0/bin/lefty matches
/gnu/store/lwxifldgdyyzd83510nzf1qffpzxbdyl-guix-0.11.0-1.4420/share/guile/site/2.0/gnu/packages/package-management.scm:              (uri (string-append "mirror://gnu/stow/stow-"
Binary file /gnu/store/lwxifldgdyyzd83510nzf1qffpzxbdyl-guix-0.11.0-1.4420/share/guile/site/2.0/gnu/packages/package-management.go matches
Binary file /gnu/store/zslzcw2wg6xylwwr9hcx2sbal0bm6kjx-util-linux-2.27/lib/libuuid.so.1.3.0 matches

Mark H Weaver wrote on 16 Oct 2016 11:03

Re: bug#24703: Store references in 8-byte chunks in compiled code

Recipients:(address . 24703@debbugs.gnu.org)

Message-ID:87k2d8sm1q.fsf@netris.org

The 8-byte chunks may appear out of order.  For example, in
'bash-static' we have this reference to:

  /gnu/store/7z3hpynjsbidxkq78xi5qi6lbcm8ndhp-glibc-intermediate-2.23

where the chunks are found in the following order:

1_/gnu/sto
2_________________ynjsbidx
3_________re/7z3hp
4_________________________________i6lbcm8n
5_________________________kq78xi5q
6_________________________________________dhp-glib
7_________________________________________________c-interm
8_________________________________________________________ediate-2
9_________________________________________________________________.23/lib/

Here's an excerpt of "objdump -d" output, annotated to show the 8-byte
constant as a string, and its position within the larger string.

======== /gnu/store/0010wvgs40kdq8chzsh403qm7la9jxq7-bash-static-4.3.42/bin/bash
  46b2c3:	48 b9 2f 67 6e 75 2f 	movabs $0x6f74732f756e672f,%rcx ; "/gnu/sto" (1)
  46b2ca:	73 74 6f 
  46b2cd:	48 01 d8             	add    %rbx,%rax
  46b2d0:	48 bb 79 6e 6a 73 62 	movabs $0x78646962736a6e79,%rbx ; "ynjsbidx" (3)
  46b2d7:	69 64 78 
  46b2da:	48 89 48 01          	mov    %rcx,0x1(%rax)
  46b2de:	48 b9 72 65 2f 37 7a 	movabs $0x7068337a372f6572,%rcx ; "re/7z3hp" (2)
  46b2e5:	33 68 70 
  46b2e8:	48 89 58 11          	mov    %rbx,0x11(%rax)
  46b2ec:	48 89 48 09          	mov    %rcx,0x9(%rax)
  46b2f0:	48 bb 69 36 6c 62 63 	movabs $0x6e386d63626c3669,%rbx ; "i6lbcm8n" (5)
  46b2f7:	6d 38 6e 
  46b2fa:	48 b9 6b 71 37 38 78 	movabs $0x713569783837716b,%rcx ; "kq78xi5q" (4)
  46b301:	69 35 71 
  46b304:	48 89 48 19          	mov    %rcx,0x19(%rax)
  46b308:	48 89 58 21          	mov    %rbx,0x21(%rax)
  46b30c:	48 b9 64 68 70 2d 67 	movabs $0x62696c672d706864,%rcx ; "dhp-glib" (6)
  46b313:	6c 69 62 
  46b316:	48 bb 63 2d 69 6e 74 	movabs $0x6d7265746e692d63,%rbx ; "c-interm" (7)
  46b31d:	65 72 6d 
  46b320:	48 89 48 29          	mov    %rcx,0x29(%rax)
  46b324:	ba 76 00 00 00       	mov    $0x76,%edx
  46b329:	48 89 58 31          	mov    %rbx,0x31(%rax)
  46b32d:	48 b9 65 64 69 61 74 	movabs $0x322d657461696465,%rcx ; "ediate-2" (8)
  46b334:	65 2d 32 
  46b337:	48 bb 2e 32 33 2f 6c 	movabs $0x2f62696c2f33322e,%rbx ; ".23/lib/" (9)
  46b33e:	69 62 2f 
  46b341:	48 89 58 41          	mov    %rbx,0x41(%rax)
  46b345:	31 f6                	xor    %esi,%esi
  46b347:	31 ff                	xor    %edi,%edi
  46b349:	c6 00 3a             	movb   $0x3a,(%rax)
  46b34c:	48 89 48 39          	mov    %rcx,0x39(%rax)
  46b350:	c7 40 49 67 63 6f 6e 	movl   $0x6e6f6367,0x49(%rax)
  46b357:	66 89 50 4d          	mov    %dx,0x4d(%rax)

Mark H Weaver wrote on 16 Oct 2016 11:25

Recipients:(address . 24703@debbugs.gnu.org)

Message-ID:87d1j0sl1l.fsf@netris.org

Here's a complex example of a reference to:

  /gnu/store/80m11l8km7bwi9iljfyr82hmscaq4xk2-unixodbc-2.3.4/etc

This also illustrates what can happen near the end of a reference.  In
this case, the last two characters are found at the end, within a 'mov'
instruction, and the previous 4 characters are found near the beginning,
after the first 8-byte chunk.

======== /gnu/store/80m11l8km7bwi9iljfyr82hmscaq4xk2-unixodbc-2.3.4/lib/libodbcinst.so.2.0.0
    8238:	48 b8 2f 67 6e 75 2f 	movabs $0x6f74732f756e672f,%rax   ; "/gnu/sto"
    823f:	73 74 6f 
    8242:	c7 05 6c fa 20 00 2e 	movl   $0x652f342e,0x20fa6c(%rip) ; ".4/e"
    8249:	34 2f 65 
    824c:	c6 05 6b fa 20 00 00 	movb   $0x0,0x20fa6b(%rip)
    8253:	48 89 05 26 fa 20 00 	mov    %rax,0x20fa26(%rip)
    825a:	48 b8 72 65 2f 38 30 	movabs $0x31316d30382f6572,%rax   ; "re/80m11"
    8261:	6d 31 31 
    8264:	c7 05 16 0a 21 00 01 	movl   $0x1,0x210a16(%rip)
    826b:	00 00 00 
    826e:	48 89 05 13 fa 20 00 	mov    %rax,0x20fa13(%rip)
    8275:	48 b8 6c 38 6b 6d 37 	movabs $0x697762376d6b386c,%rax   ; "l8km7bwi"
    827c:	62 77 69 
    827f:	48 8d 1d 42 94 00 00 	lea    0x9442(%rip),%rbx
    8286:	48 89 05 03 fa 20 00 	mov    %rax,0x20fa03(%rip)
    828d:	48 b8 39 69 6c 6a 66 	movabs $0x387279666a6c6939,%rax   ; "9iljfyr8"
    8294:	79 72 38 
    8297:	48 89 05 fa f9 20 00 	mov    %rax,0x20f9fa(%rip)
    829e:	48 b8 32 68 6d 73 63 	movabs $0x34716163736d6832,%rax   ; "2hmscaq4"
    82a5:	61 71 34 
    82a8:	48 89 05 f1 f9 20 00 	mov    %rax,0x20f9f1(%rip)
    82af:	48 b8 78 6b 32 2d 75 	movabs $0x78696e752d326b78,%rax   ; "xk2-unix"
    82b6:	6e 69 78 
    82b9:	48 89 05 e8 f9 20 00 	mov    %rax,0x20f9e8(%rip)
    82c0:	48 b8 6f 64 62 63 2d 	movabs $0x332e322d6362646f,%rax   ; "odbc-2.3"
    82c7:	32 2e 33 
    82ca:	48 89 05 df f9 20 00 	mov    %rax,0x20f9df(%rip)
    82d1:	b8 74 63 00 00       	mov    $0x6374,%eax               ; "tc"
    82d6:	66 89 05 df f9 20 00 	mov    %ax,0x20f9df(%rip)

Now imagine what will be required to graft cases like this.  Consider
what will happen for strings with an odd number of characters.  The last
character might be all by itself.

When grafting, how will we achieve confidence that we've found the
correct occurrence of the last character?  I think we will have to give
up our recently added feature of being able to change the version number
of grafts.

Oh, one more thing: I forgot to mention in my previous email, where
'bash-static' refers to 'glibc-intermediate', that on my system the
referenced 'glibc-intermediate' has been garbage-collected from my
system, so that's a real-world example of a broken link, and it happened
to be the first one I investigated :-(

       Mark

Mark H Weaver wrote on 16 Oct 2016 12:15

Recipients:(address . 24703@debbugs.gnu.org)

Message-ID:8760ossiqf.fsf@netris.org

Here's what happens with a variant of 'unixodbc' with one character
removed from its name, to make an odd number of characters.  The last
character 'c' is all by itself.  The chunks of the reference

  /gnu/store/il1bn7n0l4yj3idrii23fhvzg4nn939i-unxodbc-2.3.4/etc

are found in the following order:

1_/gnu/sto
2_________________________________________________________4/et
3_________re/il1bn
4_________________7n0l4yj3
5_________________________idrii23f
6_________________________________hvzg4nn9
7_________________________________________39i-unxo
8_________________________________________________dbc-2.3.
9_____________________________________________________________c

======== /gnu/store/y2dwj5vkbxka7y5rlakprbk4k8vrd5ij-unxodbc-2.3.4/lib/libodbcinst.so.2.0.0
    8238:	48 b8 2f 67 6e 75 2f 	movabs $0x6f74732f756e672f,%rax   ; "/gnu/sto"
    823f:	73 74 6f 
    8242:	c7 05 6c fa 20 00 34 	movl   $0x74652f34,0x20fa6c(%rip) ; "4/et"
    8249:	2f 65 74 
    824c:	c7 05 2e 0a 21 00 01 	movl   $0x1,0x210a2e(%rip)
    8253:	00 00 00 
    8256:	48 89 05 23 fa 20 00 	mov    %rax,0x20fa23(%rip)
    825d:	48 b8 72 65 2f 69 6c 	movabs $0x6e62316c692f6572,%rax   ; "re/il1bn"
    8264:	31 62 6e 
    8267:	48 8d 1d 5a 94 00 00 	lea    0x945a(%rip),%rbx
    826e:	48 89 05 13 fa 20 00 	mov    %rax,0x20fa13(%rip)
    8275:	48 b8 37 6e 30 6c 34 	movabs $0x336a79346c306e37,%rax   ; "7n0l4yj3"
    827c:	79 6a 33 
    827f:	48 89 05 0a fa 20 00 	mov    %rax,0x20fa0a(%rip)
    8286:	48 b8 69 64 72 69 69 	movabs $0x6633326969726469,%rax   ; "idrii23f"
    828d:	32 33 66 
    8290:	48 89 05 01 fa 20 00 	mov    %rax,0x20fa01(%rip)
    8297:	48 b8 68 76 7a 67 34 	movabs $0x396e6e34677a7668,%rax   ; "hvzg4nn9"
    829e:	6e 6e 39 
    82a1:	48 89 05 f8 f9 20 00 	mov    %rax,0x20f9f8(%rip)
    82a8:	48 b8 33 39 69 2d 75 	movabs $0x6f786e752d693933,%rax   ; "39i-unxo"
    82af:	6e 78 6f 
    82b2:	48 89 05 ef f9 20 00 	mov    %rax,0x20f9ef(%rip)
    82b9:	48 b8 64 62 63 2d 32 	movabs $0x2e332e322d636264,%rax   ; "dbc-2.3."
    82c0:	2e 33 2e 
    82c3:	48 89 05 e6 f9 20 00 	mov    %rax,0x20f9e6(%rip)
    82ca:	b8 63 00 00 00       	mov    $0x63,%eax                 ; "c"
    82cf:	66 89 05 e6 f9 20 00 	mov    %ax,0x20f9e6(%rip)

Ludovic Courtès wrote on 16 Oct 2016 16:42

Re: bug#24703: fontconfig keeps obfuscated reference to itself, not grafted

Recipients:(name . Mark H Weaver)(address . mhw@netris.org)(address . 24703@debbugs.gnu.org)

Message-ID:87oa2kgxti.fsf@gnu.org

Mark H Weaver <mhw@netris.org> skribis:

Toggle quote (14 lines)> It turns out there's an obfuscated self-reference to fontconfig's store
> directory.  Here's an excerpt of the output of "hexdump -C
> libfontconfig.so.1.9.0":
>
> 0000cca0  00 48 b9 2f 67 6e 75 2f  73 74 6f c6 40 48 00 45  |.H./gnu/sto.@H.E|
> 0000ccb0  31 e4 48 89 08 48 b9 72  65 2f 62 34 38 34 6e 48  |1.H..H.re/b484nH|
> 0000ccc0  89 48 08 48 b9 76 6e 39  6e 6e 72 33 64 48 89 48  |.H.H.vn9nnr3dH.H|
> 0000ccd0  10 48 b9 64 63 6c 70 7a  32 66 6d 48 89 48 18 48  |.H.dclpz2fmH.H.H|
> 0000cce0  b9 61 39 79 78 6d 69 6d  67 48 89 48 20 48 b9 32  |.a9yxmimgH.H H.2|
> 0000ccf0  6a 6a 2d 66 6f 6e 74 48  89 48 28 48 b9 63 6f 6e  |jj-fontH.H(H.con|
> 0000cd00  66 69 67 2d 32 48 89 48  30 48 b9 2e 31 31 2e 39  |fig-2H.H0H..11.9|
> 0000cd10  34 2f 65 48 89 48 38 48  b9 74 63 2f 66 6f 6e 74  |4/eH.H8H.tc/font|
> 0000cd20  73 48 89 48 40 48 8b 04  24 48 8b 18 48 89 c5 48  |sH.H@H..$H..H..H|

See commit 9d50da70608de32d9db0c29859caec6f2cddb95f in WordNet:

-fno-builtin-strcpy is the solution I found.

Weird, no? :-)

Ludo’.

Ludovic Courtès wrote on 16 Oct 2016 17:06

Recipients:(name . Mark H Weaver)(address . mhw@netris.org)(address . 24703@debbugs.gnu.org)

Message-ID:87eg3ggwog.fsf@gnu.org

ludo@gnu.org (Ludovic Courtès) skribis:

Toggle quote (19 lines)> Mark H Weaver <mhw@netris.org> skribis:
>
>> It turns out there's an obfuscated self-reference to fontconfig's store
>> directory.  Here's an excerpt of the output of "hexdump -C
>> libfontconfig.so.1.9.0":
>>
>> 0000cca0  00 48 b9 2f 67 6e 75 2f  73 74 6f c6 40 48 00 45  |.H./gnu/sto.@H.E|
>> 0000ccb0  31 e4 48 89 08 48 b9 72  65 2f 62 34 38 34 6e 48  |1.H..H.re/b484nH|
>> 0000ccc0  89 48 08 48 b9 76 6e 39  6e 6e 72 33 64 48 89 48  |.H.H.vn9nnr3dH.H|
>> 0000ccd0  10 48 b9 64 63 6c 70 7a  32 66 6d 48 89 48 18 48  |.H.dclpz2fmH.H.H|
>> 0000cce0  b9 61 39 79 78 6d 69 6d  67 48 89 48 20 48 b9 32  |.a9yxmimgH.H H.2|
>> 0000ccf0  6a 6a 2d 66 6f 6e 74 48  89 48 28 48 b9 63 6f 6e  |jj-fontH.H(H.con|
>> 0000cd00  66 69 67 2d 32 48 89 48  30 48 b9 2e 31 31 2e 39  |fig-2H.H0H..11.9|
>> 0000cd10  34 2f 65 48 89 48 38 48  b9 74 63 2f 66 6f 6e 74  |4/eH.H8H.tc/font|
>> 0000cd20  73 48 89 48 40 48 8b 04  24 48 8b 18 48 89 c5 48  |sH.H@H..$H..H..H|
>
> See commit 9d50da70608de32d9db0c29859caec6f2cddb95f in WordNet:
> -fno-builtin-strcpy is the solution I found.

Simple test case:

#include <string.h>
void foo (char *bar)
{
  strcpy (bar, "this is a very long string that's gonna be split");
}

Run “gcc -c t.c” with and without -fno-builtin-strcpy.

Ludo’.

Ludovic Courtès wrote on 16 Oct 2016 21:04

Re: bug#24703: Store references in 8-byte chunks in compiled code

Recipients:(name . Mark H Weaver)(address . mhw@netris.org)(address . 24703@debbugs.gnu.org)

Message-ID:87a8e4glot.fsf@gnu.org

Mark H Weaver <mhw@netris.org> skribis:

Toggle quote (5 lines)

> When grafting, how will we achieve confidence that we've found the

> correct occurrence of the last character? I think we will have to give

> up our recently added feature of being able to change the version number

> of grafts.

Wait, don’t jump to the conclusions. :-)

Toggle quote (6 lines)

> Oh, one more thing: I forgot to mention in my previous email, where

> 'bash-static' refers to 'glibc-intermediate', that on my system the

> referenced 'glibc-intermediate' has been garbage-collected from my

> system, so that's a real-world example of a broken link, and it happened

> to be the first one I investigated :-(

‘bash-static’ is purposefully designed to stand alone: it must have zero

references:

Toggle snippet (6 lines)

$ guix size bash-static

store item total self

/gnu/store/nxysmfi9ybchnxfpxpmpgvz4f0nh7mxs-bash-static-4.3.42 1.4 1.4 100.0%

total: 1.4 MiB

So this one is fine.

Ludo’.

Török Edwin wrote on 17 Oct 2016 09:46

Recipients:(address . 24703@debbugs.gnu.org)

Message-ID:8f2024ad-13c1-d4b1-1541-c2a5bddcb403@etorok.net

On 2016-10-16 22:04, Ludovic Courtès wrote:

Toggle quote (9 lines)

> Mark H Weaver <mhw@netris.org> skribis:

>> When grafting, how will we achieve confidence that we've found the

>> correct occurrence of the last character? I think we will have to give

>> up our recently added feature of being able to change the version number

>> of grafts.

> Wait, don’t jump to the conclusions. :-)

I've just encountered the same problem with fontconfig (after installing GuixSD, running guix pull and guix system reconfigure, --no-grafts was required).

Would it be possible for the grafts to keep a symlink (somehow registered to be part of the grafted fontconfig so that guix gc doesn't remove it) instead of patching the binaries?

/gnu/store/<old-hash>-fontconfig-2.11.94 -> /gnu/store/<grafted-hash>-fontconfig-2.11.94

Best regards,

Edwin Török | Co-founder and Lead Developer

Skylable open-source object storage: reliable, fast, secure

http://www.skylable.com

Mark H Weaver wrote on 17 Oct 2016 11:42

Re: bug#24703: Re: bug#24703: Store references in 8-byte chunks in compiled code

Recipients:(name . Török Edwin)(address . edwin+ml-guix@etorok.net)(address . 24703@debbugs.gnu.org)

Message-ID:871szfs45v.fsf@netris.org

Török Edwin <edwin+ml-guix@etorok.net> writes:

Toggle quote (14 lines)> On 2016-10-16 22:04, Ludovic Courtès wrote:
>> Mark H Weaver <mhw@netris.org> skribis:
>> 
>>> When grafting, how will we achieve confidence that we've found the
>>> correct occurrence of the last character?  I think we will have to give
>>> up our recently added feature of being able to change the version number
>>> of grafts.
>> 
>> Wait, don’t jump to the conclusions.  :-)
>
> I've just encountered the same problem with fontconfig (after
> installing GuixSD, running guix pull and guix system reconfigure,
> --no-grafts was required).

Here's how to recover, for now:

guix build --no-grafts -e '(@@ (gnu packages fontutils) fontconfig/fixed)'

Toggle quote (5 lines)

> Would it be possible for the grafts to keep a symlink (somehow

> registered to be part of the grafted fontconfig so that guix gc

> doesn't remove it) instead of patching the binaries?

> /gnu/store/<old-hash>-fontconfig-2.11.94 -> /gnu/store/<grafted-hash>-fontconfig-2.11.94

This would effectively mutate the store, and we must never do this.

Mark

Ludovic Courtès wrote on 17 Oct 2016 14:09

Re: bug#24703: Store references in 8-byte chunks in compiled code

Recipients:(name . Török Edwin)(address . edwin+ml-guix@etorok.net)

Message-ID:87h98bdvng.fsf@gnu.org

Török Edwin <edwin+ml-guix@etorok.net> skribis:

Toggle quote (14 lines)> On 2016-10-16 22:04, Ludovic Courtès wrote:
>> Mark H Weaver <mhw@netris.org> skribis:
>> 
>>> When grafting, how will we achieve confidence that we've found the
>>> correct occurrence of the last character?  I think we will have to give
>>> up our recently added feature of being able to change the version number
>>> of grafts.
>> 
>> Wait, don’t jump to the conclusions.  :-)
>
> I've just encountered the same problem with fontconfig (after installing GuixSD, running guix pull and guix system reconfigure, --no-grafts was required).
> Would it be possible for the grafts to keep a symlink (somehow registered to be part of the grafted fontconfig so that guix gc doesn't remove it) instead of patching the binaries?
> /gnu/store/<old-hash>-fontconfig-2.11.94 -> /gnu/store/<grafted-hash>-fontconfig-2.11.94

We could use a self symlink, or we could use something like

http://git.savannah.gnu.org/cgit/guix.git/commit/?id=9d50da70608de32d9db0c29859caec6f2cddb95f.

Mark, WDYT?

What remains to be seen is how many packages are affected by this issue,

and whether a generic solution needs to be found.

Ludo’.

Mark H Weaver wrote on 18 Oct 2016 05:36

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 24703@debbugs.gnu.org)

Message-ID:87k2d6qqee.fsf@netris.org

ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (26 lines)> Török Edwin <edwin+ml-guix@etorok.net> skribis:
>
>> On 2016-10-16 22:04, Ludovic Courtès wrote:
>>> Mark H Weaver <mhw@netris.org> skribis:
>>> 
>>>> When grafting, how will we achieve confidence that we've found the
>>>> correct occurrence of the last character?  I think we will have to give
>>>> up our recently added feature of being able to change the version number
>>>> of grafts.
>>> 
>>> Wait, don’t jump to the conclusions.  :-)
>>
>> I've just encountered the same problem with fontconfig (after installing GuixSD, running guix pull and guix system reconfigure, --no-grafts was required).
>> Would it be possible for the grafts to keep a symlink (somehow
>> registered to be part of the grafted fontconfig so that guix gc
>> doesn't remove it) instead of patching the binaries?
>> /gnu/store/<old-hash>-fontconfig-2.11.94 -> /gnu/store/<grafted-hash>-fontconfig-2.11.94
>
> We could use a self symlink, or we could use something like
> <http://git.savannah.gnu.org/cgit/guix.git/commit/?id=9d50da70608de32d9db0c29859caec6f2cddb95f>.
>
> Mark, WDYT?
>
> What remains to be seen is how many packages are affected by this issue,
> and whether a generic solution needs to be found.

Unfortunately, it is too widespread. As I just pointed out in

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24712#13

Among the many packages that include these obfuscated store references,

one is 'glibc-final'. My attempt to graft 'bash' in 'master' to fix

CVE-2016-0634 and CVE-2016-7543 has resulted in a system where I cannot

build *any* derivation, because 'guile-final' crashes during boot while

its 'glibc-final' tries to find its 'gconv' modules in the ungrafted

'glibc-final', which is not available in the build environment.

So, if our approach is to use -fno-builtin-strcpy, then we will have to

apply it system-wide, and rebuild all of 'core-updates' from scratch.

I've been investigating another approach: to enhance our scanner and

grafter to handle these 8-byte-chunked references. I believe it is

feasible, but only if we abandon the ability to change the file names of

grafts outside of the hash. The reason is that the hash portion of

store references are surrounded by enough other known characters on both

sides that the hash portion is almost always contained entirely within

8-byte chunks.

To be continued...

Mark

Ludovic Courtès wrote on 18 Oct 2016 10:59

Recipients:(name . Mark H Weaver)(address . mhw@netris.org)(address . 24703@debbugs.gnu.org)

Message-ID:87shrunicp.fsf@gnu.org

Mark H Weaver <mhw@netris.org> skribis:

Toggle quote (2 lines)

> ludo@gnu.org (Ludovic Courtès) writes:

[...]

Toggle quote (19 lines)>> We could use a self symlink, or we could use something like
>> <http://git.savannah.gnu.org/cgit/guix.git/commit/?id=9d50da70608de32d9db0c29859caec6f2cddb95f>.
>>
>> Mark, WDYT?
>>
>> What remains to be seen is how many packages are affected by this issue,
>> and whether a generic solution needs to be found.
>
> Unfortunately, it is too widespread.  As I just pointed out in
>
>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24712#13
>
> Among the many packages that include these obfuscated store references,
> one is 'glibc-final'.  My attempt to graft 'bash' in 'master' to fix
> CVE-2016-0634 and CVE-2016-7543 has resulted in a system where I cannot
> build *any* derivation, because 'guile-final' crashes during boot while
> its 'glibc-final' tries to find its 'gconv' modules in the ungrafted
> 'glibc-final', which is not available in the build environment.

Out of curiosity, Guile crashes while loading iconv modules, right?

This is surprising because the guile-for-build is always the ungrafted

‘guile-final’ (see ‘gnu-build’ in (guix build-system gnu)).

Toggle quote (3 lines)

> So, if our approach is to use -fno-builtin-strcpy, then we will have to

> apply it system-wide, and rebuild all of 'core-updates' from scratch.

Another approach would be to patch GCC, specifically ‘expand_movstr’ in
gcc/builtins.c, which is the part responsible for this optimization,
along these lines (untested):

--- gcc-5.3.0/gcc/builtins.c.orig	2016-10-18 10:45:35.042826368 +0200
+++ gcc-5.3.0/gcc/builtins.c	2016-10-18 10:50:46.080616285 +0200
@@ -3470,6 +3470,19 @@ expand_builtin_mempcpy_args (tree dest,
 # define CODE_FOR_movstr CODE_FOR_nothing
 #endif
 
+/* Return true if STR is a string denoting a "/gnu/store" file name.  */
+
+static bool
+store_reference_p (tree str)
+{
+  const char *store;
+
+  store = getenv ("NIX_STORE") ?: "/gnu/store";
+
+  return (TREE_STRING_LENGTH (str) > strlen (store)
+	  && strncmp (TREE_STRING_POINTER (str), store, strlen (store)));
+}
+
 /* Expand into a movstr instruction, if one is available.  Return NULL_RTX if
    we failed, the caller should emit a normal call, otherwise try to
    get the result in TARGET, if convenient.  If ENDP is 0 return the
@@ -3484,7 +3497,9 @@ expand_movstr (tree dest, tree src, rtx
   rtx dest_mem;
   rtx src_mem;
 
-  if (!HAVE_movstr)
+  /* When SRC denotes a store item, do not perform any optimization.  See
+     http://bugs.gnu.org/24703 for background.  */
+  if (!HAVE_movstr || store_reference_p (src))
     return NULL_RTX;
 
   dest_mem = get_memory_rtx (dest, NULL);

WDYT?

In the meantime, we need a workaround. The only option I can think of

is to retain a reference to the ungrafted item by adding a symlink to

it, like:

Toggle diff (20 lines)diff --git a/guix/grafts.scm b/guix/grafts.scm
index 80ae27e..4de19ae 100644
--- a/guix/grafts.scm
+++ b/guix/grafts.scm
@@ -127,7 +127,14 @@ recursively applied to dependencies of DRV."
                       files))
                    (match %outputs
                      (((names . files) ...)
-                      files))))))
+                      files)))
+
+         (for-each (match-lambda
+                     ((name . file)
+                      (symlink file
+                               (string-append (assoc-ref %outputs name)
+                                              "/ungrafted"))))
+                   old-outputs))))
 
   (define add-label
     (cut cons "x" <>))

Toggle quote (8 lines)

> I've been investigating another approach: to enhance our scanner and

> grafter to handle these 8-byte-chunked references. I believe it is

> feasible, but only if we abandon the ability to change the file names of

> grafts outside of the hash. The reason is that the hash portion of

> store references are surrounded by enough other known characters on both

> sides that the hash portion is almost always contained entirely within

> 8-byte chunks.

I think that would add complexity, would make grafting slower, and

abandoning the ability to change file names would be a regression.

So I’m more in favor of a GCC patch like above, or another compilation

tweak.

WDYT?

Thanks,

Ludo’.

Török Edwin wrote on 19 Oct 2016 23:25

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:dadebcc5-5816-4760-0bd3-72c1af95e39c@etorok.net

On 2016-10-17 12:42, Mark H Weaver wrote:

Toggle quote (5 lines)

> Here's how to recover, for now:

> guix build --no-grafts -e '(@@ (gnu packages fontutils) fontconfig/fixed)'

Thanks!

On 2016-10-17 15:09, Ludovic Courtès wrote:

Toggle quote (18 lines)> Török Edwin <edwin+ml-guix@etorok.net> skribis:
> 
>> On 2016-10-16 22:04, Ludovic Courtès wrote:
>>> Mark H Weaver <mhw@netris.org> skribis:
>>>
>>>> When grafting, how will we achieve confidence that we've found the
>>>> correct occurrence of the last character?  I think we will have to give
>>>> up our recently added feature of being able to change the version number
>>>> of grafts.
>>>
>>> Wait, don’t jump to the conclusions.  :-)
>>
>> I've just encountered the same problem with fontconfig (after installing GuixSD, running guix pull and guix system reconfigure, --no-grafts was required).
>> Would it be possible for the grafts to keep a symlink (somehow registered to be part of the grafted fontconfig so that guix gc doesn't remove it) instead of patching the binaries?
>> /gnu/store/<old-hash>-fontconfig-2.11.94 -> /gnu/store/<grafted-hash>-fontconfig-2.11.94
> 
> We could use a self symlink

I'm new here [*] and I'd like to understand what solutions would be possible, could you please explain how the self symlink would work?

I was thinking about bind mounts (the list of needed bind mounts would be maintained as a derivation, and initialized from initrd maybe?):

/gnu/store/<new-hash>-fontconfig-2.11.94 on /gnu/store/<old-hash>-fontconfig-2.11.94

or equivalently another layer of symlinks (where /gnu/store is the mutable symlink graft location, and /gnu/immutablestore is the real destination):

/gnu/store/<old-hash>-fontconfig-2.11.94 -> /gnu/immutablestore/<new-hash>-fontconfig-2.11.94

/gnu/store/<new-hash>-fontconfig-2.11.94 -> /gnu/immutablestore/<new-hash>-fontconfig-2.11.94

However bit worried of what happens to running applications: they might end up loading libs/data from both old-hash and new-hash, but

it wouldn't be worse than what happens on a traditional distribution when you upgrade say libc, you need to restart things using it eventually.

Toggle quote (3 lines)

> or we could use something like

> <http://git.savannah.gnu.org/cgit/guix.git/commit/?id=9d50da70608de32d9db0c29859caec6f2cddb95f>.

IMHO binary rewriting should be a last resort, it is too low-level and depends on the compiler/version to get it right.

[*] I've briefly tried NixOS and GuixSD in the past, but I'm using Debian and Gentoo as my main OS.

IIUC what the current grafting process does is that it reads all files of packages that depend on X from the store,

and writes them under a new hash with all references to hash of X replaced by hash of X-fixed, a bit like patchELF

except strings remain same size.

Best regards,

Ludovic Courtès wrote on 20 Oct 2016 14:25

Recipients:(name . Török Edwin)(address . edwin+ml-guix@etorok.net)

Message-ID:87twc7cin6.fsf@gnu.org

Török Edwin <edwin+ml-guix@etorok.net> skribis:

Toggle quote (29 lines)> On 2016-10-17 12:42, Mark H Weaver wrote:
>> 
>> Here's how to recover, for now:
>> 
>>   guix build --no-grafts -e '(@@ (gnu packages fontutils) fontconfig/fixed)'
>
> Thanks!
>
> On 2016-10-17 15:09, Ludovic Courtès wrote:
>> Török Edwin <edwin+ml-guix@etorok.net> skribis:
>> 
>>> On 2016-10-16 22:04, Ludovic Courtès wrote:
>>>> Mark H Weaver <mhw@netris.org> skribis:
>>>>
>>>>> When grafting, how will we achieve confidence that we've found the
>>>>> correct occurrence of the last character?  I think we will have to give
>>>>> up our recently added feature of being able to change the version number
>>>>> of grafts.
>>>>
>>>> Wait, don’t jump to the conclusions.  :-)
>>>
>>> I've just encountered the same problem with fontconfig (after installing GuixSD, running guix pull and guix system reconfigure, --no-grafts was required).
>>> Would it be possible for the grafts to keep a symlink (somehow registered to be part of the grafted fontconfig so that guix gc doesn't remove it) instead of patching the binaries?
>>> /gnu/store/<old-hash>-fontconfig-2.11.94 -> /gnu/store/<grafted-hash>-fontconfig-2.11.94
>> 
>> We could use a self symlink
>
> I'm new here [*] and I'd like to understand what solutions would be possible, could you please explain how the self symlink would work?

The grafted variant of each store item would have a symlink pointing to

its ungrafted variant.

The problem with this approach is that it would induce some storage

overhead (although deduplication probably mitigates that), and it would

make it harder to check whether a given item is using only grafted

things because ‘guix gc -R something’ would always show both the grafted

and the ungrafted variant of each dependency.

Toggle quote (3 lines)

> I was thinking about bind mounts (the list of needed bind mounts would be maintained as a derivation, and initialized from initrd maybe?):

> /gnu/store/<new-hash>-fontconfig-2.11.94 on /gnu/store/<old-hash>-fontconfig-2.11.94

That would only work on GuixSD, not on foreign distros, and it’s a

“stateful” approach, which defeats the whole reproducible approach.

Toggle quote (2 lines)

> IMHO binary rewriting should be a last resort, it is too low-level and depends on the compiler/version to get it right.

I agree. However, for fast security updates, it seems to be the best

option:

https://savannah.gnu.org/forum/forum.php?forum_id=8470

Thanks for your feedback,

Ludo’.

Leo Famulari wrote on 24 Oct 2016 21:40

Recipients:(name . Mark H Weaver)(address . mhw@netris.org)

Message-ID:20161024194022.GA1772@jasmine

On Mon, Oct 17, 2016 at 11:36:57PM -0400, Mark H Weaver wrote:

Toggle quote (10 lines)> I've been investigating another approach: to enhance our scanner and
> grafter to handle these 8-byte-chunked references.  I believe it is
> feasible, but only if we abandon the ability to change the file names of
> grafts outside of the hash.  The reason is that the hash portion of
> store references are surrounded by enough other known characters on both
> sides that the hash portion is almost always contained entirely within
> 8-byte chunks.
> 
> To be continued...

Any progress? If not, I think we should start making a list of packages

affected by the bug in order to prepare a -fno-builtin-strcpy branch.

Ludovic Courtès wrote on 24 Oct 2016 22:18

Recipients:(name . Leo Famulari)(address . leo@famulari.name)

Message-ID:87insh1oxz.fsf@gnu.org

Leo Famulari <leo@famulari.name> skribis:

Toggle quote (13 lines)> On Mon, Oct 17, 2016 at 11:36:57PM -0400, Mark H Weaver wrote:
>> I've been investigating another approach: to enhance our scanner and
>> grafter to handle these 8-byte-chunked references.  I believe it is
>> feasible, but only if we abandon the ability to change the file names of
>> grafts outside of the hash.  The reason is that the hash portion of
>> store references are surrounded by enough other known characters on both
>> sides that the hash portion is almost always contained entirely within
>> 8-byte chunks.
>> 
>> To be continued...
>
> Any progress?

I built GCC 5.3 with the patch I posted just to notice that it had no

effect (according to ‘ltrace’ it never goes through the

getenv("NIX_STORE") call that the patch adds.)

I plan to investigate further Real Soon.

Toggle quote (3 lines)

> If not, I think we should start making a list of packages affected by

> the bug in order to prepare a -fno-builtin-strcpy branch.

Yeah it would be nice to identify these packages.

I’d prefer the GCC patch over -fno-builtin-strcpy because the latter is

hacky and potentially misses the issue when it occurs with other

builtins.

Either way, we’ll have to rebuild a lot of things.

Ludo’.

Mark H Weaver wrote on 31 Oct 2016 07:35

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 24703@debbugs.gnu.org)

Message-ID:87oa21c9fe.fsf@netris.org

ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (18 lines)> Mark H Weaver <mhw@netris.org> skribis:
>
>> Unfortunately, it is too widespread.  As I just pointed out in
>>
>>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24712#13
>>
>> Among the many packages that include these obfuscated store references,
>> one is 'glibc-final'.  My attempt to graft 'bash' in 'master' to fix
>> CVE-2016-0634 and CVE-2016-7543 has resulted in a system where I cannot
>> build *any* derivation, because 'guile-final' crashes during boot while
>> its 'glibc-final' tries to find its 'gconv' modules in the ungrafted
>> 'glibc-final', which is not available in the build environment.
>
> Out of curiosity, Guile crashes while loading iconv modules, right?
>
> This is surprising because the guile-for-build is always the ungrafted
> ‘guile-final’ (see ‘gnu-build’ in (guix build-system gnu)).

Indeed. The derivations that crashed were using a grafted 'guile'.

These were not 'gnu-build' derivations, but simpler derivations such as

'module-import-compiled' derivations and some others involved with

building a 'system'.

Toggle quote (26 lines)>> So, if our approach is to use -fno-builtin-strcpy, then we will have to
>> apply it system-wide, and rebuild all of 'core-updates' from scratch.
>
> Another approach would be to patch GCC, specifically ‘expand_movstr’ in
> gcc/builtins.c, which is the part responsible for this optimization,
> along these lines (untested):
>
> --- gcc-5.3.0/gcc/builtins.c.orig	2016-10-18 10:45:35.042826368 +0200
> +++ gcc-5.3.0/gcc/builtins.c	2016-10-18 10:50:46.080616285 +0200
> @@ -3470,6 +3470,19 @@ expand_builtin_mempcpy_args (tree dest,
>  # define CODE_FOR_movstr CODE_FOR_nothing
>  #endif
>  
> +/* Return true if STR is a string denoting a "/gnu/store" file name.  */
> +
> +static bool
> +store_reference_p (tree str)
> +{
> +  const char *store;
> +
> +  store = getenv ("NIX_STORE") ?: "/gnu/store";
> +
> +  return (TREE_STRING_LENGTH (str) > strlen (store)
> +	  && strncmp (TREE_STRING_POINTER (str), store, strlen (store)));
> +}

[...]

Toggle quote (2 lines)

> WDYT?

I think it's not sufficient to apply this workaround only for string

literals that _begin_ with the store directory. In some cases, the

store name may appear only in the middle of a string.

Toggle quote (4 lines)

> In the meantime, we need a workaround. The only option I can think of

> is to retain a reference to the ungrafted item by adding a symlink to

> it, like:

I consider it a potentially serious security problem that ungrafted

outputs are being used. Papering over the problem by preventing this

buggy software from being deleted is, in my opinion, not acceptable.

I would suggest instead that we'll need to add grafts for all packages

that include these chunked references. However, due to bug 24832, it

may be that we'll need to rebuild all of 'core-updates' from scratch

anyway.

Toggle quote (16 lines)>> I've been investigating another approach: to enhance our scanner and
>> grafter to handle these 8-byte-chunked references.  I believe it is
>> feasible, but only if we abandon the ability to change the file names of
>> grafts outside of the hash.  The reason is that the hash portion of
>> store references are surrounded by enough other known characters on both
>> sides that the hash portion is almost always contained entirely within
>> 8-byte chunks.
>
> I think that would add complexity, would make grafting slower, and
> abandoning the ability to change file names would be a regression.
>
> So I’m more in favor of a GCC patch like above, or another compilation
> tweak.
>
> WDYT?

The GCC approach is okay with me in the short term, but I'll likely want

to revisit this issue in the future.

Thanks,

Mark

Ludovic Courtès wrote on 31 Oct 2016 12:37

Recipients:(name . Mark H Weaver)(address . mhw@netris.org)(address . 24703@debbugs.gnu.org)

Message-ID:87pomgrbq2.fsf@gnu.org

Hi Mark,

Mark H Weaver <mhw@netris.org> skribis:

Toggle quote (2 lines)

> ludo@gnu.org (Ludovic Courtès) writes:

[...]

Toggle quote (34 lines)>>> So, if our approach is to use -fno-builtin-strcpy, then we will have to
>>> apply it system-wide, and rebuild all of 'core-updates' from scratch.
>>
>> Another approach would be to patch GCC, specifically ‘expand_movstr’ in
>> gcc/builtins.c, which is the part responsible for this optimization,
>> along these lines (untested):
>>
>> --- gcc-5.3.0/gcc/builtins.c.orig	2016-10-18 10:45:35.042826368 +0200
>> +++ gcc-5.3.0/gcc/builtins.c	2016-10-18 10:50:46.080616285 +0200
>> @@ -3470,6 +3470,19 @@ expand_builtin_mempcpy_args (tree dest,
>>  # define CODE_FOR_movstr CODE_FOR_nothing
>>  #endif
>>  
>> +/* Return true if STR is a string denoting a "/gnu/store" file name.  */
>> +
>> +static bool
>> +store_reference_p (tree str)
>> +{
>> +  const char *store;
>> +
>> +  store = getenv ("NIX_STORE") ?: "/gnu/store";
>> +
>> +  return (TREE_STRING_LENGTH (str) > strlen (store)
>> +	  && strncmp (TREE_STRING_POINTER (str), store, strlen (store)));
>> +}
>
> [...]
>
>> WDYT?
>
> I think it's not sufficient to apply this workaround only for string
> literals that _begin_ with the store directory.  In some cases, the
> store name may appear only in the middle of a string.

Do you have examples? I think this is unlikely: the common case here is

that we’re capturing the installation prefix as in:

#define PREFIX "/gnu/store/…"

strcpy (file, PREFIX);

Toggle quote (8 lines)

>> In the meantime, we need a workaround. The only option I can think of

>> is to retain a reference to the ungrafted item by adding a symlink to

>> it, like:

> I consider it a potentially serious security problem that ungrafted

> outputs are being used. Papering over the problem by preventing this

> buggy software from being deleted is, in my opinion, not acceptable.

In practice, only data from the ungrafted input would be used, AFAICS.

I’m not saying that this is a good solution, I’m just trying to think of

solutions that we can deploy now while waiting for something better,

which may involve a full rebuild.

Toggle quote (3 lines)

> The GCC approach is okay with me in the short term, but I'll likely want

> to revisit this issue in the future.

From your message, it’s unclear to me what you are proposing:

1. as the short term solution;

2. as the long term solution.

To me short-term is symlink and long-term is GCC patch.

Thanks for your feedback,

Ludo’.

Ludovic Courtès wrote on 5 Nov 2016 00:15

Recipients:(name . Leo Famulari)(address . leo@famulari.name)

Message-ID:87a8de7s6q.fsf@inria.fr

I’ve fiddled a bit with GCC and read some code. No success yet, but

here’s a status update.

AIUI, ‘strcpy’ declarations are immediately (in the front-end)

recognized as “built-ins” as they are read (IOW, there’s no explicit

conversion from function call to built-in call.)

‘__builtin_strcpy’ calls are then converted to ‘__builtin_memcpy’ calls

(‘handle_builtin_strcpy’ in tree-ssa-strlen.c), which in turn leads to

code that uses the ‘movabs’ instruction that’s causing us problems on

x86:

Toggle snippet (16 lines)

$ echo 'extern char *strcpy(char*,const char*);void foo(char*x){ strcpy(x, "foo"); }'> t.c

$ gcc -c -fdump-tree-optimized t.c

$ cat t.c.211t.optimized

;; Function foo (foo, funcdef_no=0, decl_uid=1759, cgraph_uid=0, symbol_order=0)

foo (char * x)

{

<bb 2>:

__builtin_memcpy (x_2(D), "foo", 4);

return;

}

(Strangely, when using ‘memcpy’ in the source, the ‘movabs’ optimization

does not take place at -O0, unlike for ‘strcpy’.)

So it seems there’s no place on the front-end side where we could do

what I tried to do in my initial patch, which is to test the contents of

the literal string passed to strcpy/memcpy and turn off the

‘movabs’-producing optimization.

Instead, the knobs we have are (1) global flag to enable/disable each

built-in function (like -fno-builtin-… does), and (2) an x86-specific

knob to determine whether to use ‘movabs’ or not (‘-mmemcpy-strategy’

supposedly controls that, but ‘-mmemcpy-strategy=libcall:-1:noalign’

doesn’t seem to have any effect for instance.)

These knobs are not great because that would lead us to disable the

optimization wholesale, which is not desirable.

Other than that, I’ve also looked at all the movabs-related things in

gcc/config/i386/*.md, but to no avail.

To be continued…

Ludo’.

Leo Famulari wrote on 5 Nov 2016 19:36

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:20161105183650.GA21301@jasmine

On Sat, Nov 05, 2016 at 12:15:25AM +0100, Ludovic Courtès wrote:

Toggle quote (3 lines)

> I’ve fiddled a bit with GCC and read some code. No success yet, but

> here’s a status update.

Thanks for writing this out!

Toggle quote (6 lines)

> Instead, the knobs we have are (1) global flag to enable/disable each

> built-in function (like -fno-builtin-… does), and (2) an x86-specific

> knob to determine whether to use ‘movabs’ or not (‘-mmemcpy-strategy’

> supposedly controls that, but ‘-mmemcpy-strategy=libcall:-1:noalign’

> doesn’t seem to have any effect for instance.)

Please correct me if I paraphrase the choices incorrectly:

(1) Completely disable the strcpy optimization for all architectures

(2) Ostensibly change how strcpy is optimized on x86, except the knob

seems to have no effect

Toggle quote (3 lines)

> These knobs are not great because that would lead us to disable the

> optimization wholesale, which is not desirable.

What are the costs of (1)? Should we report (2) upstream?

Ludovic Courtès wrote on 6 Nov 2016 21:58

Recipients:(name . Leo Famulari)(address . leo@famulari.name)

Message-ID:87bmxsz5ow.fsf@gnu.org

Leo Famulari <leo@famulari.name> skribis:

Toggle quote (2 lines)

> On Sat, Nov 05, 2016 at 12:15:25AM +0100, Ludovic Courtès wrote:

[...]

Toggle quote (11 lines)>> Instead, the knobs we have are (1) global flag to enable/disable each
>> built-in function (like -fno-builtin-… does), and (2) an x86-specific
>> knob to determine whether to use ‘movabs’ or not (‘-mmemcpy-strategy’
>> supposedly controls that, but ‘-mmemcpy-strategy=libcall:-1:noalign’
>> doesn’t seem to have any effect for instance.)
>
> Please correct me if I paraphrase the choices incorrectly:
> (1) Completely disable the strcpy optimization for all architectures
> (2) Ostensibly change how strcpy is optimized on x86, except the knob
> seems to have no effect

Correct.

Toggle quote (5 lines)

>> These knobs are not great because that would lead us to disable the

>> optimization wholesale, which is not desirable.

> What are the costs of (1)? Should we report (2) upstream?

-fno-builtin-{strcpy,memcpy} disables not just use of ‘movabs’ but also
all the optimizations that GCC can make when the built-ins are being
used (see the doc for ‘-mmemcpy-strategy’).  It wouldn’t feel right to
disable that completely.

Ludo’.

Ludovic Courtès wrote on 9 Nov 2016 21:40

Recipients:(name . Leo Famulari)(address . leo@famulari.name)

Message-ID:87oa1o75ga.fsf@gnu.org

Hello!

I read much more code than I wanted just to end up in gcc/builtins.c.

In 8033772363b287ca529461e575ceb4a69d7af942 I added a patch for GCC 5.x

and 6.x that disables the ‘movabs’ optimization for strcpy/memcpy when

the source is a string constant that contains “/gnu/store” (I followed

Mark’s advice to disable the optimization for any string that contains

“/gnu/store”, rather than just for strings that start with

“/gnu/store”.)

This can be tested by compiling a file like this one (comment or

uncomment the lines that you want):

Toggle snippet (14 lines)#include <string.h>
void foo (char *x, char *y)
{
  /* memcpy(x, "this is a long string, a very long string", 42); */
  /* strcpy(x, "STRCPY THIS IS A LONG STRING, A VERY LONG STRING"); */
  strcpy(x, "STRCPY /gnu/store/THIS IS A LONG  STRING, A VERY LONG STRING");
  /* __builtin_strcpy(x, "THIS IS A LONG STRING, A VERY LONG STRING"); */
  /* strcpy(y, "/gnu/store/THIS IS A LONG STRING, A VERY LONG STRING"); */
  /* memcpy(y, "MEMCPY THIS IS A LONG STRING, A VERY LONG STRING", 30); */
  memcpy(y, "MEMCPY /gnu/store/THIS IS A LONG STRING, A VERY LONG STRING", 30);
  /* __builtin_memcpy(y, "/gnu/store/THIS IS A LONG STRING, A VERY LONG STRING", 30); */
}

… and then running “objdump -S foo.o | grep movabs”, for instance.

Now we need a plan to actually fix the bug.

The long-term goal is to rebuild everything with a compiler that has

this patch, in the next ‘core-updates’. We might as well switch to GCC

5 as the default compiler.

In the meantime, the only approach I can think of is to (1) ungraft more

frequently than we’ve done so far, and (2) when we ungraft a package,

add gcc@5 as an input such that it gets rebuilt without the problem.

Thoughts?

Thanks,

Ludo’.

Leo Famulari wrote on 10 Nov 2016 00:16

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:20161109231652.GA12060@jasmine

On Wed, Nov 09, 2016 at 09:40:05PM +0100, Ludovic Courtès wrote:

Toggle quote (28 lines)> I read much more code than I wanted just to end up in gcc/builtins.c.
> In 8033772363b287ca529461e575ceb4a69d7af942 I added a patch for GCC 5.x
> and 6.x that disables the ‘movabs’ optimization for strcpy/memcpy when
> the source is a string constant that contains “/gnu/store” (I followed
> Mark’s advice to disable the optimization for any string that contains
> “/gnu/store”, rather than just for strings that start with
> “/gnu/store”.)
> 
> This can be tested by compiling a file like this one (comment or
> uncomment the lines that you want):
> 
> --8<---------------cut here---------------start------------->8---
> #include <string.h>
> void foo (char *x, char *y)
> {
>   /* memcpy(x, "this is a long string, a very long string", 42); */
>   /* strcpy(x, "STRCPY THIS IS A LONG STRING, A VERY LONG STRING"); */
>   strcpy(x, "STRCPY /gnu/store/THIS IS A LONG  STRING, A VERY LONG STRING");
>   /* __builtin_strcpy(x, "THIS IS A LONG STRING, A VERY LONG STRING"); */
>   /* strcpy(y, "/gnu/store/THIS IS A LONG STRING, A VERY LONG STRING"); */
>   /* memcpy(y, "MEMCPY THIS IS A LONG STRING, A VERY LONG STRING", 30); */
>   memcpy(y, "MEMCPY /gnu/store/THIS IS A LONG STRING, A VERY LONG STRING", 30);
>   /* __builtin_memcpy(y, "/gnu/store/THIS IS A LONG STRING, A VERY LONG STRING", 30); */
> }
> --8<---------------cut here---------------end--------------->8---
> 
> … and then running “objdump -S foo.o | grep movabs”, for instance.

Awesome. Thank you!

Toggle quote (12 lines)> Now we need a plan to actually fix the bug.
> 
> The long-term goal is to rebuild everything with a compiler that has
> this patch, in the next ‘core-updates’.  We might as well switch to GCC
> 5 as the default compiler.
> 
> In the meantime, the only approach I can think of is to (1) ungraft more
> frequently than we’ve done so far, and (2) when we ungraft a package,
> add gcc@5 as an input such that it gets rebuilt without the problem.
> 
> Thoughts?

Sounds good to me. I wonder, with our current build farm, how often can
we do the full rebuilds required by ungrafting? I don't yet have a sense
of how long it takes.

Ludovic Courtès wrote on 10 Nov 2016 09:01

Recipients:(name . Leo Famulari)(address . leo@famulari.name)

Message-ID:87shqzyd9z.fsf@gnu.org

Leo Famulari <leo@famulari.name> skribis:

Toggle quote (2 lines)

> On Wed, Nov 09, 2016 at 09:40:05PM +0100, Ludovic Courtès wrote:

[...]

Toggle quote (14 lines)>> The long-term goal is to rebuild everything with a compiler that has
>> this patch, in the next ‘core-updates’.  We might as well switch to GCC
>> 5 as the default compiler.
>> 
>> In the meantime, the only approach I can think of is to (1) ungraft more
>> frequently than we’ve done so far, and (2) when we ungraft a package,
>> add gcc@5 as an input such that it gets rebuilt without the problem.
>> 
>> Thoughts?
>
> Sounds good to me. I wonder, with our current build farm, how often can
> we do the full rebuilds required by ungrafting? I don't yet have a sense
> of how long it takes.

I don’t know exactly. I have a feeling that the build farm has been

underused over the last couple of weeks for instance, but no precise

stats. I’d like to be able to gather usage stats so we can see when the

right time for rebuilds comes.

For now I think we’ll merge core-updates Real Soon, and then start a

staging branch or the python-updates one.

Ludo’.

Ludovic Courtès wrote on 11 Nov 2016 11:39

Recipients:(name . Leo Famulari)(address . leo@famulari.name)

Message-ID:87eg2i2tc3.fsf@gnu.org

ludo@gnu.org (Ludovic Courtès) skribis:

Toggle quote (4 lines)

> In the meantime, the only approach I can think of is to (1) ungraft more

> frequently than we’ve done so far, and (2) when we ungraft a package,

> add gcc@5 as an input such that it gets rebuilt without the problem.

As an example, I did that in commit

2c5ab05bffe5f89092ef60c3743b3941dcf92af0 for WordNet.

Ludo’.

Ludovic Courtès wrote on 3 Apr 2017 00:19

Recipients:(address . 24703-done@debbugs.gnu.org)

Message-ID:874ly68nb4.fsf@gnu.org

The patch to fix this bug was committed in ‘core-updates’ in

8033772363b287ca529461e575ceb4a69d7af942, which has now been merged.

Ludo’.

Closed

Your comment

This issue is archived.

To comment on this conversation send an email to 24703@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 24703

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date