Store references in 8-byte chunks in compiled code

  • Done
  • quality assurance status badge
Details
4 participants
  • Török Edwin
  • Leo Famulari
  • Ludovic Courtès
  • Mark H Weaver
Owner
unassigned
Submitted by
Mark H Weaver
Severity
serious
M
M
Mark H Weaver wrote on 16 Oct 2016 05:49
fontconfig keeps obfuscated reference to itself, not grafted
(address . bug-guix@gnu.org)
87mvi5lzqu.fsf@netris.org
After running "guix gc", I've found that my fonts are broken.

Gnome-terminal, Xfce4-terminal, and xterm are no longer able to find my
preferred "Dejavu Sans Mono" font. IceCat also uses a different
selection of fonts. Of the programs I've tried, only Emacs is able to
find them.

The failing programs output the following message to stderr:

Fontconfig error: Cannot load default config file

Strace reveals:

access("/gnu/store/b484nvn9nnr3ddclpz2fma9yxmimg2jj-fontconfig-2.11.94/etc/fonts/fonts.conf", R_OK) = -1 ENOENT (No such file or directory)
write(2, "Fontconfig error: ", 18Fontconfig error: ) = 18
write(2, "Cannot load default config file", 31Cannot load default config file) = 31

On my system, which includes a graft for 'bash' (not yet pushed to
Savannah), 'fontconfig' is grafted. After running 'guix gc', the
original ungrafted 'fontconfig' was removed, but libfontconfig is trying
to access it above.

It turns out there's an obfuscated self-reference to fontconfig's store
directory. Here's an excerpt of the output of "hexdump -C
libfontconfig.so.1.9.0":

0000cca0 00 48 b9 2f 67 6e 75 2f 73 74 6f c6 40 48 00 45 |.H./gnu/sto.@H.E|
0000ccb0 31 e4 48 89 08 48 b9 72 65 2f 62 34 38 34 6e 48 |1.H..H.re/b484nH|
0000ccc0 89 48 08 48 b9 76 6e 39 6e 6e 72 33 64 48 89 48 |.H.H.vn9nnr3dH.H|
0000ccd0 10 48 b9 64 63 6c 70 7a 32 66 6d 48 89 48 18 48 |.H.dclpz2fmH.H.H|
0000cce0 b9 61 39 79 78 6d 69 6d 67 48 89 48 20 48 b9 32 |.a9yxmimgH.H H.2|
0000ccf0 6a 6a 2d 66 6f 6e 74 48 89 48 28 48 b9 63 6f 6e |jj-fontH.H(H.con|
0000cd00 66 69 67 2d 32 48 89 48 30 48 b9 2e 31 31 2e 39 |fig-2H.H0H..11.9|
0000cd10 34 2f 65 48 89 48 38 48 b9 74 63 2f 66 6f 6e 74 |4/eH.H8H.tc/font|
0000cd20 73 48 89 48 40 48 8b 04 24 48 8b 18 48 89 c5 48 |sH.H@H..$H..H..H|

To be continued...

Mark
M
M
Mark H Weaver wrote on 16 Oct 2016 06:26
(address . 24703@debbugs.gnu.org)
87inssncln.fsf@netris.org
Mark H Weaver <mhw@netris.org> writes:

Toggle quote (14 lines)
> It turns out there's an obfuscated self-reference to fontconfig's store
> directory. Here's an excerpt of the output of "hexdump -C
> libfontconfig.so.1.9.0":
>
> 0000cca0 00 48 b9 2f 67 6e 75 2f 73 74 6f c6 40 48 00 45 |.H./gnu/sto.@H.E|
> 0000ccb0 31 e4 48 89 08 48 b9 72 65 2f 62 34 38 34 6e 48 |1.H..H.re/b484nH|
> 0000ccc0 89 48 08 48 b9 76 6e 39 6e 6e 72 33 64 48 89 48 |.H.H.vn9nnr3dH.H|
> 0000ccd0 10 48 b9 64 63 6c 70 7a 32 66 6d 48 89 48 18 48 |.H.dclpz2fmH.H.H|
> 0000cce0 b9 61 39 79 78 6d 69 6d 67 48 89 48 20 48 b9 32 |.a9yxmimgH.H H.2|
> 0000ccf0 6a 6a 2d 66 6f 6e 74 48 89 48 28 48 b9 63 6f 6e |jj-fontH.H(H.con|
> 0000cd00 66 69 67 2d 32 48 89 48 30 48 b9 2e 31 31 2e 39 |fig-2H.H0H..11.9|
> 0000cd10 34 2f 65 48 89 48 38 48 b9 74 63 2f 66 6f 6e 74 |4/eH.H8H.tc/font|
> 0000cd20 73 48 89 48 40 48 8b 04 24 48 8b 18 48 89 c5 48 |sH.H@H..$H..H..H|

It turns out that this is part of the compiled x86_64 code for
'FcConfigFilename' in src/fccfg.c, which copies a compile-time string
constant, 8 bytes at a time, into a buffer:

Toggle snippet (40 lines)
$ objdump -d libfontconfig.so.1.9.0 | grep -B1 -A35 '48 b9 2f 67 6e 75 2f'
cc9b: 0f 84 3f 01 00 00 je cde0 <FcConfigFilename+0x2d0>
cca1: 48 b9 2f 67 6e 75 2f movabs $0x6f74732f756e672f,%rcx
cca8: 73 74 6f
ccab: c6 40 48 00 movb $0x0,0x48(%rax)
ccaf: 45 31 e4 xor %r12d,%r12d
ccb2: 48 89 08 mov %rcx,(%rax)
ccb5: 48 b9 72 65 2f 62 34 movabs $0x6e343834622f6572,%rcx
ccbc: 38 34 6e
ccbf: 48 89 48 08 mov %rcx,0x8(%rax)
ccc3: 48 b9 76 6e 39 6e 6e movabs $0x6433726e6e396e76,%rcx
ccca: 72 33 64
cccd: 48 89 48 10 mov %rcx,0x10(%rax)
ccd1: 48 b9 64 63 6c 70 7a movabs $0x6d66327a706c6364,%rcx
ccd8: 32 66 6d
ccdb: 48 89 48 18 mov %rcx,0x18(%rax)
ccdf: 48 b9 61 39 79 78 6d movabs $0x676d696d78793961,%rcx
cce6: 69 6d 67
cce9: 48 89 48 20 mov %rcx,0x20(%rax)
cced: 48 b9 32 6a 6a 2d 66 movabs $0x746e6f662d6a6a32,%rcx
ccf4: 6f 6e 74
ccf7: 48 89 48 28 mov %rcx,0x28(%rax)
ccfb: 48 b9 63 6f 6e 66 69 movabs $0x322d6769666e6f63,%rcx
cd02: 67 2d 32
cd05: 48 89 48 30 mov %rcx,0x30(%rax)
cd09: 48 b9 2e 31 31 2e 39 movabs $0x652f34392e31312e,%rcx
cd10: 34 2f 65
cd13: 48 89 48 38 mov %rcx,0x38(%rax)
cd17: 48 b9 74 63 2f 66 6f movabs $0x73746e6f662f6374,%rcx
cd1e: 6e 74 73
cd21: 48 89 48 40 mov %rcx,0x40(%rax)
cd25: 48 8b 04 24 mov (%rsp),%rax
cd29: 48 8b 18 mov (%rax),%rbx
cd2c: 48 89 c5 mov %rax,%rbp
cd2f: 48 85 db test %rbx,%rbx
cd32: 48 89 df mov %rbx,%rdi
cd35: 75 16 jne cd4d <FcConfigFilename+0x23d>
cd37: eb 44 jmp cd7d <FcConfigFilename+0x26d>

So far, I've not been able to find any evidence of the fontconfig code
doing anything strange here. I strongly suspect that GCC is generating
this code, most likely due to an inlinable string/memory copy function
where the source is a string literal.

Obviously, this could be a serious problem for Guix (and Nix), since it
suggests that we may not be able to continue with our simplistic
assumption that references to the store in compiled code will be easy to
find and replace.

Mark
M
M
Mark H Weaver wrote on 16 Oct 2016 06:30
(no subject)
(address . control@debbugs.gnu.org)
87eg3gncfq.fsf@netris.org
retitle 24703 Store references in 8-byte chunks in compiled code
thanks
M
M
Mark H Weaver wrote on 16 Oct 2016 06:35
(address . control@debbugs.gnu.org)
877f98nc6q.fsf@netris.org
severity 24703 important
thanks
M
M
Mark H Weaver wrote on 16 Oct 2016 07:00
Re: bug#24703: fontconfig keeps obfuscated reference to itself, not grafted
(address . 24703@debbugs.gnu.org)
8737jwnb1c.fsf@netris.org
Mark H Weaver <mhw@netris.org> writes:

Toggle quote (64 lines)
> Mark H Weaver <mhw@netris.org> writes:
>
>> It turns out there's an obfuscated self-reference to fontconfig's store
>> directory. Here's an excerpt of the output of "hexdump -C
>> libfontconfig.so.1.9.0":
>>
>> 0000cca0 00 48 b9 2f 67 6e 75 2f 73 74 6f c6 40 48 00 45 |.H./gnu/sto.@H.E|
>> 0000ccb0 31 e4 48 89 08 48 b9 72 65 2f 62 34 38 34 6e 48 |1.H..H.re/b484nH|
>> 0000ccc0 89 48 08 48 b9 76 6e 39 6e 6e 72 33 64 48 89 48 |.H.H.vn9nnr3dH.H|
>> 0000ccd0 10 48 b9 64 63 6c 70 7a 32 66 6d 48 89 48 18 48 |.H.dclpz2fmH.H.H|
>> 0000cce0 b9 61 39 79 78 6d 69 6d 67 48 89 48 20 48 b9 32 |.a9yxmimgH.H H.2|
>> 0000ccf0 6a 6a 2d 66 6f 6e 74 48 89 48 28 48 b9 63 6f 6e |jj-fontH.H(H.con|
>> 0000cd00 66 69 67 2d 32 48 89 48 30 48 b9 2e 31 31 2e 39 |fig-2H.H0H..11.9|
>> 0000cd10 34 2f 65 48 89 48 38 48 b9 74 63 2f 66 6f 6e 74 |4/eH.H8H.tc/font|
>> 0000cd20 73 48 89 48 40 48 8b 04 24 48 8b 18 48 89 c5 48 |sH.H@H..$H..H..H|
>
> It turns out that this is part of the compiled x86_64 code for
> 'FcConfigFilename' in src/fccfg.c, which copies a compile-time string
> constant, 8 bytes at a time, into a buffer:
>
> $ objdump -d libfontconfig.so.1.9.0 | grep -B1 -A35 '48 b9 2f 67 6e 75 2f'
> cc9b: 0f 84 3f 01 00 00 je cde0 <FcConfigFilename+0x2d0>
> cca1: 48 b9 2f 67 6e 75 2f movabs $0x6f74732f756e672f,%rcx
> cca8: 73 74 6f
> ccab: c6 40 48 00 movb $0x0,0x48(%rax)
> ccaf: 45 31 e4 xor %r12d,%r12d
> ccb2: 48 89 08 mov %rcx,(%rax)
> ccb5: 48 b9 72 65 2f 62 34 movabs $0x6e343834622f6572,%rcx
> ccbc: 38 34 6e
> ccbf: 48 89 48 08 mov %rcx,0x8(%rax)
> ccc3: 48 b9 76 6e 39 6e 6e movabs $0x6433726e6e396e76,%rcx
> ccca: 72 33 64
> cccd: 48 89 48 10 mov %rcx,0x10(%rax)
> ccd1: 48 b9 64 63 6c 70 7a movabs $0x6d66327a706c6364,%rcx
> ccd8: 32 66 6d
> ccdb: 48 89 48 18 mov %rcx,0x18(%rax)
> ccdf: 48 b9 61 39 79 78 6d movabs $0x676d696d78793961,%rcx
> cce6: 69 6d 67
> cce9: 48 89 48 20 mov %rcx,0x20(%rax)
> cced: 48 b9 32 6a 6a 2d 66 movabs $0x746e6f662d6a6a32,%rcx
> ccf4: 6f 6e 74
> ccf7: 48 89 48 28 mov %rcx,0x28(%rax)
> ccfb: 48 b9 63 6f 6e 66 69 movabs $0x322d6769666e6f63,%rcx
> cd02: 67 2d 32
> cd05: 48 89 48 30 mov %rcx,0x30(%rax)
> cd09: 48 b9 2e 31 31 2e 39 movabs $0x652f34392e31312e,%rcx
> cd10: 34 2f 65
> cd13: 48 89 48 38 mov %rcx,0x38(%rax)
> cd17: 48 b9 74 63 2f 66 6f movabs $0x73746e6f662f6374,%rcx
> cd1e: 6e 74 73
> cd21: 48 89 48 40 mov %rcx,0x40(%rax)
> cd25: 48 8b 04 24 mov (%rsp),%rax
> cd29: 48 8b 18 mov (%rax),%rbx
> cd2c: 48 89 c5 mov %rax,%rbp
> cd2f: 48 85 db test %rbx,%rbx
> cd32: 48 89 df mov %rbx,%rdi
> cd35: 75 16 jne cd4d <FcConfigFilename+0x23d>
> cd37: eb 44 jmp cd7d <FcConfigFilename+0x26d>
>
> So far, I've not been able to find any evidence of the fontconfig code
> doing anything strange here. I strongly suspect that GCC is generating
> this code, most likely due to an inlinable string/memory copy function
> where the source is a string literal.

I've confirmed this. After building this package manually, "objdump -d
--source src/.libs/fccfg.o" reveals that the corresponding source code
is:

dir = (FcChar8 *) FONTCONFIG_PATH;
path[i] = malloc (strlen ((char *) dir) + 1);
if (!path[i])
goto bail1;
strcpy ((char *) path[i], (const char *) dir);

It is part of 'FcConfigGetPath', inlined into 'FcConfigFilename', in
src/fccfg.c. -DFONTCONFIG_PATH='"$(BASECONFIGDIR)"' is one of the flags
passed to GCC, via AM_CPPFLAGS in src/Makefile.am.

Toggle quote (5 lines)
> Obviously, this could be a serious problem for Guix (and Nix), since it
> suggests that we may not be able to continue with our simplistic
> assumption that references to the store in compiled code will be easy to
> find and replace.

Mark
M
M
Mark H Weaver wrote on 16 Oct 2016 07:10
(no subject)
(address . control@debbugs.gnu.org)
87y41olvz1.fsf@netris.org
severity 24703 serious
thanks
M
M
Mark H Weaver wrote on 16 Oct 2016 08:24
Re: Store references in 8-byte chunks in compiled code
(address . 24703@debbugs.gnu.org)
87r37gstf6.fsf_-_@netris.org
Mark H Weaver <mhw@netris.org> writes:

Toggle quote (85 lines)
> Mark H Weaver <mhw@netris.org> writes:
>
>> Mark H Weaver <mhw@netris.org> writes:
>>
>>> It turns out there's an obfuscated self-reference to fontconfig's store
>>> directory. Here's an excerpt of the output of "hexdump -C
>>> libfontconfig.so.1.9.0":
>>>
>>> 0000cca0 00 48 b9 2f 67 6e 75 2f 73 74 6f c6 40 48 00 45 |.H./gnu/sto.@H.E|
>>> 0000ccb0 31 e4 48 89 08 48 b9 72 65 2f 62 34 38 34 6e 48 |1.H..H.re/b484nH|
>>> 0000ccc0 89 48 08 48 b9 76 6e 39 6e 6e 72 33 64 48 89 48 |.H.H.vn9nnr3dH.H|
>>> 0000ccd0 10 48 b9 64 63 6c 70 7a 32 66 6d 48 89 48 18 48 |.H.dclpz2fmH.H.H|
>>> 0000cce0 b9 61 39 79 78 6d 69 6d 67 48 89 48 20 48 b9 32 |.a9yxmimgH.H H.2|
>>> 0000ccf0 6a 6a 2d 66 6f 6e 74 48 89 48 28 48 b9 63 6f 6e |jj-fontH.H(H.con|
>>> 0000cd00 66 69 67 2d 32 48 89 48 30 48 b9 2e 31 31 2e 39 |fig-2H.H0H..11.9|
>>> 0000cd10 34 2f 65 48 89 48 38 48 b9 74 63 2f 66 6f 6e 74 |4/eH.H8H.tc/font|
>>> 0000cd20 73 48 89 48 40 48 8b 04 24 48 8b 18 48 89 c5 48 |sH.H@H..$H..H..H|
>>
>> It turns out that this is part of the compiled x86_64 code for
>> 'FcConfigFilename' in src/fccfg.c, which copies a compile-time string
>> constant, 8 bytes at a time, into a buffer:
>>
>> $ objdump -d libfontconfig.so.1.9.0 | grep -B1 -A35 '48 b9 2f 67 6e 75 2f'
>> cc9b: 0f 84 3f 01 00 00 je cde0 <FcConfigFilename+0x2d0>
>> cca1: 48 b9 2f 67 6e 75 2f movabs $0x6f74732f756e672f,%rcx
>> cca8: 73 74 6f
>> ccab: c6 40 48 00 movb $0x0,0x48(%rax)
>> ccaf: 45 31 e4 xor %r12d,%r12d
>> ccb2: 48 89 08 mov %rcx,(%rax)
>> ccb5: 48 b9 72 65 2f 62 34 movabs $0x6e343834622f6572,%rcx
>> ccbc: 38 34 6e
>> ccbf: 48 89 48 08 mov %rcx,0x8(%rax)
>> ccc3: 48 b9 76 6e 39 6e 6e movabs $0x6433726e6e396e76,%rcx
>> ccca: 72 33 64
>> cccd: 48 89 48 10 mov %rcx,0x10(%rax)
>> ccd1: 48 b9 64 63 6c 70 7a movabs $0x6d66327a706c6364,%rcx
>> ccd8: 32 66 6d
>> ccdb: 48 89 48 18 mov %rcx,0x18(%rax)
>> ccdf: 48 b9 61 39 79 78 6d movabs $0x676d696d78793961,%rcx
>> cce6: 69 6d 67
>> cce9: 48 89 48 20 mov %rcx,0x20(%rax)
>> cced: 48 b9 32 6a 6a 2d 66 movabs $0x746e6f662d6a6a32,%rcx
>> ccf4: 6f 6e 74
>> ccf7: 48 89 48 28 mov %rcx,0x28(%rax)
>> ccfb: 48 b9 63 6f 6e 66 69 movabs $0x322d6769666e6f63,%rcx
>> cd02: 67 2d 32
>> cd05: 48 89 48 30 mov %rcx,0x30(%rax)
>> cd09: 48 b9 2e 31 31 2e 39 movabs $0x652f34392e31312e,%rcx
>> cd10: 34 2f 65
>> cd13: 48 89 48 38 mov %rcx,0x38(%rax)
>> cd17: 48 b9 74 63 2f 66 6f movabs $0x73746e6f662f6374,%rcx
>> cd1e: 6e 74 73
>> cd21: 48 89 48 40 mov %rcx,0x40(%rax)
>> cd25: 48 8b 04 24 mov (%rsp),%rax
>> cd29: 48 8b 18 mov (%rax),%rbx
>> cd2c: 48 89 c5 mov %rax,%rbp
>> cd2f: 48 85 db test %rbx,%rbx
>> cd32: 48 89 df mov %rbx,%rdi
>> cd35: 75 16 jne cd4d <FcConfigFilename+0x23d>
>> cd37: eb 44 jmp cd7d <FcConfigFilename+0x26d>
>>
>> So far, I've not been able to find any evidence of the fontconfig code
>> doing anything strange here. I strongly suspect that GCC is generating
>> this code, most likely due to an inlinable string/memory copy function
>> where the source is a string literal.
>
> I've confirmed this. After building this package manually, "objdump -d
> --source src/.libs/fccfg.o" reveals that the corresponding source code
> is:
>
> dir = (FcChar8 *) FONTCONFIG_PATH;
> path[i] = malloc (strlen ((char *) dir) + 1);
> if (!path[i])
> goto bail1;
> strcpy ((char *) path[i], (const char *) dir);
>
> It is part of 'FcConfigGetPath', inlined into 'FcConfigFilename', in
> src/fccfg.c. -DFONTCONFIG_PATH='"$(BASECONFIGDIR)"' is one of the flags
> passed to GCC, via AM_CPPFLAGS in src/Makefile.am.
>
>> Obviously, this could be a serious problem for Guix (and Nix), since it
>> suggests that we may not be able to continue with our simplistic
>> assumption that references to the store in compiled code will be easy to
>> find and replace.

To get an idea of how widespread this problem is, I searched for
occurrences of /gnu/sto[^r] on my GNOME/Xfce desktop system:

$ LC_ALL=C grep '/gnu/sto[^r]' --recursive /gnu/store/
[.../gnu/store/.links hits removed...]
Binary file /gnu/store/0010wvgs40kdq8chzsh403qm7la9jxq7-bash-static-4.3.42/bin/bash matches
Binary file /gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/sbin/sln matches
Binary file /gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/lib/libc-2.23.so matches
Binary file /gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/lib/libc.a matches
Binary file /gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/bin/locale matches
Binary file /gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/bin/localedef matches
Binary file /gnu/store/4fs8dg5jhf99xl3ikl093dn1va4wlv79-fontconfig-2.11.94/lib/libfontconfig.so.1.9.0 matches
Binary file /gnu/store/b484nvn9nnr3ddclpz2fma9yxmimg2jj-fontconfig-2.11.94/lib/libfontconfig.so.1.9.0 matches
Binary file /gnu/store/80m11l8km7bwi9iljfyr82hmscaq4xk2-unixodbc-2.3.4/lib/libodbcinst.so.2.0.0 matches
Binary file /gnu/store/80m11l8km7bwi9iljfyr82hmscaq4xk2-unixodbc-2.3.4/lib/libodbc.so.2.0.0 matches
Binary file /gnu/store/ld16jy012l3jpkj6azynzmldxn28cspw-ncurses-6.0/lib/libncursesw.so.6.0 matches
Binary file /gnu/store/ld16jy012l3jpkj6azynzmldxn28cspw-ncurses-6.0/lib/libncursesw.a matches
Binary file /gnu/store/lxv20br9ic8abvqd7fipszbs4mg8hkjv-ncurses-6.0/lib/libncursesw.so.6.0 matches
Binary file /gnu/store/lxv20br9ic8abvqd7fipszbs4mg8hkjv-ncurses-6.0/lib/libncursesw.a matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/gcc matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-gcc-4.9.3 matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-c++ matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-gcc matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/g++ matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-g++ matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/cpp matches
Binary file /gnu/store/frrj3bfbmg5vrd0flh9cf8j64h7cr2v4-gcc-4.9.3/bin/c++ matches
Binary file /gnu/store/sxa3b6l28ckcnyg3g7f4brvl2rdpncy1-gnuplot-5.0.4-1/bin/gnuplot matches
Binary file /gnu/store/2jvvwqz16hj2c5ws0xa46x98fvl9m90m-fontconfig-2.11.94/lib/libfontconfig.so.1.9.0 matches
Binary file /gnu/store/bxy1hwriqzdw6xk7bl28qfsw4s1s5xdq-bash-static-4.3.48/bin/bash matches
Binary file /gnu/store/3chnjjkazbv0fqbshrwahq7c3zfg42s8-ncurses-6.0/lib/libncursesw.so.6.0 matches
Binary file /gnu/store/3chnjjkazbv0fqbshrwahq7c3zfg42s8-ncurses-6.0/lib/libncursesw.a matches
Binary file /gnu/store/my7f7fq2ca5rqq4wyyrg20cw2bjrj2l4-ncurses-6.0/lib/libncursesw.so.6.0 matches
Binary file /gnu/store/my7f7fq2ca5rqq4wyyrg20cw2bjrj2l4-ncurses-6.0/lib/libncursesw.a matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/gcc matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-gcc-4.9.3 matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-c++ matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-gcc matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/g++ matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-g++ matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/cpp matches
Binary file /gnu/store/dmz9v8bmxd3davz77s4b10pmpmjnv98a-gcc-4.9.3/bin/c++ matches
Binary file /gnu/store/fjsaprcdmdn39pk39jrhbby1jl5i8rp5-xorg-server-1.18.1/bin/Xorg matches
Binary file /gnu/store/mz301kb7wqvyl9kxil4bpn8ng99ikgqy-glibc-2.23/sbin/sln matches
Binary file /gnu/store/mz301kb7wqvyl9kxil4bpn8ng99ikgqy-glibc-2.23/lib/libc-2.23.so matches
Binary file /gnu/store/mz301kb7wqvyl9kxil4bpn8ng99ikgqy-glibc-2.23/lib/libc.a matches
Binary file /gnu/store/mz301kb7wqvyl9kxil4bpn8ng99ikgqy-glibc-2.23/bin/locale matches
Binary file /gnu/store/mz301kb7wqvyl9kxil4bpn8ng99ikgqy-glibc-2.23/bin/localedef matches
Binary file /gnu/store/slhwk75g7d8bywpq2hifs7g9fxr6jx3d-bash-static-4.3.48/bin/bash matches
Binary file /gnu/store/qjsmp5s85qrba18fxf319m5lv7f8awf8-graphviz-2.38.0/bin/lefty matches
Binary file /gnu/store/6ddfp4pwlrxc5jdgf17ddh4m5wi1cldy-xorg-server-1.18.1/bin/Xorg matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/gcc matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-gcc-4.9.3 matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-c++ matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-gcc matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/g++ matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/x86_64-unknown-linux-gnu-g++ matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/cpp matches
Binary file /gnu/store/zav1zqwmzzz5xk71v22i7n6qidwh49in-gcc-4.9.3/bin/c++ matches
Binary file /gnu/store/xl19qrfzga52vrvp4ncccwjlnrjqwj95-ncurses-6.0/lib/libncursesw.so.6.0 matches
Binary file /gnu/store/xl19qrfzga52vrvp4ncccwjlnrjqwj95-ncurses-6.0/lib/libncursesw.a matches
Binary file /gnu/store/kh3awka9xslyp52dldb3gma47rr0kp2x-xorg-server-1.18.1/bin/Xorg matches
Binary file /gnu/store/121596cgx25s8zcl3yznyh2vh1f842ni-babl-0.1.18/lib/libbabl-0.1.so.0.117.1 matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-gcc-6.2.0 matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/gcc matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-c++ matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-gcc matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/g++ matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-g++ matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/cpp matches
Binary file /gnu/store/srfjnfkmjkc4xcld311xsdvhng08mmpi-gcc-6.2.0/bin/c++ matches
Binary file /gnu/store/j4q20kwzd5g1d3gv419692k66ghfzymz-gnuplot-5.0.4-1/bin/gnuplot matches
Binary file /gnu/store/mdc84lh0mfzw9n404cnzi9l1l8qr7a4r-gnuplot-5.0.4-1/bin/gnuplot matches
Binary file /gnu/store/8mj5yd1z936j64sdpx3hbqi3qkdif0c4-alsa-lib-1.0.27.1/lib/libasound.so.2.0.0 matches
Binary file /gnu/store/ninvaqcyhm6s11yp97m17h2i1q3aj24s-recode-3.7.0.201402/lib/librecode.a matches
Binary file /gnu/store/ninvaqcyhm6s11yp97m17h2i1q3aj24s-recode-3.7.0.201402/lib/librecode.so.0.0.0 matches
Binary file /gnu/store/kjybzn7az86n1qzxcm8zdz2gaypp4az6-fontconfig-2.11.94/lib/libfontconfig.so.1.9.0 matches
Binary file /gnu/store/cl2vwkvmk60s7vpamivpclzgyfxlb7wx-graphviz-2.38.0/bin/lefty matches
Binary file /gnu/store/qc8qg71k1b7gizqxa785c6ls71i8qk6d-units-2.13/bin/units matches
Binary file /gnu/store/sj8ygx2yz58hn1142yjjsb34sql4b9xr-unixodbc-2.3.4/lib/libodbcinst.so.2.0.0 matches
Binary file /gnu/store/sj8ygx2yz58hn1142yjjsb34sql4b9xr-unixodbc-2.3.4/lib/libodbc.so.2.0.0 matches
Binary file /gnu/store/571c58j8f06x8svykg4n5s0ip36kna5c-util-linux-2.27/lib/libuuid.so.1.3.0 matches
Binary file /gnu/store/1i3xmm18dw9kq6wi46f6sj9nxy9pckjl-alsa-lib-1.0.27.1/lib/libasound.so.2.0.0 matches
Binary file /gnu/store/x1kh4kkifc3f4gnlwvfgk53fm2zhfhmm-gnuplot-5.0.4-1/bin/gnuplot matches
Binary file /gnu/store/fnw55giyr1gnqyyw5yx4hf96mlrkp603-gnuplot-5.0.4-1/bin/gnuplot matches
Binary file /gnu/store/mj727bz0l8afmn97l8gsc5wh30jnql8s-babl-0.1.18/lib/libbabl-0.1.so.0.117.1 matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-gcc-6.2.0 matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/gcc matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-c++ matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-gcc matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/g++ matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/x86_64-unknown-linux-gnu-g++ matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/cpp matches
Binary file /gnu/store/0m908gszml9bb6vkikbzdqpslah2a1db-gcc-6.2.0/bin/c++ matches
Binary file /gnu/store/lqybibdq19q6ypm29p9p9s9ns0s66n0b-recode-3.7.0.201402/lib/librecode.a matches
Binary file /gnu/store/lqybibdq19q6ypm29p9p9s9ns0s66n0b-recode-3.7.0.201402/lib/librecode.so.0.0.0 matches
Binary file /gnu/store/pxdl28wbikj2jy0jjldmii245gcsh5fq-fontconfig-2.11.94/lib/libfontconfig.so.1.9.0 matches
Binary file /gnu/store/lhxgw5ynrccx9vvk9jx2k9pdsgq0yxm3-units-2.13/bin/units matches
Binary file /gnu/store/cji1mwp7n6zhdnpg73gighf3fvrh2gdi-ncurses-6.0/lib/libncursesw.so.6.0 matches
Binary file /gnu/store/cji1mwp7n6zhdnpg73gighf3fvrh2gdi-ncurses-6.0/lib/libncursesw.a matches
Binary file /gnu/store/1rcsqn8h2xwmgdra3zr33xv73d44wf1s-fontconfig-2.11.94/lib/libfontconfig.so.1.9.0 matches
Binary file /gnu/store/iiakyhsw889rvv3ghk6jl3mfdrs1wfan-fontconfig-2.11.94/lib/libfontconfig.so.1.9.0 matches
Binary file /gnu/store/zb3kh929v8yhkyg3sn8405zj4x49aza4-gnuplot-5.0.4-1/bin/gnuplot matches
Binary file /gnu/store/mkq7h60l1kx4hjclsrd0nbz8v4mnx4lv-gnuplot-5.0.4-1/bin/gnuplot matches
Binary file /gnu/store/4vhgx5xkbx0x7gnph9fq3c581rnj5ynq-libgphoto2-2.5.2/lib/libgphoto2/2.5.2/konica.so matches
Binary file /gnu/store/r6kwxzban57ghxsy8p8dqjvym6vnb2nz-graphviz-2.38.0/bin/lefty matches
/gnu/store/lwxifldgdyyzd83510nzf1qffpzxbdyl-guix-0.11.0-1.4420/share/guile/site/2.0/gnu/packages/package-management.scm: (uri (string-append "mirror://gnu/stow/stow-"
Binary file /gnu/store/lwxifldgdyyzd83510nzf1qffpzxbdyl-guix-0.11.0-1.4420/share/guile/site/2.0/gnu/packages/package-management.go matches
Binary file /gnu/store/zslzcw2wg6xylwwr9hcx2sbal0bm6kjx-util-linux-2.27/lib/libuuid.so.1.3.0 matches
M
M
Mark H Weaver wrote on 16 Oct 2016 11:03
Re: bug#24703: Store references in 8-byte chunks in compiled code
(address . 24703@debbugs.gnu.org)
87k2d8sm1q.fsf@netris.org
The 8-byte chunks may appear out of order. For example, in
'bash-static' we have this reference to:

/gnu/store/7z3hpynjsbidxkq78xi5qi6lbcm8ndhp-glibc-intermediate-2.23

where the chunks are found in the following order:

1_/gnu/sto
2_________________ynjsbidx
3_________re/7z3hp
4_________________________________i6lbcm8n
5_________________________kq78xi5q
6_________________________________________dhp-glib
7_________________________________________________c-interm
8_________________________________________________________ediate-2
9_________________________________________________________________.23/lib/

Here's an excerpt of "objdump -d" output, annotated to show the 8-byte
constant as a string, and its position within the larger string.

======== /gnu/store/0010wvgs40kdq8chzsh403qm7la9jxq7-bash-static-4.3.42/bin/bash
46b2c3: 48 b9 2f 67 6e 75 2f movabs $0x6f74732f756e672f,%rcx ; "/gnu/sto" (1)
46b2ca: 73 74 6f
46b2cd: 48 01 d8 add %rbx,%rax
46b2d0: 48 bb 79 6e 6a 73 62 movabs $0x78646962736a6e79,%rbx ; "ynjsbidx" (3)
46b2d7: 69 64 78
46b2da: 48 89 48 01 mov %rcx,0x1(%rax)
46b2de: 48 b9 72 65 2f 37 7a movabs $0x7068337a372f6572,%rcx ; "re/7z3hp" (2)
46b2e5: 33 68 70
46b2e8: 48 89 58 11 mov %rbx,0x11(%rax)
46b2ec: 48 89 48 09 mov %rcx,0x9(%rax)
46b2f0: 48 bb 69 36 6c 62 63 movabs $0x6e386d63626c3669,%rbx ; "i6lbcm8n" (5)
46b2f7: 6d 38 6e
46b2fa: 48 b9 6b 71 37 38 78 movabs $0x713569783837716b,%rcx ; "kq78xi5q" (4)
46b301: 69 35 71
46b304: 48 89 48 19 mov %rcx,0x19(%rax)
46b308: 48 89 58 21 mov %rbx,0x21(%rax)
46b30c: 48 b9 64 68 70 2d 67 movabs $0x62696c672d706864,%rcx ; "dhp-glib" (6)
46b313: 6c 69 62
46b316: 48 bb 63 2d 69 6e 74 movabs $0x6d7265746e692d63,%rbx ; "c-interm" (7)
46b31d: 65 72 6d
46b320: 48 89 48 29 mov %rcx,0x29(%rax)
46b324: ba 76 00 00 00 mov $0x76,%edx
46b329: 48 89 58 31 mov %rbx,0x31(%rax)
46b32d: 48 b9 65 64 69 61 74 movabs $0x322d657461696465,%rcx ; "ediate-2" (8)
46b334: 65 2d 32
46b337: 48 bb 2e 32 33 2f 6c movabs $0x2f62696c2f33322e,%rbx ; ".23/lib/" (9)
46b33e: 69 62 2f
46b341: 48 89 58 41 mov %rbx,0x41(%rax)
46b345: 31 f6 xor %esi,%esi
46b347: 31 ff xor %edi,%edi
46b349: c6 00 3a movb $0x3a,(%rax)
46b34c: 48 89 48 39 mov %rcx,0x39(%rax)
46b350: c7 40 49 67 63 6f 6e movl $0x6e6f6367,0x49(%rax)
46b357: 66 89 50 4d mov %dx,0x4d(%rax)
M
M
Mark H Weaver wrote on 16 Oct 2016 11:25
(address . 24703@debbugs.gnu.org)
87d1j0sl1l.fsf@netris.org
Here's a complex example of a reference to:

/gnu/store/80m11l8km7bwi9iljfyr82hmscaq4xk2-unixodbc-2.3.4/etc

This also illustrates what can happen near the end of a reference. In
this case, the last two characters are found at the end, within a 'mov'
instruction, and the previous 4 characters are found near the beginning,
after the first 8-byte chunk.

======== /gnu/store/80m11l8km7bwi9iljfyr82hmscaq4xk2-unixodbc-2.3.4/lib/libodbcinst.so.2.0.0
8238: 48 b8 2f 67 6e 75 2f movabs $0x6f74732f756e672f,%rax ; "/gnu/sto"
823f: 73 74 6f
8242: c7 05 6c fa 20 00 2e movl $0x652f342e,0x20fa6c(%rip) ; ".4/e"
8249: 34 2f 65
824c: c6 05 6b fa 20 00 00 movb $0x0,0x20fa6b(%rip)
8253: 48 89 05 26 fa 20 00 mov %rax,0x20fa26(%rip)
825a: 48 b8 72 65 2f 38 30 movabs $0x31316d30382f6572,%rax ; "re/80m11"
8261: 6d 31 31
8264: c7 05 16 0a 21 00 01 movl $0x1,0x210a16(%rip)
826b: 00 00 00
826e: 48 89 05 13 fa 20 00 mov %rax,0x20fa13(%rip)
8275: 48 b8 6c 38 6b 6d 37 movabs $0x697762376d6b386c,%rax ; "l8km7bwi"
827c: 62 77 69
827f: 48 8d 1d 42 94 00 00 lea 0x9442(%rip),%rbx
8286: 48 89 05 03 fa 20 00 mov %rax,0x20fa03(%rip)
828d: 48 b8 39 69 6c 6a 66 movabs $0x387279666a6c6939,%rax ; "9iljfyr8"
8294: 79 72 38
8297: 48 89 05 fa f9 20 00 mov %rax,0x20f9fa(%rip)
829e: 48 b8 32 68 6d 73 63 movabs $0x34716163736d6832,%rax ; "2hmscaq4"
82a5: 61 71 34
82a8: 48 89 05 f1 f9 20 00 mov %rax,0x20f9f1(%rip)
82af: 48 b8 78 6b 32 2d 75 movabs $0x78696e752d326b78,%rax ; "xk2-unix"
82b6: 6e 69 78
82b9: 48 89 05 e8 f9 20 00 mov %rax,0x20f9e8(%rip)
82c0: 48 b8 6f 64 62 63 2d movabs $0x332e322d6362646f,%rax ; "odbc-2.3"
82c7: 32 2e 33
82ca: 48 89 05 df f9 20 00 mov %rax,0x20f9df(%rip)
82d1: b8 74 63 00 00 mov $0x6374,%eax ; "tc"
82d6: 66 89 05 df f9 20 00 mov %ax,0x20f9df(%rip)

Now imagine what will be required to graft cases like this. Consider
what will happen for strings with an odd number of characters. The last
character might be all by itself.

When grafting, how will we achieve confidence that we've found the
correct occurrence of the last character? I think we will have to give
up our recently added feature of being able to change the version number
of grafts.

Oh, one more thing: I forgot to mention in my previous email, where
'bash-static' refers to 'glibc-intermediate', that on my system the
referenced 'glibc-intermediate' has been garbage-collected from my
system, so that's a real-world example of a broken link, and it happened
to be the first one I investigated :-(

Mark
M
M
Mark H Weaver wrote on 16 Oct 2016 12:15
(address . 24703@debbugs.gnu.org)
8760ossiqf.fsf@netris.org
Here's what happens with a variant of 'unixodbc' with one character
removed from its name, to make an odd number of characters. The last
character 'c' is all by itself. The chunks of the reference

/gnu/store/il1bn7n0l4yj3idrii23fhvzg4nn939i-unxodbc-2.3.4/etc

are found in the following order:

1_/gnu/sto
2_________________________________________________________4/et
3_________re/il1bn
4_________________7n0l4yj3
5_________________________idrii23f
6_________________________________hvzg4nn9
7_________________________________________39i-unxo
8_________________________________________________dbc-2.3.
9_____________________________________________________________c

======== /gnu/store/y2dwj5vkbxka7y5rlakprbk4k8vrd5ij-unxodbc-2.3.4/lib/libodbcinst.so.2.0.0
8238: 48 b8 2f 67 6e 75 2f movabs $0x6f74732f756e672f,%rax ; "/gnu/sto"
823f: 73 74 6f
8242: c7 05 6c fa 20 00 34 movl $0x74652f34,0x20fa6c(%rip) ; "4/et"
8249: 2f 65 74
824c: c7 05 2e 0a 21 00 01 movl $0x1,0x210a2e(%rip)
8253: 00 00 00
8256: 48 89 05 23 fa 20 00 mov %rax,0x20fa23(%rip)
825d: 48 b8 72 65 2f 69 6c movabs $0x6e62316c692f6572,%rax ; "re/il1bn"
8264: 31 62 6e
8267: 48 8d 1d 5a 94 00 00 lea 0x945a(%rip),%rbx
826e: 48 89 05 13 fa 20 00 mov %rax,0x20fa13(%rip)
8275: 48 b8 37 6e 30 6c 34 movabs $0x336a79346c306e37,%rax ; "7n0l4yj3"
827c: 79 6a 33
827f: 48 89 05 0a fa 20 00 mov %rax,0x20fa0a(%rip)
8286: 48 b8 69 64 72 69 69 movabs $0x6633326969726469,%rax ; "idrii23f"
828d: 32 33 66
8290: 48 89 05 01 fa 20 00 mov %rax,0x20fa01(%rip)
8297: 48 b8 68 76 7a 67 34 movabs $0x396e6e34677a7668,%rax ; "hvzg4nn9"
829e: 6e 6e 39
82a1: 48 89 05 f8 f9 20 00 mov %rax,0x20f9f8(%rip)
82a8: 48 b8 33 39 69 2d 75 movabs $0x6f786e752d693933,%rax ; "39i-unxo"
82af: 6e 78 6f
82b2: 48 89 05 ef f9 20 00 mov %rax,0x20f9ef(%rip)
82b9: 48 b8 64 62 63 2d 32 movabs $0x2e332e322d636264,%rax ; "dbc-2.3."
82c0: 2e 33 2e
82c3: 48 89 05 e6 f9 20 00 mov %rax,0x20f9e6(%rip)
82ca: b8 63 00 00 00 mov $0x63,%eax ; "c"
82cf: 66 89 05 e6 f9 20 00 mov %ax,0x20f9e6(%rip)
L
L
Ludovic Courtès wrote on 16 Oct 2016 16:42
Re: bug#24703: fontconfig keeps obfuscated reference to itself, not grafted
(name . Mark H Weaver)(address . mhw@netris.org)(address . 24703@debbugs.gnu.org)
87oa2kgxti.fsf@gnu.org
Mark H Weaver <mhw@netris.org> skribis:

Toggle quote (14 lines)
> It turns out there's an obfuscated self-reference to fontconfig's store
> directory. Here's an excerpt of the output of "hexdump -C
> libfontconfig.so.1.9.0":
>
> 0000cca0 00 48 b9 2f 67 6e 75 2f 73 74 6f c6 40 48 00 45 |.H./gnu/sto.@H.E|
> 0000ccb0 31 e4 48 89 08 48 b9 72 65 2f 62 34 38 34 6e 48 |1.H..H.re/b484nH|
> 0000ccc0 89 48 08 48 b9 76 6e 39 6e 6e 72 33 64 48 89 48 |.H.H.vn9nnr3dH.H|
> 0000ccd0 10 48 b9 64 63 6c 70 7a 32 66 6d 48 89 48 18 48 |.H.dclpz2fmH.H.H|
> 0000cce0 b9 61 39 79 78 6d 69 6d 67 48 89 48 20 48 b9 32 |.a9yxmimgH.H H.2|
> 0000ccf0 6a 6a 2d 66 6f 6e 74 48 89 48 28 48 b9 63 6f 6e |jj-fontH.H(H.con|
> 0000cd00 66 69 67 2d 32 48 89 48 30 48 b9 2e 31 31 2e 39 |fig-2H.H0H..11.9|
> 0000cd10 34 2f 65 48 89 48 38 48 b9 74 63 2f 66 6f 6e 74 |4/eH.H8H.tc/font|
> 0000cd20 73 48 89 48 40 48 8b 04 24 48 8b 18 48 89 c5 48 |sH.H@H..$H..H..H|

See commit 9d50da70608de32d9db0c29859caec6f2cddb95f in WordNet:
-fno-builtin-strcpy is the solution I found.

Weird, no? :-)

Ludo’.
L
L
Ludovic Courtès wrote on 16 Oct 2016 17:06
(name . Mark H Weaver)(address . mhw@netris.org)(address . 24703@debbugs.gnu.org)
87eg3ggwog.fsf@gnu.org
ludo@gnu.org (Ludovic Courtès) skribis:

Toggle quote (19 lines)
> Mark H Weaver <mhw@netris.org> skribis:
>
>> It turns out there's an obfuscated self-reference to fontconfig's store
>> directory. Here's an excerpt of the output of "hexdump -C
>> libfontconfig.so.1.9.0":
>>
>> 0000cca0 00 48 b9 2f 67 6e 75 2f 73 74 6f c6 40 48 00 45 |.H./gnu/sto.@H.E|
>> 0000ccb0 31 e4 48 89 08 48 b9 72 65 2f 62 34 38 34 6e 48 |1.H..H.re/b484nH|
>> 0000ccc0 89 48 08 48 b9 76 6e 39 6e 6e 72 33 64 48 89 48 |.H.H.vn9nnr3dH.H|
>> 0000ccd0 10 48 b9 64 63 6c 70 7a 32 66 6d 48 89 48 18 48 |.H.dclpz2fmH.H.H|
>> 0000cce0 b9 61 39 79 78 6d 69 6d 67 48 89 48 20 48 b9 32 |.a9yxmimgH.H H.2|
>> 0000ccf0 6a 6a 2d 66 6f 6e 74 48 89 48 28 48 b9 63 6f 6e |jj-fontH.H(H.con|
>> 0000cd00 66 69 67 2d 32 48 89 48 30 48 b9 2e 31 31 2e 39 |fig-2H.H0H..11.9|
>> 0000cd10 34 2f 65 48 89 48 38 48 b9 74 63 2f 66 6f 6e 74 |4/eH.H8H.tc/font|
>> 0000cd20 73 48 89 48 40 48 8b 04 24 48 8b 18 48 89 c5 48 |sH.H@H..$H..H..H|
>
> See commit 9d50da70608de32d9db0c29859caec6f2cddb95f in WordNet:
> -fno-builtin-strcpy is the solution I found.

Simple test case:
#include <string.h>
void foo (char *bar)
{
strcpy (bar, "this is a very long string that's gonna be split");
}
Run “gcc -c t.c” with and without -fno-builtin-strcpy.

Ludo’.
L
L
Ludovic Courtès wrote on 16 Oct 2016 21:04
Re: bug#24703: Store references in 8-byte chunks in compiled code
(name . Mark H Weaver)(address . mhw@netris.org)(address . 24703@debbugs.gnu.org)
87a8e4glot.fsf@gnu.org
Mark H Weaver <mhw@netris.org> skribis:

Toggle quote (5 lines)
> When grafting, how will we achieve confidence that we've found the
> correct occurrence of the last character? I think we will have to give
> up our recently added feature of being able to change the version number
> of grafts.

Wait, don’t jump to the conclusions. :-)

Toggle quote (6 lines)
> Oh, one more thing: I forgot to mention in my previous email, where
> 'bash-static' refers to 'glibc-intermediate', that on my system the
> referenced 'glibc-intermediate' has been garbage-collected from my
> system, so that's a real-world example of a broken link, and it happened
> to be the first one I investigated :-(

‘bash-static’ is purposefully designed to stand alone: it must have zero
references:

Toggle snippet (6 lines)
$ guix size bash-static
store item total self
/gnu/store/nxysmfi9ybchnxfpxpmpgvz4f0nh7mxs-bash-static-4.3.42 1.4 1.4 100.0%
total: 1.4 MiB

So this one is fine.

Ludo’.
T
T
Török Edwin wrote on 17 Oct 2016 09:46
(address . 24703@debbugs.gnu.org)
8f2024ad-13c1-d4b1-1541-c2a5bddcb403@etorok.net
On 2016-10-16 22:04, Ludovic Courtès wrote:
Toggle quote (9 lines)
> Mark H Weaver <mhw@netris.org> skribis:
>
>> When grafting, how will we achieve confidence that we've found the
>> correct occurrence of the last character? I think we will have to give
>> up our recently added feature of being able to change the version number
>> of grafts.
>
> Wait, don’t jump to the conclusions. :-)

I've just encountered the same problem with fontconfig (after installing GuixSD, running guix pull and guix system reconfigure, --no-grafts was required).
Would it be possible for the grafts to keep a symlink (somehow registered to be part of the grafted fontconfig so that guix gc doesn't remove it) instead of patching the binaries?
/gnu/store/<old-hash>-fontconfig-2.11.94 -> /gnu/store/<grafted-hash>-fontconfig-2.11.94

Best regards,
--
Edwin Török | Co-founder and Lead Developer

Skylable open-source object storage: reliable, fast, secure
M
M
Mark H Weaver wrote on 17 Oct 2016 11:42
Re: bug#24703: Re: bug#24703: Store references in 8-byte chunks in compiled code
(name . Török Edwin)(address . edwin+ml-guix@etorok.net)(address . 24703@debbugs.gnu.org)
871szfs45v.fsf@netris.org
Török Edwin <edwin+ml-guix@etorok.net> writes:

Toggle quote (14 lines)
> On 2016-10-16 22:04, Ludovic Courtès wrote:
>> Mark H Weaver <mhw@netris.org> skribis:
>>
>>> When grafting, how will we achieve confidence that we've found the
>>> correct occurrence of the last character? I think we will have to give
>>> up our recently added feature of being able to change the version number
>>> of grafts.
>>
>> Wait, don’t jump to the conclusions. :-)
>
> I've just encountered the same problem with fontconfig (after
> installing GuixSD, running guix pull and guix system reconfigure,
> --no-grafts was required).

Here's how to recover, for now:

guix build --no-grafts -e '(@@ (gnu packages fontutils) fontconfig/fixed)'

Toggle quote (5 lines)
> Would it be possible for the grafts to keep a symlink (somehow
> registered to be part of the grafted fontconfig so that guix gc
> doesn't remove it) instead of patching the binaries?
> /gnu/store/<old-hash>-fontconfig-2.11.94 -> /gnu/store/<grafted-hash>-fontconfig-2.11.94

This would effectively mutate the store, and we must never do this.

Mark
L
L
Ludovic Courtès wrote on 17 Oct 2016 14:09
Re: bug#24703: Store references in 8-byte chunks in compiled code
(name . Török Edwin)(address . edwin+ml-guix@etorok.net)
87h98bdvng.fsf@gnu.org
Török Edwin <edwin+ml-guix@etorok.net> skribis:

Toggle quote (14 lines)
> On 2016-10-16 22:04, Ludovic Courtès wrote:
>> Mark H Weaver <mhw@netris.org> skribis:
>>
>>> When grafting, how will we achieve confidence that we've found the
>>> correct occurrence of the last character? I think we will have to give
>>> up our recently added feature of being able to change the version number
>>> of grafts.
>>
>> Wait, don’t jump to the conclusions. :-)
>
> I've just encountered the same problem with fontconfig (after installing GuixSD, running guix pull and guix system reconfigure, --no-grafts was required).
> Would it be possible for the grafts to keep a symlink (somehow registered to be part of the grafted fontconfig so that guix gc doesn't remove it) instead of patching the binaries?
> /gnu/store/<old-hash>-fontconfig-2.11.94 -> /gnu/store/<grafted-hash>-fontconfig-2.11.94

We could use a self symlink, or we could use something like

Mark, WDYT?

What remains to be seen is how many packages are affected by this issue,
and whether a generic solution needs to be found.

Ludo’.
M
M
Mark H Weaver wrote on 18 Oct 2016 05:36
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 24703@debbugs.gnu.org)
87k2d6qqee.fsf@netris.org
ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (26 lines)
> Török Edwin <edwin+ml-guix@etorok.net> skribis:
>
>> On 2016-10-16 22:04, Ludovic Courtès wrote:
>>> Mark H Weaver <mhw@netris.org> skribis:
>>>
>>>> When grafting, how will we achieve confidence that we've found the
>>>> correct occurrence of the last character? I think we will have to give
>>>> up our recently added feature of being able to change the version number
>>>> of grafts.
>>>
>>> Wait, don’t jump to the conclusions. :-)
>>
>> I've just encountered the same problem with fontconfig (after installing GuixSD, running guix pull and guix system reconfigure, --no-grafts was required).
>> Would it be possible for the grafts to keep a symlink (somehow
>> registered to be part of the grafted fontconfig so that guix gc
>> doesn't remove it) instead of patching the binaries?
>> /gnu/store/<old-hash>-fontconfig-2.11.94 -> /gnu/store/<grafted-hash>-fontconfig-2.11.94
>
> We could use a self symlink, or we could use something like
> <http://git.savannah.gnu.org/cgit/guix.git/commit/?id=9d50da70608de32d9db0c29859caec6f2cddb95f>.
>
> Mark, WDYT?
>
> What remains to be seen is how many packages are affected by this issue,
> and whether a generic solution needs to be found.

Unfortunately, it is too widespread. As I just pointed out in


Among the many packages that include these obfuscated store references,
one is 'glibc-final'. My attempt to graft 'bash' in 'master' to fix
CVE-2016-0634 and CVE-2016-7543 has resulted in a system where I cannot
build *any* derivation, because 'guile-final' crashes during boot while
its 'glibc-final' tries to find its 'gconv' modules in the ungrafted
'glibc-final', which is not available in the build environment.

So, if our approach is to use -fno-builtin-strcpy, then we will have to
apply it system-wide, and rebuild all of 'core-updates' from scratch.

I've been investigating another approach: to enhance our scanner and
grafter to handle these 8-byte-chunked references. I believe it is
feasible, but only if we abandon the ability to change the file names of
grafts outside of the hash. The reason is that the hash portion of
store references are surrounded by enough other known characters on both
sides that the hash portion is almost always contained entirely within
8-byte chunks.

To be continued...

Mark
L
L
Ludovic Courtès wrote on 18 Oct 2016 10:59
(name . Mark H Weaver)(address . mhw@netris.org)(address . 24703@debbugs.gnu.org)
87shrunicp.fsf@gnu.org
Mark H Weaver <mhw@netris.org> skribis:

Toggle quote (2 lines)
> ludo@gnu.org (Ludovic Courtès) writes:

[...]

Toggle quote (19 lines)
>> We could use a self symlink, or we could use something like
>> <http://git.savannah.gnu.org/cgit/guix.git/commit/?id=9d50da70608de32d9db0c29859caec6f2cddb95f>.
>>
>> Mark, WDYT?
>>
>> What remains to be seen is how many packages are affected by this issue,
>> and whether a generic solution needs to be found.
>
> Unfortunately, it is too widespread. As I just pointed out in
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24712#13
>
> Among the many packages that include these obfuscated store references,
> one is 'glibc-final'. My attempt to graft 'bash' in 'master' to fix
> CVE-2016-0634 and CVE-2016-7543 has resulted in a system where I cannot
> build *any* derivation, because 'guile-final' crashes during boot while
> its 'glibc-final' tries to find its 'gconv' modules in the ungrafted
> 'glibc-final', which is not available in the build environment.

Out of curiosity, Guile crashes while loading iconv modules, right?

This is surprising because the guile-for-build is always the ungrafted
‘guile-final’ (see ‘gnu-build’ in (guix build-system gnu)).

Toggle quote (3 lines)
> So, if our approach is to use -fno-builtin-strcpy, then we will have to
> apply it system-wide, and rebuild all of 'core-updates' from scratch.

Another approach would be to patch GCC, specifically ‘expand_movstr’ in
gcc/builtins.c, which is the part responsible for this optimization,
along these lines (untested):
--- gcc-5.3.0/gcc/builtins.c.orig 2016-10-18 10:45:35.042826368 +0200
+++ gcc-5.3.0/gcc/builtins.c 2016-10-18 10:50:46.080616285 +0200
@@ -3470,6 +3470,19 @@ expand_builtin_mempcpy_args (tree dest,
# define CODE_FOR_movstr CODE_FOR_nothing
#endif
+/* Return true if STR is a string denoting a "/gnu/store" file name. */
+
+static bool
+store_reference_p (tree str)
+{
+ const char *store;
+
+ store = getenv ("NIX_STORE") ?: "/gnu/store";
+
+ return (TREE_STRING_LENGTH (str) > strlen (store)
+ && strncmp (TREE_STRING_POINTER (str), store, strlen (store)));
+}
+
/* Expand into a movstr instruction, if one is available. Return NULL_RTX if
we failed, the caller should emit a normal call, otherwise try to
get the result in TARGET, if convenient. If ENDP is 0 return the
@@ -3484,7 +3497,9 @@ expand_movstr (tree dest, tree src, rtx
rtx dest_mem;
rtx src_mem;
- if (!HAVE_movstr)
+ /* When SRC denotes a store item, do not perform any optimization. See
+ http://bugs.gnu.org/24703 for background. */
+ if (!HAVE_movstr || store_reference_p (src))
return NULL_RTX;
dest_mem = get_memory_rtx (dest, NULL);
WDYT?

In the meantime, we need a workaround. The only option I can think of
is to retain a reference to the ungrafted item by adding a symlink to
it, like:
Toggle diff (20 lines)
diff --git a/guix/grafts.scm b/guix/grafts.scm
index 80ae27e..4de19ae 100644
--- a/guix/grafts.scm
+++ b/guix/grafts.scm
@@ -127,7 +127,14 @@ recursively applied to dependencies of DRV."
files))
(match %outputs
(((names . files) ...)
- files))))))
+ files)))
+
+ (for-each (match-lambda
+ ((name . file)
+ (symlink file
+ (string-append (assoc-ref %outputs name)
+ "/ungrafted"))))
+ old-outputs))))
(define add-label
(cut cons "x" <>))
Toggle quote (8 lines)
> I've been investigating another approach: to enhance our scanner and
> grafter to handle these 8-byte-chunked references. I believe it is
> feasible, but only if we abandon the ability to change the file names of
> grafts outside of the hash. The reason is that the hash portion of
> store references are surrounded by enough other known characters on both
> sides that the hash portion is almost always contained entirely within
> 8-byte chunks.

I think that would add complexity, would make grafting slower, and
abandoning the ability to change file names would be a regression.

So I’m more in favor of a GCC patch like above, or another compilation
tweak.

WDYT?

Thanks,
Ludo’.
T
T
Török Edwin wrote on 19 Oct 2016 23:25
(name . Ludovic Courtès)(address . ludo@gnu.org)
dadebcc5-5816-4760-0bd3-72c1af95e39c@etorok.net
On 2016-10-17 12:42, Mark H Weaver wrote:
Toggle quote (5 lines)
>
> Here's how to recover, for now:
>
> guix build --no-grafts -e '(@@ (gnu packages fontutils) fontconfig/fixed)'

Thanks!

On 2016-10-17 15:09, Ludovic Courtès wrote:
Toggle quote (18 lines)
> Török Edwin <edwin+ml-guix@etorok.net> skribis:
>
>> On 2016-10-16 22:04, Ludovic Courtès wrote:
>>> Mark H Weaver <mhw@netris.org> skribis:
>>>
>>>> When grafting, how will we achieve confidence that we've found the
>>>> correct occurrence of the last character? I think we will have to give
>>>> up our recently added feature of being able to change the version number
>>>> of grafts.
>>>
>>> Wait, don’t jump to the conclusions. :-)
>>
>> I've just encountered the same problem with fontconfig (after installing GuixSD, running guix pull and guix system reconfigure, --no-grafts was required).
>> Would it be possible for the grafts to keep a symlink (somehow registered to be part of the grafted fontconfig so that guix gc doesn't remove it) instead of patching the binaries?
>> /gnu/store/<old-hash>-fontconfig-2.11.94 -> /gnu/store/<grafted-hash>-fontconfig-2.11.94
>
> We could use a self symlink

I'm new here [*] and I'd like to understand what solutions would be possible, could you please explain how the self symlink would work?

I was thinking about bind mounts (the list of needed bind mounts would be maintained as a derivation, and initialized from initrd maybe?):
/gnu/store/<new-hash>-fontconfig-2.11.94 on /gnu/store/<old-hash>-fontconfig-2.11.94

or equivalently another layer of symlinks (where /gnu/store is the mutable symlink graft location, and /gnu/immutablestore is the real destination):
/gnu/store/<old-hash>-fontconfig-2.11.94 -> /gnu/immutablestore/<new-hash>-fontconfig-2.11.94
/gnu/store/<new-hash>-fontconfig-2.11.94 -> /gnu/immutablestore/<new-hash>-fontconfig-2.11.94

However bit worried of what happens to running applications: they might end up loading libs/data from both old-hash and new-hash, but
it wouldn't be worse than what happens on a traditional distribution when you upgrade say libc, you need to restart things using it eventually.

Toggle quote (3 lines)
> or we could use something like
> <http://git.savannah.gnu.org/cgit/guix.git/commit/?id=9d50da70608de32d9db0c29859caec6f2cddb95f>.

IMHO binary rewriting should be a last resort, it is too low-level and depends on the compiler/version to get it right.

[*] I've briefly tried NixOS and GuixSD in the past, but I'm using Debian and Gentoo as my main OS.
IIUC what the current grafting process does is that it reads all files of packages that depend on X from the store,
and writes them under a new hash with all references to hash of X replaced by hash of X-fixed, a bit like patchELF
except strings remain same size.

Best regards,
L
L
Ludovic Courtès wrote on 20 Oct 2016 14:25
(name . Török Edwin)(address . edwin+ml-guix@etorok.net)
87twc7cin6.fsf@gnu.org
Török Edwin <edwin+ml-guix@etorok.net> skribis:

Toggle quote (29 lines)
> On 2016-10-17 12:42, Mark H Weaver wrote:
>>
>> Here's how to recover, for now:
>>
>> guix build --no-grafts -e '(@@ (gnu packages fontutils) fontconfig/fixed)'
>
> Thanks!
>
> On 2016-10-17 15:09, Ludovic Courtès wrote:
>> Török Edwin <edwin+ml-guix@etorok.net> skribis:
>>
>>> On 2016-10-16 22:04, Ludovic Courtès wrote:
>>>> Mark H Weaver <mhw@netris.org> skribis:
>>>>
>>>>> When grafting, how will we achieve confidence that we've found the
>>>>> correct occurrence of the last character? I think we will have to give
>>>>> up our recently added feature of being able to change the version number
>>>>> of grafts.
>>>>
>>>> Wait, don’t jump to the conclusions. :-)
>>>
>>> I've just encountered the same problem with fontconfig (after installing GuixSD, running guix pull and guix system reconfigure, --no-grafts was required).
>>> Would it be possible for the grafts to keep a symlink (somehow registered to be part of the grafted fontconfig so that guix gc doesn't remove it) instead of patching the binaries?
>>> /gnu/store/<old-hash>-fontconfig-2.11.94 -> /gnu/store/<grafted-hash>-fontconfig-2.11.94
>>
>> We could use a self symlink
>
> I'm new here [*] and I'd like to understand what solutions would be possible, could you please explain how the self symlink would work?

The grafted variant of each store item would have a symlink pointing to
its ungrafted variant.

The problem with this approach is that it would induce some storage
overhead (although deduplication probably mitigates that), and it would
make it harder to check whether a given item is using only grafted
things because ‘guix gc -R something’ would always show both the grafted
and the ungrafted variant of each dependency.

Toggle quote (3 lines)
> I was thinking about bind mounts (the list of needed bind mounts would be maintained as a derivation, and initialized from initrd maybe?):
> /gnu/store/<new-hash>-fontconfig-2.11.94 on /gnu/store/<old-hash>-fontconfig-2.11.94

That would only work on GuixSD, not on foreign distros, and it’s a
“stateful” approach, which defeats the whole reproducible approach.

Toggle quote (2 lines)
> IMHO binary rewriting should be a last resort, it is too low-level and depends on the compiler/version to get it right.

I agree. However, for fast security updates, it seems to be the best
option:


Thanks for your feedback,
Ludo’.
L
L
Leo Famulari wrote on 24 Oct 2016 21:40
(name . Mark H Weaver)(address . mhw@netris.org)
20161024194022.GA1772@jasmine
On Mon, Oct 17, 2016 at 11:36:57PM -0400, Mark H Weaver wrote:
Toggle quote (10 lines)
> I've been investigating another approach: to enhance our scanner and
> grafter to handle these 8-byte-chunked references. I believe it is
> feasible, but only if we abandon the ability to change the file names of
> grafts outside of the hash. The reason is that the hash portion of
> store references are surrounded by enough other known characters on both
> sides that the hash portion is almost always contained entirely within
> 8-byte chunks.
>
> To be continued...

Any progress? If not, I think we should start making a list of packages
affected by the bug in order to prepare a -fno-builtin-strcpy branch.
L
L
Ludovic Courtès wrote on 24 Oct 2016 22:18
(name . Leo Famulari)(address . leo@famulari.name)
87insh1oxz.fsf@gnu.org
Leo Famulari <leo@famulari.name> skribis:

Toggle quote (13 lines)
> On Mon, Oct 17, 2016 at 11:36:57PM -0400, Mark H Weaver wrote:
>> I've been investigating another approach: to enhance our scanner and
>> grafter to handle these 8-byte-chunked references. I believe it is
>> feasible, but only if we abandon the ability to change the file names of
>> grafts outside of the hash. The reason is that the hash portion of
>> store references are surrounded by enough other known characters on both
>> sides that the hash portion is almost always contained entirely within
>> 8-byte chunks.
>>
>> To be continued...
>
> Any progress?

I built GCC 5.3 with the patch I posted just to notice that it had no
effect (according to ‘ltrace’ it never goes through the
getenv("NIX_STORE") call that the patch adds.)

I plan to investigate further Real Soon.

Toggle quote (3 lines)
> If not, I think we should start making a list of packages affected by
> the bug in order to prepare a -fno-builtin-strcpy branch.

Yeah it would be nice to identify these packages.

I’d prefer the GCC patch over -fno-builtin-strcpy because the latter is
hacky and potentially misses the issue when it occurs with other
builtins.

Either way, we’ll have to rebuild a lot of things.

Ludo’.
M
M
Mark H Weaver wrote on 31 Oct 2016 07:35
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 24703@debbugs.gnu.org)
87oa21c9fe.fsf@netris.org
ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (18 lines)
> Mark H Weaver <mhw@netris.org> skribis:
>
>> Unfortunately, it is too widespread. As I just pointed out in
>>
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24712#13
>>
>> Among the many packages that include these obfuscated store references,
>> one is 'glibc-final'. My attempt to graft 'bash' in 'master' to fix
>> CVE-2016-0634 and CVE-2016-7543 has resulted in a system where I cannot
>> build *any* derivation, because 'guile-final' crashes during boot while
>> its 'glibc-final' tries to find its 'gconv' modules in the ungrafted
>> 'glibc-final', which is not available in the build environment.
>
> Out of curiosity, Guile crashes while loading iconv modules, right?
>
> This is surprising because the guile-for-build is always the ungrafted
> ‘guile-final’ (see ‘gnu-build’ in (guix build-system gnu)).

Indeed. The derivations that crashed were using a grafted 'guile'.
These were not 'gnu-build' derivations, but simpler derivations such as
'module-import-compiled' derivations and some others involved with
building a 'system'.

Toggle quote (26 lines)
>> So, if our approach is to use -fno-builtin-strcpy, then we will have to
>> apply it system-wide, and rebuild all of 'core-updates' from scratch.
>
> Another approach would be to patch GCC, specifically ‘expand_movstr’ in
> gcc/builtins.c, which is the part responsible for this optimization,
> along these lines (untested):
>
> --- gcc-5.3.0/gcc/builtins.c.orig 2016-10-18 10:45:35.042826368 +0200
> +++ gcc-5.3.0/gcc/builtins.c 2016-10-18 10:50:46.080616285 +0200
> @@ -3470,6 +3470,19 @@ expand_builtin_mempcpy_args (tree dest,
> # define CODE_FOR_movstr CODE_FOR_nothing
> #endif
>
> +/* Return true if STR is a string denoting a "/gnu/store" file name. */
> +
> +static bool
> +store_reference_p (tree str)
> +{
> + const char *store;
> +
> + store = getenv ("NIX_STORE") ?: "/gnu/store";
> +
> + return (TREE_STRING_LENGTH (str) > strlen (store)
> + && strncmp (TREE_STRING_POINTER (str), store, strlen (store)));
> +}

[...]

Toggle quote (2 lines)
> WDYT?

I think it's not sufficient to apply this workaround only for string
literals that _begin_ with the store directory. In some cases, the
store name may appear only in the middle of a string.

Toggle quote (4 lines)
> In the meantime, we need a workaround. The only option I can think of
> is to retain a reference to the ungrafted item by adding a symlink to
> it, like:

I consider it a potentially serious security problem that ungrafted
outputs are being used. Papering over the problem by preventing this
buggy software from being deleted is, in my opinion, not acceptable.

I would suggest instead that we'll need to add grafts for all packages
that include these chunked references. However, due to bug 24832, it
may be that we'll need to rebuild all of 'core-updates' from scratch
anyway.

Toggle quote (16 lines)
>> I've been investigating another approach: to enhance our scanner and
>> grafter to handle these 8-byte-chunked references. I believe it is
>> feasible, but only if we abandon the ability to change the file names of
>> grafts outside of the hash. The reason is that the hash portion of
>> store references are surrounded by enough other known characters on both
>> sides that the hash portion is almost always contained entirely within
>> 8-byte chunks.
>
> I think that would add complexity, would make grafting slower, and
> abandoning the ability to change file names would be a regression.
>
> So I’m more in favor of a GCC patch like above, or another compilation
> tweak.
>
> WDYT?

The GCC approach is okay with me in the short term, but I'll likely want
to revisit this issue in the future.

Thanks,
Mark
L
L
Ludovic Courtès wrote on 31 Oct 2016 12:37
(name . Mark H Weaver)(address . mhw@netris.org)(address . 24703@debbugs.gnu.org)
87pomgrbq2.fsf@gnu.org
Hi Mark,

Mark H Weaver <mhw@netris.org> skribis:

Toggle quote (2 lines)
> ludo@gnu.org (Ludovic Courtès) writes:

[...]

Toggle quote (34 lines)
>>> So, if our approach is to use -fno-builtin-strcpy, then we will have to
>>> apply it system-wide, and rebuild all of 'core-updates' from scratch.
>>
>> Another approach would be to patch GCC, specifically ‘expand_movstr’ in
>> gcc/builtins.c, which is the part responsible for this optimization,
>> along these lines (untested):
>>
>> --- gcc-5.3.0/gcc/builtins.c.orig 2016-10-18 10:45:35.042826368 +0200
>> +++ gcc-5.3.0/gcc/builtins.c 2016-10-18 10:50:46.080616285 +0200
>> @@ -3470,6 +3470,19 @@ expand_builtin_mempcpy_args (tree dest,
>> # define CODE_FOR_movstr CODE_FOR_nothing
>> #endif
>>
>> +/* Return true if STR is a string denoting a "/gnu/store" file name. */
>> +
>> +static bool
>> +store_reference_p (tree str)
>> +{
>> + const char *store;
>> +
>> + store = getenv ("NIX_STORE") ?: "/gnu/store";
>> +
>> + return (TREE_STRING_LENGTH (str) > strlen (store)
>> + && strncmp (TREE_STRING_POINTER (str), store, strlen (store)));
>> +}
>
> [...]
>
>> WDYT?
>
> I think it's not sufficient to apply this workaround only for string
> literals that _begin_ with the store directory. In some cases, the
> store name may appear only in the middle of a string.

Do you have examples? I think this is unlikely: the common case here is
that we’re capturing the installation prefix as in:

#define PREFIX "/gnu/store/…"
strcpy (file, PREFIX);

Toggle quote (8 lines)
>> In the meantime, we need a workaround. The only option I can think of
>> is to retain a reference to the ungrafted item by adding a symlink to
>> it, like:
>
> I consider it a potentially serious security problem that ungrafted
> outputs are being used. Papering over the problem by preventing this
> buggy software from being deleted is, in my opinion, not acceptable.

In practice, only data from the ungrafted input would be used, AFAICS.

I’m not saying that this is a good solution, I’m just trying to think of
solutions that we can deploy now while waiting for something better,
which may involve a full rebuild.

Toggle quote (3 lines)
> The GCC approach is okay with me in the short term, but I'll likely want
> to revisit this issue in the future.

From your message, it’s unclear to me what you are proposing:

1. as the short term solution;

2. as the long term solution.

To me short-term is symlink and long-term is GCC patch.

Thanks for your feedback,
Ludo’.
L
L
Ludovic Courtès wrote on 5 Nov 2016 00:15
(name . Leo Famulari)(address . leo@famulari.name)
87a8de7s6q.fsf@inria.fr
I’ve fiddled a bit with GCC and read some code. No success yet, but
here’s a status update.

AIUI, ‘strcpy’ declarations are immediately (in the front-end)
recognized as “built-ins” as they are read (IOW, there’s no explicit
conversion from function call to built-in call.)

‘__builtin_strcpy’ calls are then converted to ‘__builtin_memcpy’ calls
(‘handle_builtin_strcpy’ in tree-ssa-strlen.c), which in turn leads to
code that uses the ‘movabs’ instruction that’s causing us problems on
x86:

Toggle snippet (16 lines)
$ echo 'extern char *strcpy(char*,const char*);void foo(char*x){ strcpy(x, "foo"); }'> t.c
$ gcc -c -fdump-tree-optimized t.c
$ cat t.c.211t.optimized

;; Function foo (foo, funcdef_no=0, decl_uid=1759, cgraph_uid=0, symbol_order=0)

foo (char * x)
{
<bb 2>:
__builtin_memcpy (x_2(D), "foo", 4);
return;

}


(Strangely, when using ‘memcpy’ in the source, the ‘movabs’ optimization
does not take place at -O0, unlike for ‘strcpy’.)

So it seems there’s no place on the front-end side where we could do
what I tried to do in my initial patch, which is to test the contents of
the literal string passed to strcpy/memcpy and turn off the
‘movabs’-producing optimization.

Instead, the knobs we have are (1) global flag to enable/disable each
built-in function (like -fno-builtin-… does), and (2) an x86-specific
knob to determine whether to use ‘movabs’ or not (‘-mmemcpy-strategy’
supposedly controls that, but ‘-mmemcpy-strategy=libcall:-1:noalign’
doesn’t seem to have any effect for instance.)

These knobs are not great because that would lead us to disable the
optimization wholesale, which is not desirable.

Other than that, I’ve also looked at all the movabs-related things in
gcc/config/i386/*.md, but to no avail.

To be continued…

Ludo’.
L
L
Leo Famulari wrote on 5 Nov 2016 19:36
(name . Ludovic Courtès)(address . ludo@gnu.org)
20161105183650.GA21301@jasmine
On Sat, Nov 05, 2016 at 12:15:25AM +0100, Ludovic Courtès wrote:
Toggle quote (3 lines)
> I’ve fiddled a bit with GCC and read some code. No success yet, but
> here’s a status update.

Thanks for writing this out!

Toggle quote (6 lines)
> Instead, the knobs we have are (1) global flag to enable/disable each
> built-in function (like -fno-builtin-… does), and (2) an x86-specific
> knob to determine whether to use ‘movabs’ or not (‘-mmemcpy-strategy’
> supposedly controls that, but ‘-mmemcpy-strategy=libcall:-1:noalign’
> doesn’t seem to have any effect for instance.)

Please correct me if I paraphrase the choices incorrectly:
(1) Completely disable the strcpy optimization for all architectures
(2) Ostensibly change how strcpy is optimized on x86, except the knob
seems to have no effect

Toggle quote (3 lines)
> These knobs are not great because that would lead us to disable the
> optimization wholesale, which is not desirable.

What are the costs of (1)? Should we report (2) upstream?
L
L
Ludovic Courtès wrote on 6 Nov 2016 21:58
(name . Leo Famulari)(address . leo@famulari.name)
87bmxsz5ow.fsf@gnu.org
Leo Famulari <leo@famulari.name> skribis:

Toggle quote (2 lines)
> On Sat, Nov 05, 2016 at 12:15:25AM +0100, Ludovic Courtès wrote:

[...]

Toggle quote (11 lines)
>> Instead, the knobs we have are (1) global flag to enable/disable each
>> built-in function (like -fno-builtin-… does), and (2) an x86-specific
>> knob to determine whether to use ‘movabs’ or not (‘-mmemcpy-strategy’
>> supposedly controls that, but ‘-mmemcpy-strategy=libcall:-1:noalign’
>> doesn’t seem to have any effect for instance.)
>
> Please correct me if I paraphrase the choices incorrectly:
> (1) Completely disable the strcpy optimization for all architectures
> (2) Ostensibly change how strcpy is optimized on x86, except the knob
> seems to have no effect

Correct.

Toggle quote (5 lines)
>> These knobs are not great because that would lead us to disable the
>> optimization wholesale, which is not desirable.
>
> What are the costs of (1)? Should we report (2) upstream?

-fno-builtin-{strcpy,memcpy} disables not just use of ‘movabs’ but also
all the optimizations that GCC can make when the built-ins are being
used (see the doc for ‘-mmemcpy-strategy’). It wouldn’t feel right to
disable that completely.

Ludo’.
L
L
Ludovic Courtès wrote on 9 Nov 2016 21:40
(name . Leo Famulari)(address . leo@famulari.name)
87oa1o75ga.fsf@gnu.org
Hello!

I read much more code than I wanted just to end up in gcc/builtins.c.
In 8033772363b287ca529461e575ceb4a69d7af942 I added a patch for GCC 5.x
and 6.x that disables the ‘movabs’ optimization for strcpy/memcpy when
the source is a string constant that contains “/gnu/store” (I followed
Mark’s advice to disable the optimization for any string that contains
“/gnu/store”, rather than just for strings that start with
“/gnu/store”.)

This can be tested by compiling a file like this one (comment or
uncomment the lines that you want):

Toggle snippet (14 lines)
#include <string.h>
void foo (char *x, char *y)
{
/* memcpy(x, "this is a long string, a very long string", 42); */
/* strcpy(x, "STRCPY THIS IS A LONG STRING, A VERY LONG STRING"); */
strcpy(x, "STRCPY /gnu/store/THIS IS A LONG STRING, A VERY LONG STRING");
/* __builtin_strcpy(x, "THIS IS A LONG STRING, A VERY LONG STRING"); */
/* strcpy(y, "/gnu/store/THIS IS A LONG STRING, A VERY LONG STRING"); */
/* memcpy(y, "MEMCPY THIS IS A LONG STRING, A VERY LONG STRING", 30); */
memcpy(y, "MEMCPY /gnu/store/THIS IS A LONG STRING, A VERY LONG STRING", 30);
/* __builtin_memcpy(y, "/gnu/store/THIS IS A LONG STRING, A VERY LONG STRING", 30); */
}

… and then running “objdump -S foo.o | grep movabs”, for instance.

Now we need a plan to actually fix the bug.

The long-term goal is to rebuild everything with a compiler that has
this patch, in the next ‘core-updates’. We might as well switch to GCC
5 as the default compiler.

In the meantime, the only approach I can think of is to (1) ungraft more
frequently than we’ve done so far, and (2) when we ungraft a package,
add gcc@5 as an input such that it gets rebuilt without the problem.

Thoughts?

Thanks,
Ludo’.
L
L
Leo Famulari wrote on 10 Nov 2016 00:16
(name . Ludovic Courtès)(address . ludo@gnu.org)
20161109231652.GA12060@jasmine
On Wed, Nov 09, 2016 at 09:40:05PM +0100, Ludovic Courtès wrote:
Toggle quote (28 lines)
> I read much more code than I wanted just to end up in gcc/builtins.c.
> In 8033772363b287ca529461e575ceb4a69d7af942 I added a patch for GCC 5.x
> and 6.x that disables the ‘movabs’ optimization for strcpy/memcpy when
> the source is a string constant that contains “/gnu/store” (I followed
> Mark’s advice to disable the optimization for any string that contains
> “/gnu/store”, rather than just for strings that start with
> “/gnu/store”.)
>
> This can be tested by compiling a file like this one (comment or
> uncomment the lines that you want):
>
> --8<---------------cut here---------------start------------->8---
> #include <string.h>
> void foo (char *x, char *y)
> {
> /* memcpy(x, "this is a long string, a very long string", 42); */
> /* strcpy(x, "STRCPY THIS IS A LONG STRING, A VERY LONG STRING"); */
> strcpy(x, "STRCPY /gnu/store/THIS IS A LONG STRING, A VERY LONG STRING");
> /* __builtin_strcpy(x, "THIS IS A LONG STRING, A VERY LONG STRING"); */
> /* strcpy(y, "/gnu/store/THIS IS A LONG STRING, A VERY LONG STRING"); */
> /* memcpy(y, "MEMCPY THIS IS A LONG STRING, A VERY LONG STRING", 30); */
> memcpy(y, "MEMCPY /gnu/store/THIS IS A LONG STRING, A VERY LONG STRING", 30);
> /* __builtin_memcpy(y, "/gnu/store/THIS IS A LONG STRING, A VERY LONG STRING", 30); */
> }
> --8<---------------cut here---------------end--------------->8---
>
> … and then running “objdump -S foo.o | grep movabs”, for instance.

Awesome. Thank you!

Toggle quote (12 lines)
> Now we need a plan to actually fix the bug.
>
> The long-term goal is to rebuild everything with a compiler that has
> this patch, in the next ‘core-updates’. We might as well switch to GCC
> 5 as the default compiler.
>
> In the meantime, the only approach I can think of is to (1) ungraft more
> frequently than we’ve done so far, and (2) when we ungraft a package,
> add gcc@5 as an input such that it gets rebuilt without the problem.
>
> Thoughts?

Sounds good to me. I wonder, with our current build farm, how often can
we do the full rebuilds required by ungrafting? I don't yet have a sense
of how long it takes.
L
L
Ludovic Courtès wrote on 10 Nov 2016 09:01
(name . Leo Famulari)(address . leo@famulari.name)
87shqzyd9z.fsf@gnu.org
Leo Famulari <leo@famulari.name> skribis:

Toggle quote (2 lines)
> On Wed, Nov 09, 2016 at 09:40:05PM +0100, Ludovic Courtès wrote:

[...]

Toggle quote (14 lines)
>> The long-term goal is to rebuild everything with a compiler that has
>> this patch, in the next ‘core-updates’. We might as well switch to GCC
>> 5 as the default compiler.
>>
>> In the meantime, the only approach I can think of is to (1) ungraft more
>> frequently than we’ve done so far, and (2) when we ungraft a package,
>> add gcc@5 as an input such that it gets rebuilt without the problem.
>>
>> Thoughts?
>
> Sounds good to me. I wonder, with our current build farm, how often can
> we do the full rebuilds required by ungrafting? I don't yet have a sense
> of how long it takes.

I don’t know exactly. I have a feeling that the build farm has been
underused over the last couple of weeks for instance, but no precise
stats. I’d like to be able to gather usage stats so we can see when the
right time for rebuilds comes.

For now I think we’ll merge core-updates Real Soon, and then start a
staging branch or the python-updates one.

Ludo’.
L
L
Ludovic Courtès wrote on 11 Nov 2016 11:39
(name . Leo Famulari)(address . leo@famulari.name)
87eg2i2tc3.fsf@gnu.org
ludo@gnu.org (Ludovic Courtès) skribis:

Toggle quote (4 lines)
> In the meantime, the only approach I can think of is to (1) ungraft more
> frequently than we’ve done so far, and (2) when we ungraft a package,
> add gcc@5 as an input such that it gets rebuilt without the problem.

As an example, I did that in commit
2c5ab05bffe5f89092ef60c3743b3941dcf92af0 for WordNet.

Ludo’.
L
L
Ludovic Courtès wrote on 3 Apr 2017 00:19
(address . 24703-done@debbugs.gnu.org)
874ly68nb4.fsf@gnu.org
The patch to fix this bug was committed in ‘core-updates’ in
8033772363b287ca529461e575ceb4a69d7af942, which has now been merged.

Ludo’.
Closed
?