QEMU 4 fails to build for x86_64-linux

Done

Details

5 participants

Jan Nieuwenhuizen
Leo Famulari
Ludovic Courtès
Mathieu Othacehe
Marius Bakke

Owner: unassigned

Submitted by: Leo Famulari

Severity: important

Leo Famulari wrote on 31 Jul 2019 22:03

Recipients:(address . bug-guix@gnu.org)

Message-ID:20190731200334.GA29724@jasmine.lan

While testing the upgrade of our QEMU package to 4.0.0 or 4.1.0-rc3 on
x86_64-linux, the build fails when the header 'gnu/stubs-32.h' can't be
found:

------
In file included from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/features.h:452:0,
                 from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/bits/libc-header-start.h:33,
                 from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/stdint.h:26,
                 from linuxboot_dma.c:65:
/gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/gnu/stubs.h:7:27: fatal error: gnu/stubs-32.h: No such file or directory
compilation terminated.
make[1]: *** [/tmp/guix-build-qemu-4.1.0-rc3.drv-0/qemu-4.1.0-rc3/rules.mak:69: linuxboot_dma.o] Error 1
make[1]: Leaving directory '/tmp/guix-build-qemu-4.1.0-rc3.drv-0/qemu-4.1.0-rc3/pc-bios/optionrom'
make: *** [Makefile:519: pc-bios/optionrom/all] Error 2
------

And of course, this header is not installed by glibc when it is built
for 64-bit systems.

The header in question, 'stubs.h', looks like this:

------
#if !defined __x86_64__
# include <gnu/stubs-32.h>
#endif
#if defined __x86_64__ && defined __LP64__
# include <gnu/stubs-64.h>
#endif
#if defined __x86_64__ && defined __ILP32__
# include <gnu/stubs-x32.h>
#endif
------

When I build for i686-linux, it works as expected.

Any advice? I'm not really sure what's going on here.

Marius Bakke wrote on 1 Aug 2019 16:14

Recipients:

Message-ID:87h8712crk.fsf@devup.no

Leo Famulari <leo@famulari.name> writes:

Toggle quote (37 lines)> While testing the upgrade of our QEMU package to 4.0.0 or 4.1.0-rc3 on
> x86_64-linux, the build fails when the header 'gnu/stubs-32.h' can't be
> found:
>
> ------
> In file included from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/features.h:452:0,
>                  from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/bits/libc-header-start.h:33,
>                  from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/stdint.h:26,
>                  from linuxboot_dma.c:65:
> /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/gnu/stubs.h:7:27: fatal error: gnu/stubs-32.h: No such file or directory
> compilation terminated.
> make[1]: *** [/tmp/guix-build-qemu-4.1.0-rc3.drv-0/qemu-4.1.0-rc3/rules.mak:69: linuxboot_dma.o] Error 1
> make[1]: Leaving directory '/tmp/guix-build-qemu-4.1.0-rc3.drv-0/qemu-4.1.0-rc3/pc-bios/optionrom'
> make: *** [Makefile:519: pc-bios/optionrom/all] Error 2
> ------
>
> And of course, this header is not installed by glibc when it is built
> for 64-bit systems.
>
> The header in question, 'stubs.h', looks like this:
>
> ------
> #if !defined __x86_64__
> # include <gnu/stubs-32.h>
> #endif
> #if defined __x86_64__ && defined __LP64__
> # include <gnu/stubs-64.h>
> #endif
> #if defined __x86_64__ && defined __ILP32__
> # include <gnu/stubs-x32.h>
> #endif
> ------
>
> When I build for i686-linux, it works as expected.
>
> Any advice? I'm not really sure what's going on here.

I don't know why it fails, but it works if you give it a newer GCC such

as the one on 'core-updates'.

I tried it with 4.0.0, but got stuck on two test failures. Hopefully

4.1.0 is easier to debug... :-)

Ludovic Courtès wrote on 23 Aug 2019 14:58

Recipients:(name . Leo Famulari)(address . leo@famulari.name)

Message-ID:8736hs6ntd.fsf@gnu.org

Hi,

Leo Famulari <leo@famulari.name> skribis:

Toggle quote (19 lines)> While testing the upgrade of our QEMU package to 4.0.0 or 4.1.0-rc3 on
> x86_64-linux, the build fails when the header 'gnu/stubs-32.h' can't be
> found:
>
> ------
> In file included from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/features.h:452:0,
>                  from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/bits/libc-header-start.h:33,
>                  from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/stdint.h:26,
>                  from linuxboot_dma.c:65:
> /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/gnu/stubs.h:7:27: fatal error: gnu/stubs-32.h: No such file or directory
> compilation terminated.
> make[1]: *** [/tmp/guix-build-qemu-4.1.0-rc3.drv-0/qemu-4.1.0-rc3/rules.mak:69: linuxboot_dma.o] Error 1
> make[1]: Leaving directory '/tmp/guix-build-qemu-4.1.0-rc3.drv-0/qemu-4.1.0-rc3/pc-bios/optionrom'
> make: *** [Makefile:519: pc-bios/optionrom/all] Error 2
> ------
>
> And of course, this header is not installed by glibc when it is built
> for 64-bit systems.

That’s a “multilib header”, which our ‘glibc’ package doesn’t provide.

Normally it’s only needed when doing “gcc -m32” on x86_64 or similar.

Can you show the GCC command that leads to this error?

Thanks,

Ludo’.

Mathieu Othacehe wrote on 21 Feb 2020 12:09

Qemu 4.2.0 build for x86_64-linux fails

Recipients:(address . 36882@debbugs.gnu.org)

Message-ID:87k14gnqng.fsf@gmail.com

Hello,

On core-updates, qemu-minimal (4.2.0), fails to build. This seems to be

the same issue as this bug. The error is:

Toggle snippet (11 lines)

In file included from /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/features.h:489:0,

from /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/bits/libc-header-start.h:33,

from /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/stdint.h:26,

from linuxboot_dma.c:65:

/gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/gnu/stubs.h:7:11: fatal error: gnu/stubs-32.h: No such file or directory

# include <gnu/stubs-32.h>

^~~~~~~~~~~~~~~~

Because of this gcc command:

/gnu/store/dcnp1h3q6qyipwkm0g7l7r1bkvlqvaqa-gcc-7.5.0/bin/gcc -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/. -iquote . -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/tcg -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/tcg/i386 -I/tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/linux-headers -I/tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/linux-headers -iquote . -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0 -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/accel/tcg -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/include -I/tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0 -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fno-pie -ffreestanding -fno-stack-protector -m16 -Wa,-32 -MMD -MP -MT linuxboot_dma.o -MF ./linuxboot_dma.d -O2 -march=i486 -c -o linuxboot_dma.o linuxboot_dma.c

Toggle snippet (8 lines)

Any idea? Do we still need to add "multilib header" support to our

glibc build?

Thanks,

Mathieu

Ludovic Courtès wrote on 21 Feb 2020 21:29

Recipients:(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)

Message-ID:87mu9b3crd.fsf@gnu.org

Hi,

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (15 lines)> On core-updates, qemu-minimal (4.2.0), fails to build. This seems to be
> the same issue as this bug. The error is:
>
> In file included from /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/features.h:489:0,
>                  from /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/bits/libc-header-start.h:33,
>                  from /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/stdint.h:26,
>                  from linuxboot_dma.c:65:
> /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/gnu/stubs.h:7:11: fatal error: gnu/stubs-32.h: No such file or directory
>  # include <gnu/stubs-32.h>
>            ^~~~~~~~~~~~~~~~
>
> Because of this gcc command:
>
> /gnu/store/dcnp1h3q6qyipwkm0g7l7r1bkvlqvaqa-gcc-7.5.0/bin/gcc -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/. -iquote . -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/tcg -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/tcg/i386 -I/tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/linux-headers -I/tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/linux-headers -iquote . -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0 -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/accel/tcg -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/include -I/tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0 -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fno-pie -ffreestanding   -fno-stack-protector   -m16   -Wa,-32 -MMD -MP -MT linuxboot_dma.o -MF ./linuxboot_dma.d -O2 -march=i486  -c -o linuxboot_dma.o linuxboot_dma.c

How was this bug initially reported against QEMU 4.0.0 fixed?

On master there’s pretty much the same command as above, with ‘-m16’,

and “yet it works”.

https://ci.guix.gnu.org/log/ymzp5yz2r3zfw4xczwwlykyjv2kqcqs0-qemu-4.2.0

Ludo’.

Mathieu Othacehe wrote on 22 Feb 2020 20:13

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87a75a5taw.fsf@gmail.com

Hello,

Toggle quote (5 lines)

> On master there’s pretty much the same command as above, with ‘-m16’,

> and “yet it works”.

> https://ci.guix.gnu.org/log/ymzp5yz2r3zfw4xczwwlykyjv2kqcqs0-qemu-4.2.0

On master, when building qemu, this is the gcc include path (obtained

with "gcc -v -x c -E /dev/null"):

Toggle snippet (8 lines)

/gnu/store/adm2cx3ayabn1sp84nnjsk0672m800ip-flex-2.6.4/include

/gnu/store/l86azr7r3p5631wj3kk329jl1y1mpjgy-bzip2-1.0.6/include

...

/gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.4.0/include [1]

/gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.4.0/include-fixed

/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/include [2]

We see that glibc comes last. So when building a program with "gcc

-ffreestanding -m16", the first <stdint.h> header picked is the one from

gcc (marked [1] above). This header contains:

Toggle snippet (16 lines)#ifndef _GCC_WRAP_STDINT_H
  #if __STDC_HOSTED__ <-- Set to 0 because of -ffreestanding option.
    # if defined __cplusplus && __cplusplus >= 201103L
      #  undef __STDC_LIMIT_MACROS
      #  define __STDC_LIMIT_MACROS
      #  undef __STDC_CONSTANT_MACROS
      #  define __STDC_CONSTANT_MACROS
    # endif
    # include_next <stdint.h>
  #else
    # include "stdint-gcc.h" <-- This one gets included.
  #endif
  #define _GCC_WRAP_STDINT_H
#endif

So the <stdint.h> from glibc (marked [2] above) is neved included. And

that's good because this header would try to include <gnu/stubs-32.h>

that our glibc does not provide.

Now the question is, why glibc comes last, whereas it is a part of

CPATH? The "gcc -v -x c -E /dev/null" command says:

Toggle snippet (4 lines)

ignoring duplicate directory "/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/include"

as it is a non-system directory that duplicates a system directory

So the glibc from CPATH is ignored and the last glibc, hardcoded into

gcc is kept.

Now, what's happening on core-updates? Because of CPATH ->

C_INCLUDE_PATH transition, we have a different behaviour. The same

command as above states:

Toggle snippet (10 lines)

ignoring duplicate directory "/gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include"

...

/gnu/store/kyypmcn2miyzllqqmcx28gvj89qnvhpi-flex-2.6.4/include

...

/gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include [1]

...

/gnu/store/wl7zanqf0s9bfppggmv8qxqan71fjfw8-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/include [2]

/gnu/store/wl7zanqf0s9bfppggmv8qxqan71fjfw8-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/include-fixed

The glibc ignored seems to be the hardcoded one. The one from the

C_INCLUDE_PATH is kept. So the <stdint.h> included is the one from glibc

(marked [1] above), which triggers <gnu/stubs-32.h> inclusion and makes

build fail.

So here's how it fails, but I'm not sure how to fix it. Removing glibc

from C_INCLUDE_PATH, forces gcc to use the hardcoded one that comes last

and fixes the build. However, there are maybe other implications?

Thanks,

Mathieu

Ludovic Courtès wrote on 23 Feb 2020 12:32

Recipients:(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)

Message-ID:87o8tptu7u.fsf@gnu.org

Hi Mathieu,

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (35 lines)> On master, when building qemu, this is the gcc include path (obtained
> with "gcc -v -x c -E /dev/null"):
>
>  /gnu/store/adm2cx3ayabn1sp84nnjsk0672m800ip-flex-2.6.4/include
>  /gnu/store/l86azr7r3p5631wj3kk329jl1y1mpjgy-bzip2-1.0.6/include
>  ...
>  /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.4.0/include [1]
>  /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.4.0/include-fixed
>  /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/include [2]
>
>
> We see that glibc comes last. So when building a program with "gcc
> -ffreestanding -m16", the first <stdint.h> header picked is the one from
> gcc (marked [1] above). This header contains:
>
> #ifndef _GCC_WRAP_STDINT_H
>   #if __STDC_HOSTED__ <-- Set to 0 because of -ffreestanding option.
>     # if defined __cplusplus && __cplusplus >= 201103L
>       #  undef __STDC_LIMIT_MACROS
>       #  define __STDC_LIMIT_MACROS
>       #  undef __STDC_CONSTANT_MACROS
>       #  define __STDC_CONSTANT_MACROS
>     # endif
>     # include_next <stdint.h>
>   #else
>     # include "stdint-gcc.h" <-- This one gets included.
>   #endif
>   #define _GCC_WRAP_STDINT_H
> #endif
>
>
> So the <stdint.h> from glibc (marked [2] above) is neved included. And
> that's good because this header would try to include <gnu/stubs-32.h>
> that our glibc does not provide.

[...]

Toggle quote (18 lines)> Now, what's happening on core-updates? Because of CPATH ->
> C_INCLUDE_PATH transition, we have a different behaviour. The same
> command as above states:
>
> ignoring duplicate directory "/gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include"
> ...
>  /gnu/store/kyypmcn2miyzllqqmcx28gvj89qnvhpi-flex-2.6.4/include
>  ...
>  /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include [1]
>  ...
>  /gnu/store/wl7zanqf0s9bfppggmv8qxqan71fjfw8-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/include [2]
>  /gnu/store/wl7zanqf0s9bfppggmv8qxqan71fjfw8-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/include-fixed
>
> The glibc ignored seems to be the hardcoded one. The one from the
> C_INCLUDE_PATH is kept. So the <stdint.h> included is the one from glibc
> (marked [1] above), which triggers <gnu/stubs-32.h> inclusion and makes
> build fail.

Woow, thanks for the analysis.

It’s a bug on ‘core-updates’ that GCC’s headers come after libc’s. This

is what should be fixed IMO.

So is it expected that C_INCLUDE_PATH comes before the hard-coded GCC

include directory? How can we work around that?

Thanks!

Ludo’.

Mathieu Othacehe wrote on 24 Feb 2020 10:36

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87ftf0nx7n.fsf@gmail.com

Hey,

Toggle quote (3 lines)

> So is it expected that C_INCLUDE_PATH comes before the hard-coded GCC

> include directory? How can we work around that?

Turns out, the best source of documentation here is

gcc/incpath.c. Here's a summary of my understanding.

Header search list:

* QUOTE

-> -iquote

* BRACKET

-> -I goes here

-> CPATH goes here

* SYSTEM

-> -isystem goes here

-> C_INCLUDE_PATH goes here

-> Hardcoded includes (gcc + glibc) goes here

* AFTER

-> -idirafter goes here.

Duplicates inside SYSTEM are deleted, the first occurence is kept. So as

long as we have glibc in C_INCLUDE_PATH it will trigger deletion of

glibc in hardcoded includes and AFTER section.

So I can see only two solutions here.

1. Go back to using CPATH (sad!), because when there is duplication

between BRACKET and SYSTEM, the include from SYSTEM is kept (why it

works on master).

2. Find a way to remove glibc from C_INCLUDE_PATH, but I have no clue

how to do this properly. Maybe using some kind of filter on

search-path-specifications.

WDYT?

Mathieu

Ludovic Courtès wrote on 24 Feb 2020 11:29

control message for bug #36882

Recipients:(address . control@debbugs.gnu.org)

Message-ID:877e0cnuqk.fsf@gnu.org

severity 36882 important

quit

Ludovic Courtès wrote on 24 Feb 2020 15:25

Re: Qemu 4.2.0 build for x86_64-linux fails

Recipients:(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)

Message-ID:87tv3gm59r.fsf@gnu.org

Hi,

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (34 lines)>> So is it expected that C_INCLUDE_PATH comes before the hard-coded GCC
>> include directory?  How can we work around that?
>
> Turns out, the best source of documentation here is
> gcc/incpath.c. Here's a summary of my understanding.
>
> Header search list:
>
> * QUOTE
>  -> -iquote 
> * BRACKET
>  -> -I goes here
>  -> CPATH goes here
> * SYSTEM
>  -> -isystem goes here
>  -> C_INCLUDE_PATH goes here
>  -> Hardcoded includes (gcc + glibc) goes here
> * AFTER
> -> -idirafter goes here.
>
> Duplicates inside SYSTEM are deleted, the first occurence is kept. So as
> long as we have glibc in C_INCLUDE_PATH it will trigger deletion of
> glibc in hardcoded includes and AFTER section.
>
> So I can see only two solutions here.
>
> 1. Go back to using CPATH (sad!), because when there is duplication
> between BRACKET and SYSTEM, the include from SYSTEM is kept (why it
> works on master).
>
> 2. Find a way to remove glibc from C_INCLUDE_PATH, but I have no clue
> how to do this properly. Maybe using some kind of filter on
> search-path-specifications.

I’d rather go for #2. To do that, we could modify the ‘set-paths’ phase

to manually remove glibc from C_INCLUDE_PATH (fragile), or we could

modify GCC (perhaps removing the ‘remove_duplicates’ call for SYSTEM).

Either way, this wouldn’t work well with ‘guix environment’, where glibc

ends up in /gnu/store/…-profile, so it does not appear as duplicate to

GCC.

On ‘core-updates’, I see:

Toggle snippet (36 lines)$ git log | head -3
commit 5afcb5caa53615c0a432e0c1781155398d747218
Author: Ludovic Courtès <ludo@gnu.org>
Date:   Sat Feb 22 21:39:27 2020 +0100
$ ./pre-inst-env guix environment -C -e '(@@ (gnu packages commencement) coreutils-final)'
[env]$ gcc -v -x c -E /dev/null
Using built-in specs.
COLLECT_GCC=gcc
Target: x86_64-unknown-linux-gnu
Configured with: 
Thread model: posix
gcc version 7.5.0 (GCC) 
COLLECT_GCC_OPTIONS='-v' '-E' '-mtune=generic' '-march=x86-64'
 /gnu/store/0pjrnr8vhp94ykvarysd9wg7hcvfqkl5-gcc-7.5.0/libexec/gcc/x86_64-unknown-linux-gnu/7.5.0/cc1 -E -quiet -v /dev/null -mtune=generic -march=x86-64
ignoring duplicate directory "/gnu/store/k9l4v4530p1a69j8qs0aijbmn8lwak20-profile/include"
ignoring nonexistent directory "/no-gcc-local-prefix/include"
ignoring nonexistent directory "/gnu/store/adrw71v03nawqwyblxc0mdhcc41j5vnn-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../../../../../x86_64-unknown-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /gnu/store/k9l4v4530p1a69j8qs0aijbmn8lwak20-profile/include
 /gnu/store/adrw71v03nawqwyblxc0mdhcc41j5vnn-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/include
 /gnu/store/adrw71v03nawqwyblxc0mdhcc41j5vnn-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/include-fixed
 /gnu/store/ccjj5wg0qkjs1zfjr98nhb8cpr434izw-glibc-2.31/include
End of search list.
# 1 "/dev/null"
# 1 "<built-in>"
# 1 "<command-line>"
# 31 "<command-line>"
# 1 "/gnu/store/k9l4v4530p1a69j8qs0aijbmn8lwak20-profile/include/stdc-predef.h" 1 3
# 32 "<command-line>" 2
# 1 "/dev/null"
COMPILER_PATH=/gnu/store/0pjrnr8vhp94ykvarysd9wg7hcvfqkl5-gcc-7.5.0/libexec/gcc/x86_64-unknown-linux-gnu/7.5.0/:/gnu/store/0pjrnr8vhp94ykvarysd9wg7hcvfqkl5-gcc-7.5.0/libexec/gcc/x86_64-unknown-linux-gnu/7.5.0/:/gnu/store/0pjrnr8vhp94ykvarysd9wg7hcvfqkl5-gcc-7.5.0/libexec/gcc/x86_64-unknown-linux-gnu/:/gnu/store/adrw71v03nawqwyblxc0mdhcc41j5vnn-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/:/gnu/store/adrw71v03nawqwyblxc0mdhcc41j5vnn-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/
LIBRARY_PATH=/gnu/store/k9l4v4530p1a69j8qs0aijbmn8lwak20-profile/lib/:/gnu/store/k9l4v4530p1a69j8qs0aijbmn8lwak20-profile/lib/:/gnu/store/adrw71v03nawqwyblxc0mdhcc41j5vnn-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/:/gnu/store/adrw71v03nawqwyblxc0mdhcc41j5vnn-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../:/gnu/store/ccjj5wg0qkjs1zfjr98nhb8cpr434izw-glibc-2.31/lib
COLLECT_GCC_OPTIONS='-v' '-E' '-mtune=generic' '-march=x86-64'

Looking at ‘cppdefault.c’ in GCC, I don’t see where glibc-2.31/include

comes from; it seems that ‘INCLUDE_DEFAULTS’ is undefined on glibc

systems.

Thoughts?

Incidentally, do we have problems building anything other than QEMU?

Thanks,

Ludo’.

Mathieu Othacehe wrote on 25 Feb 2020 15:34

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:875zfuag6v.fsf@gmail.com

Hey!

Toggle quote (8 lines)

> I’d rather go for #2. To do that, we could modify the ‘set-paths’ phase

> to manually remove glibc from C_INCLUDE_PATH (fragile), or we could

> modify GCC (perhaps removing the ‘remove_duplicates’ call for SYSTEM).

> Either way, this wouldn’t work well with ‘guix environment’, where glibc

> ends up in /gnu/store/…-profile, so it does not appear as duplicate to

> GCC.

[...]

Toggle quote (4 lines)

> Looking at ‘cppdefault.c’ in GCC, I don’t see where glibc-2.31/include

> comes from; it seems that ‘INCLUDE_DEFAULTS’ is undefined on glibc

> systems.

It's indeed undefined and glibc comes from NATIVE_SYSTEM_HEADER_DIR at

the end of the file you mentioned. It is a consequence of passing

--with-native-system-header-dir=glibc in (gnu packages gcc).

About the environment issue, we have the same problem on master. You can

run the following command:

Toggle quote (7 lines)>
> Thoughts?
>
> Incidentally, do we have problems building anything other than QEMU?
>
> Thanks,
> Ludo’.

Mathieu Othacehe wrote on 25 Feb 2020 15:46

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:874kveafns.fsf@gmail.com

Oops wrong shortcut, sorry!

Toggle quote (8 lines)

> I’d rather go for #2. To do that, we could modify the ‘set-paths’ phase

> to manually remove glibc from C_INCLUDE_PATH (fragile), or we could

> modify GCC (perhaps removing the ‘remove_duplicates’ call for SYSTEM).

> Either way, this wouldn’t work well with ‘guix environment’, where glibc

> ends up in /gnu/store/…-profile, so it does not appear as duplicate to

> GCC.

[...]

Toggle quote (4 lines)

> Looking at ‘cppdefault.c’ in GCC, I don’t see where glibc-2.31/include

> comes from; it seems that ‘INCLUDE_DEFAULTS’ is undefined on glibc

> systems.

It's indeed undefined and glibc comes from NATIVE_SYSTEM_HEADER_DIR at

the end of the file you mentioned. It is a consequence of passing

--with-native-system-header-dir=glibc in (gnu packages gcc).

About the environment issue, we have the same problem on master. You can

run the following command:

Toggle snippet (5 lines)

./pre-inst-env guix environment -C -e '(@@ (gnu packages commencement)

coreutils-final)' -- echo -e '#include <stdint.h>\n int main() {return

0;}' > test.c && gcc -m16 -ffreestanding test.c

and see that in takes stdint.h from the profile glibc header:

Toggle snippet (9 lines)

In file included from /gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/features.h:474:0,

from /gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/bits/libc-header-start.h:33,

from /gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/stdint.h:26,

from test.c:1:

/gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/gnu/stubs.h:7:11: fatal error: gnu/stubs-32.h: No such file or directory

# include <gnu/stubs-32.h>

^~~~~~~~~~~~~~~~

So if it's ok for you, I'll try to implement a GCC hack so that we can

keep using C_INCLUDE_PATH on core-updates and have QEMU building, as you

proposed.

About the environment use-case, it's getting really tricky, but as it is

not a regression, we can maybe postpone the resolution.

Toggle quote (2 lines)

> Incidentally, do we have problems building anything other than QEMU?

I don't know, but potentially any program building with -m<something>

and -ffreestanding fails on core-updates.

Thanks for your help :),

Mathieu

Ludovic Courtès wrote on 26 Feb 2020 21:55

Recipients:(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)

Message-ID:874kvdcbl5.fsf@gnu.org

Hi,

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (18 lines)> About the environment issue, we have the same problem on master. You can
> run the following command:
>
>  ./pre-inst-env guix environment -C -e '(@@ (gnu packages commencement)
>  coreutils-final)' -- echo -e '#include <stdint.h>\n int main() {return
>  0;}' > test.c && gcc -m16 -ffreestanding test.c
>
>
> and see that in takes stdint.h from the profile glibc header:
>
> In file included from /gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/features.h:474:0,
>                  from /gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/bits/libc-header-start.h:33,
>                  from /gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/stdint.h:26,
>                  from test.c:1:
> /gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/gnu/stubs.h:7:11: fatal error: gnu/stubs-32.h: No such file or directory
>  # include <gnu/stubs-32.h>
>            ^~~~~~~~~~~~~~~~

Indeed.

Toggle quote (7 lines)

> So if it's ok for you, I'll try to implement a GCC hack so that we can

> keep using C_INCLUDE_PATH on core-updates and have QEMU building, as you

> proposed.

> About the environment use-case, it's getting really tricky, but as it is

> not a regression, we can maybe postpone the resolution.

Yes, both make sense to me.

Toggle quote (5 lines)

>> Incidentally, do we have problems building anything other than QEMU?

> I don't know, but potentially any program building with -m<something>

> and -ffreestanding fails on core-updates.

The evil idea I was getting at was that, if that’s just a couple of

packages, we can fix them locally. Evil plan in case the better hack

turns out to be tricky. :-)

Ludo’.

Marius Bakke wrote on 26 Feb 2020 22:12

Recipients:(address . 36882@debbugs.gnu.org)

Message-ID:87lfop2gu0.fsf@devup.no

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (2 lines)

> Incidentally, do we have problems building anything other than QEMU?

The only other regression I've noticed with the C_INCLUDE_PATH change is

that GHC 8.4 fails to build -- previously we at least got to GHC 8.6.

The error message does not make much sense to me (something about

libffi.so not found), but it must be related to the C_INCLUDE_PATH

change since it built before.

So I think we can work around these problems for now and try to fix it

properly in the next core-updates cycle (rumor has it that it includes a

switch to GCC 9 as well).

Marius Bakke wrote on 2 Mar 2020 23:01

Recipients:(address . 36882@debbugs.gnu.org)

Message-ID:874kv6ju1c.fsf@devup.no

Hi Mathieu,

Mathieu Othacehe <m.othacehe@gmail.com> writes:

Toggle quote (4 lines)

> So if it's ok for you, I'll try to implement a GCC hack so that we can

> keep using C_INCLUDE_PATH on core-updates and have QEMU building, as you

> proposed.

Did you get anywhere with this?  As Ludovic mentioned, it might make
sense to work around it in gnu-build-system too if patching GCC turns
out to be difficult.

If we can't find a workaround in the coming weeks, perhaps we should
postpone the issue and add package-specific workarounds in the mean
time?

Mathieu Othacehe wrote on 3 Mar 2020 08:39

Recipients:(name . Marius Bakke)(address . mbakke@fastmail.com)

Message-ID:878skh3n0o.fsf@gmail.com

Hey Marius,

Toggle quote (4 lines)

> Did you get anywhere with this? As Ludovic mentioned, it might make

> sense to work around it in gnu-build-system too if patching GCC turns

> out to be difficult.

Yup, turned out patching GCC was too difficult. I'm experimenting a

filter over inputs passed to set-path-environment-variable in set-paths.

I hope to send this hack to review later this week.

Thanks,

Mathieu

Mathieu Othacehe wrote on 3 Mar 2020 12:55

Recipients:(name . Marius Bakke)(address . mbakke@fastmail.com)

Message-ID:871rq9ejpb.fsf@gmail.com

Toggle quote (3 lines)

> Yup, turned out patching GCC was too difficult. I'm experimenting a

> filter over inputs passed to set-path-environment-variable in set-paths.

This is also quite tricky, because the "libc" input passed to set-paths

must not be removed from C_INCLUDE_PATH in (gnu packages commencement)

for mes packages.

So, maybe the best option is to patch QEMU directly, as proposed in

the attached patch. WDYT?

Thanks,

Mathieu

From 6d5dea746ce7b501cdb5ba41c5eff0c9ea7eb0b0 Mon Sep 17 00:00:00 2001

This fixes https://issues.guix.info/issue/36882.

* gnu/packages/virtualization.scm (qemu-minimal)[arguments]: Add a

remove-glibc phase to remove glibc from C_INCLUDE_PATH.

---

gnu/packages/virtualization.scm | 23 ++++++++++++++++++++++-

1 file changed, 22 insertions(+), 1 deletion(-)

Toggle diff (52 lines)diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.scm
index e16405fc9c..6ab8c3802c 100644
--- a/gnu/packages/virtualization.scm
+++ b/gnu/packages/virtualization.scm
@@ -11,6 +11,7 @@
 ;;; Copyright © 2018 Sou Bunnbu <iyzsong@member.fsf.org>
 ;;; Copyright © 2018 Julien Lepiller <julien@lepiller.eu>
 ;;; Copyright © 2019 Guy Fleury Iteriteka <hoonandon@gmail.com>
+;;; Copyright © 2020 Mathieu Othacehe <m.othacehe@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -124,7 +125,7 @@
                "1w38hzlw7xp05gcq1nhga7hxvndxy6dfcnzi7q2il8ff110isj6k"))))
     (build-system gnu-build-system)
     (arguments
-     '(;; Running tests in parallel can occasionally lead to failures, like:
+     `(;; Running tests in parallel can occasionally lead to failures, like:
        ;; boot_sector_test: assertion failed (signature == SIGNATURE): (0x00000000 == 0x0000dead)
        #:parallel-tests? #f
        #:configure-flags (list "--enable-usb-redir" "--enable-opengl"
@@ -135,8 +136,28 @@
                                "--audio-drv-list=alsa,pa,sdl")
        ;; Make build and test output verbose to facilitate investigation upon failure.
        #:make-flags '("V=1")
+       #:modules ((srfi srfi-1)
+                  (ice-9 match)
+                  ,@%gnu-build-system-modules)
        #:phases
        (modify-phases %standard-phases
+         ;; Work around https://issues.guix.info/issue/36882.  We need to
+         ;; remove glibc from C_INCLUDE_PATH so that the one hardcoded in GCC,
+         ;; at the bottom of GCC include search-path is used.
+         (add-after 'set-paths 'remove-glibc
+          (lambda* (#:key inputs #:allow-other-keys)
+            (let* ((filters '("libc"))
+                   (input-directories
+                    (filter-map (lambda (input)
+                                  (match input
+                                    ((name . dir)
+                                     (and (not (member name filters))
+                                          dir))))
+                                inputs)))
+              (set-path-environment-variable "C_INCLUDE_PATH"
+                                             '("include")
+                                             input-directories)
+              #t)))
          (replace 'configure
            (lambda* (#:key inputs outputs (configure-flags '())
                            #:allow-other-keys)
-- 
2.25.1

Marius Bakke wrote on 3 Mar 2020 21:26

Recipients:(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)

Message-ID:87v9nli3rj.fsf@devup.no

Hello Mathieu & Ludo,

(+ janneke)

Mathieu Othacehe <m.othacehe@gmail.com> writes:

Toggle quote (7 lines)

>> Yup, turned out patching GCC was too difficult. I'm experimenting a

>> filter over inputs passed to set-path-environment-variable in set-paths.

> This is also quite tricky, because the "libc" input passed to set-paths

> must not be removed from C_INCLUDE_PATH in (gnu packages commencement)

> for mes packages.

If that is the only issue preventing us from solving this problem for

all build systems, I'd say let's fix that. We could rename it obviously,

though there are a few places that expect the "libc" key so it may become

unwieldly.

Can we find another way to detect the Mes toolchain, perhaps directly in

gnu-build-system where we apply this workaround? E.g. look for inputs

where (string-contains (package-name input) "mesboot")?

Suggestions?

Marius Bakke wrote on 3 Mar 2020 21:37

Recipients:(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)

Message-ID:87pndti39k.fsf@devup.no

Marius Bakke <mbakke@fastmail.com> writes:

Toggle quote (24 lines)> Hello Mathieu & Ludo,
>
> (+ janneke)
>
> Mathieu Othacehe <m.othacehe@gmail.com> writes:
>
>>> Yup, turned out patching GCC was too difficult. I'm experimenting a
>>> filter over inputs passed to set-path-environment-variable in set-paths.
>>
>> This is also quite tricky, because the "libc" input passed to set-paths
>> must not be removed from C_INCLUDE_PATH in (gnu packages commencement)
>> for mes packages.
>
> If that is the only issue preventing us from solving this problem for
> all build systems, I'd say let's fix that.  We could rename it obviously,
> though there are a few places that expect the "libc" key so it may become
> unwieldly.
>
> Can we find another way to detect the Mes toolchain, perhaps directly in
> gnu-build-system where we apply this workaround?  E.g. look for inputs
> where (string-contains (package-name input) "mesboot")?
>
> Suggestions?

I realize we don't have access to PACKAGE-NAME in the builder, though we

could check for the string in the store item names. 'mesboot' contains

an 'e' and is thus guaranteed not to end up in the hash.

Not a very elegant solution though, hoping for other suggestions. :-)

Jan Nieuwenhuizen wrote on 3 Mar 2020 22:09

Re: bug#36882: Qemu 4.2.0 build for x86_64-linux fails

Recipients:(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)

Message-ID:87k141rvq3.fsf@gnu.org

Mathieu Othacehe <m.othacehe@gmail.com> writes:

Hello!

Toggle quote (7 lines)

>> Yup, turned out patching GCC was too difficult. I'm experimenting a

>> filter over inputs passed to set-path-environment-variable in set-paths.

> This is also quite tricky, because the "libc" input passed to set-paths

> must not be removed from C_INCLUDE_PATH in (gnu packages commencement)

> for mes packages.

Do you have a patch/branch where I can see this related error? If mes

builds are an exception, it could be nice to solve it there? Or is

this a Qemu thing and is Qemu the only package that suffers from this?

Trying to help here, but I'm not sure if I understand what's going on...

Greetings,

janneke

Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org

Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com

Mathieu Othacehe wrote on 4 Mar 2020 09:16

Recipients:(name . Jan (janneke) Nieuwenhuizen)(address . janneke@gnu.org)

Message-ID:87v9nkedr2.fsf@gmail.com

Hello Janneke,

Toggle quote (6 lines)

> Do you have a patch/branch where I can see this related error? If mes

> builds are an exception, it could be nice to solve it there? Or is

> this a Qemu thing and is Qemu the only package that suffers from this?

> Trying to help here, but I'm not sure if I understand what's going on...

To summarize the situation, we are considering adding a dirty hack in

gnu-build-system so that "libc" input is removed from

C_INCLUDE_PATH. The reason is that GCC has already a libc hardcoded in

its include search-path. Having both libc, causes build failures for

QEMU on core-updates and possibly other programs playing around

-m<something> and -ffreestanding in GCC.

Anyway, Mes does not break anything, it's just that removing "libc"

input unconditionally in set-paths breaks Mes packages. That's why

Marius was suggesting renaming "libc" to "mes-libc" or something, so

that we can proceed.

Thanks,

Mathieu

Ludovic Courtès wrote on 5 Mar 2020 17:36

Re: Qemu 4.2.0 build for x86_64-linux fails

Recipients:(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)

Message-ID:87v9ni92tf.fsf@gnu.org

Hi!

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (7 lines)

>> Yup, turned out patching GCC was too difficult. I'm experimenting a

>> filter over inputs passed to set-path-environment-variable in set-paths.

> This is also quite tricky, because the "libc" input passed to set-paths

> must not be removed from C_INCLUDE_PATH in (gnu packages commencement)

> for mes packages.

Yeah, fiddling with ‘set-paths’ sounds like opening a can of worms.

Toggle quote (3 lines)

> So, maybe the best option is to patch QEMU directly, as proposed in

> the attached patch. WDYT?

I’m all in favor of that for now.  It’s likely that QEMU is the only
package having problems, as Marius noted (the GHC error sounds like it
could be something different), so I’d say go for the package-specific
solution for now.

Thanks,
Ludo’.

Marius Bakke wrote on 5 Mar 2020 17:42

Recipients:(address . 36882@debbugs.gnu.org)

Message-ID:87a74ulpnl.fsf@devup.no

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (21 lines)> Hi!
>
> Mathieu Othacehe <m.othacehe@gmail.com> skribis:
>
>>> Yup, turned out patching GCC was too difficult. I'm experimenting a
>>> filter over inputs passed to set-path-environment-variable in set-paths.
>>
>> This is also quite tricky, because the "libc" input passed to set-paths
>> must not be removed from C_INCLUDE_PATH in (gnu packages commencement)
>> for mes packages.
>
> Yeah, fiddling with ‘set-paths’ sounds like opening a can of worms.
>
>> So, maybe the best option is to patch QEMU directly, as proposed in
>> the attached patch. WDYT?
>
> I’m all in favor of that for now.  It’s likely that QEMU is the only
> package having problems, as Marius noted (the GHC error sounds like it
> could be something different), so I’d say go for the package-specific
> solution for now.

Indeed, I will push Mathieus commit shortly.

clang-runtime requires a similar fix for CPLUS_INCLUDE_PATH. No idea

what's up with GHC@8.4 yet, but unsurprisingly removing libc from the

include paths does not make a difference.

Mathieu Othacehe wrote on 6 Mar 2020 08:25

Recipients:(name . Marius Bakke)(address . mbakke@fastmail.com)

Message-ID:87tv32djxg.fsf@gmail.com

Hey,

Toggle quote (4 lines)

> clang-runtime requires a similar fix for CPLUS_INCLUDE_PATH. No idea

> what's up with GHC@8.4 yet, but unsurprisingly removing libc from the

> include paths does not make a difference.

Closing this one, thanks for pushing!

Mathieu

Closed

Your comment

This issue is archived.

To comment on this conversation send an email to 36882@debbugs.gnu.org

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date