clang: missing default include paths for C++

Hi,

I noticed the following bug in clang when installed through guix:

Compiling C++ programs does not work because the include path is not set

correctly.

I will use the following test program for compling:

?????

? #include <iostream>

?

? int

? main()

? {

? std::cout << "Hello, World\n";

? }

?????

When I now compile using clang I get the following error message:

?????

? which clang++

? clang++ test.cc 2>&1

? exit 0

?????

?????

? /home/tibbe/.guix-profile/bin/clang++

? test.cc:1:10: fatal error: 'iostream' file not found

? #include <iostream>

? ^~~~~~~~~~

? 1 error generated.

?????

?????

? which g++

? g++ test.cc 2>&1

? exit 0

?????

?????

? /home/tibbe/.guix-profile/bin/g++

?????

As you can see g++ has no problem compiling the code, but clang++ cannot

find the `<iostream>' header. This is due to the already mentioned bad

include paths:

?????

? g++ -v test.cc 2>&1

? clang++ -v test.cc 2>&1

? exit 0

?????

?????

? Using built-in specs.

? COLLECT_GCC=g++

?

COLLECT_LTO_WRAPPER=/gnu/store/758qfjvn4wnsrhrr08pmwfr77vmcq4q2-gcc-8.2.0/libexec/gcc/x86_64-unknown-linux-gnu/8.2.0/lto-wrapper

? Target: x86_64-unknown-linux-gnu

? Configured with:

? Thread model: posix

? gcc version 8.2.0 (GCC)

? COLLECT_GCC_OPTIONS='-v' '-shared-libgcc' '-mtune=generic' '-march=x86-64'

?

/gnu/store/758qfjvn4wnsrhrr08pmwfr77vmcq4q2-gcc-8.2.0/libexec/gcc/x86_64-unknown-linux-gnu/8.2.0/cc1plus

-quiet -v -D_GNU_SOURCE test.cc -quiet -dumpbase test.cc -mtune=generic

-march=x86-64 -auxbase test -version -o /tmp/cceSqDtK.s

? GNU C++14 (GCC) version 8.2.0 (x86_64-unknown-linux-gnu)

? compiled by GNU C version 8.2.0, GMP version 6.1.2, MPFR version

4.0.1, MPC version 1.1.0, isl version isl-0.18-GMP

?

? GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072

? ignoring nonexistent directory "/no-gcc-local-prefix/include"

? ignoring nonexistent directory

"/gnu/store/bmaxmigwnlbdpls20px2ipq1fll36ncd-gcc-8.2.0-lib/lib/gcc/x86_64-unknown-linux-gnu/8.2.0/../../../../../../../x86_64-unknown-linux-gnu/include"

? #include "..." search starts here:

? #include <...> search starts here:

? /home/tibbe/.guix-profile/include

? /gnu/store/758qfjvn4wnsrhrr08pmwfr77vmcq4q2-gcc-8.2.0/include/c++

?

/gnu/store/758qfjvn4wnsrhrr08pmwfr77vmcq4q2-gcc-8.2.0/include/c++/x86_64-unknown-linux-gnu

?

/gnu/store/758qfjvn4wnsrhrr08pmwfr77vmcq4q2-gcc-8.2.0/include/c++/backward

?

/gnu/store/bmaxmigwnlbdpls20px2ipq1fll36ncd-gcc-8.2.0-lib/lib/gcc/x86_64-unknown-linux-gnu/8.2.0/include

?

/gnu/store/bmaxmigwnlbdpls20px2ipq1fll36ncd-gcc-8.2.0-lib/lib/gcc/x86_64-unknown-linux-gnu/8.2.0/include-fixed

? /gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/include

? End of search list.

? GNU C++14 (GCC) version 8.2.0 (x86_64-unknown-linux-gnu)

? compiled by GNU C version 8.2.0, GMP version 6.1.2, MPFR version

4.0.1, MPC version 1.1.0, isl version isl-0.18-GMP

?

? GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072

? Compiler executable checksum: 238b7d99644945f4ccaa2a89b02dcd25

? COLLECT_GCC_OPTIONS='-v' '-shared-libgcc' '-mtune=generic' '-march=x86-64'

? as -v --64 -o /tmp/ccZKb9XQ.o /tmp/cceSqDtK.s

? GNU assembler version 2.30 (x86_64-unknown-linux-gnu) using BFD

version (GNU Binutils) 2.30

?

COMPILER_PATH=/gnu/store/758qfjvn4wnsrhrr08pmwfr77vmcq4q2-gcc-8.2.0/libexec/gcc/x86_64-unknown-linux-gnu/8.2.0/:/gnu/store/758qfjvn4wnsrhrr08pmwfr77vmcq4q2-gcc-8.2.0/libexec/gcc/x86_64-unknown-linux-gnu/8.2.0/:/gnu/store/758qfjvn4wnsrhrr08pmwfr77vmcq4q2-gcc-8.2.0/libexec/gcc/x86_64-unknown-linux-gnu/:/gnu/store/bmaxmigwnlbdpls20px2ipq1fll36ncd-gcc-8.2.0-lib/lib/gcc/x86_64-unknown-linux-gnu/8.2.0/:/gnu/store/bmaxmigwnlbdpls20px2ipq1fll36ncd-gcc-8.2.0-lib/lib/gcc/x86_64-unknown-linux-gnu/

?

LIBRARY_PATH=/home/tibbe/.guix-profile/lib/:/home/tibbe/.guix-profile/lib/:/gnu/store/bmaxmigwnlbdpls20px2ipq1fll36ncd-gcc-8.2.0-lib/lib/gcc/x86_64-unknown-linux-gnu/8.2.0/:/gnu/store/bmaxmigwnlbdpls20px2ipq1fll36ncd-gcc-8.2.0-lib/lib/gcc/x86_64-unknown-linux-gnu/8.2.0/../../../:/gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/lib

? COLLECT_GCC_OPTIONS='-v' '-shared-libgcc' '-mtune=generic' '-march=x86-64'

?

/gnu/store/758qfjvn4wnsrhrr08pmwfr77vmcq4q2-gcc-8.2.0/libexec/gcc/x86_64-unknown-linux-gnu/8.2.0/collect2

-plugin

/gnu/store/758qfjvn4wnsrhrr08pmwfr77vmcq4q2-gcc-8.2.0/libexec/gcc/x86_64-unknown-linux-gnu/8.2.0/liblto_plugin.so

-plugin-opt=/gnu/store/758qfjvn4wnsrhrr08pmwfr77vmcq4q2-gcc-8.2.0/libexec/gcc/x86_64-unknown-linux-gnu/8.2.0/lto-wrapper

-plugin-opt=-fresolution=/tmp/ccKNZuuX.res

-plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc

-plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc

-plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc

--eh-frame-hdr -m elf_x86_64 -dynamic-linker

/gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/lib/ld-linux-x86-64.so.2

/home/tibbe/.guix-profile/lib/crt1.o

/home/tibbe/.guix-profile/lib/crti.o

/gnu/store/bmaxmigwnlbdpls20px2ipq1fll36ncd-gcc-8.2.0-lib/lib/gcc/x86_64-unknown-linux-gnu/8.2.0/crtbegin.o

-L/home/tibbe/.guix-profile/lib -L/home/tibbe/.guix-profile/lib

-L/gnu/store/bmaxmigwnlbdpls20px2ipq1fll36ncd-gcc-8.2.0-lib/lib/gcc/x86_64-unknown-linux-gnu/8.2.0

-L/gnu/store/bmaxmigwnlbdpls20px2ipq1fll36ncd-gcc-8.2.0-lib/lib/gcc/x86_64-unknown-linux-gnu/8.2.0/../../..

-L/gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/lib

/tmp/ccZKb9XQ.o -lstdc++ -lm -lgcc_s -lgcc

-L/gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/lib

-rpath=/gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/lib

-rpath=/gnu/store/bmaxmigwnlbdpls20px2ipq1fll36ncd-gcc-8.2.0-lib/lib

-lgcc_s -lc -lgcc_s -lgcc

/gnu/store/bmaxmigwnlbdpls20px2ipq1fll36ncd-gcc-8.2.0-lib/lib/gcc/x86_64-unknown-linux-gnu/8.2.0/crtend.o

/home/tibbe/.guix-profile/lib/crtn.o

? COLLECT_GCC_OPTIONS='-v' '-shared-libgcc' '-mtune=generic' '-march=x86-64'

? clang version 6.0.0 (tags/RELEASE_600/final)

? Target: x86_64-unknown-linux-gnu

? Thread model: posix

? InstalledDir: /home/tibbe/.guix-profile/bin

? Found candidate GCC installation:

/gnu/store/bgc2ckrkyz5fg4sp278msyzxs5a30pwa-gcc-5.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/5.5.0

? Selected GCC installation:

/gnu/store/bgc2ckrkyz5fg4sp278msyzxs5a30pwa-gcc-5.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/5.5.0

? Candidate multilib: .;@m64

? Selected multilib: .;@m64

?

"/gnu/store/dy7lcm3d0dbh4510vkc0dv35g48pis50-clang-6.0.0/bin/clang-6.0"

-cc1 -triple x86_64-unknown-linux-gnu -emit-obj -mrelax-all

-disable-free -disable-llvm-verifier -discard-value-names

-main-file-name test.cc -mrelocation-model static -mthread-model posix

-mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases

-munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info

-debugger-tuning=gdb -v -resource-dir

/gnu/store/dy7lcm3d0dbh4510vkc0dv35g48pis50-clang-6.0.0/lib/clang/6.0.0

-I/home/tibbe/.guix-profile/include -internal-isystem /usr/local/include

-internal-isystem

/gnu/store/dy7lcm3d0dbh4510vkc0dv35g48pis50-clang-6.0.0/lib/clang/6.0.0/include

-internal-externc-isystem

/gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/include

-fdeprecated-macro -fdebug-compilation-dir /home/tibbe/src/hello-world

-ferror-limit 19 -fmessage-length 0 -fobjc-runtime=gcc -fcxx-exceptions

-fexceptions -fdiagnostics-show-option -o /tmp/test-b83cb1.o -x c++ test.cc

? clang -cc1 version 6.0.0 based upon LLVM 6.0.0 default target

x86_64-unknown-linux-gnu

? #include "..." search starts here:

? #include <...> search starts here:

? /home/tibbe/.guix-profile/include

? /usr/local/include

?

/gnu/store/dy7lcm3d0dbh4510vkc0dv35g48pis50-clang-6.0.0/lib/clang/6.0.0/include

? /gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/include

? End of search list.

? test.cc:1:10: fatal error: 'iostream' file not found

? #include <iostream>

? ^~~~~~~~~~

? 1 error generated.

?????

As you can see from the output, clang is missing some include paths that

gcc has. Specifying a custom `CPLUS_INCLUDE_PATH' fixes the problem:

?????

?

CPLUS_INCLUDE_PATH=$HOME/.guix-profile/include/c++:$HOME/.guix-profile/include/c++/x86_64-unknown-linux-gnu/

clang++ test.cc

? ./a.out

?????

?????

? Hello, World

?????

This is already done in the package definition for the

`C_INCLUDE_PATH'. It is not done for C++ because clang does not

implement a feature or build system variable for changing it.

Fixing this problem would probably include an upstream patch enabling a

similar feature for C++ for what is already done in C and configuring

this variable in build phase to add the same include paths that g++ has.

Tim.

On 19.09.2018 20:33, Pjotr Prins wrote:

Yes, that seems to be quite the same to my workaround.

Its just that this is a workaround that is difficult to get working in

some contexts: I want to use emacs-irony-mode through guix which is not

really useable because it won't autocomplete any std::* stuff.

If you take all packages that might want to use libclang and other

features of clang it might be a better solution to find a proper fix for

this problem. Also both workarounds need a user profile that is

cluttered with all include files.

I had a quick look into clangs source code how C_INCLUDE_DIRS is

implemented. It should be more or less easy to add the same option for

C++ (even C_INCLUDE_DIRS seems to be tinkered in to me).

I just wanted to file a bug about this because fixing this is not

trivial and I am not sure whether I will find the time right away to fix it.

Fixing it would also have the benefit that I could send the patch to the

LLVM mailing list and we might see the change upstream in the next LLVM

version.

Tim.

Hi Tim,

Tim Gesthuizen <tim.gesthuizen@yahoo.de> writes:

Another solution, maybe simpler than a new environment variable, is to

have clang use the C++ include path from its gcc input. On FHS systems,

clang can find C++ headers using the GCC_INSTALL_PREFIX configure

option, but it doesn't work under Guix because the GCC package puts

headers and libraries in separate outputs. Guix already patches clang to

hardcode some library directories; maybe something similar could be done

for C++ headers. (I think the function to modify for this would be

Linux::addLibStdCxxIncludePaths in lib/Driver/ToolChains/Linux.cpp.)

On 22.09.2018 02:58, Robin Templeton wrote:

Hi Robin,

I also found that section and the environment variable while debugging

clang. I did not know about that variable even though its documented and

Guix uses it. I've created a debug build of clang for investigating and

pointing GCC_INSTALL_PREFIX to the GCC input and not the lib part fixes

the problem for me. I don't know from where this build pulls crt1.o. I

will try changing GCC_INSTALL_PREFIX in the guix package definition and

see whether that fixes the bug.

Tim.

Tim Gesthuizen writes:

Just put everything into `C_INCLUDE_DIRS' should make it work, see:

Using clang/clang++ from above link to compile a single translation unit should

work. And there is `cquery' in `code.scm' within that repository, which relies

on clang to work, you can try it out.

It shouldn't take long to make LLVM and Clang in there ready to be merge in

core-update or staging (I don't know yet), it's just I don't feel comfortable

with the `clang-from-llvm' in guix, maybe someone could help carrying it out.

Jiaming

---

gnu/packages/llvm.scm | 5 +++++

1 file changed, 5 insertions(+)

Hi David!

This patch is most welcome, I’m wondering if you think it’d be possible to add more details in the comments for future readers of the code. See below.

I believe the reason why clang’s “auto detection” fails here is because it expects the /include dir containing c++ to be under the same directory as LibDir, correct? Perhaps we can note that since in Guix, we put the libs in a separate output, we invalidate the aforementioned expectation, and this is why we need to manually add this path.

Another point: Perhaps we can add the path on a separate line instead of it being on the same line?

Otherwise, looks great!

Cheers,

Carl Dong

Hello David,

This patch is a real progress, thank you! However, I still have a

linking error when compiling a trivial test program with clang:

mathieu@elbruz ~/guix [env]$ ./pre-inst-env guix environment --ad-hoc clang --pure

mathieu@elbruz ~/guix [env]$ clang ~/tmp/t.cpp

clang-8: error: unable to execute command: Executable "ld" doesn't exist!

clang-8: error: linker command failed with exit code 1 (use -v to see invocation)

This is because the linker provided by "binutils" is not accessible. I

guess we need to make something analog to make-gcc-toolchain.

WDYT?

Mathieu

David Truby writes:

Yes propagating gcc-toolchain would work but, would also cause gcc to be

available as a side effect. Maybe the patch attached, on top of yours,

would work?

Thanks,

Mathieu

Hi Mathieu,

Interesting, for some reason this worked for me. Although now that I

think about it I may not have used a pure environment, because I do

remember seeing this before.

I think adding gcc-toolchain to propagated-inputs should work in

principle, other than the fact that the gcc used for gcc-lib and

libstdc++ need to be the same as the propagated gcc-toolchain. I did

have a fix for this but I'm not sure if it's the best way of writing

it.

I'll send another patch with the fix though so at least others can

comment on it!

David Truby

Hi Mathieu,

Applying both your patch and my patch works for me, in both pure and

non-pure environments. I would suggest taht we also hide clang in the

same way as gcc is hidden (in favour of clang-toolchain) to avoid

confusion, but otherwise it seems these two patches together at least

get things to a working state!

I'd like to see make-clang-toolchain allow you to pick between a

specific libstdc++ or libc++, but that requires more thinking so I

think ideally it'd be better to apply these patches first to get things

to a fixed state.

Thanks

David Truby

On Thu, 2019-11-14 at 14:40 +0100, Mathieu Othacehe wrote:

Great!

I agree. Let's wait to see if someone has objections and I'll push both

patches in a few days.

Thanks for your support on this :)

Mathieu

No problem, happy to help!

I'm also vaguely thinking if there's a way to upstream some of this

stuff in to clang so that we aren't patching it manually here. The same

file that we're patching has stuff specifically for other distros (e.g.

it has some Gentoo and Cray Linux specific fixes) so I don't think

they'd be opposed to accepting patches that are specific to Guix.

I'm not sure what the answer is there though, as we still need to patch

in the full path to the headers... maybe a cmake option and associated

macro to be used in that file?

Not that this has any bearing on accepting these patches now, it's just

another thought for the future :)

David Truby

On Thu, 2019-11-14 at 17:54 +0100, Mathieu Othacehe wrote:

They already have a CMake option to select libc++ or libstdc++, so

adding another option to specify the path of the selected C++ library

would be really nice.

Plus it would benefit Nix and other distros where the path of those

libraries cannot be guessed.

Mathieu

Hello,

I edited your patch to include a copyright line, adapted commit log

message to our conventions and pushed it! Please tell me if you want a

different copyright.

I also pushed my patch adding clang-toolchain packages.

Thanks,

Mathieu

Hello,

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Isn’t it enough to add ‘ld-wrapper’ to the environment?

Thanks,

Ludo’.

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Nice, LGTM!

Thanks,

Ludo’.

Hi Mathier,

Thanks for merging this! Would it be possible to change the copyright

to Arm Ltd. but leave my email as the point of contact? Please let me

know if this is an issue.

Thanks

David Truby

On Sun, 2019-11-17 at 12:21 +0100, Mathieu Othacehe wrote:

No I guess its ok. Pushed!

Mathieu