Hi all! TLDR if you just want to know what's the final patch that worked, grep for"final patch which works is the following" This is a long email, but it is only long because I want to be very clear and Iinclude a lot of snippets of code and logs. I originally wrote my NSIS package (introduced in e214a22007) on top of7150743522, which was before the GREAT CORE-UPDATES MERGE. However, afterrebasing it over the GREAT CORE-UPDATES MERGE, the package stopped working. Myinvestigations into this failure has revealed some insights which might beuseful for other cross-compiling packages and for resolving the long-standing"important" issue #30756. For some context, the NSIS package is somewhat special. It requires that bothnative compilers and mingw-w64 cross-compilers be available, as NSIS is aprogram used to create installers meant to be run on Windows. My original method of achieving this in pre-GREAT CORE-UPDATES MERGE Guix can besee in e214a22007, whereby I added a 'fix-env phase. In that 'fix-env phase, Iwould remove mingw-w64 paths from CPLUS_INCLUDE_PATH, C_INCLUDE_PATH, andLIBRARY_PATH, and put them in CROSS_CPLUS_INCLUDE_PATH, CROSS_C_INCLUDE_PATH,and CROSS_LIBRARY_PATH respectively. This way, the native compilers won't thinkthat the mingw-w64 paths were something they were supposed to use and getconfused. As part of the GREAT CORE-UPDATES MERGE, it would seem that CPLUS_INCLUDE_PATHand C_INCLUDE_PATH are no longer set in the build environment. Rather, onlyCPATH is set. Now, you would think that the solution here is just to replaceCPLUS_INCLUDE_PATH and C_INCLUDE_PATH in my 'fix-env phase with CPATH like so:
Hi Carl, On Thu, 17 Oct 2019 21:56:44 +0000Carl Dong <email@example.com> wrote:
Toggle quote (4 lines)> Note that the reason mingw-w64-x86_64-6.0.0/include is in this list at all is> because of the 'fix-env phase I added, which plucked it from CPATH and plopped> it into CROSS_CPATH.
Yeah, I did the same in sunxi-tools--which was also broken after the core-updatesmerge, but had been working fine before.It's using an x86_64->ARM cross compiler.But just changing it to CROSS_CPATH works, there.
Toggle quote (3 lines)> 3. Does this reveal something more fundamentally wrong with how we build our> search paths in the first place that should be addressed
I think so. I can't figure out why Guix is not just setting up CROSS_CPATHon its own in the first place.gnu/packages/cross-base.scm DOES have a search-path specification for CROSS_CPATH. Notes: * CPATH is something like "-I", but CPATH applies after all command-line "-I"s.* C_INCLUDE_PATH, CPLUS_INCLUDE_PATH and OBJC_INCLUDE_PATH are something like"-isystem", but they apply after all command-line "-isystem"s. Note that an empty (colon-separated) element in those environment variablesmeans "current working directory". https://gcc.gnu.org/onlinedocs/gcc-6.2.0/gcc/Directory-Options.html:
Toggle quote (9 lines)>If a standard system include directory, or a directory specified with>-isystem, is also specified with -I, the -I option is ignored. The>directory is still searched but as a system directory at its normal>position in the system include chain. This is to ensure that GCC's>procedure to fix buggy system headers and the ordering for the>include_next directive are not inadvertently changed. If you really>need to change the search order for system directories, use the>-nostdinc and/or -isystem options.
Toggle quote (7 lines)>The lookup order is as follows:> For the quote form of the include directive, the directory of the current file is searched first.> For the quote form of the include directive, the directories specified by -iquote options are searched in LTR order.> Directories specified with -I options are scanned in left-to-right order.> Directories specified with -isystem options are scanned in left-to-right order.> Standard system directories are scanned[, except if -nostdinc[++] is specified].> Directories specified with -idirafter options are scanned in left-to-right order.
Hi Danny, Thank you so much for the links and quotes, I'm definitely going to refer backto them in the future and you probably saved me dozens of hours :-)
Toggle quote (5 lines)> I think so. I can't figure out why Guix is not just setting up CROSS_CPATH> on its own in the first place.> gnu/packages/cross-base.scm DOES have a search-path specification for> CROSS_CPATH.
Perhaps Ludovic can confirm this, but I believe the reason why Guix is notsetting up CROSS_CPATH is because it doesn't _know_ it's cross-compiling. Guixonly sets up CROSS_CPATH when we invoke on the command line with--target=x86_64-w64-mingw32 or something like that. I'm not exactly sure what aclean solution to this is, but I'd hope we can find one in the future. I'm thinking that the reason why my final solution involved explicitly settingthe exact ordering in my CROSS_CPLUS_INCLUDE_PATH was because mingw-w64 isconsidered to be libc and that makes it special somehow. Not 100% sure though. Cheers,Carl Dongcontact@carldong.me"I fight for the users"
Hi, On Fri, 18 Oct 2019 14:02:33 +0000Carl Dong <firstname.lastname@example.org> wrote:
Toggle quote (3 lines)> Perhaps Ludovic can confirm this, but I believe the reason why Guix is not> setting up CROSS_CPATH is because it doesn't _know_ it's cross-compiling.
Ah, that makes a lot of sense! But the "cross-gcc" returns a cross compilerthat should know that we're cross-compiling (but it sets up SEARCH-PATHSwith CROSS_LIBRARY_PATH and CROSS_CPATH, but not NATIVE-SEARCH-PATHS. Hmm). But the fundamental problem remains that guix host-side can't know whetherwe are cross compiling from this. It has its own cross-compiling supportat the toplevel, at the guix command line. The following is only tangentially related to your issue, so maybe not souseful to you: I've also tried (define (xcross base-package) (package (inherit base-package) (search-paths '()) (native-search-paths (list (search-path-specification (variable "CROSS_LIBRARY_PATH") (files '("lib" "lib64"))) (search-path-specification (variable "CROSS_CPATH") (files '("lib" "lib64"))))))) and (native-inputs `(("pkg-config" ,pkg-config) ("cross-gcc" , (xcross (cross-gcc "arm-linux-gnueabihf" #:xbinutils (cross-binutils "arm-linux-gnueabihf") #:libc (xcross (cross-libc "arm-linux-gnueabihf"))))))) ;("cross-libc" ,(xcross (cross-libc "arm-linux-gnueabihf"))) ; header files ("cross-libc-static" ,(xcross (cross-libc "arm-linux-gnueabihf")) "static") ("libusb" ,libusb))) for sunxi-tools. But that didn't work correctly either. What I'm trying to do is a little different from what you are trying to do. sunxi-tools has some parts that are supposed to be run on the embedded system inquestion and some that are supposed to run on the host (both are GNU systems).For convenience, I'm (and upstream are) trying to provide both in thederivation--I think because the tools can send a program via USB to the embeddedsystem. So if you compile sunxi-tools on x86_64, you get host tools for x86_64 and targettools for ARM. If you compile sunxi-tools on x86_64 for RISC-V, you get host tools forRISC-V and target tools for ARM. So it basically has to override the "which cross compiler" setting later inthe compilation process. The more I think about it the more I'm getting the feeling that I should stoptrying to fit a square peg into a round hole and just add an extra packagefor the target tools. So I did the latter.See guix-patches patch# 37823 which is now very nice. Thanks for bringing the cross compilation topic up :)