[core-updates-frozen] CMake fails to build on i686-linux

  • Done
  • quality assurance status badge
Details
4 participants
  • Guillaume Le Vaillant
  • Ludovic Courtès
  • Ludovic Courtès
  • Maxime Devos
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 16 Sep 2021 11:41
(address . bug-guix@gnu.org)
87y27w26yy.fsf@inria.fr
On ‘core-updates-frozen’, CMake has one test failure on i686-linux when

Toggle snippet (82 lines)
545/558 Test #518: RunCMake.CPack_TXZ ................................***Failed 3.79 sec
-- MINIMAL-MONOLITHIC-type - PASSED
-- TXZ/MINIMAL-MONOLITHIC-type - PASSED
-- MINIMAL-COMPONENT-type - PASSED
-- TXZ/MINIMAL-COMPONENT-type - PASSED
-- MINIMAL-package-target-MONOLITHIC-type - PASSED
-- TXZ/MINIMAL-package-target-MONOLITHIC-type - PASSED
-- MINIMAL-package-target-COMPONENT-type - PASSED
-- TXZ/MINIMAL-package-target-COMPONENT-type - PASSED
-- THREADED_ALL-package-target-MONOLITHIC-type - PASSED
CMake Error at RunCMake.cmake:201 (message):
TXZ/THREADED_ALL-package-target-MONOLITHIC-type - FAILED:

Result is [1], not [0].

stderr does not match that expected.

Command was:

command> "/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/bin/cmake" "-DRunCMake_TEST=THREADED_ALL-package-target-MONOLITHIC-type" "-DRunCMake_TEST_FILE_PREFIX=THREADED_ALL" "-DRunCMake_SUBTEST_SUFFIX=" "-DGENERATOR_TYPE=TXZ" "-DPACKAGING_TYPE=MONOLITHIC" "-Dsrc_dir=/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/CPack" "-Dbin_dir=/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/TXZ/CPack/THREADED_ALL-package-target-build" "-Dconfig_file=/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/CPack/conf/TXZ_config.cmake" "-P" "/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/CPack/VerifyResult.cmake"

Actual stdout:

actual-out>

Expected stderr to match:

expect-err> ^$

Actual stderr:

actual-err> CMake Error at /tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/CPack/VerifyResult.cmake:88 (message):
actual-err> Found more than one file for file No. '1'! Found files count '0'. Files:
actual-err> '' Globbing expression: 'threaded_all-0.1.1-*.tar.xz'
actual-err>
actual-err> CPack output: 'make[1]: Entering directory
actual-err> '/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/TXZ/CPack/THREADED_ALL-package-target-build'
actual-err>
actual-err>
actual-err> Run CPack packaging tool...
actual-err>
actual-err> CPack: Create package using TXZ
actual-err>
actual-err> CPack: Install projects
actual-err>
actual-err> CPack: - Run preinstall target for:
actual-err> THREADED_ALL-package-target-MONOLITHIC-type
actual-err>
actual-err> CPack: - Install project: THREADED_ALL-package-target-MONOLITHIC-type []
actual-err>
actual-err> CPack: Create package
actual-err>
actual-err> make[1]: Leaving directory
actual-err> '/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/TXZ/CPack/THREADED_ALL-package-target-build'
actual-err>
actual-err>
actual-err> '
actual-err>
actual-err> CPack error: 'CPack Error: Problem to open archive
actual-err> </tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/TXZ/CPack/THREADED_ALL-package-target-build/_CPack_Packages/Linux/TXZ/threaded_all-0.1.1-Linux.tar.xz>,
actual-err> ERROR = archive_write_open: Internal error initializing compression
actual-err> library: Cannot allocate memory
actual-err>
actual-err> CPack Error: Problem compressing the directory
actual-err>
actual-err> CPack Error: Error when generating package: threaded_all
actual-err>
actual-err> make[1]: *** [Makefile:146: package] Error 1
actual-err>
actual-err> ';
actual-err>
actual-err> CPack result: '';
actual-err>
actual-err> config file: ''

Call Stack (most recent call first):
RunCMake.cmake:215 (run_cmake)
CPack/CPackTestHelpers.cmake:119 (run_cmake_command)
CPack/CPackTestHelpers.cmake:143 (run_cpack_test_common_)
CPack/RunCMakeTest.cmake:24 (run_cpack_test_package_target)

I cannot reproduce it on hardware with 32 cores. I suspect it has to do
with the number of threads used for xz compression, which defaults to
the number of cores, and some of the build machines on berlin have way
more cores.

Ludo’.
G
G
Guillaume Le Vaillant wrote on 16 Sep 2021 15:13
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)(address . 50617@debbugs.gnu.org)
87k0jgoe2v.fsf@kitej
Ludovic Courtès <ludovic.courtes@inria.fr> skribis:

Toggle quote (15 lines)
> On ‘core-updates-frozen’, CMake has one test failure on i686-linux when
> building on berlin (e.g., <https://ci.guix.gnu.org/build/790602/log/raw>):
>
> --8<---------------cut here---------------start------------->8---
> 545/558 Test #518: RunCMake.CPack_TXZ ................................***Failed 3.79 sec
> [...]
> --8<---------------cut here---------------end--------------->8---
>
> I cannot reproduce it on hardware with 32 cores. I suspect it has to do
> with the number of threads used for xz compression, which defaults to
> the number of cores, and some of the build machines on berlin have way
> more cores.
>
> Ludo’.

I tried a few times on a machine with 16 cores, and I can't reproduce
either. The build succeeded every time.
-----BEGIN PGP SIGNATURE-----

iIUEAREKAC0WIQTLxZxm7Ce5cXlAaz5r6CCK3yH+PwUCYUNDuA8cZ2x2QHBvc3Rl
by5uZXQACgkQa+ggit8h/j9U5QEAhA6zRrbOIlUl9vDvrLXiOlqIGzxbgOYSMoom
PoL6aBQA/0SnuYiLKa665+i+z4QaFag0Rgy5TM9d8ZF1GYxs46Yq
=w9Na
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 16 Sep 2021 17:20
(name . Guillaume Le Vaillant)(address . glv@posteo.net)(address . 50617@debbugs.gnu.org)
87h7ek1r9s.fsf@inria.fr
Hi,

Guillaume Le Vaillant <glv@posteo.net> skribis:

Toggle quote (20 lines)
> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>
>> On ‘core-updates-frozen’, CMake has one test failure on i686-linux when
>> building on berlin (e.g., <https://ci.guix.gnu.org/build/790602/log/raw>):
>>
>> --8<---------------cut here---------------start------------->8---
>> 545/558 Test #518: RunCMake.CPack_TXZ ................................***Failed 3.79 sec
>> [...]
>> --8<---------------cut here---------------end--------------->8---
>>
>> I cannot reproduce it on hardware with 32 cores. I suspect it has to do
>> with the number of threads used for xz compression, which defaults to
>> the number of cores, and some of the build machines on berlin have way
>> more cores.
>>
>> Ludo’.
>
> I tried a few times on a machine with 16 cores, and I can't reproduce
> either. The build succeeded every time.

My latest attempt is to go as far as setting the number of threads used
by libarchive to 1 (patch attached). However, that still fails
systematically on berlin’s 32-core build machines.

The Internet doesn’t seem to have much to say about this problem.

Ideas anyone?

Ludo’.
Toggle diff (50 lines)
diff --git a/gnu/packages/cmake.scm b/gnu/packages/cmake.scm
index f76ee3ff3a..a235d922e0 100644
--- a/gnu/packages/cmake.scm
+++ b/gnu/packages/cmake.scm
@@ -92,7 +92,7 @@ using the CMake build system.")
;;; Build phases shared between 'cmake-bootstrap' and the later variants
;;; that use cmake-build-system.
-(define %common-build-phases
+(define (%common-build-phases)
`((add-after 'unpack 'split-package
;; Remove files that have been packaged in other package recipes.
(lambda _
@@ -100,6 +100,18 @@ using the CMake build system.")
(substitute* "Auxiliary/CMakeLists.txt"
((".*cmake-mode.el.*") ""))
#t))
+ ,@(let ((system (or (%current-target-system) (%current-system))))
+ (if (or (string-prefix? "i686-" system)
+ (string-prefix? "i586-" system))
+ '((add-after 'unpack 'clamp-xz-thread-number
+ (lambda _
+ ;; By default, 'CPack' would use as many threads as the
+ ;; number of available cores for xz compression.
+ ;; However, this leads to test failures: <XXX>.
+ (substitute* "Source/cmArchiveWrite.cxx"
+ (("case CompressXZ:")
+ "case CompressXZ: numThreads = 1;\n")))))
+ '()))
(add-before 'configure 'patch-bin-sh
(lambda _
;; Replace "/bin/sh" by the right path in... a lot of
@@ -188,7 +200,7 @@ using the CMake build system.")
" --exclude-regex ^\\(" (string-join skipped-tests "\\|") "\\)$")))
#:phases
(modify-phases %standard-phases
- ,@%common-build-phases
+ ,@(%common-build-phases)
(add-before 'configure 'set-paths
(lambda _
;; Help cmake's bootstrap process to find system libraries
@@ -295,7 +307,7 @@ and workspaces that can be used in the compiler environment of your choice.")
#:build-type "Release"
#:phases
(modify-phases %standard-phases
- ,@%common-build-phases
+ ,@(%common-build-phases)
(add-after 'install 'delete-help-documentation
(lambda* (#:key outputs #:allow-other-keys)
(delete-file-recursively
M
M
Maxime Devos wrote on 16 Sep 2021 18:21
(address . 50617@debbugs.gnu.org)
848bb43e75e0513ed31734d660fe958c1b3f8eaa.camel@telenet.be
Ludovic Courtès schreef op do 16-09-2021 om 17:20 [+0200]:
Toggle quote (4 lines)
> + ,@(let ((system (or (%current-target-system) (%current-system))))
> + (if (or (string-prefix? "i686-" system)
> + (string-prefix? "i586-" system))

(Unrelated to the build failure) you could use target-x86-32? here
to make the code a little simpler:

,@(if (target-x86-32?)
'((add-after ....))
'()

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYUNvCxccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7nsGAP9r27M9UW7qzB6Cwmb5sqUALyrh
tOo8MzJKyluP3zXmsQEAsKQUE8QxDX8BccJsaRvOZlG8N0PhvshAfv93LpkLnAc=
=TuvP
-----END PGP SIGNATURE-----


L
L
Ludovic Courtès wrote on 17 Sep 2021 09:05
(name . Maxime Devos)(address . maximedevos@telenet.be)
87zgsby95r.fsf@inria.fr
Maxime Devos <maximedevos@telenet.be> skribis:

Toggle quote (12 lines)
> Ludovic Courtès schreef op do 16-09-2021 om 17:20 [+0200]:
>> + ,@(let ((system (or (%current-target-system) (%current-system))))
>> + (if (or (string-prefix? "i686-" system)
>> + (string-prefix? "i586-" system))
>
> (Unrelated to the build failure) you could use target-x86-32? here
> to make the code a little simpler:
>
> ,@(if (target-x86-32?)
> '((add-after ....))
> '()

Ah yes, definitely! Thanks for the hint.

Ludo’.
L
L
Ludovic Courtès wrote on 21 Sep 2021 10:21
(name . Guillaume Le Vaillant)(address . glv@posteo.net)(address . 50617-done@debbugs.gnu.org)
87mto6nxt9.fsf@gnu.org
Hi,

Guillaume Le Vaillant <glv@posteo.net> skribis:

Toggle quote (20 lines)
> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>
>> On ‘core-updates-frozen’, CMake has one test failure on i686-linux when
>> building on berlin (e.g., <https://ci.guix.gnu.org/build/790602/log/raw>):
>>
>> --8<---------------cut here---------------start------------->8---
>> 545/558 Test #518: RunCMake.CPack_TXZ ................................***Failed 3.79 sec
>> [...]
>> --8<---------------cut here---------------end--------------->8---
>>
>> I cannot reproduce it on hardware with 32 cores. I suspect it has to do
>> with the number of threads used for xz compression, which defaults to
>> the number of cores, and some of the build machines on berlin have way
>> more cores.
>>
>> Ludo’.
>
> I tried a few times on a machine with 16 cores, and I can't reproduce
> either. The build succeeded every time.

Since the bug only seems to manifest on specific machines for reasons
that escape me, I went ahead and arranged to just skip it in commit
f762d63ec072500cd327ec5342c2f5434c267222.

Ludo’.
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 50617@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 50617
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch