bug#50617: [core-updates-frozen] CMake fails to build on i686-linux

[Ludovic Courtès]

Hi,

Guillaume Le Vaillant  skribis:

> Ludovic Courtès  skribis:
>
>> On ‘core-updates-frozen’, CMake has one test failure on i686-linux when
>> building on berlin (e.g., ):
>>
>> --8<---cut here---start->8---
>> 545/558 Test #518: RunCMake.CPack_TXZ 
>> ***Failed3.79 sec
>> [...]
>> --8<---cut here---end--->8---
>>
>> I cannot reproduce it on hardware with 32 cores.  I suspect it has to do
>> with the number of threads used for xz compression, which defaults to
>> the number of cores, and some of the build machines on berlin have way
>> more cores.
>>
>> Ludo’.
>
> I tried a few times on a machine with 16 cores, and I can't reproduce
> either. The build succeeded every time.

Since the bug only seems to manifest on specific machines for reasons
that escape me, I went ahead and arranged to just skip it in commit
f762d63ec072500cd327ec5342c2f5434c267222.

Ludo’.

bug#50617: [core-updates-frozen] CMake fails to build on i686-linux

[Ludovic Courtès]

Maxime Devos  skribis:

> Ludovic Courtès schreef op do 16-09-2021 om 17:20 [+0200]:
>> +,@(let ((system (or (%current-target-system) (%current-system
>> +(if (or (string-prefix? "i686-" system)
>> +(string-prefix? "i586-" system))
>
> (Unrelated to the build failure) you could use target-x86-32? here
> to make the code a little simpler:
>
> ,@(if (target-x86-32?)
>   '((add-after ))
>   '()

Ah yes, definitely!  Thanks for the hint.

Ludo’.

bug#50617: [core-updates-frozen] CMake fails to build on i686-linux

[Maxime Devos]

Ludovic Courtès schreef op do 16-09-2021 om 17:20 [+0200]:
> +,@(let ((system (or (%current-target-system) (%current-system
> +(if (or (string-prefix? "i686-" system)
> +(string-prefix? "i586-" system))

(Unrelated to the build failure) you could use target-x86-32? here
to make the code a little simpler:

,@(if (target-x86-32?)
  '((add-after ))
  '()

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

bug#50617: [core-updates-frozen] CMake fails to build on i686-linux

[Ludovic Courtès]

Hi,

Guillaume Le Vaillant  skribis:

> Ludovic Courtès  skribis:
>
>> On ‘core-updates-frozen’, CMake has one test failure on i686-linux when
>> building on berlin (e.g., ):
>>
>> --8<---cut here---start->8---
>> 545/558 Test #518: RunCMake.CPack_TXZ 
>> ***Failed3.79 sec
>> [...]
>> --8<---cut here---end--->8---
>>
>> I cannot reproduce it on hardware with 32 cores.  I suspect it has to do
>> with the number of threads used for xz compression, which defaults to
>> the number of cores, and some of the build machines on berlin have way
>> more cores.
>>
>> Ludo’.
>
> I tried a few times on a machine with 16 cores, and I can't reproduce
> either. The build succeeded every time.

My latest attempt is to go as far as setting the number of threads used
by libarchive to 1 (patch attached).  However, that still fails
systematically on berlin’s 32-core build machines.

The Internet doesn’t seem to have much to say about this problem.

Ideas anyone?

Ludo’.

diff --git a/gnu/packages/cmake.scm b/gnu/packages/cmake.scm
index f76ee3ff3a..a235d922e0 100644
--- a/gnu/packages/cmake.scm
+++ b/gnu/packages/cmake.scm
@@ -92,7 +92,7 @@ using the CMake build system.")
 
 ;;; Build phases shared between 'cmake-bootstrap' and the later variants
 ;;; that use cmake-build-system.
-(define %common-build-phases
+(define (%common-build-phases)
   `((add-after 'unpack 'split-package
   ;; Remove files that have been packaged in other package recipes.
   (lambda _
@@ -100,6 +100,18 @@ using the CMake build system.")
 (substitute* "Auxiliary/CMakeLists.txt"
   ((".*cmake-mode.el.*") ""))
 #t))
+,@(let ((system (or (%current-target-system) (%current-system
+(if (or (string-prefix? "i686-" system)
+(string-prefix? "i586-" system))
+'((add-after 'unpack 'clamp-xz-thread-number
+(lambda _
+  ;; By default, 'CPack' would use as many threads as the
+  ;; number of available cores for xz compression.
+  ;; However, this leads to test failures: .
+  (substitute* "Source/cmArchiveWrite.cxx"
+(("case CompressXZ:")
+ "case CompressXZ: numThreads = 1;\n")
+'()))
 (add-before 'configure 'patch-bin-sh
   (lambda _
 ;; Replace "/bin/sh" by the right path in... a lot of
@@ -188,7 +200,7 @@ using the CMake build system.")
" --exclude-regex ^\\(" (string-join skipped-tests "\\|") "\\)$")))
#:phases
(modify-phases %standard-phases
- ,@%common-build-phases
+ ,@(%common-build-phases)
  (add-before 'configure 'set-paths
(lambda _
  ;; Help cmake's bootstrap process to find system libraries
@@ -295,7 +307,7 @@ and workspaces that can be used in the compiler environment of your choice.")
#:build-type "Release"
#:phases
(modify-phases %standard-phases
- ,@%common-build-phases
+ ,@(%common-build-phases)
  (add-after 'install 'delete-help-documentation
(lambda* (#:key outputs #:allow-other-keys)
  (delete-file-recursively

bug#50617: [core-updates-frozen] CMake fails to build on i686-linux

[Guillaume Le Vaillant]

Ludovic Courtès  skribis:

> On ‘core-updates-frozen’, CMake has one test failure on i686-linux when
> building on berlin (e.g., ):
>
> --8<---cut here---start->8---
> 545/558 Test #518: RunCMake.CPack_TXZ 
> ***Failed3.79 sec
> [...]
> --8<---cut here---end--->8---
>
> I cannot reproduce it on hardware with 32 cores.  I suspect it has to do
> with the number of threads used for xz compression, which defaults to
> the number of cores, and some of the build machines on berlin have way
> more cores.
>
> Ludo’.

I tried a few times on a machine with 16 cores, and I can't reproduce
either. The build succeeded every time.


signature.asc
Description: PGP signature

bug#50617: [core-updates-frozen] CMake fails to build on i686-linux

[Ludovic Courtès]

On ‘core-updates-frozen’, CMake has one test failure on i686-linux when
building on berlin (e.g., ):

--8<---cut here---start->8---
545/558 Test #518: RunCMake.CPack_TXZ ***Failed 
   3.79 sec
-- MINIMAL-MONOLITHIC-type - PASSED
-- TXZ/MINIMAL-MONOLITHIC-type - PASSED
-- MINIMAL-COMPONENT-type - PASSED
-- TXZ/MINIMAL-COMPONENT-type - PASSED
-- MINIMAL-package-target-MONOLITHIC-type - PASSED
-- TXZ/MINIMAL-package-target-MONOLITHIC-type - PASSED
-- MINIMAL-package-target-COMPONENT-type - PASSED
-- TXZ/MINIMAL-package-target-COMPONENT-type - PASSED
-- THREADED_ALL-package-target-MONOLITHIC-type - PASSED
CMake Error at RunCMake.cmake:201 (message):
  TXZ/THREADED_ALL-package-target-MONOLITHIC-type - FAILED:

  Result is [1], not [0].

  stderr does not match that expected.

  Command was:

   command> 
"/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/bin/cmake" 
"-DRunCMake_TEST=THREADED_ALL-package-target-MONOLITHIC-type" 
"-DRunCMake_TEST_FILE_PREFIX=THREADED_ALL" "-DRunCMake_SUBTEST_SUFFIX=" 
"-DGENERATOR_TYPE=TXZ" "-DPACKAGING_TYPE=MONOLITHIC" 
"-Dsrc_dir=/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/CPack"
 
"-Dbin_dir=/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/TXZ/CPack/THREADED_ALL-package-target-build"
 
"-Dconfig_file=/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/CPack/conf/TXZ_config.cmake"
 "-P" 
"/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/CPack/VerifyResult.cmake"

  Actual stdout:

   actual-out>

  Expected stderr to match:

   expect-err> ^$

  Actual stderr:

   actual-err> CMake Error at 
/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/CPack/VerifyResult.cmake:88
 (message):
   actual-err>   Found more than one file for file No.  '1'! Found files count 
'0'.  Files:
   actual-err>   '' Globbing expression: 'threaded_all-0.1.1-*.tar.xz'
   actual-err>
   actual-err>   CPack output: 'make[1]: Entering directory
   actual-err>   
'/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/TXZ/CPack/THREADED_ALL-package-target-build'
   actual-err>
   actual-err>
   actual-err>   Run CPack packaging tool...
   actual-err>
   actual-err>   CPack: Create package using TXZ
   actual-err>
   actual-err>   CPack: Install projects
   actual-err>
   actual-err>   CPack: - Run preinstall target for:
   actual-err>   THREADED_ALL-package-target-MONOLITHIC-type
   actual-err>
   actual-err>   CPack: - Install project: 
THREADED_ALL-package-target-MONOLITHIC-type []
   actual-err>
   actual-err>   CPack: Create package
   actual-err>
   actual-err>   make[1]: Leaving directory
   actual-err>   
'/tmp/guix-build-cmake-bootstrap-3.20.2.drv-0/cmake-3.20.2/Tests/RunCMake/TXZ/CPack/THREADED_ALL-package-target-build'
   actual-err>
   actual-err>
   actual-err>   '
   actual-err>
   actual-err>   CPack error: 'CPack Error: Problem to open archive
   actual-err>   
,
   actual-err>   ERROR = archive_write_open: Internal error initializing 
compression
   actual-err>   library: Cannot allocate memory
   actual-err>
   actual-err>   CPack Error: Problem compressing the directory
   actual-err>
   actual-err>   CPack Error: Error when generating package: threaded_all
   actual-err>
   actual-err>   make[1]: *** [Makefile:146: package] Error 1
   actual-err>
   actual-err>   ';
   actual-err>
   actual-err>   CPack result: '';
   actual-err>
   actual-err>   config file: ''

Call Stack (most recent call first):
  RunCMake.cmake:215 (run_cmake)
  CPack/CPackTestHelpers.cmake:119 (run_cmake_command)
  CPack/CPackTestHelpers.cmake:143 (run_cpack_test_common_)
  CPack/RunCMakeTest.cmake:24 (run_cpack_test_package_target)
--8<---cut here---end--->8---

I cannot reproduce it on hardware with 32 cores.  I suspect it has to do
with the number of threads used for xz compression, which defaults to
the number of cores, and some of the build machines on berlin have way
more cores.

Ludo’.