Thanks. Is there a commit for that I could cherry-pick, or a diff I
could pick up in the interim?
Regarding the command length overflow on Windows and Linux, I've had a
look and it's due to the amount of POST_BUILD custom commands which are
being created. These are concatenated into a single mega-command, which
exceeds the OS argv limits. The attached patch reduces it sufficiently
by converting these into regular dependencies and targets to alleviate
the problem, so they are then regular separate dependency-driven
actions, but likely needs doing across the board--it's not intended to
apply right now but rather to demonstrate a possible solution. I'd be
happy to work on a more comprehensive solution by extending this to all
custom command usage if that's useful to you.
With this patch, I can use the Ninja generator on Windows. There are
some unrelated failures I still need to investigate which I'll follow up
with separately.
If the test data is modified by the tests and needs re-copying, then
this strategy won't be sufficient on its own. But ctest runs appear to
be idempotent, so this doesn't look like an issue. If it is, then we
could do the copying in a test wrapper (see below).
Could I possibly also point you to this example:
https://github.com/ome/ome-cmake-superbuild/blob/develop/packages/bzip2/patches/cmake.diff#L205
https://github.com/ome/ome-cmake-superbuild/blob/develop/packages/bzip2/patches/cmake.diff#L226
This is a wrapper to run tests, which resolves all the needed run-time
paths via cmake generator expressions, and which makes the tests
completely generator-agnostic and work across the board on Windows. A
similar test wrapper could be written for hdf5, with a hdf5_add_test
around add_test to make use of it. Again, I'd be happy to look into
doing this if you'd find this useful.
Is hdf5 in a public VCS? I couldn't see it on your github HDFGroup
group, unless I overlooked it or it's under a different name?
Kind regards,
Roger
On 07/06/2016 18:05, Allen Byrne wrote:
There is a typo in the CMake test call - it should be using the "latest"
version of the hdf5 file. This will be fixed in the next release of 1.10.
Allen
On Tuesday, June 07, 2016 04:16:51 PM Roger Leigh wrote:
Hi folks,
I'm looking at the preliminary work needed to add HDF5 to our project
builds, but encountered a few errors which I was hoping someone might be
able to provide pointers or suggestions for.
Logs are here:
https://gist.github.com/rleigh-dundee/1337ed94f446ba7afc3e1a7a94da6f44
On Linux (Ubuntu 16.04) and MacOS X (10.11 with latest Xcode), I get a
single test failure. In both cases, I built with CMake (Unix Makefiles
on Ubuntu, Ninja on MacOS X, to make sure it wasn't a generator-specific
bug), and I didn't specify any hdf5-specific configuration options, just
the install prefix:
Start 632: H5CLEAR-clear_open_chk-latest_log_v3_F
632: Test command:
/Users/rleigh/code/ome-cmake-superbuild/h/hdf5-build/bin/clear_open_chk
"h5clear_log_v3.h5"
632: Test timeout computed to be: 600
632/2317 Test #632: H5CLEAR-clear_open_chk-latest_log_v3_F
This doesn't look platform- or compiler-specific since it fails in
exactly the same way on both. Is this a known issue?
On Windows with CMake/Ninja, I get
[4/1416] Running utility command for h5watch-files
FAILED: cmd.exe /C "cd /D
V:\ome-cmake-superbuild\b\hdf5-build\hl\tools\h5watch && "C:\Program
Files (x86)\CMake\bin\cmake.exe" -E copy_if_different
V:/ome-cmake-superbuild/b/hdf5-source/hl/tools/testfiles/w-help1.ddl
V:/ome-cmake-superbuild/b/hdf5-build/hl/tools/h5watch/testfiles/w-help1.ddl
&& ...
[ elided 10 kilobytes of commands! ]
..."C:\Program Files (x86)\CMake\bin\cmake.exe" -E
copy_if_different
V:/ome-cmake-superbuild/b/hdf5-source/hl/tools/testfiles/w-ext-two-width.ddl
V:/ome-cmake-superbuild/b/hdf5-build/hl/tools/h5watch/testfiles/w-ext-two-w
idth.ddl" The command line is too long.
(I also had a similar failure on Linux, but don't have a log for it
right now.)
Is the CMake logic here correct? Or could this be split up into a
number of separate custom commands/targets which are run separately
rather than in a single shot which is exceeding the practical limits of
at least two systems.
Kind regards,
Roger Leigh
--
Dr Roger Leigh -- Open Microscopy Environment
Wellcome Trust Centre for Gene Regulation and Expression,
School of Life Sciences, University of Dundee, Dow Street,
Dundee DD1 5EH Scotland UK Tel: (01382) 386364
The University of Dundee is a registered Scottish Charity, No: SC015096
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
--
Dr Roger Leigh -- Open Microscopy Environment
Wellcome Trust Centre for Gene Regulation and Expression,
School of Life Sciences, University of Dundee, Dow Street,
Dundee DD1 5EH Scotland UK Tel: (01382) 386364
The University of Dundee is a registered Scottish Charity, No: SC015096
diff -urN hdf5-1.10.0-patch1.orig/hl/tools/h5watch/CMakeTests.cmake
hdf5-1.10.0-patch1/hl/tools/h5watch/CMakeTests.cmake
--- hdf5-1.10.0-patch1.orig/hl/tools/h5watch/CMakeTests.cmake 2016-05-02
16:56:10.000000000 +0000
+++ hdf5-1.10.0-patch1/hl/tools/h5watch/CMakeTests.cmake 2016-06-08
09:42:17.739503700 +0000
@@ -45,18 +45,22 @@
# make test dir
file (MAKE_DIRECTORY "${PROJECT_BINARY_DIR}/testfiles")
-add_custom_target(h5watch-files ALL COMMENT "Copying files needed by h5watch
tests")
foreach (h5watch_file ${H5WATCH_TEST_FILES})
+ set (src "${HDF5_HL_TOOLS_DIR}/testfiles/${h5watch_file}")
set (dest "${PROJECT_BINARY_DIR}/testfiles/${h5watch_file}")
#message (STATUS " Copying ${h5watch_file}")
add_custom_command (
- TARGET h5watch-files
- POST_BUILD
- COMMAND ${CMAKE_COMMAND}
- ARGS -E copy_if_different
${HDF5_HL_TOOLS_DIR}/testfiles/${h5watch_file} ${dest}
+ OUTPUT "${dest}"
+ COMMAND "${CMAKE_COMMAND}"
+ ARGS -E copy "${src}" "${dest}"
+ DEPENDS "${src}"
)
-endforeach (h5watch_file ${H5WATCH_TEST_FILES})
+ list(APPEND H5WATCH_BINARY_TEST_FILES "${dest}")
+endforeach ()
+
+add_custom_target(h5watch-files ALL COMMENT "Copying files needed by h5watch
tests"
+ DEPENDS ${H5WATCH_BINARY_TEST_FILES})
##############################################################################
##############################################################################
diff -urN hdf5-1.10.0-patch1.orig/tools/h5format_convert/CMakeTests.cmake
hdf5-1.10.0-patch1/tools/h5format_convert/CMakeTests.cmake
--- hdf5-1.10.0-patch1.orig/tools/h5format_convert/CMakeTests.cmake
2016-05-02 16:56:08.000000000 +0000
+++ hdf5-1.10.0-patch1/tools/h5format_convert/CMakeTests.cmake 2016-06-08
09:04:46.185991800 +0000
@@ -58,30 +58,36 @@
)
file (MAKE_DIRECTORY "${PROJECT_BINARY_DIR}/testfiles")
- add_custom_target(h5fc-files ALL COMMENT "Copying files needed by h5fc
tests")
foreach (ddl_file ${HDF5_REFERENCE_FILES})
+ set (ddlsrc "${HDF5_TOOLS_H5FC_SOURCE_DIR}/testfiles/${ddl_file}")
set (ddldest "${PROJECT_BINARY_DIR}/testfiles/${ddl_file}")
#message (STATUS " Translating ${ddl_file}")
add_custom_command (
- TARGET h5fc-files
- POST_BUILD
- COMMAND ${CMAKE_COMMAND}
- ARGS -E copy_if_different
${HDF5_TOOLS_H5FC_SOURCE_DIR}/testfiles/${ddl_file} ${ddldest}
+ OUTPUT "${ddldest}"
+ COMMAND "${CMAKE_COMMAND}"
+ ARGS -E copy_if_different "${ddlsrc}" "${ddldest}"
+ DEPENDS "${ddlsrc}"
)
- endforeach (ddl_file ${HDF5_REFERENCE_FILES})
+ list(APPEND HDF5_BINARY_REFERENCE_FILES "${ddldest}")
+ endforeach ()
foreach (h5_file ${HDF5_REFERENCE_TEST_FILES})
+ set (src "${HDF5_TOOLS_H5FC_SOURCE_DIR}/testfiles/${h5_file}")
set (dest "${PROJECT_BINARY_DIR}/testfiles/${h5_file}")
#message (STATUS " Copying ${h5_file}")
add_custom_command (
- TARGET h5fc-files
- POST_BUILD
- COMMAND ${CMAKE_COMMAND}
- ARGS -E copy_if_different
${HDF5_TOOLS_H5FC_SOURCE_DIR}/testfiles/${h5_file} ${dest}
+ OUTPUT "${dest}"
+ COMMAND "${CMAKE_COMMAND}"
+ ARGS -E copy_if_different "${src}" "${dest}"
+ DEPENDS "${src}"
)
+ list(APPEND HDF5_BINARY_REFERENCE_TEST_FILES "${dest}")
endforeach (h5_file ${HDF5_REFERENCE_TEST_FILES})
+ add_custom_target(h5fc-files ALL COMMENT "Copying files needed by h5fc tests"
+ DEPENDS ${HDF5_BINARY_REFERENCE_FILES}
${HDF5_BINARY_REFERENCE_TEST_FILES})
+
##############################################################################
##############################################################################
### T H E T E S T S M A C R O S ###
diff -urN hdf5-1.10.0-patch1.orig/tools/perform/CMakeTests.cmake
hdf5-1.10.0-patch1/tools/perform/CMakeTests.cmake
--- hdf5-1.10.0-patch1.orig/tools/perform/CMakeTests.cmake 2016-03-28
16:26:44.000000000 +0000
+++ hdf5-1.10.0-patch1/tools/perform/CMakeTests.cmake 2016-06-08
09:42:07.819495800 +0000
@@ -5,11 +5,14 @@
##############################################################################
##############################################################################
-add_custom_command (
- TARGET zip_perf
- POST_BUILD
- COMMAND ${CMAKE_COMMAND}
- ARGS -E copy_if_different
${HDF5_TOOLS_SRC_DIR}/testfiles/tfilters.h5 ${PROJECT_BINARY_DIR}/tfilters.h5
+add_custom_command(
+ OUTPUT "${PROJECT_BINARY_DIR}/tfilters.h5"
+ COMMAND "${CMAKE_COMMAND}" -E copy
"${HDF5_TOOLS_SRC_DIR}/testfiles/tfilters.h5"
"${PROJECT_BINARY_DIR}/tfilters.h5"
+ DEPENDS "${HDF5_TOOLS_SRC_DIR}/testfiles/tfilters.h5"
+)
+
+add_custom_target(zip_perf_file ALL COMMENT "Copying files needed by perform
tests"
+ DEPENDS "${PROJECT_BINARY_DIR}/tfilters.h5"
)
#-----------------------------------------------------------------------------
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
