Re: [cmake-developers] [CMake] Integrating ExternalData with Artifactory
On Mon, Sep 22, 2014 at 10:07 AM, Brad King wrote: > Perhaps: > > list(APPEND ExternalData_URL_TEMPLATES > "ExternalDataCustomScript://MyFetch/%(algo)/%(hash)" > ) > set(ExternalData_CUSTOM_SCRIPT_MyFetch /path/to/MyFetch.cmake) > > The script would be include()-ed in place of the current call > to _ExternalData_download_file with the part of the URL after > the "MyFetch/" already parsed into some variable(s). > This all sounds reasonable to me. I'd love to see this land in a future CMake release. I'm willing to help if you need a tester/reviewer. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] [CMake] Integrating ExternalData with Artifactory
On 09/19/2014 06:07 PM, Taylor Braun-Jones wrote: > On Fri, Sep 19, 2014 at 3:04 PM, Brad King wrote: >> I think it can be activated by a special format of an entry in >> ExternalData_URL_TEMPLATES that specifies a lookup key that maps >> to some kind of custom configuration of a .cmake script to include >> or command to launch with execute_process. > > How about defining new URL scheme like: > > externaldatacommand://<*file|module*>/<*command*> Yes, something along these lines is what I had in mind. > list(APPEND ExternalData_URL_TEMPLATES > > "externaldatacommand://ArtifactoryExternalData/ARTIFACTORY_DOWNLOAD_FILE?ALGO=%(algo)&" > ) > > And inside Testing/CMake/ArtifactoryExternalData.cmake it would look like: > > function(ARTIFACTORY_DOWNLOAD_FILE file algo hash) Since CMake does not support variable evaluation in a command name, this would require configuring a file with code in it to launch for each entry of this form in the url templates list. Also CMAKE_MODULE_PATH will not be available to the ExternalData module when it is launched at build time to do the download. The include() will have to work with a full path, perhaps stored in the configured copy of "ExternalData_config.cmake.in". It could be indexed with a key. Perhaps: list(APPEND ExternalData_URL_TEMPLATES "ExternalDataCustomScript://MyFetch/%(algo)/%(hash)" ) set(ExternalData_CUSTOM_SCRIPT_MyFetch /path/to/MyFetch.cmake) The script would be include()-ed in place of the current call to _ExternalData_download_file with the part of the URL after the "MyFetch/" already parsed into some variable(s). -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] [CMake] Integrating ExternalData with Artifactory
Shifting discussion to cmake-developers.
On Fri, Sep 19, 2014 at 3:04 PM, Brad King wrote:
> On 09/19/2014 12:23 PM, Taylor Braun-Jones wrote:
> > integrate Artifactory with the ExternalData framework?
> >
> > the Artifactory REST API is a two step process
>
> I think it can be activated by a special format of an entry in
> ExternalData_URL_TEMPLATES that specifies a lookup key that maps
> to some kind of custom configuration of a .cmake script to include
> or command to launch with execute_process.
>
How about defining new URL scheme like:
externaldatacommand://<*file|module*>/<*command*>
Where *file|module* is something that can be passed to the include command
and defines a function or macro named *command*. *command* would then be
called like:
command(outputfile algo hash)
So, using Artifactory as an example case, a projects CMakeLists.txt might
look like:
list(APPEND CMAKE_MODULE_PATH
${CMAKE_SOURCE_DIR}/Testing/CMake
)
set(ARTIFACTORY_BASE_URL https://example.com/artifactory)
list(APPEND ExternalData_URL_TEMPLATES
"externaldatacommand://ArtifactoryExternalData/ARTIFACTORY_DOWNLOAD_FILE?ALGO=%(algo)&"
)
And inside Testing/CMake/ArtifactoryExternalData.cmake it would look like:
function(ARTIFACTORY_DOWNLOAD_FILE file algo hash)
set(json_response_file
${CMAKE_BINARY_DIR}/artifactory_search_${algo}_${hash}.json)
set(artifact_search_query
${ARTIFACTORY_BASE_URL}/api/search/checksum?${algo}=${hash})
message("Query URI: ${artifact_search_query}")
file(DOWNLOAD ${artifact_search_query} ${json_response_file} STATUS
status)
list(GET status 0 error_code)
list(GET status 1 error_message)
if(error_code)
file(REMOVE ${json_response_file})
message(FATAL_ERROR "Artifactory query failed with error ${error_code}:
${error_message}")
endif()
# For testing
file(WRITE ${json_response_file} "{\"results\":[{\"uri\":\"
https://example.com/artifactory/api/storage/repo1-cache/lena.png\"}]}";)
file(READ ${json_response_file} json_response)
file(REMOVE ${json_response_file})
# Normalize the JSON response by removing any whitespace (makes it easier
to parse)
string(REGEX REPLACE "[ \t\n\r]+" "" json_response ${json_response})
if(NOT json_response MATCHES "\"uri\":")
message(FATAL_ERROR "Artifactory file was not found")
endif()
string(REGEX MATCH "https?://[^\"]+" artifact_uri ${json_response})
message("Artifact URI: ${artifact_uri}")
file(DOWNLOAD ${artifact_uri} ${file} STATUS status)
list(GET status 0 error_code)
list(GET status 1 error_message)
if(error_code)
message(FATAL_ERROR "Artifactory download failed with error
${error_code}: ${error_message}")
endif()
endfunction()
Then if you had:
ExternalData_Add_Test(MyProjectData
NAME SmoothingTest
COMMAND SmoothingExe DATA{Input/Image.png}
SmoothedImage.png
)
The ExternalData framework would execute:
ARTIFACTORY_DOWNLOAD_FILE(${ExternalData_BINARY_ROOT}/Input/Image.png md5
1234567890abcdef1234567890abcdef)
Thoughts?
Taylor
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers
