Re: [cmake-developers] Custom commands with Ninja on Windows
On 05/23/2016 10:24 AM, Brad King wrote: > Please use cmGeneratedFileStream. Also please place the script files in the > CMakeFiles/ directory. See cmake::GetCMakeFilesDirectoryPostSlash callers > for examples. [snip] > Please also update the call sites for PRE_BUILD, PRE_LINK, and POST_BUILD > custom commands. Please also look at adding a case to the test suite, perhaps under Tests/RunCMake/Ninja, to try running a really long custom command line that forces use of the script. Thanks, -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Custom commands with Ninja on Windows
On 05/23/2016 06:48 AM, Martin Ankerl wrote: > Here is an updated patch Thanks! > +#include This looks unnecessary. > +const std::string cmdFile = hasher.HashString(cmd.str()) + ".cmd"; > + > +// TODO fail if can't open file > +std::ofstream fout(cmdFile); Please use cmGeneratedFileStream. Also please place the script files in the CMakeFiles/ directory. See cmake::GetCMakeFilesDirectoryPostSlash callers for examples. > - this->BuildCommandLine(cmdLines), this->ConstructComment(ccg), > + this->BuildCommandLine(cmdLines, false), this->ConstructComment(ccg), Please also update the call sites for PRE_BUILD, PRE_LINK, and POST_BUILD custom commands. Thanks, -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Custom commands with Ninja on Windows
Thanks for the feedback! Here is an updated patch that uses a hash as filename, .cmd as extension, boolean argument, and some error checks. Martin On Fri, May 20, 2016 at 3:40 PM Brad King wrote: > On 05/20/2016 03:41 AM, Martin Ankerl wrote: > > From what I have understood it seems more safe to use .bat and not > > .cmd, because the behavior of the errorlevel is different. From > > > http://waynes-world-it.blogspot.co.at/2008/08/difference-between-bat-and-cmd.html > : > > > >> The differences between .CMD and .BAT as far as CMD.EXE is concerned > are: > >> With extensions enabled, PATH/APPEND/PROMPT/SET/ASSOC in .CMD files > will set > >> ERRORLEVEL regardless of error. .BAT sets ERRORLEVEL only on errors. > > I read that to mean the opposite: cmd is better than bat because it will > always set ERRORLEVEL even if it is to zero, so one can reliably determine > whether a command worked (Git's wrapper on windows is git.cmd and not > git.bat). > > > I've created a first attempt of this implementation, please see the > attached > > patch. > > Good start. > > > + // TODO fail if command is too long and no file specified > > Several of the call sites of BuildCommandLine are constructing > commands using ninja's $VAR reference/placeholder syntax. For > these call sites it is never safe to use a separate cmdFile. > Fortunately they tend to be for compiler/linker invocations > that can use the normal RSP_FILE approach. > > Instead of adding a cmdFile argument to BuildCommandLine, add > a boolean (or enum) to indicate whether the commands hold any > placeholders. Only when no placeholders are used is it safe to > activate the new behavior. When activated, I think it is > cleaner to use a hash of the command line as the cmdFile name. > That way when the command changes Ninja will be aware of the > change and re-run it. > > Thanks, > -Brad > > -- Martin 0001-Support-for-many-custom-commands-in-Windows.patch Description: Binary data -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Custom commands with Ninja on Windows
On 05/20/2016 03:41 AM, Martin Ankerl wrote: > From what I have understood it seems more safe to use .bat and not > .cmd, because the behavior of the errorlevel is different. From > http://waynes-world-it.blogspot.co.at/2008/08/difference-between-bat-and-cmd.html: > >> The differences between .CMD and .BAT as far as CMD.EXE is concerned are: >> With extensions enabled, PATH/APPEND/PROMPT/SET/ASSOC in .CMD files will set >> ERRORLEVEL regardless of error. .BAT sets ERRORLEVEL only on errors. I read that to mean the opposite: cmd is better than bat because it will always set ERRORLEVEL even if it is to zero, so one can reliably determine whether a command worked (Git's wrapper on windows is git.cmd and not git.bat). > I've created a first attempt of this implementation, please see the attached > patch. Good start. > + // TODO fail if command is too long and no file specified Several of the call sites of BuildCommandLine are constructing commands using ninja's $VAR reference/placeholder syntax. For these call sites it is never safe to use a separate cmdFile. Fortunately they tend to be for compiler/linker invocations that can use the normal RSP_FILE approach. Instead of adding a cmdFile argument to BuildCommandLine, add a boolean (or enum) to indicate whether the commands hold any placeholders. Only when no placeholders are used is it safe to activate the new behavior. When activated, I think it is cleaner to use a hash of the command line as the cmdFile name. That way when the command changes Ninja will be aware of the change and re-run it. Thanks, -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Custom commands with Ninja on Windows
sorry, that DEP_FILE problem was my fault. It's working correctly. Martin On Fri, May 20, 2016 at 9:41 AM Martin Ankerl wrote: > From what I have understood it seems more safe to use .bat and not .cmd, > because the behavior of the errorlevel is different. From > http://waynes-world-it.blogspot.co.at/2008/08/difference-between-bat-and-cmd.html > : > > > The differences between .CMD and .BAT as far as CMD.EXE is concerned > are: With extensions enabled, PATH/APPEND/PROMPT/SET/ASSOC in .CMD files > will set > ERRORLEVEL regardless of error. .BAT sets ERRORLEVEL only on errors. > > I've created a first attempt of this implementation, please see the > attached patch. It seems to do the trick for my test case, but I couldn't > test it well because unfortunately I've run into a different unrelated > problem: the slashes for the DEP_FILE are in the git version / instead of \ > which makes the ninja build fail. This was not the case with version 3.5.2 > > Martin > > > > On Thu, May 19, 2016 at 10:46 PM Brad King wrote: > >> On 05/19/2016 04:31 PM, Martin Ankerl wrote: >> > I didn't think about just writing a .cmd (or .bat?) with cmake >> >> The ".cmd" extension is a modern version of ".bat". >> >> > that sounds like the simplest solution! >> >> Yes, assuming we never have a need for ninja placeholder substitution. >> Why are one-line response files generated by Ninja not a solution? >> Does cmd support them? >> >> > if %errorlevel% neq 0 exit /b %errorlevel% >> >> Yes. >> >> > I have no experience with cmake implementation though, how would you >> > find a name for the filename? >> >> First, it can be made conditional on when the command line is really long. >> Second, you could just put it in CMakeFiles/ and name it using a hash >> of its content (or of the list of outputs). See Source/cmCryptoHash.h >> for example. >> >> Thanks, >> -Brad >> >> -- > Martin > -- Martin -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Custom commands with Ninja on Windows
>From what I have understood it seems more safe to use .bat and not .cmd, because the behavior of the errorlevel is different. From http://waynes-world-it.blogspot.co.at/2008/08/difference-between-bat-and-cmd.html : > The differences between .CMD and .BAT as far as CMD.EXE is concerned are: With extensions enabled, PATH/APPEND/PROMPT/SET/ASSOC in .CMD files will set ERRORLEVEL regardless of error. .BAT sets ERRORLEVEL only on errors. I've created a first attempt of this implementation, please see the attached patch. It seems to do the trick for my test case, but I couldn't test it well because unfortunately I've run into a different unrelated problem: the slashes for the DEP_FILE are in the git version / instead of \ which makes the ninja build fail. This was not the case with version 3.5.2 Martin On Thu, May 19, 2016 at 10:46 PM Brad King wrote: > On 05/19/2016 04:31 PM, Martin Ankerl wrote: > > I didn't think about just writing a .cmd (or .bat?) with cmake > > The ".cmd" extension is a modern version of ".bat". > > > that sounds like the simplest solution! > > Yes, assuming we never have a need for ninja placeholder substitution. > Why are one-line response files generated by Ninja not a solution? > Does cmd support them? > > > if %errorlevel% neq 0 exit /b %errorlevel% > > Yes. > > > I have no experience with cmake implementation though, how would you > > find a name for the filename? > > First, it can be made conditional on when the command line is really long. > Second, you could just put it in CMakeFiles/ and name it using a hash > of its content (or of the list of outputs). See Source/cmCryptoHash.h > for example. > > Thanks, > -Brad > > -- Martin 0001-Support-for-many-custom-commands-in-Windows.patch Description: Binary data -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Custom commands with Ninja on Windows
On 05/19/2016 04:31 PM, Martin Ankerl wrote: > I didn't think about just writing a .cmd (or .bat?) with cmake The ".cmd" extension is a modern version of ".bat". > that sounds like the simplest solution! Yes, assuming we never have a need for ninja placeholder substitution. Why are one-line response files generated by Ninja not a solution? Does cmd support them? > if %errorlevel% neq 0 exit /b %errorlevel% Yes. > I have no experience with cmake implementation though, how would you > find a name for the filename? First, it can be made conditional on when the command line is really long. Second, you could just put it in CMakeFiles/ and name it using a hash of its content (or of the list of outputs). See Source/cmCryptoHash.h for example. Thanks, -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Custom commands with Ninja on Windows
I didn't think about just writing a .cmd (or .bat?) with cmake, that sounds like the simplest solution! After each each custom command we just need to add something like this to have the same behaviour as the &&: if %errorlevel% neq 0 exit /b %errorlevel% I have no experience with cmake implementation though, how would you find a name for the filename? Martin On Thu, May 19, 2016 at 4:03 PM Brad King wrote: > On 05/19/2016 01:43 AM, Martin Ankerl wrote: > > It seems to me that there are two possible solution: > > > > * use a response file (rspfile and rspfile_content). > > Can cmd.exe /C even use a response file? Perhaps the whole command could > simply be written to a .cmd file instead. I don't think our generated > custom commands ever use ninja placeholders so we could have CMake write > the commands to a file instead. Ninja would only see a call to that file. > The drawback is that we may not see the whole command from `ninja -v`. > > > only possible when modifying ninja to support multiline response files > > In what case do we end up with a multiline command? Isn't it just > one long command line with && chaining? > > > * Add a build target for each custom command, e.g. instead of > > It may be tricky to get the same build semantics with that approach. > > Thanks, > -Brad > > -- Martin -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Custom commands with Ninja on Windows
On 05/19/2016 01:43 AM, Martin Ankerl wrote: > It seems to me that there are two possible solution: > > * use a response file (rspfile and rspfile_content). Can cmd.exe /C even use a response file? Perhaps the whole command could simply be written to a .cmd file instead. I don't think our generated custom commands ever use ninja placeholders so we could have CMake write the commands to a file instead. Ninja would only see a call to that file. The drawback is that we may not see the whole command from `ninja -v`. > only possible when modifying ninja to support multiline response files In what case do we end up with a multiline command? Isn't it just one long command line with && chaining? > * Add a build target for each custom command, e.g. instead of It may be tricky to get the same build semantics with that approach. Thanks, -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Custom commands with Ninja on Windows
Alright, I'll see what I can do when I find some free time to work on this then! /Florent On Mon, Jun 15, 2015 at 3:14 PM, Brad King wrote: > On 06/12/2015 06:01 AM, Florent Castelli wrote: > > So I've been thinking that on Windows, instead of concatenating > > everything, we should be a script that is run using a response > > file that just runs all the command with error checking. > > Eventually, we could reuse > cmLocalVisualStudioGenerator::ConstructScript() > > for that matter. > > Yes, I think that would be a reasonable approach. > > Thanks, > -Brad > > -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Custom commands with Ninja on Windows
On 06/12/2015 06:01 AM, Florent Castelli wrote: > So I've been thinking that on Windows, instead of concatenating > everything, we should be a script that is run using a response > file that just runs all the command with error checking. > Eventually, we could reuse cmLocalVisualStudioGenerator::ConstructScript() > for that matter. Yes, I think that would be a reasonable approach. Thanks, -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Custom commands with Ninja on Windows
> I'm having some issues with Ninja on Windows with long custom commands > (or actually a long succession of short commands appended to the same > target). The problem is that they get concatenated in one single command > using " && " and it is pretty easy to go over the 8k command line size > limit on Windows. > > So I've been thinking that on Windows, instead of concatenating > everything, we should be a script that is run using a response file that > just runs all the command with error checking. Eventually, we could > reuse cmLocalVisualStudioGenerator::ConstructScript() for that matter. > > Any thought about this? Any hint for implementing this myself in the > Ninja generator? Do you think of another way to fix this issue? came across this issue as well: http://www.itk.org/Bug/view.php?id=15612 -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
