Re: [cmake-developers] daemon-mode meeting last Tuesday
Hello everyone, On Wed, Jun 29, 2016 at 12:41 AM, Stephen Kelly wrote: > Tobias Hunger wrote: > Name > > > Last year before publication I was calling this feature a 'metadata server' > (class called cmMetadataServer etc). I probably made a mistake renaming it > to a daemon. Now, 'metadata server' might be better and more specific than > 'server' alone. I do not care what we end up calling the baby, but considering that I do not expect more server modes to pop up in cmake (and considering that we do not yet fully know where we will end up with our little server), I think server is a fine name. > Change Notification > --- > > I wrote previously that I think change notification is an essential feature > of the first version of the protocol - otherwise the output is no better > than a static generated file. > > > http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/16589/focus=16648 > > Tobias was concerned that implementing filesystem watching for all necessary > platforms is an unreasonable burden and he would prefer something like > QFileSystemWatcher. > > libuv has a portable abstraction for filesystem change notification similar > in a way to QFileSystemWatcher. I implemented a basic use of the libuv API > fs notification API last year. > > Action item for me is to dig out that work or do it again. Then the metadata > server can emit a message on the protocol when the source directory content > changes. Creator will watch files itself. It has rather nice logic to save file descriptors when doing that. And since file descriptors are a scarce system resource I will need to turn file watching in server mode off. I do see a use-case where having the cmake server do the watching for its users. I am in no way opposed to doing that, but for me it is definitely not a feature required for the first version. > Feature Flags > - > >> 1.2 Protocol versions >> >> * We need the daemon-mode to support different protocol versions. >> >> * We want to have the option to mark protocol versions as experimental. > > As far as I know, Chrome has a development model whereby unfinished features > are checked into master immediately and developed there, instead of being > developed in a branch. The new/experimental features are disabled by default > with 'feature flags' so that people/CI can enable and test features which > are candidates to be enabled by default to everyone. > > We discussed whether this would make sense for the server instead of > 'experimental' protocol versions. We decided that such feature flags would > make sense if there would be many experimental features in flight from > multiple contributors over multiple cmake releases. We don't expect this to > be the case, so we consider feature flags to be over-engineered for the > server for now. Thanks for documenting that:-) I remembered that I had forgot to mention something we agreed on to not do, but I totally forgot what that was. > InReplyTo > - > > Tobias told me that the only reason for this is to make it easier for people > developing the actual protocol to paste blocks of text which contain > multiple server messages. For example a developer could paste a 'configure' > and a 'compute' and a 'codeModel' message over and over while implementing > the 'codeModel' message. If the 'configure' or 'compute' fail for some > reason with an error Tobias thinks it is confusing to figure out which one > had the error after the fact. I don't agree that that is hard or confusing. > > To be clear: The inReplyTo will never be used by client tools. It is only > for the above use-case. > > My concern is that if we add something only for a use-case like that, client > tools will start to use it for something it is not designed for. Then we > will regret that we did not design something better for the clients actual > need (because inReplyTo isn't exactly what they actually need). > > So, I am strongly in favor of keeping the protocol content to the absolute > minimum - only include things which we know client use-cases for. > Downstreams abusing CMake script APIs for things they were not intended for > is a long term problem for CMake. We should avoid introducing that problem > in the server protocol. > > Especially as inReplyTo, even in its current form is 'fuzzy' to me: Is it in > spontaneous messages which are not in response to anything (message() > output, change notifications etc)? Does it have 'special' values? "inReplyTo" is set in all messages generated by the server in reply to a request received by the client. If the server sends a signal that is not in reply to a request by the client (e.g. filesystem notification), then that will not have inReplyTo set. Considering that some requests trigger a whole lot of messages (e.g. "configure" triggers messages and progress notifications to be sent before the final reply), I find this field highly useful. > Initialization Granularity > ---
Re: [cmake-developers] daemon-mode meeting last Tuesday
Tobias Hunger wrote: > Hello CMake Developers, > > Stephen Kelly and me met last Tuesday to talk about the daemon-mode patch > we both have been working on. It was a very productive meeting: We managed > to resolve almost all the differences of opinion we had. Thanks Tobias for the summary and everyone else for the discussion. Here are a few additional notes that I took when meeting Tobias last week. These notes don't change anything in the discussion, but expands on a few things to put some meat on the bones and lists some discarded design options. cmMessenger --- > * Having cmMessanger would be nice and will make things easier. In particular - a cmMessenger class can have some virtual methods implemented to print to the console as cmake::IssueMessage currently does. It would be inherited by a cmServerMessenger which would override the virtual methods to send them over the server IPC protocol instead of printing them on stdout. This has the advantage of not losing semantic information - a backtrace becomes a JSON array, meaning clients don't have to try to parse the backtrace out of the stdout message as they currently do. That becomes possible with just a IssueMessage virtual. See http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15607 Additionally, the cmMessenger can have additional high-level virtual methods, such as for find_package results making it easier for user tools and IDEs to get a list of packages searched for and make it possible to give users use-case specific ways of specifying where packages are. Similarly it can have a virtual method for try_compile results (somehow - this might require extending the message type for scripting API) which also give user tools more opportunities to present the user with semantic information and not discard the intent of the person writing the buildsystem. Another place where a virtual method on cmMessenger would make sense is for policy warnings, giving user tools a chance to present them to the user without trying to parse them from stdout. There may be other messages which have semantics whose meaning we currently discard by printing to stdout. All of this is currently blocked by a clean-up in the parser needing further review and perhaps further implementation work: http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15607/focus=16663 Name Last year before publication I was calling this feature a 'metadata server' (class called cmMetadataServer etc). I probably made a mistake renaming it to a daemon. Now, 'metadata server' might be better and more specific than 'server' alone. Change Notification --- I wrote previously that I think change notification is an essential feature of the first version of the protocol - otherwise the output is no better than a static generated file. http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/16589/focus=16648 Tobias was concerned that implementing filesystem watching for all necessary platforms is an unreasonable burden and he would prefer something like QFileSystemWatcher. libuv has a portable abstraction for filesystem change notification similar in a way to QFileSystemWatcher. I implemented a basic use of the libuv API fs notification API last year. Action item for me is to dig out that work or do it again. Then the metadata server can emit a message on the protocol when the source directory content changes. Feature Flags - > 1.2 Protocol versions > > * We need the daemon-mode to support different protocol versions. > > * We want to have the option to mark protocol versions as experimental. As far as I know, Chrome has a development model whereby unfinished features are checked into master immediately and developed there, instead of being developed in a branch. The new/experimental features are disabled by default with 'feature flags' so that people/CI can enable and test features which are candidates to be enabled by default to everyone. We discussed whether this would make sense for the server instead of 'experimental' protocol versions. We decided that such feature flags would make sense if there would be many experimental features in flight from multiple contributors over multiple cmake releases. We don't expect this to be the case, so we consider feature flags to be over-engineered for the server for now. InReplyTo - > * Currently a reply (and error and all other messages in response to a > request) contain a "inReplyTo" with the type string from the request > triggering the reply. > > Stephen thinks that is not necessary and argues anything that is not > strictly necessary should not be in the first version of the protocol. Tobias told me that the only reason for this is to make it easier for people developing the actual protocol to paste blocks of text which contain multiple server messages. For example a developer could paste a 'config
Re: [cmake-developers] daemon-mode meeting last Tuesday
Hi Brad, Am 27.06.2016 7:18 nachm. schrieb "Brad King" : > I will take care of making libuv available within CMake's implementation > prior to merging the server mode work. See the "libuv in cmake" branch > of this thread. Perfect. Sorry, I only noticed the other thread after sending my mail. Best regards, Tobias -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
On 06/27/2016 11:49 AM, Tobias Hunger wrote: > Brad, Daniel: You both mentioned wanting to use libuv. Is somebody > already looking into merging this into cmake? So far I just link > against the library and am done with it... > > Stephen said 3rd party libraries would need to be checked into cmake > and that scripts exist to keep these libraries up-to-date. What can I > do to help getting this library available to all cmake users (so that > I can further shrink my patch set;-)? I will take care of making libuv available within CMake's implementation prior to merging the server mode work. See the "libuv in cmake" branch of this thread. -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
He everybody, sorry, I am currently not making the progress I want. Qt Creator has feature freeze coming up and I need to push some patches and fix some bugs. Brad, Daniel: You both mentioned wanting to use libuv. Is somebody already looking into merging this into cmake? So far I just link against the library and am done with it... Stephen said 3rd party libraries would need to be checked into cmake and that scripts exist to keep these libraries up-to-date. What can I do to help getting this library available to all cmake users (so that I can further shrink my patch set;-)? On Mon, Jun 27, 2016 at 5:03 PM, Brad King wrote: > On 06/23/2016 05:19 PM, Tobias Hunger wrote: >> I'll create a task to rename it to "server" then. > > Sounds good. > >>> Would each type of query have a known type of response? >> >> It has a type: "reply" (or "error"). I so far use the inReplyTo field to >> specify what the request a reply refers to. Stephen thinks that is not >> necessary as there is always one request in flight and the client can just >> figure things out without additional information. > > Okay. If the field makes debugging or other use cases easier I see no reason > not to include it. Good, I'll keep inReplyTo for now. >>> Also, doesn't the cookie allow the query/response pairs to be matched? >> >> In theory yes. But a protocol should work without having to reply on cookies. > > Yes, I see. > >> The header currently is the type, inReplyTo and the cookie. I did not see >> the need to separate those. > > Okay. This may be easier to review in context when the time comes. Ok, I'll keep things the way they are for now then. Changing that should not be too hard anyway. >>> Currently cmake-gui supports switching generators, build trees, etc., so >>> there is some precedent for such switching within a single process. If >>> we have (re-)initialization bugs they should simply be fixed. >> >> So you think we should keep that? > > No. See response in another branch of this thread. OK, going for the command line switching model then. >>> I'm not sure we have that information. IIRC CMake only adds settings to the >>> generated build system to tell the tools where to put the .pdb and what to >>> call it if it happens to be created. >> >> I think CMake should know what is generated and should not leave decisions >> like that up to generators. > > Yes, but that will take some additional investigation and work to achieve. > CMake will have to be taught more about which tools/platforms actually > produce the .pdb files. They are not first-class artifacts in CMake's > model right now. I currently list the PDB files in the artifacts (if there is a .lib file). I'll keep that for the time being. Best Regards, Tobias -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
On 06/23/2016 05:19 PM, Tobias Hunger wrote: > I'll create a task to rename it to "server" then. Sounds good. >> Would each type of query have a known type of response? > > It has a type: "reply" (or "error"). I so far use the inReplyTo field to > specify what the request a reply refers to. Stephen thinks that is not > necessary as there is always one request in flight and the client can just > figure things out without additional information. Okay. If the field makes debugging or other use cases easier I see no reason not to include it. >> Also, doesn't the cookie allow the query/response pairs to be matched? > > In theory yes. But a protocol should work without having to reply on cookies. Yes, I see. > The header currently is the type, inReplyTo and the cookie. I did not see > the need to separate those. Okay. This may be easier to review in context when the time comes. >> Currently cmake-gui supports switching generators, build trees, etc., so >> there is some precedent for such switching within a single process. If >> we have (re-)initialization bugs they should simply be fixed. > > So you think we should keep that? No. See response in another branch of this thread. >> I'm not sure we have that information. IIRC CMake only adds settings to the >> generated build system to tell the tools where to put the .pdb and what to >> call it if it happens to be created. > > I think CMake should know what is generated and should not leave decisions > like that up to generators. Yes, but that will take some additional investigation and work to achieve. CMake will have to be taught more about which tools/platforms actually produce the .pdb files. They are not first-class artifacts in CMake's model right now. > I like "cmakeInputs" Sounds good. Thanks, -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
On 06/24/2016 11:43 AM, Daniel Pfeifer wrote: > On Fri, Jun 24, 2016, Tobias Hunger wrote: So you opt for enabling the cmake-server to switch between different build directories? >>> >>> Clients who need this feature could simply restart the server in the >>> new directory. >> >> yes, that was what Stephen and me thought, too. I understood Brad to >> imply that we should just do everything inside server-mode and fix any >> fallout we run into instead. > > To me it feels like the only reason why there is precedent for such > switching within a single process, is that cmake-gui links against > libcmake instead of using the cmake-server. I don't see any technical > reason apart from that it hasn't been split into two processes yet. Yes. I was just pointing out that it is possible to use a single process if we want. If it is simpler to make the src/bld/gen settings part of the startup command line then I think we should do that. -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
On 06/23/2016 04:48 PM, Ben Boeckel wrote: > cmake-oracle sounds reasonable for what it's doing to me. That name conflicts with a well-known vendor behind a platform on which CMake runs so IMO would be confusing. -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
On Fri, Jun 24, 2016 at 5:24 PM, Tobias Hunger wrote: > Hi Daniel, > > On Fri, Jun 24, 2016 at 5:08 PM, Daniel Pfeifer > wrote: Currently cmake-gui supports switching generators, build trees, etc., so there is some precedent for such switching within a single process. If we have (re-)initialization bugs they should simply be fixed. >>> >>> So you opt for enabling the cmake-server to switch between different >>> build directories? >> >> Clients who need this feature could simply restart the server in the >> new directory. > > yes, that was what Stephen and me thought, too. I understood Brad to > imply that we should just do everything inside server-mode and fix any > fallout we run into instead. I just opened cmake-gui and checked what exactly can be changed. There is no graphical option to change the generator (you can change the option sting in the cache, of course). When you change the binary directory, it reads the cache from that directory, with the effect that the source directory is changed too. To me, that feels like a hard reset/restart. To me it feels like the only reason why there is precedent for such switching within a single process, is that cmake-gui links against libcmake instead of using the cmake-server. I don't see any technical reason apart from that it hasn't been split into two processes yet. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
Hi Daniel, On Fri, Jun 24, 2016 at 5:08 PM, Daniel Pfeifer wrote: >>> Currently cmake-gui supports switching generators, build trees, etc., so >>> there is some precedent for such switching within a single process. If >>> we have (re-)initialization bugs they should simply be fixed. >> >> So you opt for enabling the cmake-server to switch between different >> build directories? > > Clients who need this feature could simply restart the server in the > new directory. yes, that was what Stephen and me thought, too. I understood Brad to imply that we should just do everything inside server-mode and fix any fallout we run into instead. That is why I wanted to have this clarified, as this does cause quite a few changes in the implementation. Best Regards, Tobias -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
On Fri, Jun 24, 2016 at 4:56 PM, Tobias Hunger wrote: > Hi Brad, > > On Thu, Jun 23, 2016 at 10:46 PM, Brad King wrote: >>> * This is currently used to set sourcedirectory, builddirectory and >>> generator. >>> >>> These three should be passed in via the command line instead, again >>> to avoid trouble with users switching to different build >>> directories/etc. >>> while the daemon is running, which might trigger problems with state >>> cleanup, etc. >> >> Currently cmake-gui supports switching generators, build trees, etc., so >> there is some precedent for such switching within a single process. If >> we have (re-)initialization bugs they should simply be fixed. > > So you opt for enabling the cmake-server to switch between different > build directories? Clients who need this feature could simply restart the server in the new directory. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
Hi Brad, On Thu, Jun 23, 2016 at 10:46 PM, Brad King wrote: >> * This is currently used to set sourcedirectory, builddirectory and >> generator. >> >> These three should be passed in via the command line instead, again >> to avoid trouble with users switching to different build directories/etc. >> while the daemon is running, which might trigger problems with state >> cleanup, etc. > > Currently cmake-gui supports switching generators, build trees, etc., so > there is some precedent for such switching within a single process. If > we have (re-)initialization bugs they should simply be fixed. So you opt for enabling the cmake-server to switch between different build directories? That would have the following side-effects: * We would need the reset command to get the server back into a defined state * We need to keep it possible to set source directory, build directory and generator from inside server mode * We can keep the existing generators query-able from inside server mode. If they are set inside server mode, then there is no need to know them before starting server mode after all. * The commandline way of querying generators is not necessary. We might still want that, but it does not belong into this patch set anymore. Is this the way to do? I will need to back out some changes I did since speaking with Stephen then. No problem, git has everything;-) Best Regards, Tobias -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
Hallo Daniel, On Fri, Jun 24, 2016 at 9:59 AM, Daniel Pfeifer wrote: > On Thu, Jun 23, 2016 at 2:27 PM, Tobias Hunger wrote: >> * We both think it only makes sense to merge the infrastructure part into >> cmake (if it passes review first of course) once we have some >> functionality >> that is genuinely useful. So we want to aim at having the infrastructure >> and the codemodel merged in one go. > > Please think about adding libuv earlier. As Brad wrote before, libuv > could replace some #ifdef code that we currently have (process > handling, file operations). I am currently refactoring the code around > the cmOutputConverter and I would like to introduce a cmPath class (I > will write more about that later, but I can already say that libuv > might come handy for some functionality). Feel free to merge it at your own pace. I'll just use your copy then:-) Currently I just link to an external version of said library and have not yet tried to figure out what the policy is wrt. 3rd party code in cmake. Stephen suggested that we would need to include a copy into the cmake repository and hinted at a script that keeps these copies updated. >> 2.7 cache (report contents of CMakeCache.txt file) >> >> * Review by other potential users would be appreciated, but no obvious >> problems seen. > > Is this needed? If the CMakeCache.txt file continues to be written, > clients could just parse that file. If clients want to make changes > (ie. cache editors), they can rewrite the file. Is that a problem? I > assume it is, because CMake needs to reconfigure when it rereads it. > In that case, the server should provide a way to report the contents, > but also ways to modify it. Yes, this is needed. I do not want 3rd party code to interact with what is basically a file internal to cmake. And the syntax of that file is not as trivial as the comment at its top claims it is. Just check how properties on values are encoded. Best Regards, Tobias -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
On Thu, Jun 23, 2016 at 2:27 PM, Tobias Hunger wrote: > * We both think it only makes sense to merge the infrastructure part into > cmake (if it passes review first of course) once we have some > functionality > that is genuinely useful. So we want to aim at having the infrastructure > and the codemodel merged in one go. Please think about adding libuv earlier. As Brad wrote before, libuv could replace some #ifdef code that we currently have (process handling, file operations). I am currently refactoring the code around the cmOutputConverter and I would like to introduce a cmPath class (I will write more about that later, but I can already say that libuv might come handy for some functionality). > 2.7 cache (report contents of CMakeCache.txt file) > > * Review by other potential users would be appreciated, but no obvious > problems seen. Is this needed? If the CMakeCache.txt file continues to be written, clients could just parse that file. If clients want to make changes (ie. cache editors), they can rewrite the file. Is that a problem? I assume it is, because CMake needs to reconfigure when it rereads it. In that case, the server should provide a way to report the contents, but also ways to modify it. Cheers, Daniel PS: Next meeting in a Biergarten in Munich? -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
Hi Rolf, Am 23.06.2016 23:00 schrieb "Rolf Eike Beer" : > What about just passing --server-fd= into it? That way the client can set > up whatever if wants to pass there and CMake has nothing to do with any setup. > And if one really wants to get messy one could pass a tty there. Not sure whether that is even possible on all platforms we need. Libuv (which the server is using) apparently has a cross-platform solution for setting up a communication channel like the one we need, so I want to give that a try. Best Regards, Tobias -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
Hi Ben, Am 23.06.2016 22:48 schrieb "Ben Boeckel" : > cmake-oracle sounds reasonable for what it's doing to me. I hope this thing will provide information more reliable than e.g. the Oracle of Delphi in ancient Greece:-) Best Regards, Tobias -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
Am 23.06.2016 22:46 schrieb "Brad King" : > > On 06/23/2016 08:27 AM, Tobias Hunger wrote: > Great! Thanks for the detailed summary of the meeting and status of the design. You are welcome. > > * Some names could be improved. Suggestions welcome. > > We'll cover these in more detail during review. The most important name is > the name of the feature, and IMO "daemon" is not an accurate description of > it (it is never re-parented to 'init' and has a lifetime tied to the client). > I think "server" would be better but am open to other suggestions. Even if > it later really becomes a daemon the name "server" will still be appropriate. I'll create a task to rename it to "server" then. > > * Currently a reply (and error and all other messages in response to a > > request) contain a "inReplyTo" with the type string from the request > > triggering the reply. > > > > Stephen thinks that is not necessary and argues anything that is not > > strictly necessary should not be in the first version of the protocol. > > > > I like having this as it makes it helps to identify what a reply/error is > > related to. I often paste a whole string of requests into the daemon and > > if there is an error it is really easy to see how for things got. > > Would each type of query have a known type of response? Shouldn't the response > objects have a type too? It has a type: "reply" (or "error"). I so far use the inReplyTo field to specify what the request a reply refers to. Stephen thinks that is not necessary as there is always one request in flight and the client can just figure things out without additional information. > Also, doesn't the cookie allow the query/response > pairs to be matched? In theory yes. But a protocol should work without having to reply on cookies. A cookie is user data attached to an request. The idea is that e.g. an IDE can run a request for some more details on some node in the project tree view and pass some form of ID of the node the information is for along with the query. That can simplify client code a lot. > > * Should we have a split between header (cookie/type/inReplyTo) and data? > > > > Stephen suggested passing all data that is from/for the client in a > > "body" attribute to make messages more uniform. > > > > I think that is just adding one more level into the JSON file for no > > real benefit. > > I agree with Tobias that there is no reason to put the body fields in > a sub-object. However, I think the header fields should go in a sub-object > named by a "header" field. That will isolate the header field names from > the body fields (good for sorted views, future addition of headers, etc.). The header currently is the type, inReplyTo and the cookie. I did not see the need to separate those. > > * This is currently used to set sourcedirectory, builddirectory and > > generator. > > > > These three should be passed in via the command line instead, again > > to avoid trouble with users switching to different build directories/etc. > > while the daemon is running, which might trigger problems with state > > cleanup, etc. > > Currently cmake-gui supports switching generators, build trees, etc., so > there is some precedent for such switching within a single process. If > we have (re-)initialization bugs they should simply be fixed. So you think we should keep that? I do not have an opinion here and need to follow the advise of people more knowledgeable about CMake code than I am. > > * Figure out whether a PDB file is actually going to be build. Currently > > the PDB file is added to the list of artifacts whenever there is a .lib > > file, which is probably wrong. Brad: Do you have an idea how to > > get this information? > > I'm not sure we have that information. IIRC CMake only adds settings to the > generated build system to tell the tools where to put the .pdb and what to > call it if it happens to be created. We don't have install rules for them > so there has never been a need to know whether they will actually be created. > Perhaps our format for the list of artifacts needs a way to indicate whether > an artifact is optionally present. I think CMake should know what is generated and should not leave decisions like that up to generators. > > 2.8 buildsystem (return cmake files) > > > > * Return a list of cmake files (CMakeLists.txt, etc.) in different categories > > (in source directory/in build directory/in the system). > > > > * Stephen suggested to rename this to "cmake_rerun_depends". I am not sure > > that name is much better than what is currently used. > > "cmake_rerun_depends" describes the current use case for these. I don't think > the name "buildsystem" is clear though. Other ideas: > > * cmake_files > * cmake_inputs > * input_files I like "cmakeInputs":-) I'll make a task to change it if nobody opposes this suggestion. Best Regards, Tobias -- Powe
Re: [cmake-developers] daemon-mode meeting last Tuesday
> 1.1 Communication: > > * Clients should talk to daemon mode via a socket (libuv supports that), > but we also want to support stdin/stdout at least for a "debug mode" * > There should be a separate command line switch to start stdin/out mode and > for socket mode > * This is needed before it makes sense to merge the patch set. What about just passing --server-fd= into it? That way the client can set up whatever if wants to pass there and CMake has nothing to do with any setup. And if one really wants to get messy one could pass a tty there. Eike signature.asc Description: This is a digitally signed message part. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
On Thu, Jun 23, 2016 at 16:46:48 -0400, Brad King wrote: > On 06/23/2016 08:27 AM, Tobias Hunger wrote: > > * Figure out whether a PDB file is actually going to be build. Currently > > the PDB file is added to the list of artifacts whenever there is a .lib > > file, which is probably wrong. Brad: Do you have an idea how to > > get this information? > > I'm not sure we have that information. IIRC CMake only adds settings to the > generated build system to tell the tools where to put the .pdb and what to > call it if it happens to be created. We don't have install rules for them > so there has never been a need to know whether they will actually be created. > Perhaps our format for the list of artifacts needs a way to indicate whether > an artifact is optionally present. Well, there is this issue: https://gitlab.kitware.com/cmake/cmake/issues/10636 which requests that they always be generated :) . > > 2.8 buildsystem (return cmake files) > > > > * Return a list of cmake files (CMakeLists.txt, etc.) in different > > categories > > (in source directory/in build directory/in the system). > > > > * Stephen suggested to rename this to "cmake_rerun_depends". I am not sure > > that name is much better than what is currently used. > > "cmake_rerun_depends" describes the current use case for these. I don't think > the name "buildsystem" is clear though. Other ideas: > > * cmake_files > * cmake_inputs > * input_files Also possible: regen_trigger_files --Ben -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
On Thu, Jun 23, 2016 at 16:46:48 -0400, Brad King wrote: > On 06/23/2016 08:27 AM, Tobias Hunger wrote: > > * Some names could be improved. Suggestions welcome. > > We'll cover these in more detail during review. The most important name is > the name of the feature, and IMO "daemon" is not an accurate description of > it (it is never re-parented to 'init' and has a lifetime tied to the client). > I think "server" would be better but am open to other suggestions. Even if > it later really becomes a daemon the name "server" will still be appropriate. > > As you said before there has been a lot of PR with the name "daemon", but I > don't think it is too late to change it. Those interested in the features > will see the new name when it appears in release notes and should easily be > able to recognize it. cmake-oracle sounds reasonable for what it's doing to me. --Ben -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] daemon-mode meeting last Tuesday
On 06/23/2016 08:27 AM, Tobias Hunger wrote: > Stephen Kelly and me met last Tuesday to talk about the daemon-mode patch we > both have been working on. It was a very productive meeting: We managed to > resolve almost all the differences of opinion we had. Great! Thanks for the detailed summary of the meeting and status of the design. > * Some names could be improved. Suggestions welcome. We'll cover these in more detail during review. The most important name is the name of the feature, and IMO "daemon" is not an accurate description of it (it is never re-parented to 'init' and has a lifetime tied to the client). I think "server" would be better but am open to other suggestions. Even if it later really becomes a daemon the name "server" will still be appropriate. As you said before there has been a lot of PR with the name "daemon", but I don't think it is too late to change it. Those interested in the features will see the new name when it appears in release notes and should easily be able to recognize it. > * Clients should talk to daemon mode via a socket (libuv supports that), > but we also want to support stdin/stdout at least for a "debug mode" > * There should be a separate command line switch to start stdin/out mode >and for socket mode > * This is needed before it makes sense to merge the patch set. Given the noise we have on stdout it makes sense to offer a dedicated communication path for clients. > * Currently a reply (and error and all other messages in response to a > request) contain a "inReplyTo" with the type string from the request > triggering the reply. > > Stephen thinks that is not necessary and argues anything that is not > strictly necessary should not be in the first version of the protocol. > > I like having this as it makes it helps to identify what a reply/error is > related to. I often paste a whole string of requests into the daemon and > if there is an error it is really easy to see how for things got. Would each type of query have a known type of response? Shouldn't the response objects have a type too? Also, doesn't the cookie allow the query/response pairs to be matched? > * Should we have a split between header (cookie/type/inReplyTo) and data? > > Stephen suggested passing all data that is from/for the client in a > "body" attribute to make messages more uniform. > > I think that is just adding one more level into the JSON file for no > real benefit. I agree with Tobias that there is no reason to put the body fields in a sub-object. However, I think the header fields should go in a sub-object named by a "header" field. That will isolate the header field names from the body fields (good for sorted views, future addition of headers, etc.). > * This is currently used to set sourcedirectory, builddirectory and > generator. > > These three should be passed in via the command line instead, again > to avoid trouble with users switching to different build directories/etc. > while the daemon is running, which might trigger problems with state > cleanup, etc. Currently cmake-gui supports switching generators, build trees, etc., so there is some precedent for such switching within a single process. If we have (re-)initialization bugs they should simply be fixed. > * Figure out whether a PDB file is actually going to be build. Currently > the PDB file is added to the list of artifacts whenever there is a .lib > file, which is probably wrong. Brad: Do you have an idea how to > get this information? I'm not sure we have that information. IIRC CMake only adds settings to the generated build system to tell the tools where to put the .pdb and what to call it if it happens to be created. We don't have install rules for them so there has never been a need to know whether they will actually be created. Perhaps our format for the list of artifacts needs a way to indicate whether an artifact is optionally present. > 2.8 buildsystem (return cmake files) > > * Return a list of cmake files (CMakeLists.txt, etc.) in different > categories > (in source directory/in build directory/in the system). > > * Stephen suggested to rename this to "cmake_rerun_depends". I am not sure > that name is much better than what is currently used. "cmake_rerun_depends" describes the current use case for these. I don't think the name "buildsystem" is clear though. Other ideas: * cmake_files * cmake_inputs * input_files -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/hel
[cmake-developers] daemon-mode meeting last Tuesday
Hello CMake Developers, Stephen Kelly and me met last Tuesday to talk about the daemon-mode patch we both have been working on. It was a very productive meeting: We managed to resolve almost all the differences of opinion we had. Here is a short summary of what we talked about: 0) General * We both want a daemon mode:-) * Some names could be improved. Suggestions welcome. * Having cmMessanger would be nice and will make things easier. * We agreed that we should get the basic infrastructure part reviewed first, followed by the functionality up to the project structure (codemodel). * We both think it only makes sense to merge the infrastructure part into cmake (if it passes review first of course) once we have some functionality that is genuinely useful. So we want to aim at having the infrastructure and the codemodel merged in one go. 1) Infrastructure: 1.1 Communication: * Clients should talk to daemon mode via a socket (libuv supports that), but we also want to support stdin/stdout at least for a "debug mode" * There should be a separate command line switch to start stdin/out mode and for socket mode * This is needed before it makes sense to merge the patch set. 1.2 Protocol versions * We need the daemon-mode to support different protocol versions. These need to be versioned independently of the cmake binary itself and we will try to keep the protocol versions working as long as that is reasonable, but would try to support them for at least a year or two (if there are no overriding concerns). * We want to have the option to mark protocol versions as experimental. Experimental protocol versions come with *NO* guarantees and can change *AT ANY TIME*. If a experimental protocol ends up in a cmake release there still are no guarantees. I would like to see experimental protocol versions in cmake releases, since that makes it really easy for other parties to evaluate changes. In my experience hardly anybody builds custom branches:-/ All this would need to be communicated by cmake e.g. in the release notes. 1.3 Basic Protocol * Client/daemon talk via JSON messages. * All messages have a "type" with a string specifying the type of message. * We (mostly) have a request/reply model. The client sends a request message and will receive a reply or an error. * Requests contain a type, and an optional cookie (which will be returned together with any message sent by the daemon in response to the request). A cookie can be any JSON object. * A reply is a JSON message of type "reply". It contains the cookie of its request and any data that the daemon wants to send its client. * An error is a JSON message of type "error". It contains the cookie of its request and an "errorMessage". * There are more JSON messages for progress information and messages, etc. and we can add more as needed. TO DECIDE: * Currently a reply (and error and all other messages in response to a request) contain a "inReplyTo" with the type string from the request triggering the reply. Stephen thinks that is not necessary and argues anything that is not strictly necessary should not be in the first version of the protocol. I like having this as it makes it helps to identify what a reply/error is related to. I often paste a whole string of requests into the daemon and if there is an error it is really easy to see how for things got. * Should we have a split between header (cookie/type/inReplyTo) and data? Stephen suggested passing all data that is from/for the client in a "body" attribute to make messages more uniform. I think that is just adding one more level into the JSON file for no real benefit. 2) Basic Command Set 2.1) Reset (Reset to a state right after daemon start) * Remove this. This is error prone as data structures may not be cleaned out properly. 2.2) globalSettings (Query generators, debug flags, etc.) * Move generators and other static information into a separate cmake commandline invocation (http://public.kitware.com/Bug/view.php?id=15462): * Generators with information on them * CMake version information * Module path * Daemon mode support all should be available in a machine readable way from that command. TODO: Figure out how to get all the generator information, e.g. which values the flags some generator taken can have (architecture, etc.). Stephen volunteered to check into this. * We can report the selected generator/extragenerator here. * Undecided about whether it makes sense to report source/builddirectory here 2.3) setGlobalSettings (set the information from globalSettings) * This is currently used to set sourcedirectory, builddirectory and
