Re: [cmake-developers] Using CMake as a library from Python
Charles, I'd like to comment on a minor issue: CMakeLists.txt files don't specify user-specific data like source and build dirs (for a reason) yet your POC listfile contains this line: cmake.init("Ninja", "/media/dev/src/cmaketest", "/media/dev/build/cmaketest") which sets the source and build directories. Is this some temporary solution for the POC only? Tamas On Mon, Jan 4, 2016 at 8:41 AM, Charles Huetwrote: > Hi, > > Because of the above, I worry that you are basing your work on an old >> version of CMake. As of April 2015 cmState is used as the interface to the >> cache. > > > Yeah, that is sadly right. I wanted to rebase on master before publishing, > thinking it should not be too hard, and I have lots to re-do, mostly > understand how things are done now. > > I did something similar some years ago with QML instead of python, so it >> sounds interesting to me. > > > I'm trying to be as declarative as possible, because really like how > readable simple QML programs are, and I think it would be perfect for a > buildsystem. > > My guess is that you are using python to instantiate a cmAddLibraryCommand >> and then executing it. > > > Actually, I'm directly using the cmMakefile, because I did not want to > wrap all the commands, and it seemed backwards to me to wrap them. > > Even much of cmMakefile shouldn't be used by a new language. Instead >> certain >> parts of cmMakefile should be extracted as other classes >> (cmVariableExpander >> which should have the ExpandVariablesInString and ConfigureString stuff, >> cmMessenger for the IssueMessage stuff, some other class for the >> CompileFeature stuff etc). Then cmMakefile would be just about executing >> and >> scoping the CMake language. A new language would not need that, but would >> use the refactored extracted classes. > > > Ah, this is very interesting, thanks. > > > Having said all that, Brad favors Lua I believe, and he favors a different >> approach (which no one is working on as far as I know) to adding a new >> language. So wait to hear from him to know whether it is something that >> would be upstreamable. > > > Have any details on the approach in question ? SWIG would allow for Lua > bindings as easily, but I don't think having multiple languages would be a > good idea. > I went with Python because I'm familiar with it and have already written > bindings for it with SWIG. Also, buildbot is written in python and it could > provide a really interesting integration I think. > > I would guide/support you in refactoring cmake as needed. The refactoring >> part would definitely be upstreamable. I would very much like to see a >> proof >> of concept alternative language even if that wasn't upstreamed. It would >> prove that another language is possible, and that's one of the steps to >> replacing the current cmake language I think. > > > I will need to work on it to make it work again with master, but I'll try > and do this soon. > > Here is what my test POC looked like for generating a simple shared > library: > > #!/usr/bin/env python >> # -*- coding: utf-8 -*- >> import cmake >> cmake.init("Ninja", "/media/dev/src/cmaketest", >> "/media/dev/build/cmaketest") >> myProject = cmake.Project("MyTestProject") >> myProject.targets = [ cmake.SharedLibrary("testLibrary", ["lib.cxx"]) ] >> cmake.generate() > > > Thanks a lot for your comments, I'm happy to see this was not just a dumb > idea, and that others have thought about it. > > I'll update with the github as soon as I get it working. > > Best > > > Le lun. 4 janv. 2016 à 01:16, Stephen Kelly a écrit : > >> Charles Huet wrote: >> >> > * cmCacheManager::AddCacheEntry was made public, as cmake::Configure >> > cannot be used from python (it check for the existence of a >> CMakeLists.txt >> > file, which does not exist in this scenario) and the cache variables it >> > sets seem to be necessary. >> >> Because of the above, I worry that you are basing your work on an old >> version of CMake. As of April 2015 cmState is used as the interface to the >> cache. >> >> https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a6b1ad13 >> >> Also note that the C++ interfaces in CMake are not stable. In the last >> year >> they have been changed utterly and that will continue to happen without >> notice. I'm sure you know this, but I thought I'd say it anyway. >> >> > I will try to make a layer of abstraction on top of the python bindings >> > before publishing this work on github, in order to show the syntactic >> > sugar python can provide, but I will publish this before if anybody is >> > interested in working on this. >> >> I would be interested in seeing it. >> >> > Now, does anyone beside me think this is a good idea ? >> >> I did something similar some years ago with QML instead of python, so it >> sounds interesting to me. >> >> In fact, one of the reasons for introducing cmState and doing all the >> refactoring I did in cmake was to make it possible to
Re: [cmake-developers] Using CMake as a library from Python
I took the time to make it work again, so I pushed it here: https://github.com/packadal/CMake/tree/cmake-python The whole branch is ugly as can be, and because I started with an old CMake and recently rebased, the python abstraction might be bloated already, but it works in the nominal case, and it would be pretty easy to add most of the CMake basic functionalities. I am not used to writing python, so it might not be idiomatic either, but hey, it works as a simple proof of concept, and it's all I wanted to do for now :) Le lun. 4 janv. 2016 à 08:41, Charles Hueta écrit : > Hi, > > Because of the above, I worry that you are basing your work on an old >> version of CMake. As of April 2015 cmState is used as the interface to the >> cache. > > > Yeah, that is sadly right. I wanted to rebase on master before publishing, > thinking it should not be too hard, and I have lots to re-do, mostly > understand how things are done now. > > I did something similar some years ago with QML instead of python, so it >> sounds interesting to me. > > > I'm trying to be as declarative as possible, because really like how > readable simple QML programs are, and I think it would be perfect for a > buildsystem. > > My guess is that you are using python to instantiate a cmAddLibraryCommand >> and then executing it. > > > Actually, I'm directly using the cmMakefile, because I did not want to > wrap all the commands, and it seemed backwards to me to wrap them. > > Even much of cmMakefile shouldn't be used by a new language. Instead >> certain >> parts of cmMakefile should be extracted as other classes >> (cmVariableExpander >> which should have the ExpandVariablesInString and ConfigureString stuff, >> cmMessenger for the IssueMessage stuff, some other class for the >> CompileFeature stuff etc). Then cmMakefile would be just about executing >> and >> scoping the CMake language. A new language would not need that, but would >> use the refactored extracted classes. > > > Ah, this is very interesting, thanks. > > > Having said all that, Brad favors Lua I believe, and he favors a different >> approach (which no one is working on as far as I know) to adding a new >> language. So wait to hear from him to know whether it is something that >> would be upstreamable. > > > Have any details on the approach in question ? SWIG would allow for Lua > bindings as easily, but I don't think having multiple languages would be a > good idea. > I went with Python because I'm familiar with it and have already written > bindings for it with SWIG. Also, buildbot is written in python and it could > provide a really interesting integration I think. > > I would guide/support you in refactoring cmake as needed. The refactoring >> part would definitely be upstreamable. I would very much like to see a >> proof >> of concept alternative language even if that wasn't upstreamed. It would >> prove that another language is possible, and that's one of the steps to >> replacing the current cmake language I think. > > > I will need to work on it to make it work again with master, but I'll try > and do this soon. > > Here is what my test POC looked like for generating a simple shared > library: > > #!/usr/bin/env python >> # -*- coding: utf-8 -*- >> import cmake >> cmake.init("Ninja", "/media/dev/src/cmaketest", >> "/media/dev/build/cmaketest") >> myProject = cmake.Project("MyTestProject") >> myProject.targets = [ cmake.SharedLibrary("testLibrary", ["lib.cxx"]) ] >> cmake.generate() > > > Thanks a lot for your comments, I'm happy to see this was not just a dumb > idea, and that others have thought about it. > > I'll update with the github as soon as I get it working. > > Best > > > Le lun. 4 janv. 2016 à 01:16, Stephen Kelly a écrit : > >> Charles Huet wrote: >> >> > * cmCacheManager::AddCacheEntry was made public, as cmake::Configure >> > cannot be used from python (it check for the existence of a >> CMakeLists.txt >> > file, which does not exist in this scenario) and the cache variables it >> > sets seem to be necessary. >> >> Because of the above, I worry that you are basing your work on an old >> version of CMake. As of April 2015 cmState is used as the interface to the >> cache. >> >> https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a6b1ad13 >> >> Also note that the C++ interfaces in CMake are not stable. In the last >> year >> they have been changed utterly and that will continue to happen without >> notice. I'm sure you know this, but I thought I'd say it anyway. >> >> > I will try to make a layer of abstraction on top of the python bindings >> > before publishing this work on github, in order to show the syntactic >> > sugar python can provide, but I will publish this before if anybody is >> > interested in working on this. >> >> I would be interested in seeing it. >> >> > Now, does anyone beside me think this is a good idea ? >> >> I did something similar some years ago with QML instead of python, so it >>
Re: [cmake-developers] Using CMake as a library from Python
Yes, this is POC-only. I felt too lazy to make an argparse object and properly perform these tasks. Obviously, the generator and source dir should be arguments. The build dir should be the execution dir of the script, to mimic CMake behavior. This should not be difficult, but as I said I wanted first to get this disussion started. Le lun. 4 janv. 2016 à 11:54, Tamás Kenéza écrit : > Charles, > > I'd like to comment on a minor issue: > > CMakeLists.txt files don't specify user-specific data like source and > build dirs (for a reason) yet your POC listfile contains this line: > > cmake.init("Ninja", "/media/dev/src/cmaketest", > "/media/dev/build/cmaketest") > > which sets the source and build directories. Is this some temporary > solution for the POC only? > > Tamas > > On Mon, Jan 4, 2016 at 8:41 AM, Charles Huet > wrote: > >> Hi, >> >> Because of the above, I worry that you are basing your work on an old >>> version of CMake. As of April 2015 cmState is used as the interface to >>> the >>> cache. >> >> >> Yeah, that is sadly right. I wanted to rebase on master before >> publishing, thinking it should not be too hard, and I have lots to re-do, >> mostly understand how things are done now. >> >> I did something similar some years ago with QML instead of python, so it >>> sounds interesting to me. >> >> >> I'm trying to be as declarative as possible, because really like how >> readable simple QML programs are, and I think it would be perfect for a >> buildsystem. >> >> My guess is that you are using python to instantiate a cmAddLibraryCommand >>> and then executing it. >> >> >> Actually, I'm directly using the cmMakefile, because I did not want to >> wrap all the commands, and it seemed backwards to me to wrap them. >> >> Even much of cmMakefile shouldn't be used by a new language. Instead >>> certain >>> parts of cmMakefile should be extracted as other classes >>> (cmVariableExpander >>> which should have the ExpandVariablesInString and ConfigureString stuff, >>> cmMessenger for the IssueMessage stuff, some other class for the >>> CompileFeature stuff etc). Then cmMakefile would be just about executing >>> and >>> scoping the CMake language. A new language would not need that, but would >>> use the refactored extracted classes. >> >> >> Ah, this is very interesting, thanks. >> >> >> Having said all that, Brad favors Lua I believe, and he favors a different >>> approach (which no one is working on as far as I know) to adding a new >>> language. So wait to hear from him to know whether it is something that >>> would be upstreamable. >> >> >> Have any details on the approach in question ? SWIG would allow for Lua >> bindings as easily, but I don't think having multiple languages would be a >> good idea. >> I went with Python because I'm familiar with it and have already written >> bindings for it with SWIG. Also, buildbot is written in python and it could >> provide a really interesting integration I think. >> >> I would guide/support you in refactoring cmake as needed. The refactoring >>> part would definitely be upstreamable. I would very much like to see a >>> proof >>> of concept alternative language even if that wasn't upstreamed. It would >>> prove that another language is possible, and that's one of the steps to >>> replacing the current cmake language I think. >> >> >> I will need to work on it to make it work again with master, but I'll try >> and do this soon. >> >> Here is what my test POC looked like for generating a simple shared >> library: >> >> #!/usr/bin/env python >>> # -*- coding: utf-8 -*- >>> import cmake >>> cmake.init("Ninja", "/media/dev/src/cmaketest", >>> "/media/dev/build/cmaketest") >>> myProject = cmake.Project("MyTestProject") >>> myProject.targets = [ cmake.SharedLibrary("testLibrary", ["lib.cxx"]) ] >>> cmake.generate() >> >> >> Thanks a lot for your comments, I'm happy to see this was not just a dumb >> idea, and that others have thought about it. >> >> I'll update with the github as soon as I get it working. >> >> Best >> >> >> Le lun. 4 janv. 2016 à 01:16, Stephen Kelly a >> écrit : >> >>> Charles Huet wrote: >>> >>> > * cmCacheManager::AddCacheEntry was made public, as cmake::Configure >>> > cannot be used from python (it check for the existence of a >>> CMakeLists.txt >>> > file, which does not exist in this scenario) and the cache variables it >>> > sets seem to be necessary. >>> >>> Because of the above, I worry that you are basing your work on an old >>> version of CMake. As of April 2015 cmState is used as the interface to >>> the >>> cache. >>> >>> https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a6b1ad13 >>> >>> Also note that the C++ interfaces in CMake are not stable. In the last >>> year >>> they have been changed utterly and that will continue to happen without >>> notice. I'm sure you know this, but I thought I'd say it anyway. >>> >>> > I will try to make a layer of abstraction
Re: [cmake-developers] Using CMake as a library from Python
Charles Huet wrote: > * cmCacheManager::AddCacheEntry was made public, as cmake::Configure > cannot be used from python (it check for the existence of a CMakeLists.txt > file, which does not exist in this scenario) and the cache variables it > sets seem to be necessary. Because of the above, I worry that you are basing your work on an old version of CMake. As of April 2015 cmState is used as the interface to the cache. https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a6b1ad13 Also note that the C++ interfaces in CMake are not stable. In the last year they have been changed utterly and that will continue to happen without notice. I'm sure you know this, but I thought I'd say it anyway. > I will try to make a layer of abstraction on top of the python bindings > before publishing this work on github, in order to show the syntactic > sugar python can provide, but I will publish this before if anybody is > interested in working on this. I would be interested in seeing it. > Now, does anyone beside me think this is a good idea ? I did something similar some years ago with QML instead of python, so it sounds interesting to me. In fact, one of the reasons for introducing cmState and doing all the refactoring I did in cmake was to make it possible to cleanly replace the language, some day. My guess is that you are using python to instantiate a cmAddLibraryCommand and then executing it. I think a better approach would be to leave all the cm*Command classes behind as they carry a lot of backward compatibility baggage and policies which a new language shouldn't be burdened with. Even much of cmMakefile shouldn't be used by a new language. Instead certain parts of cmMakefile should be extracted as other classes (cmVariableExpander which should have the ExpandVariablesInString and ConfigureString stuff, cmMessenger for the IssueMessage stuff, some other class for the CompileFeature stuff etc). Then cmMakefile would be just about executing and scoping the CMake language. A new language would not need that, but would use the refactored extracted classes. So, you would implement new cmPython*Command or whatever which operate on those classes and cmState. cmState is designed to be a language-agnostic store of data, like a database of buildsystem state. As I said though, this would require further refactoring of the cmake code. I can provide guidance on how to do that if you are interested in pursuing that route. It would take some months I think, but be very valuable for many reasons and long-term cmake features. Another example of the kind of refactoring I mean is putting target state in cmState and making it accessible through cmState::Target, similar to how cmState::Directory currently works. Having said all that, Brad favors Lua I believe, and he favors a different approach (which no one is working on as far as I know) to adding a new language. So wait to hear from him to know whether it is something that would be upstreamable. I would prefer an approach similar to what I described above, which is close to what you are doing, so as a proof of concept I would like to see your work. I would guide/support you in refactoring cmake as needed. The refactoring part would definitely be upstreamable. I would very much like to see a proof of concept alternative language even if that wasn't upstreamed. It would prove that another language is possible, and that's one of the steps to replacing the current cmake language I think. Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Using CMake as a library from Python
Hi, Because of the above, I worry that you are basing your work on an old > version of CMake. As of April 2015 cmState is used as the interface to the > cache. Yeah, that is sadly right. I wanted to rebase on master before publishing, thinking it should not be too hard, and I have lots to re-do, mostly understand how things are done now. I did something similar some years ago with QML instead of python, so it > sounds interesting to me. I'm trying to be as declarative as possible, because really like how readable simple QML programs are, and I think it would be perfect for a buildsystem. My guess is that you are using python to instantiate a cmAddLibraryCommand > and then executing it. Actually, I'm directly using the cmMakefile, because I did not want to wrap all the commands, and it seemed backwards to me to wrap them. Even much of cmMakefile shouldn't be used by a new language. Instead certain > parts of cmMakefile should be extracted as other classes > (cmVariableExpander > which should have the ExpandVariablesInString and ConfigureString stuff, > cmMessenger for the IssueMessage stuff, some other class for the > CompileFeature stuff etc). Then cmMakefile would be just about executing > and > scoping the CMake language. A new language would not need that, but would > use the refactored extracted classes. Ah, this is very interesting, thanks. Having said all that, Brad favors Lua I believe, and he favors a different > approach (which no one is working on as far as I know) to adding a new > language. So wait to hear from him to know whether it is something that > would be upstreamable. Have any details on the approach in question ? SWIG would allow for Lua bindings as easily, but I don't think having multiple languages would be a good idea. I went with Python because I'm familiar with it and have already written bindings for it with SWIG. Also, buildbot is written in python and it could provide a really interesting integration I think. I would guide/support you in refactoring cmake as needed. The refactoring > part would definitely be upstreamable. I would very much like to see a > proof > of concept alternative language even if that wasn't upstreamed. It would > prove that another language is possible, and that's one of the steps to > replacing the current cmake language I think. I will need to work on it to make it work again with master, but I'll try and do this soon. Here is what my test POC looked like for generating a simple shared library: #!/usr/bin/env python > # -*- coding: utf-8 -*- > import cmake > cmake.init("Ninja", "/media/dev/src/cmaketest", > "/media/dev/build/cmaketest") > myProject = cmake.Project("MyTestProject") > myProject.targets = [ cmake.SharedLibrary("testLibrary", ["lib.cxx"]) ] > cmake.generate() Thanks a lot for your comments, I'm happy to see this was not just a dumb idea, and that others have thought about it. I'll update with the github as soon as I get it working. Best Le lun. 4 janv. 2016 à 01:16, Stephen Kellya écrit : > Charles Huet wrote: > > > * cmCacheManager::AddCacheEntry was made public, as cmake::Configure > > cannot be used from python (it check for the existence of a > CMakeLists.txt > > file, which does not exist in this scenario) and the cache variables it > > sets seem to be necessary. > > Because of the above, I worry that you are basing your work on an old > version of CMake. As of April 2015 cmState is used as the interface to the > cache. > > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a6b1ad13 > > Also note that the C++ interfaces in CMake are not stable. In the last year > they have been changed utterly and that will continue to happen without > notice. I'm sure you know this, but I thought I'd say it anyway. > > > I will try to make a layer of abstraction on top of the python bindings > > before publishing this work on github, in order to show the syntactic > > sugar python can provide, but I will publish this before if anybody is > > interested in working on this. > > I would be interested in seeing it. > > > Now, does anyone beside me think this is a good idea ? > > I did something similar some years ago with QML instead of python, so it > sounds interesting to me. > > In fact, one of the reasons for introducing cmState and doing all the > refactoring I did in cmake was to make it possible to cleanly replace the > language, some day. > > My guess is that you are using python to instantiate a cmAddLibraryCommand > and then executing it. I think a better approach would be to leave all the > cm*Command classes behind as they carry a lot of backward compatibility > baggage and policies which a new language shouldn't be burdened with. > > Even much of cmMakefile shouldn't be used by a new language. Instead > certain > parts of cmMakefile should be extracted as other classes > (cmVariableExpander > which should have the ExpandVariablesInString and ConfigureString stuff, > cmMessenger for the
[cmake-developers] Using CMake as a library from Python
Hi, I'd like to have a discussion about using CMake as a library from python, and by that I mean not writing CMakeLists.txt files, but python code that then uses the CMake generators and Find*.cmake in order to leverage both the power of CMake and Python. First, a bit of background on the idea itself. I've used CMake since 2010 in different projects at work, and I love it. I'm the CMake guy at work, because most of the colleagues cannot be bothered with it. In fact I highly prefer doing the CMake maintenance myself, as most of the colleagues do not make any effort in trying to apply best practices of to factorize the code (which they do with the C++ code). When talking about CMake, the first reaction most of them have is about the language itself, because they don't know it, don't like some of the idioms, and have trouble accomplishing what they want with it. Nobody argues that CMake is awesome at generating either Visual Studio solutions or Unix Makefiles or whatever, but the language is a high barrier of entry. I have to admit that doing seemingly simple stuff can be a bit confusing in cmake, e.g. checking if the value of a variable is greater than another. Thus I thought that using a language that is well-known and has a closer syntax to what we are used to (not to mention we use python quite a bit for custom scripts) would greatly ease the maintenance of our buildsystem. This would allow for a few things that are not straightforward in cmake, such as functions that return a value. I prototyped around this idea these last weeks (a few minutes or hours at a time) and got it in a proof of concept state, where I cant do a add_library in python. I think this would be a great step forward for writing buildsystems, as lots of tasks would be greatly simplified. From the top of my mind: * reading files * writing files * downloading files * copying files (CMake's copy is not POSIX-compliant, as I learned recently when copying symbolic links) Also, the syntax could be made Object-Oriented very easily, as the python bindings will need a layer of abstraction. Now, for the technical realisation My prototype required a few modifications into CMake itself: * I added cmSystemTools::SetCMakeRoot, as the system root cannot be determined automatically using the current methods * cmLinkItem's typedef of std::string was made public because swig has trouble with it. There may be a better way to solve the problem, but this was the quickest for a POC * cmCacheManager::AddCacheEntry was made public, as cmake::Configure cannot be used from python (it check for the existence of a CMakeLists.txt file, which does not exist in this scenario) and the cache variables it sets seem to be necessary. A cmake.i swig interface file, and a python file that does the initialization of CMake, and I have a simple library compiled. I will try to make a layer of abstraction on top of the python bindings before publishing this work on github, in order to show the syntactic sugar python can provide, but I will publish this before if anybody is interested in working on this. Now, does anyone beside me think this is a good idea ? Cheers -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers