Hi Marc-Antoine, On Apr 18, 2010, at 10:47 AM, Marc-Antoine Ruel wrote:
> 2010/4/17 Kevin Ollivier <kev...@theolliviers.com> > Hi Marc-Antoine, > > On Apr 17, 2010, at 11:26 AM, Marc-Antoine Ruel wrote: > >> 2010/4/17 Kevin Ollivier <kev...@theolliviers.com> >> Hi Maciej, >> >> On Apr 16, 2010, at 3:34 PM, Maciej Stachowiak wrote: >> >> > >> > On Apr 16, 2010, at 8:09 AM, Nikolas Zimmermann wrote: >> > >> >> >> >> Am 16.04.2010 um 16:44 schrieb Adam Treat: >> >> >> >>> I am very skeptical that it is feasible to write a gyp generator that >> >>> would >> >>> output QMake files. There is a log of magic in those QMake files. My >> >>> sense is >> >>> that it would not be trivial by any means. >> >>> >> >>> Plus, I don't like the idea of a meta-meta generators. Seems way to >> >>> mickey- >> >>> mouse to me. >> >> >> >> Agreed to a certain degree. Using gyp/whatever to generate qmake files, >> >> to generate Makefile/Xcode files etc seems akward to me as well. >> >> >> >> What we really need to resolve is adding/removing files from compilation, >> >> that's the most common >> >> task that has to be done in 5+ build systems at the moment. >> > >> > Besides adding, removing and renaming, the other thing that's really hard >> > is adding a new generated source rule. Although this is not needed as >> > frequently, I think anyone adding a new code generator script that has to >> > work across all WebKit ports would have a hellish time of it right now. >> > >> > If I had to do it myself, I would just skip any ports that don't use >> > DerivedSources.make. >> > >> > >> >> So I have a new proposal: >> >> >> >> 1) Maintain a list of headers/source files to be compiled for ALL >> >> platforms (ie. dom/Node.cpp, etc..) >> >> >> >> 2) Keep all existing Makefile.am, WebCore.pro etc files as "templates", >> >> ie. WebCore.pro.template, with a special >> >> variable somewhere marking the $$HEADER_LIST$$ and the $$SOURCE_LIST$$ >> >> >> >> 3) Use a script that generates individual build files (eg. WebCore.pro) >> >> from WebCore.pro.template, it only >> >> needs to insert the file list with the correct syntax in the correct >> >> places >> >> >> >> 4) Keep all platform specific files to be compiled in the individual >> >> build system files (eg. WebCore.pro.template) >> >> >> >> I think we'll never find a consensus on a single build system, there are >> >> too many different needs. I only care >> >> about the most repetitive work in order to keep the build system up2date: >> >> adding/removing cross-platform files. >> > >> > I think the proposal above does not handle the derived sources problem. It >> > also doesn't handle files that are shared between multiple ports but not >> > all ports. It also doesn't provide project files that are directly usable >> > by IDEs, on platforms where that is the standard way to do development. >> >> Converting, say, a JSON list of files to some XML-based output format would >> not be difficult at all (and I >> >> Like this? >> http://trac.webkit.org/browser/trunk/WebCore/WebCore.gypi >> >> - It is currently not JSON compliant. It's in fact a python file but that >> can be fixed: s/'/"/g and removing the extra commas should be sufficient. >> - It is currently chromium specific, that's what I meant by >> "de-chromification" of the gyp files. It's mainly adding more stuff and >> fixing the regexp if I'm not mistaken. I don't mind if it doesn't become the >> golden file, the goal is just to hopefully reduce the number of build >> system, nothing more. > > Yes, precisely why I mentioned JSON and later gyp, though I don't know if > Chromium-specific is the right word here. It even has wx port files in it, > which I don't think are built by Chromium. :) I suppose you somehow filter > out which ones Chromium should build after the fact? If so, where does that > logic reside? > > mac, qt, gtk, wx, symbian and even some JSC files are listed. I'm 100% sure > the lists aren't complete and I'll need to fix them along the way, especially > JSC stuff which is a show stopper. > > .gypi files are header files. They can be imported by other .gypi or a .gyp > file. Each .gyp file will generate a .sln, .xcodeproj, Makefile or > SConstruct, depending on the selected build system. Each "target" inside a > .gyp will generate a .vcproj or .scons. > > The regex logic lives in inside "sources" entries inside a target. You can > have a list of file names, 'include' or 'exclude' with an array of regexp. > Please take a look at the very bottom of this file for an example: > http://trac.webkit.org/browser/trunk/WebCore/WebCore.gyp/WebCore.gyp May I ask why it is being done this way, that is, having one really large list of build files and using include / exclude regexs to trim the list? i.e. why is there not a WebCoreCommon.gypi, WebCoreWinCommon.gypi, WebCoreChromium.gypi, etc.? Are you guys using scripts to create WebCore.gypi? > The main gyp caveat is some of its shortcut notation like /, ! and +++. > Please see the documentation for more info if interested. > > WebCore.gyp already brings a lot of conditions which can make the whole thing > messy as the number of platforms increases but these can be moved off > individual .gypi files to keep the whole thing bearable. In the end, it may > even not make sense to share the gyp files accross platform if they don't > share enough build logic, it's no big deal as long as people can add and > remove files easily. > > BTW, I don't why WebCore.gypi lists the idl, svg idl, source files, > webinspector and images as individual lists since they all have clearly > simple regexp to split them off. It still makes sense as a logical separation > though, I just want to raise the point that it could be entirely doable to > just have a one whole file list. > > > Anyway, the fact that this file is actually Python (I had forgotten the > format was JSON-like rather than straight JSON) makes things even better, as > we only really need to handle export now, and not parse some import file > list. i.e. for WebKitTools/Scripts/update-sources-list.py, > getWebCoreFilesDict() basically becomes a very small script that execfile's > the gypi file, then we run whatever filtering mechanism on it (waf has a list > of ports we could use to do the filtering that I could probably move into > WebKitTools/Scripts, if we don't have a pre-made Chromium solution here), and > then passes the final source list along to generateWebCoreSourcesGTKandQT to > generate the sources / includes for those platforms, and actually this > solution should work for Android.mk and possibly jam too, as their syntax > looks largely similar. Then we'd add some XML parsing code to grab the node > for common file groups and update them for the MSVC projects, and then I > think that mostly leaves XCode, which I think would be pretty similar in > nature. > > For MSVC and XCode, it makes more sense to use the native gyp support since > it makes really clean projects. FYI, VS2010 is not supported for now. Yes, if all MSVC and XCode projects are planning to switch over anyway, it's probably not worth bothering with. Though if some choose not to, we could actually re-use the chunk of Gyp code that creates the file lists and just inject that into the project file. > Beside that point, I agree with the rest. As you said, for other platforms, > "whatever is preferred by the committers" which may mean "don't use gyp at > all" or "use WebCore.gypi with custom scripts, not gyp". > > > As long as people are willing to test out this solution with their build > systems and help with any debugging, I would be willing to help out in > hacking it together, though I can't promise anything in the way of time since > this is not an immediate concern for me personally. > > Thanks, I'll ping you later as questions show up. Sure. :) Thanks, Kevin > --- > > Since this email sounds like "gyp solve world hunger", here's a short irc log: > > 13:08 < maruel> manyoso: if you want to try out cmake first before I dive > into gyp stuff, feel free as I don't plan to do it in the next two weeks > 13:18 < manyoso> maruel: ok, i'm waiting on othermaciej to tell me whether > apple has cmake installed in build farm > 13:18 < manyoso> maruel: is potentially a show stopper apparently > 13:20 < maruel> manyoso: to be clear, I really don't mind whatever is used, > as long as it can improve the productivity of my team (and by > extension the whole community) > 13:20 < manyoso> maruel: understood. and fwiw I am of the same mindset > 13:21 < manyoso> maruel: if cmake turns out to have real showstoppers i am > considering helping with gyp port of apple's windows port > 13:21 < manyoso> as i feel that build system proliferation is a real issue > for the whole project > 13:22 < manyoso> as well as proliferation and ongoing existence of platform > specific tests > 13:22 < manyoso> as the project continues to scale those two are going to > bite us in the ass more and more > 13:23 < maruel> yep > 13:42 < manyoso> we have ~16000 platform specific expected.txt files in > LayoutTests right now > > > -- > M-A
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
