Re: I/O effect on builds [was Moving Away from Makefile's]

2012-09-06 Thread Gregory Szorc
On 9/4/2012 11:09 PM, Nicholas Nethercote wrote: On Tue, Sep 4, 2012 at 9:39 PM, Mike Hommey m...@glandium.org wrote: On the other hand, the ccache miss rate is pretty high, at least it is for me, so in the end, ccache might not be a win at all. That's what I've found, for my compiling

Re: I/O effect on builds [was Moving Away from Makefile's]

2012-09-05 Thread Nicholas Nethercote
On Tue, Sep 4, 2012 at 9:39 PM, Mike Hommey m...@glandium.org wrote: On the other hand, the ccache miss rate is pretty high, at least it is for me, so in the end, ccache might not be a win at all. That's what I've found, for my compiling patterns. Nick

Re: Moving Away from Makefile's

2012-09-04 Thread Gregory Szorc
On 9/3/2012 6:45 PM, qheaden wrote: On Sunday, September 2, 2012 5:15:40 PM UTC-4, Gregory Szorc wrote: A decision has been made: we will be using Python files executing in a sandboxed environment (using the technique that Hanno posted). Supporting this decision are Ted (build system owner)

Re: Moving Away from Makefile's

2012-08-29 Thread Robert Kaiser
Gregory Szorc schrieb: * if is ifneq (,$(foo)) or even ifneq (,$(strip $(foo))) in case some extra whitespace snuck in there. One thing that was hoped we could achieve with pymake was to possibly switch to it everywhere at some time and then be able to replace those ugly conditionals with

I/O effect on builds [was Moving Away from Makefile's]

2012-08-29 Thread Karl Tomlinson
On 8/22/12 6:10 AM, Robert Kaiser wrote: [...] esp. given that basic file I/O is often costly (from watching my CPU usage, a lot of the build time is spent in I/O wait when using spinning disks - SSDs improve that hugely). On Wed, 22 Aug 2012 12:29:24 -0700, Gregory Szorc wrote: Yes, I/O

Re: Moving Away from Makefile's

2012-08-24 Thread qheaden
On Tuesday, August 21, 2012 7:36:26 PM UTC-4, Gregory Szorc wrote: tl;dr We're proposing moving away from Makefile's as the sole source of the build system definition. This will lead to faster build times. Bikeshedding^wFeedback on the file format is requested. The existing build

Re: Moving Away from Makefile's

2012-08-24 Thread qheaden
On Friday, August 24, 2012 10:32:46 AM UTC-4, Ted Mielczarek wrote: On Fri, Aug 24, 2012 at 9:17 AM, qheaden qhea...@phaseshiftsoftware.com wrote: Is there any special reason why an existing build system such as SCcons couldn't be used as a new build system for Mozilla? I know the

Re: Moving Away from Makefile's

2012-08-24 Thread Brian Smith
Gregory Szorc wrote: 4. Native support for list and maps. Make files only support strings. The hacks this results in are barely tolerable. 5. Ability to handle conditionals. We need to be able to conditionally define things based on the presence or value of certain variables. e.g. if the

Re: Moving Away from Makefile's

2012-08-23 Thread Mike Hommey
On Wed, Aug 22, 2012 at 01:38:22PM -0700, Gregory Szorc wrote: On 8/22/12 1:00 PM, Ben Hearsum wrote: On 08/22/12 12:43 PM, Jeff Hammel wrote: If we do go with python, it would be nice to keep the configuration files as much configuration as possible. The reason I question having any full

Re: Moving Away from Makefile's

2012-08-23 Thread Ben Hearsum
On 08/23/12 12:05 AM, Robert O'Callahan wrote: On Thu, Aug 23, 2012 at 8:40 AM, Ben Hearsum bhear...@mozilla.com wrote: On 08/22/12 04:38 PM, Gregory Szorc wrote: Let's think of what can be done to secure/limit Python. Disabling import has already been mentioned. That's a start. I think

Re: Moving Away from Makefile's

2012-08-23 Thread Mike Hommey
On Thu, Aug 23, 2012 at 08:43:31AM -0400, Ben Hearsum wrote: Yeah, I think I agree. My experience in RelEng has biased me strongly towards not allowing even temporary hacks like that, because it's rare that we ever remove them, and end up with a pile of very difficult to maintain hacks

Re: Moving Away from Makefile's

2012-08-22 Thread Neil
Gregory Szorc wrote: Up until now, the focus has been on making Makefile.in's themselves generic and data-driven. We would use pymake's API to parse, load, and extract data from Makefile.in's to construct the build definition. In the long run, we'd realize that using make files for data

Re: Moving Away from Makefile's

2012-08-22 Thread Blair McBride
On 22/08/2012 5:58 p.m., Mike Hommey wrote: +1 for Lua - it seems perfect for this. For #1, I find it far easier to read (and write) than Gyp, when it comes to things like conditionals. For #8, we could just ship the entire runtime in the tree for Tier 1 platforms (its small enough!), then its

Re: Moving Away from Makefile's

2012-08-22 Thread Robert Kaiser
Gregory Szorc schrieb: We could go the route of GYP and shoehorn conditionals into a static document (JSON) [3]. JSON is a good format for data for the most part, but IMHO we *really* want comments in those files, and unfortunately JSON doesn't have those and therefore probably must be

Re: Moving Away from Makefile's

2012-08-22 Thread Mike Hommey
On Thu, Aug 23, 2012 at 01:07:34AM +1200, Blair McBride wrote: On 22/08/2012 5:58 p.m., Mike Hommey wrote: +1 for Lua - it seems perfect for this. For #1, I find it far easier to read (and write) than Gyp, when it comes to things like conditionals. For #8, we could just ship the entire runtime

Re: Moving Away from Makefile's

2012-08-22 Thread Ehsan Akhgari
On 12-08-22 12:43 PM, Jeff Hammel wrote: TL;DR - python if we want/need a full language, JSON if we can get away with POD I think JSON is the wrong choice here. It will not satisfy the need for conditionals, which causes us to invent terrible hacks inside the JSON like we had to do with the

Re: Moving Away from Makefile's

2012-08-22 Thread Justin Wood (Callek)
Jeff Hammel wrote: While a bit of an unfair example, our buildbot-configs fall into this category. IMO not unfair at all. (p.s. to stay on topic, +1 to all else you said) ~Justin Wood (Callek) ___ dev-platform mailing list

Re: Moving Away from Makefile's

2012-08-22 Thread Jeff Hammel
On 08/22/2012 12:29 PM, Gregory Szorc wrote: Switching to something else also has another advantage: the opportunity for a clean slate. I'm hoping we'll use the opportunity to scrape away 10+ years of cruft. I support the sentiment, though this reasoning always makes me cautious, as it

Re: Moving Away from Makefile's

2012-08-22 Thread Ben Hearsum
On 08/22/12 04:38 PM, Gregory Szorc wrote: Let's think of what can be done to secure/limit Python. Disabling import has already been mentioned. That's a start. I think it's worth noting that even if you *do* limit what you can do through some technical means, you still have the option to change

Re: Moving Away from Makefile's

2012-08-22 Thread Ian Bicking
On Wed, Aug 22, 2012 at 3:38 PM, Gregory Szorc g...@mozilla.com wrote: Let's think of what can be done to secure/limit Python. Disabling import has already been mentioned. That's a start. What about the ast module [1]? I /think/ it could be used to ensure parsed code conforms to whatever we

Re: Moving Away from Makefile's

2012-08-22 Thread Robert O'Callahan
On Thu, Aug 23, 2012 at 8:40 AM, Ben Hearsum bhear...@mozilla.com wrote: On 08/22/12 04:38 PM, Gregory Szorc wrote: Let's think of what can be done to secure/limit Python. Disabling import has already been mentioned. That's a start. I think it's worth noting that even if you *do* limit

Moving Away from Makefile's

2012-08-21 Thread Gregory Szorc
tl;dr We're proposing moving away from Makefile's as the sole source of the build system definition. This will lead to faster build times. Bikeshedding^wFeedback on the file format is requested. The existing build system is defined by Makefile.in's scattered around the source tree (typically

Re: Moving Away from Makefile's

2012-08-21 Thread xunxun
于 2012/8/22 7:36, Gregory Szorc 写道: tl;dr We're proposing moving away from Makefile's as the sole source of the build system definition. This will lead to faster build times. Bikeshedding^wFeedback on the file format is requested. The existing build system is defined by Makefile.in's

Re: Moving Away from Makefile's

2012-08-21 Thread Gregory Szorc
On 8/21/2012 6:12 PM, xunxun wrote: 于 2012/8/22 7:36, Gregory Szorc 写道: Up until now, the focus has been on making Makefile.in's themselves generic and data-driven [1]. We would use pymake's API to parse, load, and extract data from Makefile.in's to construct the build definition. In the long

Re: Moving Away from Makefile's

2012-08-21 Thread Blair McBride
On 22/08/2012 11:36 a.m., Gregory Szorc wrote: I think Lua is perfect for this (it was invented to be a configuration language after all). But, I'm not sure it satisfies #1 nor #8. +1 for Lua - it seems perfect for this. For #1, I find it far easier to read (and write) than Gyp, when it comes

Re: Moving Away from Makefile's

2012-08-21 Thread Mike Hommey
On Wed, Aug 22, 2012 at 03:56:33PM +1200, Blair McBride wrote: On 22/08/2012 11:36 a.m., Gregory Szorc wrote: I think Lua is perfect for this (it was invented to be a configuration language after all). But, I'm not sure it satisfies #1 nor #8. +1 for Lua - it seems perfect for this. For #1,