Re: Moving Away from Makefile's
Gregory Szorc wrote: We want feedback on the file format to use for the new build manifest files. Is there a good reason to shoehorn everything into a single file format? -- Warning: May contain traces of nuts. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Gecko switched to use standard integer types
Makoto Kato wrote: How about PRUptrdiff? Most cases can changes to uintptr_t. Which surprised me, but then I guess NSPR didn't have a true uintptr_t type. I did find one use of PRUptrdiff that should have been size_t though. -- Warning: May contain traces of nuts. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: New XPIDL attribute: [infallible]
Justin Lebar wrote: So now you can do nsCOMPtr foo; int32_t f = foo->GetFoo(); Why was I expecting this to be Foo()? (Perhaps unreasonably.) I rejected the first approach because it meant that every call to GetFoo from XPCOM would need to go through two virtual calls: GetFoo(int32_t*) and then GetFoo(). And also because MSVC would have messed up the vtable. -- Warning: May contain traces of nuts. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: New XPIDL attribute: [infallible]
Justin Lebar wrote:
%{C++
inline int32_t GetFoo() {
int32_t result;
nsresult rv = GetFoo(&result);
MOZ_ASSERT(NS_SUCCEEDED(rv));
return result;
}
%}
Alternative approach?
inline int32_t Foo(int32_t result = 0) {
GetFoo(&result);
return result;
}
--
Warning: May contain traces of nuts.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: New XPIDL attribute: [infallible]
On 08/24/2012 02:42 AM, Neil wrote: Justin Lebar wrote: So now you can do nsCOMPtr foo; int32_t f = foo->GetFoo(); Why was I expecting this to be Foo()? (Perhaps unreasonably.) Yeah, it should be Foo(). File a bug? I rejected the first approach because it meant that every call to GetFoo from XPCOM would need to go through two virtual calls: GetFoo(int32_t*) and then GetFoo(). And also because MSVC would have messed up the vtable. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Moving Away from Makefile's
On Tuesday, August 21, 2012 7:36:26 PM UTC-4, Gregory Szorc wrote: > tl;dr We're proposing moving away from Makefile's as the sole source of > > the build system definition. This will lead to faster build times. > > Bikeshedding^wFeedback on the file format is requested. > > > > The existing build system is defined by Makefile.in's scattered around > > the source tree (typically one Makefile.in per directory). At configure > > time, these Makefile.in's get preprocessed into Makefile's using simple > > variable substitution. Then make/pymake is let loose on the result. It > > is a very traditional model. > > > > We are attempting to move to a model where the build definition is > > generic and data-driven. By treating the build definition as data > > (rather than a glorified shell script that is Makefiles), this will > > allow us to take that data and convert it into formats understood by > > other, better/faster build backends, such as non-recursive make files, > > Tup, Ninja, or even Visual Studio. > > > > Up until now, the focus has been on making Makefile.in's themselves > > generic and data-driven [1]. We would use pymake's API to parse, load, > > and extract data from Makefile.in's to construct the build definition. > > In the long run, we'd realize that using make files for data definition > > was silly (and a large foot gun) and thus we would switch to something else. > > > > After a long IRC conversation, Mike Hommey and I concluded that we want > > to begin the transition away from Makefile.in's ASAP. > > > > Essentially, the proposal is to move (not duplicate) some data from > > Makefile.in's into new files. Initially, this would include things like > > subdirectories to descend into and files to copy/preprocess. Simple > > stuff to start with. Eventually, scope would likely increase to cover > > the entirety of the build system definition (like compiling), rendering > > Makefile.in's obsolete. But, it will take a *long* time before we get there. > > > > In the new world, the source of truth for the build system is jointly > > defined by existing Makefile.in's and whatever these new files are that > > we create. I'll call these not-yet-existing files "build manifest > > files." Somewhere in the build process we read in the build manifest > > files and generate output for the build backend of choice. > > > > Our existing non-recursive make backend should integrate with this > > seamlessly. Instead of a dumb variable substitution phase for > > configuring the build backend, we'll have some additional logic to write > > out new make files derived from the contents of the build manifest > > files. This is similar to the approach I've taken in build splendid [2]. > > The only difference is the build definition is living in somewhere not > > Makefile.in's. > > > > We don't have details on how exactly the migration will be carried > > about. But, it should be seamless. So, unless you touch the build > > system, you should be able to continue living in blissful ignorance. > > > > If you have any concerns over this transition, please voice them. > > > > File Format > > === > > > > I hinted at bikeshedding in the tl;dr. We want feedback on the file > > format to use for the new build manifest files. The requirements are as > > follows (feel free to push back on these): > > > > 1. Easy for humans to grok and edit. An existing and well-known format > > is preferred. We don't want a steep learning curve here. > > 2. Simple for computers to parse. We will use Python to load the build > > manifest files. Python can do just about anything, so I'm not too > > worried here. > > 3. Efficient for computers to load. As these files need to be consulted > > to perform builds, we want to minimize the overhead for reading them > > into (Python) data structures. > > 4. Native support for list and maps. Make files only support strings. > > The hacks this results in are barely tolerable. > > 5. Ability to handle conditionals. We need to be able to conditionally > > define things based on the presence or value of certain "variables." > > e.g. "if the current OS is Linux, append this value to this list." I > > quote "variables" because there may not be a full-blown variable system > > here, just magic values that come from elsewhere and are addressed by > > some convention. > > 6. Ability to perform ancillary functionality, such as basic string > > transforms. I'm not sure exactly what would be needed here. Looking at > > make's built-in functions might be a good place to start. We may be able > > to work around this by shifting functionality to side-effects from > > specially named variables, function calls, etc. I really don't know. > > 7. Evaluation must be free from unknown side-effects. If there are > > unknown side-effects from evaluation, this could introduce race >
Re: Moving Away from Makefile's
On Fri, Aug 24, 2012 at 9:17 AM, qheaden wrote: > Is there any special reason why an existing build system such as SCcons > couldn't be used as a new build system for Mozilla? I know the Mozilla source > has a lot of special build instructions, but SCons does allow you to create > your own special builders in Python code. Build systems like SCons are just a different coat of paint over make. They wouldn't really solve any of our problems, it'd just be busy-work. In addition, SCons (among other build systems) tries to solve more problems than we need, by providing the features of autoconf as well as make. Finally, for SCons in particular, I have doubts about its ability to scale to a project of Mozilla's size. KDE tried to switch to SCons and failed, and wound up using CMake. In short, most build systems suck at large scale. Almost any will suffice for a small project, but for a project of Mozilla's size there's no perfect solution. -Ted ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Moving Away from Makefile's
On Friday, August 24, 2012 10:32:46 AM UTC-4, Ted Mielczarek wrote: > On Fri, Aug 24, 2012 at 9:17 AM, qheaden > wrote: > > > Is there any special reason why an existing build system such as SCcons > > couldn't be used as a new build system for Mozilla? I know the Mozilla > > source has a lot of special build instructions, but SCons does allow you to > > create your own special builders in Python code. > > > > Build systems like SCons are just a different coat of paint over make. > > They wouldn't really solve any of our problems, it'd just be > > busy-work. In addition, SCons (among other build systems) tries to > > solve more problems than we need, by providing the features of > > autoconf as well as make. Finally, for SCons in particular, I have > > doubts about its ability to scale to a project of Mozilla's size. KDE > > tried to switch to SCons and failed, and wound up using CMake. > > > > In short, most build systems suck at large scale. Almost any will > > suffice for a small project, but for a project of Mozilla's size > > there's no perfect solution. > > > > -Ted Yes, I guess you are right about that. A custom, simplified build system would be the best choice in this situation. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: New XPIDL attribute: [infallible]
>>> nsCOMPtr foo; >>> int32_t f = foo->GetFoo(); >>> >> Why was I expecting this to be Foo()? (Perhaps unreasonably.) > > Yeah, it should be Foo(). > File a bug? I considered Foo(), but my concern was that, when we extend this to attributes which return interfaces (e.g. nsIFoo), then Foo() versus GetFoo() has a particular meaning (in parts of our code): Foo() must never return null. But I was expecting that an infallible attribute could correctly return null. We can tweak this: attributes which return primitives are always Foo(), and attributes which return interfaces are GetFoo() unless we also have [nevernull] on the attribute. I'm not sure if the complexity there is worth the cost. >>> I rejected the first approach because it meant that every call to GetFoo >>> from XPCOM would need to go through two virtual calls: GetFoo(int32_t*) and >>> then GetFoo(). >>> >> And also because MSVC would have messed up the vtable. >> > > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: New XPIDL attribute: [infallible]
On Fri, Aug 24, 2012 at 2:44 AM, Neil wrote:
> Justin Lebar wrote:
>
>> %{C++
>> inline int32_t GetFoo() {
>>int32_t result;
>>nsresult rv = GetFoo(&result);
>>MOZ_ASSERT(NS_SUCCEEDED(rv));
>>return result;
>> }
>> %}
>>
>
> Alternative approach?
> inline int32_t Foo(int32_t result = 0) {
>GetFoo(&result);
>return result;
>
> }
I think it would be better to bake the default params into the IDL, if
we wanted that. 0 isn't always what you want.
But I also think that [infallible] should mean "always gives you a
result," so you shouldn't have to worry about what the default is.
I'd be OK doing
> inline int32_t GetFoo() {
>int32_t result = 0;
>nsresult rv = GetFoo(&result);
>MOZ_ASSERT(NS_SUCCEEDED(rv));
>return result;
> }
so that we never return uninitialized memory in release builds. 0 may
or may not be right, but it should be no worse than a random value!
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Moving Away from Makefile's
Gregory Szorc wrote:
> 4. Native support for list and maps. Make files only support strings.
> The hacks this results in are barely tolerable.
>
> 5. Ability to handle conditionals. We need to be able to
> conditionally define things based on the presence or value of certain
> "variables."
> e.g. "if the current OS is Linux, append this value to this list." I
> quote "variables" because there may not be a full-blown variable
> system here, just magic values that come from elsewhere and are
> addressed by some convention.
>
> 6. Ability to perform ancillary functionality, such as basic string
> transforms. I'm not sure exactly what would be needed here. Looking
> at make's built-in functions might be a good place to start. We may
> be able to work around this by shifting functionality to side-effects
> from specially named variables, function calls, etc. I really don't
> know.
>
> 7. Evaluation must be free from unknown side-effects. If there are
> unknown side-effects from evaluation, this could introduce race
> conditions, order dependency, etc. We don't want that. Evaluation
> must either be sandboxed to ensure nothing can happen or must be able
> to be statically analyzed by computers to ensure it doesn't do anything
> it isn't supposed to.
...
> On the other end of the spectrum, we could have the build manifest
> files be Python "scripts." This solves a lot of problems around
> needing functionality in the manifest files. But, it would be a
> potential foot gun. See requirement #7.
I do not think it is reasonable to require support for alternate build systems
for all of Gecko/Firefox.
But, let's say were were to divide the build into three phases:
1. Generate any generated C/C++ source files.
2. Build all the C/C++ code into libraries and executables
3. Do everything else (build omnijar, etc.)
(I imagine phase 3 could probably run 100% concurrently with the first two
phases).
It would be very nice if phase #2 ONLY could support msbuild (building with
Visual Studio project files, basically), because this would allow smart
editors'/IDEs' code completion and code navigation features to work very well,
at least for the C/C++ source code. I think this would also greatly simplify
the deployment of any static analysis tools that we would develop.
In addition, potentially it would allow Visual Studio's "Edit and Continue"
feature to work. ("Edit and Continue" is a feature that allows you to make
changes to the C++ source code and relink those changes into a running
executable while execution is paused at a breakpoint, without restarting the
executable.)
I think that if you look at the limitations of gyp, some (all?) of them are at
least partially driven by the desire to provide such support. I am sure the
advanced features that you list in (4), (5), (6), (7) are helpful, but they may
make it difficult to support these secondary use cases.
That said, getting the build system to build as fast as it can is much more
important.
Cheers,
Brian
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Moving Away from Makefile's
On 8/24/12 12:42 PM, Brian Smith wrote:
I do not think it is reasonable to require support for alternate build systems
for all of Gecko/Firefox.
But, let's say were were to divide the build into three phases:
1. Generate any generated C/C++ source files.
2. Build all the C/C++ code into libraries and executables
3. Do everything else (build omnijar, etc.)
(I imagine phase 3 could probably run 100% concurrently with the first two
phases).
It would be very nice if phase #2 ONLY could support msbuild (building with
Visual Studio project files, basically), because this would allow smart
editors'/IDEs' code completion and code navigation features to work very well,
at least for the C/C++ source code. I think this would also greatly simplify
the deployment of any static analysis tools that we would develop.
In addition, potentially it would allow Visual Studio's "Edit and Continue" feature to
work. ("Edit and Continue" is a feature that allows you to make changes to the C++ source
code and relink those changes into a running executable while execution is paused at a breakpoint,
without restarting the executable.)
I think that if you look at the limitations of gyp, some (all?) of them are at
least partially driven by the desire to provide such support. I am sure the
advanced features that you list in (4), (5), (6), (7) are helpful, but they may
make it difficult to support these secondary use cases.
That said, getting the build system to build as fast as it can is much more
important.
Agreed.
Changing how we define the build config would enable us to do everything
you mentioned and more. With the current "architecture" of our
Makefile's, we effectively have different build phases called tiers. See
[1] for more. As much as I would love to split things up into more
distinct phases/tiers, the overhead for recursive make traversal would
be prohibitive. As far as prioritizing work to enable basic Visual
Studio project generation, I'm all for that: my build-splendid branch
[2] had its roots in VS generation after all (if you go back far enough
in the history you can still see this)!
Once we treat the build system as a giant data structure, we are free to
transform that any way we want. We feed that data structure into a
generator and spit something out the other side. This is very similar to
GYP's model. Essentially what we are proposing is reinventing GYP, but
with a different frontend. It's entirely possible we will implement
things using GYP's APIs so we can reuse GYP's existing generators! Time
will tell.
[1] http://gregoryszorc.com/blog/2012/07/29/mozilla-build-system-overview/
[2] https://github.com/indygreg/mozilla-central/tree/build-splendid
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Proposed policy change: reusability of tests by other browsers
Aryeh Gregor wrote: > 1) Decide on guidelines for whether a test is internal or reusable. > As a starting point, I suggest that all tests that are regular > webpages that don't use any Mozilla-specific features should be > candidates for reuse. Examples of internal tests would be tests > written in XUL and unit tests. In particular, I think we should > write > tests for reuse if they cover anything that other browsers implement > or might implement, even if there's currently no standard for it. > Other browsers should still be able to run these tests, even if they > might decide not to follow them. Also, tests that currently use > prefixed web-exposed properties should still be made reusable, since > the properties should eventually be unprefixed. Which other browser makers are going to follow these guidelines, so that we benefit from them? Generally, this is a great idea if it makes it faster and easier to improve Firefox. But, like Asa, I also interpreted this proposal along the lines of "Spend resources, and slow down Firefox development, to help other browsers." That seems totally in line with our values, but doesn't seem great as far as competitiveness is concerned. Also, are you saying "if you are going to write a mochitest, then try to write a reusable test" or "if you are going to write a test, write a reusable test?" The reason I ask is that we're supposed to write xpcshell tests in preference to mochitests when possible, and I'd hate the preference to change to be in favor of mochitests, because xpcshell tests are much more convenient (and faster) to write and run. Thanks, Brian ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposed policy change: reusability of tests by other browsers
On Fri, Aug 24, 2012 at 2:08 PM, Brian Smith wrote: > Also, are you saying "if you are going to write a mochitest, then try to > write a reusable test" or "if you are going to write a test, write a > reusable test?" The reason I ask is that we're supposed to write xpcshell > tests in preference to mochitests when possible, and I'd hate the > preference to change to be in favor of mochitests, because xpcshell tests > are much more convenient (and faster) to write and run. > Most things that this is relevant to (things visible to web content) can't be tested from xpcshell anyways, so this shouldn't affect xpcshell vs. mochitests. - Kyle ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: STR Needed Keyword?
For those interested, I've gone ahead and filed a bug to get the keyword added: https://bugzilla.mozilla.org/show_bug.cgi?id=785519 Thanks to everyone who provided feedback. On 12-08-17 10:30 AM, Ralph Giles wrote: On 12-08-16 6:01 PM, Anthony Hughes wrote: (CCing dev-quality to reach a broader audience -- please direct responses to dev-platform) It has come to my attention that we lack a keyword in Bugzilla for when steps-to-reproduce are needed (a very common request). However, we do have keywords for when a testcase, regression range, or URLs are wanted. I find it to be extremely useful when someone requesting qawanted pairs it with a keyword indicating what is being requested. It's certainly more efficient then having to parse the comments to interpret the request. Assuming support for such a keyword here are some proposed names: * steps-wanted * str-wanted * needSTR * need-steps Would people find this keyword useful? If so, I can file a bug to get it added. I think this would be useful, and improves workflow clarity along the lines of dbaron's blog post yesterday. * steps-wanted This is the most obvious of the options you gave. -r -- Anthony Hughes Quality Engineer Mozilla QA (Desktop) ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
