[gwt-contrib] Re: RFC: sharded linking

2010-02-10 Thread Alex Moffat
I've replied before but don't see it here, if it turns up ignore this dupe. I don't maintain any linkers but I have experimented with multi- machine builds. The current Precompile, CompilePerms, and Link implementation has the nice feature that the CompilePerms step does not require access to the

[gwt-contrib] Re: RFC: sharded linking

2010-02-10 Thread Alex Moffat
I may have misunderstood the proposal but I've experimented a little with multi machine builds so I'll comment based on that. One very nice feature of the current system is that the CompilePerms step does not need access to the source code being compiled. This is a significant benefit as it makes

[gwt-contrib] Re: RFC: sharded linking

2010-02-11 Thread Brendan Kenny
On Feb 11, 6:43 pm, Scott Blum wrote: > I have a few comments, but first I wanted to raise the point that I'm not > sure why we're having this argument about maximally sharded Precompiles at > all.  For one thing, it's already implemented, and optional, via > "-XshardPrecompile".  I can't think of

[gwt-contrib] Re: RFC: sharded linking

2010-02-12 Thread Alex Moffat
Where can I read a description of what -XshardPrecompile, or see the code for it, it sounds very useful to me personally? It's not in 2.0.0 as far as I can see. My concerns about the sharded linking proposal came from what I understood the original flow to be from my looking at it and from the orig

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-10 Thread Lex Spoon
What you describe, Alex, is available via the "Compiler" entry point, though it hasn't been particularly well documented. There is a PermutationWorkerFactory that can create CompilePerms workers. The default worker factory spawns Java VMs on the same machine, but it is possible to write a replace

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-10 Thread John Tamplin
On Wed, Feb 10, 2010 at 10:45 AM, Lex Spoon wrote: > What you describe, Alex, is available via the "Compiler" entry point, > though it hasn't been particularly well documented. There is a > PermutationWorkerFactory that can create CompilePerms workers. The default > worker factory spawns Java V

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-10 Thread James Northrup
there's a fairly large repository based elephant in the room named maven. On Feb 10, 2010, at 7:58 AM, John Tamplin wrote: > On Wed, Feb 10, 2010 at 10:45 AM, Lex Spoon wrote: > What you describe, Alex, is available via the "Compiler" entry point, though > it hasn't been particularly well docum

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-10 Thread John Tamplin
On Wed, Feb 10, 2010 at 11:01 AM, James Northrup wrote: > there's a fairly large repository based elephant in the room named maven. > I'm not sure what that has to do with sharding a compile of a GWT application across a build farm. -- John A. Tamplin Software Engineer (GWT), Google -- http:/

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-10 Thread James Northrup
i'm only chiming in on the 3 letters RFC in the topic. the usecases being described as a point of deliberation, defining dependancies, repository access, and bundling automation, are well solved items in the maven stable. how hard can it be to define a multiproject descriptor, assign "channels

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-11 Thread Lex Spoon
I've posted a patch for review here: http://gwt-code-reviews.appspot.com/141811 Let's make it work well for everyone it impacts. -Lex -- http://groups.google.com/group/Google-Web-Toolkit-Contributors

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-11 Thread Lex Spoon
On Wed, Feb 10, 2010 at 10:58 AM, John Tamplin wrote: > On Wed, Feb 10, 2010 at 10:45 AM, Lex Spoon wrote: >> >> Is copying source code so inconvenient that it would be worth having a >> slower build? I would have thought any of the following would work to move >> source code from one machine t

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-11 Thread John Tamplin
On Thu, Feb 11, 2010 at 5:33 PM, Lex Spoon wrote: > >> Besides, what is gained by having the user have to arrange this copying >> themselves rather than the current method of sending it as part of the >> compile process? For example, distributed C/C++ compilers send the >> preprocessed source to

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-11 Thread Lex Spoon
On Wed, Feb 10, 2010 at 11:25 AM, James Northrup wrote: > the usecases being described as a point of deliberation, defining > dependancies, repository access, and bundling automation, are well solved > items in the maven stable. how hard can it be to define a multiproject > descriptor, assign "ch

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-12 Thread James Northrup
in this comment i also mentioned use of 'synthetic' maven poms. heriein lies the scope and garbage collection features, where by semaphore or lack of semaphore, almost any single process can decorate a repo with artifacts and precursor artifacts using the synthesized repo for the project or bu

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-12 Thread Lex Spoon
On Fri, Feb 12, 2010 at 9:50 AM, Alex Moffat wrote: > Where can I read a description of what -XshardPrecompile, or see the > code for it, it sounds very useful to me personally? -XshardPrecompile is an experiment that everyone wants to change, so it seems unlikely to be released in its current

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-12 Thread Lex Spoon
On Thu, Feb 11, 2010 at 8:58 PM, Brendan Kenny wrote: > > If this is indeed the direction to go in (and I'm a big fan of the > goals as well), it's probably also worth making a more formal > definition for "won't step on each other's toes". As a use case, I'm > working on a PRE linker that (curren

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-12 Thread Lex Spoon
On Thu, Feb 11, 2010 at 5:42 PM, John Tamplin wrote: > That is exactly my point -- the C++ example sends the preprocessed source > to the worker nodes, so they don't have to have the dependencies or the > right include path or whatever. The analogy here would be for GWT to send > all of the coll

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-16 Thread Scott Blum
On Fri, Feb 12, 2010 at 7:00 PM, Lex Spoon wrote: > On Fri, Feb 12, 2010 at 9:50 AM, Alex Moffat wrote: > >> Where can I read a description of what -XshardPrecompile, or see the >> code for it, it sounds very useful to me personally? > > > -XshardPrecompile is an experiment that everyone wants to

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-17 Thread Lex Spoon
On Tue, Feb 16, 2010 at 3:32 PM, Scott Blum wrote: > On Fri, Feb 12, 2010 at 7:00 PM, Lex Spoon wrote: > >> On Fri, Feb 12, 2010 at 9:50 AM, Alex Moffat wrote: >> >>> Where can I read a description of what -XshardPrecompile, or see the >>> code for it, it sounds very useful to me personally? >>

Re: [gwt-contrib] Re: RFC: sharded linking

2010-02-18 Thread Scott Blum
On Wed, Feb 17, 2010 at 6:17 PM, Lex Spoon wrote: > On Tue, Feb 16, 2010 at 3:32 PM, Scott Blum wrote: > >> On Fri, Feb 12, 2010 at 7:00 PM, Lex Spoon wrote: >> >>> On Fri, Feb 12, 2010 at 9:50 AM, Alex Moffat wrote: >>> Where can I read a description of what -XshardPrecompile, or see the