[gwt-contrib] Re: SDM "-XcompilePerFile": initial feedback

[John Stalcup]

On Monday, August 18, 2014 7:14:38 AM UTC-7, Ivan Markov wrote:
>
> Dear GWT team (and John in particular),
>
> Lately I was checking from time to time the progress of the SDM 
> incremental compiler in GWT 2.7.0.
>
> Let me first say - great work so far! 
> This morning our project was - for the first time - compilable with the 
> new incremental SDM code-path, without internal compiler errors or visible 
> malfunctioning.
>
That's great to hear :). We're pretty happy with the recent progress as 
well. 

>
> And this with NO changes to the codebase. Also the fact that I don't have 
> to split our codebase into very many small acyclic modules (not that it is 
> not a good practice anyway) is obviously a HUGE PLUS.
>
> What I am observing in terms of recompilation times is encouraging (25 
> seconds with 2.7 vs. 47 with 2.6.1) but what is even more interesting is 
> that the compilation itself (up until where it says "Compile of 
> permutations succeeded") seems to take only 7 seconds, which is less than 
> 30% of the recompilation time.
>
> In other words, with the new incremental compiler, the biggest slowdown 
> during recompilation seems to be the linker now, right?
>

Yes, we've come to the same conclusion and Roberto is working on the linker 
speed right now.
 

> If the linking time can go down to, say, a couple of seconds, then SDM is 
> there (IMO).
>
> So out of pure curiosity - are there obvious optimizations in the linker 
> which are still pending?
>
> For example, I noticed that a huge amount of time (> 50% of the whole 
> linking time) is spend in "Saving source with extras", i.e. SourceSaver. 
> Given that these sources are only needed by the browser during a 
> sourcemap-based debugging session, can't they be served on the fly by the 
> code server rather than being pregenerated on disk during the 
> recompilation? There are debugging scenarios (in-IDE debugging with SDBG), 
> where these source files are not even needed and never requested from the 
> codeserver.
>
> Also, is there any way to accelerate the linker which is in the control of 
> the developer?
> For example, can I turn off the above source files pregeneration to see 
> how much time is saved during recompilation?
>
> = 
>
> Finally one side question:
> Does the incremental compiler preserve the names of the generated 
> JavaScript types/functions across recompiles? Including the names of the 
> types/functions which ARE recompiled (as long as the programmer did not 
> change the name of the corresponding Java method/class, that is)?
>
Yes, names in this mode are now consistent for the life of that 
SuperDevMode process (see 
https://gwt.googlesource.com/gwt/+/master/dev/core/src/com/google/gwt/dev/js/JsPersistentPrettyNamer.java).
 

>
> Because if it does, this opens interesting possibilities:
>
> (a) Javascript hot-swapping 
> One disadvantage of the current SDM recompilation is that it triggers page 
> reload in the browser.
> If the application being debugged is a stateful, single-paged one, this 
> means that even for a very small change, the developer essentially 
> starts-over the debugging, as the application state is lost. 
>
> Now, the debugging protocol of Blink allows the debugger to hot-swap a 
> piece of JavaScript with a newer version, right?
> If the incremental compiler preserves the names of the generated 
> Javascript types/functions, then a debugger like SDBG can detect the SDM 
> recompilation and can pro-actively push the newly generated JavaScript into 
> the browser. Imagine how cool is this: you stay on the page, spot a bug 
> (say, in an onclick event handler), fix the bug in the Java code, hit the 
> Recompile button (or Eclipse/SDBG does it for you behind the scenes when 
> you save your Java file!) and there you have it - you can click again and 
> now the correct, recompiled event handler will be triggered. All of this 
> without leaving the application page and starting over.
>
> Of course, for the above to work, additional changes may be necessary. For 
> example, contrary to production, the generated javascript file should 
> probably always be named the same way. Perhaps, it should also not contain 
> one big "eval" statement. Finally, for big changes, the above hot-swapping 
> will likely not work, but the idea is not to make it work in all cases, but 
> for the majority of the small, incremental changes.
>
It does sound like an interesting idea. If at some point the browser reload 
time (as opposed to the compile time) is the longest part then we will 
start looking at ways to speed that up. One similarish idea we've discussed 
is to change the linkers so that the output JS exists as separates files. 
This would allow the browser to hold on to it's precompiled javascript 
cache for parts of the application that haven't changed (thus reload much 
faster). But as Ray mentions, we're pretty heads down on an earlier stage 
of development right now. So I probably won't be look

[gwt-contrib] Re: SDM "-XcompilePerFile": initial feedback

[Roberto Lublinerman]

>
> So out of pure curiosity - are there obvious optimizations in the linker 
> which are still pending?
>

We have been thinking about strategies to incrementalize the work done by 
the linkers, one idea is to have a linker that outputs js for different 
Java classes to different resources and that could be incrementalized.

>
> For example, I noticed that a huge amount of time (> 50% of the whole 
> linking time) is spend in "Saving source with extras", i.e. SourceSaver. 
> Given that these sources are only needed by the browser during a 
> sourcemap-based debugging session, can't they be served on the fly by the 
> code server rather than being pregenerated on disk during the 
> recompilation? There are debugging scenarios (in-IDE debugging with SDBG), 
> where these source files are not even needed and never requested from the 
> codeserver.
>
> I am currently working on this one. There are two options here: 1) serving 
source code directly without copying and 2) copying only "changed" sources. 
Option 1) would be the optimal in terms of compile time but it might result 
of a worse debugging experience (if you modify a source while debugging the 
source mapping will be out of date).
 

-- 
You received this message because you are subscribed to the Google Groups "GWT 
Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit-contributors+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-web-toolkit-contributors/cc62d937-5db6-451a-b2d5-65982dcc9f5a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [gwt-contrib] SDM "-XcompilePerFile": initial feedback

[Ivan Markov]

>
>
>  
>
>> For example, I noticed that a huge amount of time (> 50% of the whole 
>> linking time) is spend in "Saving source with extras", i.e. SourceSaver. 
>> Given that these sources are only needed by the browser during a 
>> sourcemap-based debugging session, can't they be served on the fly by the 
>> code server rather than being pregenerated on disk during the 
>> recompilation? There are debugging scenarios (in-IDE debugging with SDBG), 
>> where these source files are not even needed and never requested from the 
>> codeserver.
>>
>>
> Actually, saving sources isn't strictly needed to make source maps work, 
> especially for non-Googlers, so we can probably default this to false.
>
>
This is likely to break the ChromeDevTools debugger, as it does need and 
does request those source files from the codeserver, isn't it?
(SDBG does not need these as it has the "real" source files in the Eclipse 
IDE.)

Therefore the perfect scenario would be if these files are indeed 
materialized by the codeserver lazily, on a file-by-file basis, once such a 
file is requested by the browser.
A compromise would be if there is an option to switch their generation off, 
but it defaults to ON (generating them).


>> Finally one side question:
>> Does the incremental compiler preserve the names of the generated 
>> JavaScript types/functions across recompiles? Including the names of the 
>> types/functions which ARE recompiled (as long as the programmer did not 
>> change the name of the corresponding Java method/class, that is)?
>>
>>
> Yes, within a session. If you restart SDM, then no. Hot-patching and 
> expression evaluation are definitely things we've discussed (I want to 
> build an Apple XCode/Swift Playground-like REPL for fun myself), but first 
> we've got to make the normal SDM experience out of the box great, including 
> IDE integration and super-fast recompile.  Once we get there, hopefully we 
> can do the other things and reach a state that was better than DevMode was.
>
> I think within a session is good enough.
And yes, super-fast recompile is the most important thing of all. 
Everything else can wait.

 

-- 
You received this message because you are subscribed to the Google Groups "GWT 
Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit-contributors+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-web-toolkit-contributors/fd6bda6d-1a6e-4e8c-9df3-7d62bf7bbc77%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [gwt-contrib] SDM "-XcompilePerFile": initial feedback

['Ray Cromwell' via GWT Contributors]

On Mon, Aug 18, 2014 at 7:14 AM, Ivan Markov  wrote:

> Dear GWT team (and John in particular),
>
>
> In other words, with the new incremental compiler, the biggest slowdown
> during recompilation seems to be the linker now, right?
> If the linking time can go down to, say, a couple of seconds, then SDM is
> there (IMO).
>
>
Yes, we are working on speeding up the linkers right now. Most of the
linkers just write a file to disk, which should happen in less than a
second.


> For example, I noticed that a huge amount of time (> 50% of the whole
> linking time) is spend in "Saving source with extras", i.e. SourceSaver.
> Given that these sources are only needed by the browser during a
> sourcemap-based debugging session, can't they be served on the fly by the
> code server rather than being pregenerated on disk during the
> recompilation? There are debugging scenarios (in-IDE debugging with SDBG),
> where these source files are not even needed and never requested from the
> codeserver.
>
>
Actually, saving sources isn't strictly needed to make source maps work,
especially for non-Googlers, so we can probably default this to false.


> Finally one side question:
> Does the incremental compiler preserve the names of the generated
> JavaScript types/functions across recompiles? Including the names of the
> types/functions which ARE recompiled (as long as the programmer did not
> change the name of the corresponding Java method/class, that is)?
>
>
Yes, within a session. If you restart SDM, then no. Hot-patching and
expression evaluation are definitely things we've discussed (I want to
build an Apple XCode/Swift Playground-like REPL for fun myself), but first
we've got to make the normal SDM experience out of the box great, including
IDE integration and super-fast recompile.  Once we get there, hopefully we
can do the other things and reach a state that was better than DevMode was.


 --
> You received this message because you are subscribed to the Google Groups
> "GWT Contributors" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-web-toolkit-contributors+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/google-web-toolkit-contributors/5a2a4436-5c2d-4e60-9d5d-314db0045b7d%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "GWT 
Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit-contributors+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-web-toolkit-contributors/CAPVRV7fS17GY57Qsx_%3DyCqpi9vtz8SJud92j2kKFrh7v2kOFRA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[gwt-contrib] Re: Compiling an application with current master

[Johannes Barop]

Thanks. That solved the issue!

Am Montag, 18. August 2014 16:12:24 UTC+2 schrieb Thomas Broyer:
>
>
>
> On Monday, August 18, 2014 3:31:35 PM UTC+2, Johannes Barop wrote:
>>
>> Hi,
>>
>> I was trying out the latest changes in the master branch. Is there 
>> something special I have take into account when using the gwt-maven-plugin?
>>
>> When doing a gwt:compile I get this error.
>>
>> [INFO] Loading inherited module 'GwtPlayground'
>> [INFO]Loading inherited module 'com.google.gwt.user.User'
>> [INFO]   Loading inherited module 'com.google.gwt.media.Media'
>> [INFO]  Loading inherited module 'com.google.gwt.user.UI'
>> [INFO] Loading inherited module 
>> 'com.google.gwt.uibinder.UiBinder'
>> [INFO][ERROR] Line 20: Unexpected element 'resource'
>> [INFO][ERROR] Failure while parsing XML
>>
>> It works with (S)DM. Any suggestions?
>>
>
> The error would mean you still have an older gwt-dev in the classpath. 
> Could it be that you forgot to override the GWT dependencies of the 
> gwt-maven-plugin itself? 
> http://mojo.codehaus.org/gwt-maven-plugin/user-guide/using-different-gwt-sdk-version.html
> or maybe a dependency that transitively depends on gwt-dev?
>

-- 
You received this message because you are subscribed to the Google Groups "GWT 
Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit-contributors+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-web-toolkit-contributors/0e523536-d2fb-4ca0-b379-8edd3eb8e8a6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[gwt-contrib] SDM "-XcompilePerFile": initial feedback

[Ivan Markov]

Dear GWT team (and John in particular),

Lately I was checking from time to time the progress of the SDM incremental 
compiler in GWT 2.7.0.

Let me first say - great work so far! 
This morning our project was - for the first time - compilable with the new 
incremental SDM code-path, without internal compiler errors or visible 
malfunctioning.

And this with NO changes to the codebase. Also the fact that I don't have 
to split our codebase into very many small acyclic modules (not that it is 
not a good practice anyway) is obviously a HUGE PLUS.

What I am observing in terms of recompilation times is encouraging (25 
seconds with 2.7 vs. 47 with 2.6.1) but what is even more interesting is 
that the compilation itself (up until where it says "Compile of 
permutations succeeded") seems to take only 7 seconds, which is less than 
30% of the recompilation time.

In other words, with the new incremental compiler, the biggest slowdown 
during recompilation seems to be the linker now, right?
If the linking time can go down to, say, a couple of seconds, then SDM is 
there (IMO).

So out of pure curiosity - are there obvious optimizations in the linker 
which are still pending?

For example, I noticed that a huge amount of time (> 50% of the whole 
linking time) is spend in "Saving source with extras", i.e. SourceSaver. 
Given that these sources are only needed by the browser during a 
sourcemap-based debugging session, can't they be served on the fly by the 
code server rather than being pregenerated on disk during the 
recompilation? There are debugging scenarios (in-IDE debugging with SDBG), 
where these source files are not even needed and never requested from the 
codeserver.

Also, is there any way to accelerate the linker which is in the control of 
the developer?
For example, can I turn off the above source files pregeneration to see how 
much time is saved during recompilation?

= 

Finally one side question:
Does the incremental compiler preserve the names of the generated 
JavaScript types/functions across recompiles? Including the names of the 
types/functions which ARE recompiled (as long as the programmer did not 
change the name of the corresponding Java method/class, that is)?

Because if it does, this opens interesting possibilities:

(a) Javascript hot-swapping 
One disadvantage of the current SDM recompilation is that it triggers page 
reload in the browser.
If the application being debugged is a stateful, single-paged one, this 
means that even for a very small change, the developer essentially 
starts-over the debugging, as the application state is lost. 

Now, the debugging protocol of Blink allows the debugger to hot-swap a 
piece of JavaScript with a newer version, right?
If the incremental compiler preserves the names of the generated Javascript 
types/functions, then a debugger like SDBG can detect the SDM recompilation 
and can pro-actively push the newly generated JavaScript into the browser. 
Imagine how cool is this: you stay on the page, spot a bug (say, in an 
onclick event handler), fix the bug in the Java code, hit the Recompile 
button (or Eclipse/SDBG does it for you behind the scenes when you save 
your Java file!) and there you have it - you can click again and now the 
correct, recompiled event handler will be triggered. All of this without 
leaving the application page and starting over.

Of course, for the above to work, additional changes may be necessary. For 
example, contrary to production, the generated javascript file should 
probably always be named the same way. Perhaps, it should also not contain 
one big "eval" statement. Finally, for big changes, the above hot-swapping 
will likely not work, but the idea is not to make it work in all cases, but 
for the majority of the small, incremental changes.

(b) On the fly evaluation of Java expressions in the debugger
The challenge here is that the expression being evaluated must be compiled 
as if it is already part of the GWT application (i.e.  if it refers to 
application's classes, methods fields etc., it should be with their correct 
javascript names). 
This sounds like a perfect fit for the incremental compiler - how about a 
new HTTP service point in the codeserver? Say, something like 
http://localhost:?eval=, 
which outputs, as a response, a valid JavaScript block of code which can be 
evaluated by the debugger? SDBG (or other debugger) will then use the SDM 
codeserver as a compiler of sorts for the expression entered by the user 
during debugging.

I'm surely missing something - for example the evaluated expression needs 
to be capable of accessing the variables on the currently selected 
stackframe in the debugger, but if Eclipse's Java debugger is capable of 
doing that without special support in the Java debugging protocol, then it 
should be possible for our case as well.

What do you think?

-- 
You received this message because you are subscribed to the Google Groups "GWT 
Contributors" grou

[gwt-contrib] Re: Compiling an application with current master

[Thomas Broyer]

On Monday, August 18, 2014 3:31:35 PM UTC+2, Johannes Barop wrote:
>
> Hi,
>
> I was trying out the latest changes in the master branch. Is there 
> something special I have take into account when using the gwt-maven-plugin?
>
> When doing a gwt:compile I get this error.
>
> [INFO] Loading inherited module 'GwtPlayground'
> [INFO]Loading inherited module 'com.google.gwt.user.User'
> [INFO]   Loading inherited module 'com.google.gwt.media.Media'
> [INFO]  Loading inherited module 'com.google.gwt.user.UI'
> [INFO] Loading inherited module 
> 'com.google.gwt.uibinder.UiBinder'
> [INFO][ERROR] Line 20: Unexpected element 'resource'
> [INFO][ERROR] Failure while parsing XML
>
> It works with (S)DM. Any suggestions?
>

The error would mean you still have an older gwt-dev in the classpath. 
Could it be that you forgot to override the GWT dependencies of the 
gwt-maven-plugin 
itself? 
http://mojo.codehaus.org/gwt-maven-plugin/user-guide/using-different-gwt-sdk-version.html
or maybe a dependency that transitively depends on gwt-dev?

-- 
You received this message because you are subscribed to the Google Groups "GWT 
Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit-contributors+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-web-toolkit-contributors/eb432ee2-7f57-448b-9b53-7a77574e4cfb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[gwt-contrib] Compiling an application with current master

[Johannes Barop]

Hi,

I was trying out the latest changes in the master branch. Is there 
something special I have take into account when using the gwt-maven-plugin?

When doing a gwt:compile I get this error.

[INFO] Loading inherited module 'GwtPlayground'
[INFO]Loading inherited module 'com.google.gwt.user.User'
[INFO]   Loading inherited module 'com.google.gwt.media.Media'
[INFO]  Loading inherited module 'com.google.gwt.user.UI'
[INFO] Loading inherited module 
'com.google.gwt.uibinder.UiBinder'
[INFO][ERROR] Line 20: Unexpected element 'resource'
[INFO][ERROR] Failure while parsing XML

It works with (S)DM. Any suggestions?


Cheers,
Johannes

-- 
You received this message because you are subscribed to the Google Groups "GWT 
Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit-contributors+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-web-toolkit-contributors/52a73975-7c39-4b21-8bab-7f8708dfaa31%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.