[gwt-contrib] Re: RR : Chunk the JavaScript in the initial fragment
It still sounds to me like the route Scott suggested and Bob agreed with would be better, so as to avoid the whole issue of re-lexing the JS during link. On Tue, Apr 14, 2009 at 1:19 AM, Vitali Lovich vlov...@gmail.com wrote: Should've mentioned this in the original post, but probably the TokenStream.javahttp://mxr.mozilla.org/mozilla/source/js/rhino/src/org/mozilla/javascript/TokenStream.javaclass is the scanner you would be interested in I believe. On Tue, Apr 14, 2009 at 1:14 AM, Vitali Lovich vlov...@gmail.com wrote: Hey guys, Rhino http://www.mozilla.org/rhino/ (one of Mozilla's javascript engines) is written entirely in Java supports JS 1.7 if that helps. I'm sure there's a parser component in there that can be extracted if the license is compatible (MPL/GPL). There's also GromJS http://www.bauk.ws/gromjs.html (public domain I think). On the other hand these are probably overkill for your needs. On Tue, Apr 14, 2009 at 12:33 AM, Lex Spoon sp...@google.com wrote: On Mon, Apr 13, 2009 at 1:53 PM, John Tamplin j...@google.com wrote: On Mon, Apr 13, 2009 at 3:51 PM, Scott Blum sco...@google.com wrote: If we were playing Mao, I would give you a card penalty for stating the obvious. :-) But uh, reliably tracking whether or not you're in a string literal is about as much fun as writing a JavaScript parser. In fact, it might be *exactly* as fun, if you know what I mean. String literals have to be known at parse time, so it seems to me all you have to look for is start/stop quotes (of both sorts) while handling embedded ones of the wrong variety and escaped ones of the right variety. That looks like it can be handled with a single character lookahead scanner, and very little complexity -- what am I missing? It would be awesome if there were a simple lexer available for JavaScript. Given a token stream, brace counting would sound feasible. However, from what I hear it's pretty complicated even to tokenize JavaScript. I don't know exactly what is so hard, but last time I asked, regex literals came up. Lex --~--~-~--~~~---~--~~ http://groups.google.com/group/Google-Web-Toolkit-Contributors -~--~~~~--~~--~--~---
[gwt-contrib] Re: RR : Chunk the JavaScript in the initial fragment
The code LG, one high-level comment about the design. Assuming we're ok
with the design implications, this seems good.
http://gwt-code-reviews.appspot.com/21801/diff/1/2
File
dev/core/src/com/google/gwt/core/ext/linker/impl/SelectionScriptLinker.java
(right):
http://gwt-code-reviews.appspot.com/21801/diff/1/2#newcode419
Line 419: private void segmentJs(StringBuilder b, CharSequence js) {
It's true that we presently never emit comments into the JS, but we need
to be keenly aware that this technique will break in the face of
comments because this depth-tracking technique won't be reliable.
http://gwt-code-reviews.appspot.com/21801
--~--~-~--~~~---~--~~
http://groups.google.com/group/Google-Web-Toolkit-Contributors
-~--~~~~--~~--~--~---
[gwt-contrib] Re: RR : Chunk the JavaScript in the initial fragment
On Mon, Apr 13, 2009 at 2:40 PM, sco...@google.com wrote: The code LG, one high-level comment about the design. Assuming we're ok with the design implications, this seems good. The never-fail way to do this would be to have JsSourceGeneration visitor indicate offset values of top-level statements in the strings that it generates. Then the offset data needs to be threaded back from JavaToJavaScriptCompiler back through the PermutationResult and then onto the Linker API. It's doable, but somehow it feels like it's littering more junk around the compiler. So does anyone want to veto the block-depth-counting approach? -- Bob Vawter Google Web Toolkit Team --~--~-~--~~~---~--~~ http://groups.google.com/group/Google-Web-Toolkit-Contributors -~--~~~~--~~--~--~---
[gwt-contrib] Re: RR : Chunk the JavaScript in the initial fragment
On Mon, Apr 13, 2009 at 2:55 PM, Scott Blum sco...@google.com wrote: Actually, I am going to have to torpedo this (sorry!). It just occurred to me that while comments are only a hypothetical problem, string literals are a real, actual problem that could cause problems today. A string literal with mismatched curly braces would break this. Duh. Right, time to rewrite this. -- Bob Vawter Google Web Toolkit Team --~--~-~--~~~---~--~~ http://groups.google.com/group/Google-Web-Toolkit-Contributors -~--~~~~--~~--~--~---
[gwt-contrib] Re: RR : Chunk the JavaScript in the initial fragment
If we were playing Mao, I would give you a card penalty for stating the obvious. :-) But uh, reliably tracking whether or not you're in a string literal is about as much fun as writing a JavaScript parser. In fact, it might be *exactly* as fun, if you know what I mean. On Mon, Apr 13, 2009 at 3:13 PM, John Tamplin j...@google.com wrote: On Mon, Apr 13, 2009 at 2:55 PM, Scott Blum sco...@google.com wrote: Actually, I am going to have to torpedo this (sorry!). It just occurred to me that while comments are only a hypothetical problem, string literals are a real, actual problem that could cause problems today. A string literal with mismatched curly braces would break this. You could avoid that problem by tracking when you are in a string literal. -- John A. Tamplin Software Engineer (GWT), Google --~--~-~--~~~---~--~~ http://groups.google.com/group/Google-Web-Toolkit-Contributors -~--~~~~--~~--~--~---
[gwt-contrib] Re: RR : Chunk the JavaScript in the initial fragment
On Mon, Apr 13, 2009 at 3:51 PM, Scott Blum sco...@google.com wrote: If we were playing Mao, I would give you a card penalty for stating the obvious. :-) But uh, reliably tracking whether or not you're in a string literal is about as much fun as writing a JavaScript parser. In fact, it might be *exactly* as fun, if you know what I mean. String literals have to be known at parse time, so it seems to me all you have to look for is start/stop quotes (of both sorts) while handling embedded ones of the wrong variety and escaped ones of the right variety. That looks like it can be handled with a single character lookahead scanner, and very little complexity -- what am I missing? -- John A. Tamplin Software Engineer (GWT), Google --~--~-~--~~~---~--~~ http://groups.google.com/group/Google-Web-Toolkit-Contributors -~--~~~~--~~--~--~---
[gwt-contrib] Re: RR : Chunk the JavaScript in the initial fragment
On Mon, Apr 13, 2009 at 1:53 PM, John Tamplin j...@google.com wrote: On Mon, Apr 13, 2009 at 3:51 PM, Scott Blum sco...@google.com wrote: If we were playing Mao, I would give you a card penalty for stating the obvious. :-) But uh, reliably tracking whether or not you're in a string literal is about as much fun as writing a JavaScript parser. In fact, it might be *exactly* as fun, if you know what I mean. String literals have to be known at parse time, so it seems to me all you have to look for is start/stop quotes (of both sorts) while handling embedded ones of the wrong variety and escaped ones of the right variety. That looks like it can be handled with a single character lookahead scanner, and very little complexity -- what am I missing? And couldn't you treat comments similarly without much increase in complexity? (I haven't looked at the patch so sorry if this is irrelevant.) Ian --~--~-~--~~~---~--~~ http://groups.google.com/group/Google-Web-Toolkit-Contributors -~--~~~~--~~--~--~---
[gwt-contrib] Re: RR : Chunk the JavaScript in the initial fragment
On Mon, Apr 13, 2009 at 1:53 PM, John Tamplin j...@google.com wrote: On Mon, Apr 13, 2009 at 3:51 PM, Scott Blum sco...@google.com wrote: If we were playing Mao, I would give you a card penalty for stating the obvious. :-) But uh, reliably tracking whether or not you're in a string literal is about as much fun as writing a JavaScript parser. In fact, it might be *exactly* as fun, if you know what I mean. String literals have to be known at parse time, so it seems to me all you have to look for is start/stop quotes (of both sorts) while handling embedded ones of the wrong variety and escaped ones of the right variety. That looks like it can be handled with a single character lookahead scanner, and very little complexity -- what am I missing? It would be awesome if there were a simple lexer available for JavaScript. Given a token stream, brace counting would sound feasible. However, from what I hear it's pretty complicated even to tokenize JavaScript. I don't know exactly what is so hard, but last time I asked, regex literals came up. Lex --~--~-~--~~~---~--~~ http://groups.google.com/group/Google-Web-Toolkit-Contributors -~--~~~~--~~--~--~---
[gwt-contrib] Re: RR : Chunk the JavaScript in the initial fragment
Hey guys, Rhino http://www.mozilla.org/rhino/ (one of Mozilla's javascript engines) is written entirely in Java supports JS 1.7 if that helps. I'm sure there's a parser component in there that can be extracted if the license is compatible (MPL/GPL). There's also GromJS http://www.bauk.ws/gromjs.html (public domain I think). On the other hand these are probably overkill for your needs. On Tue, Apr 14, 2009 at 12:33 AM, Lex Spoon sp...@google.com wrote: On Mon, Apr 13, 2009 at 1:53 PM, John Tamplin j...@google.com wrote: On Mon, Apr 13, 2009 at 3:51 PM, Scott Blum sco...@google.com wrote: If we were playing Mao, I would give you a card penalty for stating the obvious. :-) But uh, reliably tracking whether or not you're in a string literal is about as much fun as writing a JavaScript parser. In fact, it might be *exactly* as fun, if you know what I mean. String literals have to be known at parse time, so it seems to me all you have to look for is start/stop quotes (of both sorts) while handling embedded ones of the wrong variety and escaped ones of the right variety. That looks like it can be handled with a single character lookahead scanner, and very little complexity -- what am I missing? It would be awesome if there were a simple lexer available for JavaScript. Given a token stream, brace counting would sound feasible. However, from what I hear it's pretty complicated even to tokenize JavaScript. I don't know exactly what is so hard, but last time I asked, regex literals came up. Lex --~--~-~--~~~---~--~~ http://groups.google.com/group/Google-Web-Toolkit-Contributors -~--~~~~--~~--~--~---
