Reviewers: ulan,
Message:
Committed patchset #1 manually as r17414 (presubmit successful).
Description:
Experimental scanner: support latin1 (not just ascii).
BUG=
R=u...@chromium.org
Committed: https://code.google.com/p/v8/source/detail?r=17414
Please review this at https://codereview.chromium.org/48533002/
SVN Base: https://v8.googlecode.com/svn/branches/experimental/parser
Affected files (+3, -3 lines):
M src/lexer/lexer.re
Index: src/lexer/lexer.re
diff --git a/src/lexer/lexer.re b/src/lexer/lexer.re
index
91ebc0b40c6e75133571ce32bda72e8ea8a1b64b..28f1dfc1f1f209f7b0f623290918cb4a0e339ad5
100644
--- a/src/lexer/lexer.re
+++ b/src/lexer/lexer.re
@@ -257,10 +257,10 @@ start_:
eof = "\000";
any = [\000-\377];
- whitespace_char = [ \t\v\f\r];
+ whitespace_char = [ \t\v\f\r\240];
whitespace = whitespace_char+;
- identifier_start = [$_a-zA-Z];
- identifier_char = [$_a-zA-Z0-9];
+ identifier_start = [$_a-zA-Z\300-\377];
+ identifier_char = [$_a-zA-Z0-9\300-\377];
not_identifier_char = any\identifier_char\[\\];
line_terminator = [\n\r]+;
digit = [0-9];
--
--
v8-dev mailing list
v8-dev@googlegroups.com
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups "v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to v8-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
