https://issues.dlang.org/show_bug.cgi?id=14641
briancsch...@gmail.com changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |briancsch...@gmail.com
--- Comment #1 from briancsch...@gmail.com ---
The best way to do this that I've found is to skip everything other than a set
of bytes that varies based on the comment being lexed:
For /* */ comments:
0x0c (\n)
0x0d (\r)
0x2a (*)
0x2f (/)
x0e2 (Beginning of multi-byte UTF-8 newline)
For /+ +/ comments:
0x0c (\n)
0x0d (\r)
0x2b (+)
0x2f (/)
x0e2 (Beginning of multi-byte UTF-8 newline)
For // comments:
0x0c (\n)
0x0d (\r)
x0e2 (Beginning of multi-byte UTF-8 newline)
The instruction used in libdparse to do this is "pcmpestri", which requires
SSE4.2 (First released in 2008 according to wikipedia). My advice is to leave
most of the logic intact and implement the assembly code such that it may
advance the lexer 0 or more bytes, so that the rest of the algorithm is not
disrupted on machines that don't support SSE4.2.
--
