On 03/28/2013 12:06 PM, Mohammad Al Houssami (Alumni) wrote:
Hello everyone.
I was wondering if there is some sort of tests for the Tokenizer along with the
correct output of tokens as well as a way of representing tokens.
What I have in mind is running the tokenizer on some HTML input and printing
the tokens in the same way the correct output is written.
I will then be comparing the result I have with the correct one provided
character by character. :)
http://code.google.com/p/html5lib/source/browse/#hg%2Ftestdata%2Ftokenizer
http://wiki.whatwg.org/wiki/Parser_tests has some documentation of the
format.
