Thank you Nikita for take this subject here! On Fri, Sep 9, 2011 at 10:01, Ferenc Kovacs <tyr...@gmail.com> wrote:
> don't break there but for the next ';'. You can also just count the number of semantic token after T_HALT_COMPILER (ie excluding whitespace and comments) and once you hit 3, halt. less confusing solution would be to explicitly add '(', ')' and ';' to the > result in the T_HALT_COMPILER condition before breking out of the > loop. > If you mean verifying that '(', ')' and (';' or T_CLOSE_TAG) are effectively following T_HALT_COMPILER, I think that's part of the syntax analyser's job, not tokenizer's. If you're ok with this argument, then just couting 3 tokens is really the most basic "syntax analysis" we have to do to fix the pb, don't you think? > could there be other important tokens after the __halt_compiler() > which should be present in the token_get_all() result? > Maybe the binary data itself, as a big T_INLINE_HTML for example ? Also, if token_get_all you be made binary safe, that would be very cool ! (no more eating of \x00-\x1F inside regular code) :) Nicolas