On Thu, Jan 14, 2016 at 7:59 PM, Andrea Faulds <a...@ajf.me> wrote: >> 2. Make a new extension to prototype this PhpToken class outside of >> the php-src tree, add all the extra goodies we want (array of token >> return, iterator of token return, extra info, limited info, etc...) >> 3. When this extension is good and solid, propose merging into core. >> >> I had reserved the github.com/phplang project for the specification, >> but didn't end up using it. We can share a repo there if you'd like. > > > Like Stas, I think this sounds good. I might even look over what's created, > if I find the time. > > Regarding arrays versus iterators, you can create the former from the > latter, but not the other way around. I'd go for just having an iterator and > no array.. It maps better to how PHP actually works, anyway. > > But enough discussion here, I shall await your git repository... > Okay, i've got a starting point up at https://github.com/phplang/php-token . It's basically just token_get_all() and token_name() as they currently stand (I didn't even bother overriding the behavior in token_name() yet). I did change some of the codeflow layout a bit (and add support for scanning files in the process), but that's it.
Unfortunately... I realized a few issues with my plans to add support for callbacks and iterators... The tokenizer uses a global state that is REALLY easy to invalidate, so once we allow userspace code to resume, we're going to have to be careful about how we preserve/restore/guard the tokenizer's state. This is usually not an issue since there are no scanner callbacks in PHP currently. I've added TazeTSchnitzel(Andrea), smalyshev(Stas), and nikic(Nikita) to the ACL so far. Poke me here or on twitter with your github id if you're interested in contributing... -Sara -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
