I've attached an updated version of the tokeniser documentation patch. Applied.
I'll start restructuring the code to match that API too (except the token writer, since no such code has been written yet). Ok. Many thanks. -- Jose E. Marchesi <[email protected]> http://www.jemarch.net GNU Project http://www.gnu.org
