User "MaxSem" posted a comment on MediaWiki.r97542. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/97542#c22797 Commit summary:
Rewrote tokenizer so that it can actually be understood by mortal humans. All previous tokenizer's glitches are kept intact FOR NOW, except for: * Slightly different handling of invalid phonemes/hieroglyphs. Garbage in, garbage out. * Normalised handling of "!" and "." - now they are handled uniformly no matter if they're separated from other tokens or not, e.g. "a!b" === "a ! b". * Removed the attempt to handle parenthetical grouping pending a proper fix as requested by bug 31000. Comment: Forgot another thing: this also fixed the handling of stuff like <nowiki><hiero>[[ ]]</hiero> and <hiero>[& &]</hiero></nowiki> which was previously broken. _______________________________________________ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
