[Citadel Development] Re:SVN commit log: revision 5376

2007-08-10 Thread IGnatius T Foobar
dothebart: I think we've got a problem. extract_token() is crashing citserver now.

[Citadel Development] Re:SVN commit log: revision 5376

2007-08-10 Thread IGnatius T Foobar
>so |foo|bar|bla > >gimme number 3 > >will give you nada? The tokenizer understands zero-length tokens. So in the example you gave: Token 0 is "" Token 1 is "foo" Token 2 is "bar" Token 3 is "bla" And if you supply it a string that doesn't contain