Hello, I am having trouble finding how to remove/ignore whitespace when indexing. The only answer I have found suggested that it is necessary to write my own tokenizer. Is this true? I want to remove whitespace and special characters from the phrase and create N-grams from the result.
Ultimately, the effect I am after is that searching "bobdole" would match "Bob Dole", "Bo B. Dole", and maybe "Bobdo". Maybe there is a better way... can anyone lend some assistance? Thanks! Dev B
