custom token filter generates empty tokens

Hi :) I wrote a custom token filter which removes special characters. Sometimes, all characters of the token are removed so the filter procudes an empty token. I would like to remove this token from the tokenstream but i'm not sure how to do that. Is there something missing in my custom toke

RE: custom token filter generates empty tokens

search you will need to adjust the position attribute to account for the now-empty token. -Original Message- From: G.Long [mailto:jde...@gmail.com] Sent: Thursday, October 09, 2014 7:54 AM To: java-user@lucene.apache.org Subject: custom token filter generates empty tokens Hi :) I wrot

Re: custom token filter generates empty tokens

Hi G.Long, You can use TrimFilter+LengthFilter to remove empty/whitespace tokens. Ahmet On Thursday, October 9, 2014 5:54 PM, G.Long wrote: Hi :) I wrote a custom token filter which removes special characters. Sometimes, all characters of the token are removed so the filter procudes an empt

Re: custom token filter generates empty tokens

java-user@lucene.apache.org Subject: custom token filter generates empty tokens Hi :) I wrote a custom token filter which removes special characters. Sometimes, all characters of the token are removed so the filter procudes an empty token. I would like to remove this token from the tokenstream b