Yeah, that behavior is consistent with what I documented in my e-book for
Solr. The dot is kept only if between two digits or two letters.
-- Jack Krupansky
-----Original Message-----
From: Jorge Luis BetancourtGonzález
Sent: Sunday, November 2, 2014 4:34 PM
To: solr-user@lucene.apache.org
Subject: Question about StandardTokenizer in Solr 4.9
Hi all:
From the description of the StandardTokenizer, it should Recognizes Internet
domain names and email addresses and preserves them as a single token, which
works great, but I've detected that in cases like this:
socks25.domain.com it outputs 2 tokens: socks25 | domain.com
if the URL doesn't have any numbers:
socks.domain.com it outputs a single token: socks.domain.com
The same happens if the number is not at the end an URL part:
so2cks.domain.com it outputs a single token: so2cks.domain.com
Is this an intended behavior? The odd part is that without the number at the
end of an URL part it works fine.
Regards,