I'd be interested in seeing it, sure. It looks like this has been implemented in Perl as well: http://search.cpan.org/author/MPIOTR/Lingua-Ident-1.4/ ...looks trivial. I remember the author's name from a few years back. His thesis was about language recognition, I believe. I still have the printout somewhere.
Otis --- Neil Couture <[EMAIL PROTECTED]> wrote: > Snowball stemmer is not of very good quality. I think the best would > be to build a lemmatizer from ispell more precisely from the ispell > rules syntax. As for the language identifier the best overall > language identifier is based on Ted Dunning. You can find the source > code on the web. > > > its c code but can easily be ported to java. Also of interest is the > Mozilla source code, there is code that do encoding detection. In > fact I devellloped a java lib starting from that source code. Its > based upon the LGPL license would you be interested to merge that > source code in Lucene? > > > -Neil > > > > -----Original Message----- > From: Otis Gospodnetic [mailto:[EMAIL PROTECTED]] > Sent: 7 janvier, 2003 12:06 > To: [EMAIL PROTECTED] > Subject: language identifier contrib > > > Now that Doug put Snowball's stemmer's in Lucene Sandbox, it would be > nice to have that language recognition contribution that somebody > mentioned a month or two ago. > > Ah, here it is, the original email that mentions this language > identifier: > http://nagoya.apache.org/eyebrowse/ReadMsg?[EMAIL PROTECTED]&msgNo=2695 > > There's also this: > http://frank.spieleck.de/ngram/ > > Thanks, > Otis > > > __________________________________________________ > Do you Yahoo!? > Yahoo! Mail Plus - Powerful. Affordable. Sign up now. > http://mailplus.yahoo.com > > -- > To unsubscribe, e-mail: > <mailto:[EMAIL PROTECTED]> > For additional commands, e-mail: > <mailto:[EMAIL PROTECTED]> > > > -- > To unsubscribe, e-mail: > <mailto:[EMAIL PROTECTED]> > For additional commands, e-mail: > <mailto:[EMAIL PROTECTED]> > __________________________________________________ Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now. http://mailplus.yahoo.com -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
