What you need to do is to create your own tokenizer. Just copy the code from the StandardTokenizer to your XYZTokenizer and make your changes. Then you need to create your own Analyzer class (again copy the code from the StandardAnalyzer) and user your XYZTokenizer in the new XYZAnalyzer you created.
HTH Aviran http://www.aviransplace.com http://shaveh.co.il -----Original Message----- From: bhecht [mailto:[EMAIL PROTECTED] Sent: Monday, May 21, 2007 2:59 PM To: java-user@lucene.apache.org Subject: Implement a tokenizer Hi there, I was interested in changing the StandardTokenzier so it will not remove the "+" (plus) sign from my stream. Looking in the code and documentation, it reads: "If this tokenizer does not suit your application, please consider copying this source code directory to your project and maintaining your own grammar-based tokenizer." I can't understand from this code where I should jump in, and do my change. Can someone point me out to where I should look at in order perform my change? Thanks in advanced -- View this message in context: http://www.nabble.com/Implement-a-tokenizer-tf3792172.html#a10724827 Sent from the Lucene - Java Users mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
