Our business has a need to allow for multiple values for a single field. For example, we have an index of employers where an employer often has multiple ways people refer to it. For example, the company "Wal-mart" is referred to as:
1) Wal-mart 2) Wal-mart Stores 3) Walmart I would like a search for any of these 3 terms to match the Wal-mart employer. I've tried two different approaches for this. Approach 1: Create multiple values for the same field. So the document has these three fields: 1) name=Wal-mart 2) name=Wal-mart Stores 3) name=Walmart The problem with this is Lucene seems to treat the 3 different fields as one long field of "Wal-mart Wal-mart Stores Walmart". This is problematic b/c term frequencies is 2 when a user searches for "Wal-mart". Approach 2: Create different named fields for each value so the document has these 3 fields: 1) name1=Wal-mart 2) name2=Wal-mart Stores 3) name3=Walmart This fixes the issue above but introduces a different problem. The idf calculation is incorrect b/c idf is calculated per field. Most employers only have one name or maybe 2 names. So the name3 fields idf ends up being much higher b/c there are fewer docs with a given term in the name3 field. For now, I'm going with approach 2 but overriding the IndexReader. IndexReader.docFreq(Term t) method always returns the doc frequency from the name1 field even if the Term t is actually for name2 or name3, etc. But this doesn't feel like a clean solution. Any suggestions on how to deal with this? Any ideas would be appreciated. Ryan Aylward
