Hello,
we have a small internet shop which uses lucene for product search.
With increasing traffic we have continuos problem with literaly
hundreds of threads being BLOCKED in lucene code:
here is an example taken with jstack on prod system:
"ajp-0.0.0.0-8009-1504" daemon prio=10 tid=0x7f81a
Split the data into 2 fields, timestamp and content. Store one lucene
document per line with the 2 fields, timestamp stored and not indexed
(unless you want to search on it), content stored and analyzed. Use
StandardAnalyzer unless you have special requirements.
Then close the IndexWriter, open
