Re: [fw-general] RE: Zend_Search_Lucene - Best Practices for Indexing 100k+ articles

2007-05-14 Thread Alexander Veremyev
webshark27 wrote: Hi Alexander, 1. I Optimized using Luke 0.6 - so there is 1 segment (183mb) a couple of days ago. 2. The search takes 5 seconds before I display any results, just this line: $hits = $index->find($query); And it returns a ton of data, not just the Document's ID. Here: http:

Re: [fw-general] RE: Zend_Search_Lucene - Best Practices for Indexing 100k+ articles

2007-05-14 Thread webshark27
Hi Alexander, 1. I Optimized using Luke 0.6 - so there is 1 segment (183mb) a couple of days ago. 2. The search takes 5 seconds before I display any results, just this line: $hits = $index->find($query); And it returns a ton of data, not just the Document's ID. Here: http://www.articlesbase.

Re: [fw-general] RE: Zend_Search_Lucene - Best Practices for Indexing 100k+ articles

2007-05-14 Thread Alexander Veremyev
1) Index should be optimized (have only one segment) to make search faster. 2) Large search result is a cause of slow searching. Do you retrieve any stored field of returned hits? Note: Search itself only collects documents' IDs, but retrieving any stored field causes full document retrieving.

Re: [fw-general] RE: Zend_Search_Lucene - Best Practices for Indexing 100k+ articles

2007-05-13 Thread Simon Gelfand
Hi Craig, You can see a test here with 130,000 articles indexed I am getting slow searching - 5,6 seconds. I have added paging + max 250 hits displayed + memory caching to speed browsing after an initial search. Here is an example: http://www.articlesbase.com/test-search.php?q=business+consulti

Re: [fw-general] RE: Zend_Search_Lucene - Best Practices for Indexing 100k+ articles

2007-05-09 Thread Craig Slusher
webshark27, When you get your articles indexed, it would be really great if you can share your experience with searching against it. I would love to know how well the Zend implementation of Lucene handles the load. On 5/8/07, webshark27 <[EMAIL PROTECTED]> wrote: Hi Chris, Thanks for the quic

[fw-general] RE: Zend_Search_Lucene - Best Practices for Indexing 100k+ articles

2007-05-08 Thread webshark27
Hi Chris, Thanks for the quick response. Doesn't the "$doc = new Zend_Search_Lucene_Document();" just overwrite the old one? Also I think the $index->addDocument($doc) is filling up the memory fast, I don't know exactly how to play with the MergeFactor, MaxMergeDocs and MaxBufferedDocs effects