Re: [fw-general] Lucene limit for pagination
I've been using the SPL LimitIterator:-
$index = Zend_Search_Lucene::open($this-_index);
$query = Zend_Search_Lucene_Search_QueryParser::parse($query);
$hits = new ArrayObject($index-find($query));
if (!count($hits)) {
return null;
}
$hits = new LimitIterator($hits-getIterator(), ($pg - 1) *
$pp, $pp);
return $hits;
(Assuming you have already defined $pp as per-page and $pg as the page
number).
Is there a way to limit the number of rows for pagination in lucene?
IF the file has millions of rows and returns say about 10k rows for
a search (all rows), the server's memory would bog down to load 10 k
rows into memory, right? How are others handling this situation?
--
Simon Mundy | Director | PEPTOLAB
202/258 Flinders Lane | Melbourne | Victoria | Australia | 3000
Voice +61 (0) 3 9654 4324 | Mobile 0438 046 061 | Fax +61 (0) 3 9654
4124
http://www.peptolab.com
Re: [fw-general] Lucene limit for pagination
Hi, You must return all the results, as Lucene must score each document to sort it correctly. Lucene is optimized so that data is only read on demand. When you do a -find(), Lucene returns just the object IDs and score. When you then ask for information about the document, it looks up the document in the index and returns that information. iffy guy wrote: Is there a way to limit the number of rows for pagination in lucene? IF the file has millions of rows and returns say about 10k rows for a search (all rows), the server's memory would bog down to load 10 k rows into memory, right? How are others handling this situation? -- View this message in context: http://www.nabble.com/Lucene-limit-for-pagination-tp14667540s16154p14680486.html Sent from the Zend Framework mailing list archive at Nabble.com.
RE: [fw-general] Lucene limit for pagination
If you cached the result you could save having to perform the query over and over again -Original Message- From: Carl.Vondrick [mailto:[EMAIL PROTECTED] Sent: Tuesday, 8 January 2008 11:33 AM To: fw-general@lists.zend.com Subject: Re: [fw-general] Lucene limit for pagination Hi, You must return all the results, as Lucene must score each document to sort it correctly. Lucene is optimized so that data is only read on demand. When you do a -find(), Lucene returns just the object IDs and score. When you then ask for information about the document, it looks up the document in the index and returns that information. iffy guy wrote: Is there a way to limit the number of rows for pagination in lucene? IF the file has millions of rows and returns say about 10k rows for a search (all rows), the server's memory would bog down to load 10 k rows into memory, right? How are others handling this situation? -- View this message in context: http://www.nabble.com/Lucene-limit-for-pagination-tp14667540s16154p14680486. html Sent from the Zend Framework mailing list archive at Nabble.com.
Re: [fw-general] Lucene limit for pagination
So if there are 10k matching rows, would it return an array of 10k IDs/score? Would that size not bog the memory? Thanks to all for the feedbak. On Jan 7, 2008 7:33 PM, Carl.Vondrick [EMAIL PROTECTED] wrote: Hi, You must return all the results, as Lucene must score each document to sort it correctly. Lucene is optimized so that data is only read on demand. When you do a -find(), Lucene returns just the object IDs and score. When you then ask for information about the document, it looks up the document in the index and returns that information. iffy guy wrote: Is there a way to limit the number of rows for pagination in lucene? IF the file has millions of rows and returns say about 10k rows for a search (all rows), the server's memory would bog down to load 10 k rows into memory, right? How are others handling this situation? -- View this message in context: http://www.nabble.com/Lucene-limit-for-pagination-tp14667540s16154p14680486.html Sent from the Zend Framework mailing list archive at Nabble.com.
