RE: solr 4.0 - pagination
Then what will be when we filter out only some result and want to group ,how your index time group count will help. - Grijesh -- View this message in context: http://lucene.472066.n3.nabble.com/solr-4-0-pagination-tp1812384p2124747.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: solr 4.0 - pagination
Well, right now, I'm using SOLR in a LOT of my projects. I'm VERY fond of it, proud of it and VERY happy that such a team exists to make it work. Of course the pagination issue is a bit frustrating on the field collapsing... But... heck... I'm currently de-normalizing my postgresql database and... I'm just counting the total unique rows using SQL :D And let SOLR do the rest of the job So... As a generic idea for the SOLR JAVA expert-people, I think the counting of groups per each field should be something to be done at indexing time rather than at query-time. So basically when you index each document, you compute the grouping thingie IF the user would so choose to have his field as eligible for grouping inside his schema.xml... If you guys geather my meaning... So I would have a field like this: Or something like that... and the indexer will know to compute the number of groups that exist for this field, so that number would be available at query-time without too much stress on the memory or CPU. Hope this helps In the meanwhile... just count the total from SQL... And keep your index in-sync often :D Cip. -- View this message in context: http://lucene.472066.n3.nabble.com/solr-4-0-pagination-tp1812384p2123255.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: solr 4.0 - pagination
I see. Let's assume that there are 1000 groups. Can I use safely (with no negative impact on memory usage or slowness) the start = 990, rows = 10 to get the latest page? Or this will not work, due you will need to compute all the groups till 1000, in order to return the last 10, and because of this the whole will be slow / memory usage will increase considerably. regards, Rich -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Sunday, November 07, 2010 21:54 To: Papp Richard Cc: solr-user@lucene.apache.org Subject: Re: solr 4.0 - pagination On Sun, Nov 7, 2010 at 2:45 PM, Papp Richard wrote: > Hi Yonik, > > I've just tried the latest stable version from nightly build: > apache-solr-4.0-2010-11-05_08-06-28.war > > I have some concerns however: I have 3 documents; 2 in the first group, 1 > in the 2nd group. > > 1. I got for matches 3 - which is good, but I still don't know how many > groups I have. (using start = 0, rows = 10) > 2. as far as I see the start / rows is working now, but the matches is > returned incorrectly => it said matches = 3 instead of = 1, when I used > start = 1, rows = 1 "matches" is the number of documents before grouping, so start/rows or group.offset/group.limit will not affect this number. > so can you help me, how to compute how many pages I'll have, because the > matches can't use for this. Solr doesn't even know given the current algorithm, hence it can't return that info. The issue is that to calculate the total number of groups, we would need to keep each group in memory (which could cause a big blowup if there are tons of groups). The current algorithm only keeps the top 10 groups (assuming rows=10) in memory at any one time, hence it has no idea what the total number of groups is. -Yonik http://www.lucidimagination.com __ Information from ESET NOD32 Antivirus, version of virus signature database 5599 (20101107) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __ Information from ESET NOD32 Antivirus, version of virus signature database 5599 (20101107) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com
RE: solr 4.0 - pagination
Hey Yonik, Sorry, I think the matches is ok - because it probably returns always the total document number - however I don't know how to compute the number of pages. thanks, Rich -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Sunday, November 07, 2010 18:04 To: Papp Richard Cc: solr-user@lucene.apache.org Subject: Re: solr 4.0 - pagination On Sun, Nov 7, 2010 at 10:55 AM, Papp Richard wrote: > this is fantastic, but can you tell any time it will be ready ? It already is ;-) Grab the latest trunk or the latest nightly build. -Yonik http://www.lucidimagination.com __ Information from ESET NOD32 Antivirus, version of virus signature database 5598 (20101107) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __ Information from ESET NOD32 Antivirus, version of virus signature database 5599 (20101107) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com
Re: solr 4.0 - pagination
On Sun, Nov 7, 2010 at 2:45 PM, Papp Richard wrote: > Hi Yonik, > > I've just tried the latest stable version from nightly build: > apache-solr-4.0-2010-11-05_08-06-28.war > > I have some concerns however: I have 3 documents; 2 in the first group, 1 > in the 2nd group. > > 1. I got for matches 3 - which is good, but I still don't know how many > groups I have. (using start = 0, rows = 10) > 2. as far as I see the start / rows is working now, but the matches is > returned incorrectly => it said matches = 3 instead of = 1, when I used > start = 1, rows = 1 "matches" is the number of documents before grouping, so start/rows or group.offset/group.limit will not affect this number. > so can you help me, how to compute how many pages I'll have, because the > matches can't use for this. Solr doesn't even know given the current algorithm, hence it can't return that info. The issue is that to calculate the total number of groups, we would need to keep each group in memory (which could cause a big blowup if there are tons of groups). The current algorithm only keeps the top 10 groups (assuming rows=10) in memory at any one time, hence it has no idea what the total number of groups is. -Yonik http://www.lucidimagination.com
RE: solr 4.0 - pagination
Hi Yonik, I've just tried the latest stable version from nightly build: apache-solr-4.0-2010-11-05_08-06-28.war I have some concerns however: I have 3 documents; 2 in the first group, 1 in the 2nd group. 1. I got for matches 3 - which is good, but I still don't know how many groups I have. (using start = 0, rows = 10) 2. as far as I see the start / rows is working now, but the matches is returned incorrectly => it said matches = 3 instead of = 1, when I used start = 1, rows = 1 so can you help me, how to compute how many pages I'll have, because the matches can't use for this. regards, Rich -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Sunday, November 07, 2010 18:04 To: Papp Richard Cc: solr-user@lucene.apache.org Subject: Re: solr 4.0 - pagination On Sun, Nov 7, 2010 at 10:55 AM, Papp Richard wrote: > this is fantastic, but can you tell any time it will be ready ? It already is ;-) Grab the latest trunk or the latest nightly build. -Yonik http://www.lucidimagination.com __ Information from ESET NOD32 Antivirus, version of virus signature database 5598 (20101107) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __ Information from ESET NOD32 Antivirus, version of virus signature database 5599 (20101107) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com
RE: solr 4.0 - pagination
thank you very much Yonik! you are a magician! regards, Rich -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Sunday, November 07, 2010 18:04 To: Papp Richard Cc: solr-user@lucene.apache.org Subject: Re: solr 4.0 - pagination On Sun, Nov 7, 2010 at 10:55 AM, Papp Richard wrote: > this is fantastic, but can you tell any time it will be ready ? It already is ;-) Grab the latest trunk or the latest nightly build. -Yonik http://www.lucidimagination.com __ Information from ESET NOD32 Antivirus, version of virus signature database 5598 (20101107) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __ Information from ESET NOD32 Antivirus, version of virus signature database 5598 (20101107) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com
Re: solr 4.0 - pagination
On Sun, Nov 7, 2010 at 10:55 AM, Papp Richard wrote: > this is fantastic, but can you tell any time it will be ready ? It already is ;-) Grab the latest trunk or the latest nightly build. -Yonik http://www.lucidimagination.com
RE: solr 4.0 - pagination
Dear Yonik, this is fantastic, but can you tell any time it will be ready ? I would need this feature in two weeks. Is it possible to finish and make an update in this time or should I look for another solution cocerning the pgaination (like implement just "more results" link instead of pagination) ? best regards, Rich -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Saturday, October 30, 2010 19:29 To: solr-user@lucene.apache.org Subject: Re: solr 4.0 - pagination On Sat, Oct 30, 2010 at 12:22 PM, Papp Richard wrote: > I'm using Solr 4.0 with grouping (field collapsing), but unfortunately I > can't solve the pagination. It's not implemented yet, but I'm working on that right now. -Yonik http://www.lucidimagination.com __ Information from ESET NOD32 Antivirus, version of virus signature database 5576 (20101029) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __ Information from ESET NOD32 Antivirus, version of virus signature database 5598 (20101107) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com
Re: solr 4.0 - pagination
See: https://issues.apache.org/jira/browse/SOLR-2207 <https://issues.apache.org/jira/browse/SOLR-2207>You can track the progress here, but it hasn't been committed to trunk yet. Your options are: 1> get a recent trunk source tree and apply the patch and compile. 2> wait until it gets put in the trunk and get a nightly build. 3> volunteer to test it a lot ... Best Erick On Sat, Oct 30, 2010 at 1:27 PM, Papp Richard wrote: > Can you estimate please when it will be done? > > > thanks, > Rich > > -Original Message- > From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik > Seeley > Sent: Saturday, October 30, 2010 19:29 > To: solr-user@lucene.apache.org > Subject: Re: solr 4.0 - pagination > > On Sat, Oct 30, 2010 at 12:22 PM, Papp Richard wrote: > > I'm using Solr 4.0 with grouping (field collapsing), but unfortunately I > > can't solve the pagination. > > It's not implemented yet, but I'm working on that right now. > > -Yonik > http://www.lucidimagination.com > > > __ Information from ESET NOD32 Antivirus, version of virus > signature > database 5576 (20101029) __ > > The message was checked by ESET NOD32 Antivirus. > > http://www.eset.com > > > > __ Information from ESET NOD32 Antivirus, version of virus > signature > database 5576 (20101029) __ > > The message was checked by ESET NOD32 Antivirus. > > http://www.eset.com > > >
RE: solr 4.0 - pagination
Can you estimate please when it will be done? thanks, Rich -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Saturday, October 30, 2010 19:29 To: solr-user@lucene.apache.org Subject: Re: solr 4.0 - pagination On Sat, Oct 30, 2010 at 12:22 PM, Papp Richard wrote: > I'm using Solr 4.0 with grouping (field collapsing), but unfortunately I > can't solve the pagination. It's not implemented yet, but I'm working on that right now. -Yonik http://www.lucidimagination.com __ Information from ESET NOD32 Antivirus, version of virus signature database 5576 (20101029) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __ Information from ESET NOD32 Antivirus, version of virus signature database 5576 (20101029) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com
Re: solr 4.0 - pagination
On Sat, Oct 30, 2010 at 12:22 PM, Papp Richard wrote: > I'm using Solr 4.0 with grouping (field collapsing), but unfortunately I > can't solve the pagination. It's not implemented yet, but I'm working on that right now. -Yonik http://www.lucidimagination.com
