RE: Result grouping performance

2017-11-23 Thread Kempelen , Ákos
case. Testing environment: Solr 7.0.1, 1 core with 70GB data (~1.000.000 documents), no shards Akos -Original Message- From: Mikhail Khludnev [mailto:m...@apache.org] Sent: Thursday, November 23, 2017 8:38 AM To: solr-user Subject: Re: Result grouping performance Akos, Can you provide

Re: Result grouping performance

2017-11-22 Thread Mikhail Khludnev
> > I am migrating our codebase from Solr 4.7 to 7.0.1 but the performance of > result grouping seems very poor using the newer Solr. > For example a simple MatchAllDocsQuery takes 5 sec on Solr4.7, and 21 sec > on Solr7. > I wonder what causes the x4 difference in time? We hoped that

Re: Result grouping performance

2017-11-22 Thread Erick Erickson
Have you enabled docValues (and reindexed from scratch) on the field you're grouping on? Best, Erick On Wed, Nov 22, 2017 at 5:13 AM, Kempelen, Ákos wrote: > Hello, > > I am migrating our codebase from Solr 4.7 to 7.0.1 but the performance of > result grouping seems very poo

Result grouping performance

2017-11-22 Thread Kempelen , Ákos
Hello, I am migrating our codebase from Solr 4.7 to 7.0.1 but the performance of result grouping seems very poor using the newer Solr. For example a simple MatchAllDocsQuery takes 5 sec on Solr4.7, and 21 sec on Solr7. I wonder what causes the x4 difference in time? We hoped that newer Solr

Re: Result Grouping vs. Collapsing Query Parser -- Can one be deprecated?

2016-10-20 Thread Jeff Wartes
I’ll also mention the choice to improve processing speed by allocating more memory, which increases the importance of GC tuning. This bit me when I tried using it on a larger index. https://issues.apache.org/jira/browse/SOLR-9125 I don’t know if the result grouping feature shares the same

Re: Result Grouping vs. Collapsing Query Parser -- Can one be deprecated?

2016-10-19 Thread Joel Bernstein
omething to fix this particular situation in >> the future. >> >> Solr currently has two ways to get grouped results (so far!). You can >> either use Result Grouping or you can use the Collapsing Query Parser. >> Result grouping seems like the obvious way to go. It

Re: Result Grouping vs. Collapsing Query Parser -- Can one be deprecated?

2016-10-19 Thread Joel Bernstein
> the future. > > Solr currently has two ways to get grouped results (so far!). You can > either use Result Grouping or you can use the Collapsing Query Parser. > Result grouping seems like the obvious way to go. It's well documented, the > parameters are clear, it doesn't

Re: Result Grouping vs. Collapsing Query Parser -- Can one be deprecated?

2016-10-19 Thread John Bickerstaff
perhaps see if we can do something to fix this particular situation in > the future. > > Solr currently has two ways to get grouped results (so far!). You can > either use Result Grouping or you can use the Collapsing Query Parser. > Result grouping seems like the obvious way to

Result Grouping vs. Collapsing Query Parser -- Can one be deprecated?

2016-10-19 Thread Mike Lissner
Hi all, I've had a rotten day today because of Solr. I want to share my experience and perhaps see if we can do something to fix this particular situation in the future. Solr currently has two ways to get grouped results (so far!). You can either use Result Grouping or you can use the Colla

Re: result grouping in sharded index

2016-06-15 Thread Jay Potharaju
aju > wrote: > > Any suggestions on how to handle result grouping in sharded index? > > > > > > On Mon, Jun 13, 2016 at 1:15 PM, Jay Potharaju > > wrote: > > > >> Hi, > >> I am working on a functionality that would require me to group docum

Re: result grouping in sharded index

2016-06-15 Thread Tom Evans
Do you have to group, or can you collapse instead? https://cwiki.apache.org/confluence/display/solr/Collapse+and+Expand+Results Cheers Tom On Tue, Jun 14, 2016 at 4:57 PM, Jay Potharaju wrote: > Any suggestions on how to handle result grouping in sharded index? > > > On Mon, Jun 1

Re: result grouping in sharded index

2016-06-14 Thread Jay Potharaju
Any suggestions on how to handle result grouping in sharded index? On Mon, Jun 13, 2016 at 1:15 PM, Jay Potharaju wrote: > Hi, > I am working on a functionality that would require me to group documents > by a id field. I read that the ngroups feature would not work in a sharded >

result grouping in sharded index

2016-06-13 Thread Jay Potharaju
Hi, I am working on a functionality that would require me to group documents by a id field. I read that the ngroups feature would not work in a sharded index. Can someone recommend how to handle this in a sharded index? Solr Version: 5.5 https://cwiki.apache.org/confluence/display/solr/Result

Re: [CONF] Apache Solr Reference Guide > Result Grouping

2015-11-04 Thread Zheng Lin Edwin Yeo
For me, I'm using the signature field grouping method, as shown from this website: https://cwiki.apache.org/confluence/display/solr/De-Duplication You can set the signatureField to be "title", then during the query, instead of using &group=true&group.field=title, you can use &group=true&group.fiel

Re: [CONF] Apache Solr Reference Guide > Result Grouping

2015-11-04 Thread Jan Høydahl
I second Toke’s recommendation to ensure you have a pure string-version of your title. For pure de-duplication you could also consider the lighter-weight CollapseComponent Instead of &group=true&group.field=title, use &fq={!collapse field=title_string} See https://cwiki.apache.org/confluence/d

Re: [CONF] Apache Solr Reference Guide > Result Grouping

2015-11-03 Thread Toke Eskildsen
On Tue, 2015-11-03 at 14:53 +0530, vishal raut wrote: > I have indexed various videos in solr which I have in my database. I want > to search for those video titles, but there can be duplicate video titles > as well (If the video is same but source is different, this will have > separate entry in s

[CONF] Apache Solr Reference Guide > Result Grouping

2015-11-03 Thread vishal raut
tps://cwiki.apache.org/confluence/display/%7Evishalraut20> I am using Result Grouping in my search. If I am searching normally Solr gives proper results, but when I am using "group=true" and other grouping options solr returns following error : Too many values for UnInvertedField f

Re: result grouping on all documents

2015-10-21 Thread Emir Arnautovic
Hi Christian, It seems to me that you can use range faceting to get counts. Thanks, Emir -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ On 20.10.2015 17:05, Christian Reuschling wrote: Hi, we try to get the number

result grouping on all documents

2015-10-20 Thread Christian Reuschling
Hi, we try to get the number of documents for given time slots in the index efficiently. For this, we query the solr index like this: http://localhost:8014/solr/myCore/query?q=*:*&rows=1&fl=id&group=true&group.query=modified:[201103010%20TO%20201302010]&group.query=modified:[20

Re: Is docValues required in Solr 5.x for distributed result grouping?

2015-09-24 Thread Oliver Schrenk
The error message looks a lot like this bug https://issues.apache.org/jira/browse/SOLR-7495 group.faceting is broken for numeric values. Does it mean that I have to enable docvalues for every field that I want to facet on? On 24 Sep 2015, at 17:02, Tomoko Uchida mailto:tomoko.uchida.1...@gma

Re: Is docValues required in Solr 5.x for distributed result grouping?

2015-09-24 Thread Tomoko Uchida
Hi, > Of course the Doc Values data structure can be used by anything who wants > to retrieve the column base view of documents per field, but is anywhere > documented all the ways it's used in Solr ? According to the Solr guide about docvalues, it is used in faceting, sorting, and grouping. http

Re: Is docValues required in Solr 5.x for distributed result grouping?

2015-09-24 Thread Alessandro Benedetti
I didn't know DocValues to be used apart from sorting and faceting (fc algorithm) on fields. Of course the Doc Values data structure can be used by anything who wants to retrieve the column base view of documents per field, but is anywhere documented all the ways it's used in Solr ? Cheers 2015-0

Re: Is docValues required in Solr 5.x for distributed result grouping?

2015-09-23 Thread Shawn Heisey
On 9/18/2015 3:27 PM, Shawn Heisey wrote: > A query that works fine in Solr 4.9.1 doesn't work in 5.2.1 with the > same schema. The field that I am grouping on does not have docValues. > I get this exception: > > java.lang.IllegalStateException: unexpected docvalues type SORTED_SET > for field '

Cost of using group.cache.percent parameters in Result Grouping

2015-09-20 Thread Zheng Lin Edwin Yeo
Hi, I've been trying to improve the speed of my Result Grouping, and I've found that by setting the parameter group.cache.percent to 100 actually does improve the speed, especially for the longer query string. But I would like to find out is whether is there any cost in doing so? Lik

Is docValues required in Solr 5.x for distributed result grouping?

2015-09-18 Thread Shawn Heisey
A query that works fine in Solr 4.9.1 doesn't work in 5.2.1 with the same schema. The field that I am grouping on does not have docValues. I get this exception: java.lang.IllegalStateException: unexpected docvalues type SORTED_SET for field 'ip' (expected=SORTED). Use UninvertingReader or index

Result Grouping: Number of results in group is not according to specs

2015-08-17 Thread Arnon Yogev
When using result grouping, Solr specs state the following about the "rows" and "group.limit" params: rows - The number of groups to return. group.limit - Number of rows to return in each group. We are using Solr cloud with a single collection and 64 shards. When grouping

Re: Unexpected docvalues type error using result grouping - Use UninvertingReader or index with docvalues

2015-07-25 Thread Erick Erickson
Simply put, trying to cut corners and intuit what would be OK when changing the schema by NOT reindexing from scratch when you are _not_ completely familiar with the low-level details of Lucene is an recipe for problems. As you are finding out and Shawn explained. Think of it this way. The schema.

Re: Unexpected docvalues type error using result grouping - Use UninvertingReader or index with docvalues

2015-07-24 Thread Shawn Heisey
On 7/24/2015 3:48 PM, shamik wrote: > Here's the part which I'm not able to understand. I've for e.g. Source A, B, > C and D in index. Each source contains "n" number of documents. Now, out of > these, a bunch of documents in A and B are tagged with MediaType. I took the > following steps: > > 1.

Re: Unexpected docvalues type error using result grouping - Use UninvertingReader or index with docvalues

2015-07-24 Thread shamik
aType ? I'll follow your recommendation of creating a new collection, do a full index and delete original collection. -- View this message in context: http://lucene.472066.n3.nabble.com/Unexpected-docvalues-type-error-using-result-grouping-Use-UninvertingReader-or-index-with-docvalues-tp4218

Re: Unexpected docvalues type error using result grouping - Use UninvertingReader or index with docvalues

2015-07-24 Thread Erick Erickson
this message in context: > http://lucene.472066.n3.nabble.com/Unexpected-docvalues-type-error-using-result-grouping-Use-UninvertingReader-or-index-with-docvalues-tp4218939p4219089.html > Sent from the Solr - User mailing list archive at Nabble.com.

Re: Unexpected docvalues type error using result grouping - Use UninvertingReader or index with docvalues

2015-07-24 Thread shamik
ng from scratch ? -- View this message in context: http://lucene.472066.n3.nabble.com/Unexpected-docvalues-type-error-using-result-grouping-Use-UninvertingReader-or-index-with-docvalues-tp4218939p4219089.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Unexpected docvalues type error using result grouping - Use UninvertingReader or index with docvalues

2015-07-23 Thread William Bell
You could try stopping SOLR, going into the data directory and rm -rf * and starting SOLR again. Did you use the schema REST api? Residual ? On Thu, Jul 23, 2015 at 6:57 PM, Shamik Bandopadhyay wrote: > Hi, > > I'm facing this weird error while running result grouping queries.

Unexpected docvalues type error using result grouping - Use UninvertingReader or index with docvalues

2015-07-23 Thread Shamik Bandopadhyay
Hi, I'm facing this weird error while running result grouping queries. This started when I turned on "docvalues" for an existing facet field and indexed the documents. Looking at the exception, I reverted back the change and re-indexed the documents again. But I'm still g

SUM in result grouping

2015-05-25 Thread Abhijit Deka
Is it possible to SUM in a Group query?I am using the Solr group function and it's retrieving the results.Now, I want to SUM the numeric field.Is it possible. My query is like this-http://localhost:8983/solr/glaas/select?%20q=abc%20&fl=TotalInvoices%20&wt=json%20&indent=true%20&debugQuery=true%2

Re: Solr 5.0 --> "IllegalStateException: unexpected docvalues type NONE" on result grouping

2015-03-24 Thread Shawn Heisey
On 3/12/2015 3:36 PM, Alexandre Rafalovitch wrote: Manual optimize is no longer needed for modern Solr. It does great optimization automatically. The only reason I recommended it here is to make sure that all segments are brought up to the latest version and the deleted documents are purged. That

Re: Solr 5.0 --> "IllegalStateException: unexpected docvalues type NONE" on result grouping

2015-03-12 Thread Alexandre Rafalovitch
Manual optimize is no longer needed for modern Solr. It does great optimization automatically. The only reason I recommended it here is to make sure that all segments are brought up to the latest version and the deleted documents are purged. That's something that also would happen automatically eve

Re: Solr 5.0 --> "IllegalStateException: unexpected docvalues type NONE" on result grouping

2015-03-12 Thread shamik
Wow, "optimize" worked like a charm. This really addressed the docvalues issue. A follow-up question, is it recommended to run optimize in a Production Solr index ? Also, in a Sorl cloud mode, do we need to run optimize on each instance / each shard / any instance ? Appreciate your help Alex. -

Re: Solr 5.0 --> "IllegalStateException: unexpected docvalues type NONE" on result grouping

2015-03-12 Thread Alexandre Rafalovitch
Do you have any really old segments in that index? Could be worth trying to optimize them down to one in latest format first. Like Shawn, this is just a "one more idea" proposal. Regards, Alex. Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: http://www.solr-start.com/ O

Re: Solr 5.0 --> "IllegalStateException: unexpected docvalues type NONE" on result grouping

2015-03-12 Thread Shawn Heisey
On 3/11/2015 4:45 PM, shamik wrote: > multiValued="false" required="false" omitNorms="true" docValues="true" > /> 3/11/2015, 2:14:30 PM ERROR SolrDispatchFilter > null:java.lang.IllegalStateException: unexpected docvalues type NONE > for field 'DocumentType' (expected=SORTED). Use UninvertingReade

Re: Solr 5.0 --> "IllegalStateException: unexpected docvalues type NONE" on result grouping

2015-03-12 Thread shamik
Well, I think I've narrowed down the issue. The error is happening when I'm trying to do a rolling update from Solr 4.7 (which is our current version) to 5.0 . I'm able to re-produce this couple of times. If I do a fresh index on a 5.0, it works. Not sure if there's any other way to mitigate it.

Solr 5.0 --> "IllegalStateException: unexpected docvalues type NONE" on result grouping

2015-03-12 Thread Shamik Bandopadhyay
Hi, I've a field which is being used for result grouping. Here's the field definition. This started once I did a rolling update from 4.7 to 5.0. I started getting the error on any group by query --> "SolrDispatchFilter null:java.lang. IllegalStateException: unexpected d

Re: Solr 5.0 --> "IllegalStateException: unexpected docvalues type NONE" on result grouping

2015-03-11 Thread shamik
n context: http://lucene.472066.n3.nabble.com/Solr-5-0-IllegalStateException-unexpected-docvalues-type-NONE-on-result-grouping-tp4192477p4192537.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 5.0 --> "IllegalStateException: unexpected docvalues type NONE" on result grouping

2015-03-11 Thread Chris Hostetter
ge 3)providing a w/o the makes it hard to guess exactly what settings you are using (ie: is "docValuesFormat" specified on the fieldType?) 4) You said you are using result grouping, but you haven't actually shown us an example of what your request (with all params) looks like ..

Re: Solr 5.0 --> "IllegalStateException: unexpected docvalues type NONE" on result grouping

2015-03-11 Thread shamik
ucene.472066.n3.nabble.com/Solr-5-0-IllegalStateException-unexpected-docvalues-type-NONE-on-result-grouping-tp4192477p4192529.html Sent from the Solr - User mailing list archive at Nabble.com.

Solr MLT with result grouping

2014-12-04 Thread Min L
Hi all: Has anyone made solr Morelikethis working with results grouping? Thanks in advance. M

Solr Result Grouping and Sorting

2014-11-21 Thread Sanja Zivotic
r of vehicles per maker in the city and in the given price range (faceting). I tried something like this (using solr grouping https://cwiki.apache.org/confluence/display/solr/Result+Grouping): I group the records with the same "vehicle_full_name" field ('group.field': 'v

Solr Result Grouping and Sorting

2014-11-21 Thread Sanja Zivotic
r of vehicles per maker in the city and in the given price range (faceting). I tried something like this (using solr grouping https://cwiki.apache.org/confluence/display/solr/Result+Grouping): I group the records with the same "vehicle_full_name" field ('group.field': 'v

RE: Result grouping using externalfilefield

2014-10-14 Thread Cario, Elaine
Message- From: Sudhakar Maddineni [mailto:maddineni...@gmail.com] Sent: Friday, September 19, 2014 2:10 PM To: solr-user@lucene.apache.org Subject: Result grouping using externalfilefield Hi, Just trying to understand grouping feature with solrcloud 4.2 and I have 4 node/2 shard cluster setup.I

Result grouping using externalfilefield

2014-09-19 Thread Sudhakar Maddineni
Hi, Just trying to understand grouping feature with solrcloud 4.2 and I have 4 node/2 shard cluster setup.I am trying to group my search results based on an externalfilefield that I defined. Here is my configuration: core1: == schema.xml: solrconfig.xml: And, created empty file

Re: Getting min and max of a solr field for each group while doing field collapsing/result grouping

2014-05-01 Thread Varun Gupta
Hi Ahmet, Thanks for the information! But as per Solr documentation, group.truncate is not supported in distributed searches and I am looking for a solution that can work on SolrCloud. -- Varun Gupta On Thu, May 1, 2014 at 4:12 PM, Ahmet Arslan wrote: > Hi Varun, > > I think you can use group.

Re: Getting min and max of a solr field for each group while doing field collapsing/result grouping

2014-05-01 Thread Ahmet Arslan
Hi Varun, I think you can use group.truncate=true with stats component http://wiki.apache.org/solr/StatsComponent     If true, facet counts are based on the most relevant document of each group matching the query. Same applies for StatsComponent. Default is false. Solr3.4 Supported from Solr

Getting min and max of a solr field for each group while doing field collapsing/result grouping

2014-05-01 Thread Varun Gupta
Hi, I am using SolrCloud for getting results grouped by a particular field. Now, I also want to get min and max value for a particular field for each group. For example, if I am grouping results by city, then I also want to get the minimum and maximum price for each city. Is this possible to do w

Re: numFound is not correct while using Result Grouping

2014-04-22 Thread rachun
I'm still having the same problem which is can not get ngroups when set group.main=true. Is this problem fix? I'm usgin Solr 4.6.0 than you very much, Chun. -- View this message in context: http://lucene.472066.n3.nabble.com/numFound-is-not-correct-while-using-Resul

Re: SolrCloud Result Grouping vs CollapsingQParserPlugin

2014-01-15 Thread shamik
Thanks Joel, really appreciate your help. I'll keep an eye on the 4.6.1 release. -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-Result-Grouping-vs-CollapsingQParserPlugin-tp4111331p4111486.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud Result Grouping vs CollapsingQParserPlugin

2014-01-15 Thread Joel Bernstein
ss multiple shards be specifying the number of bits to use from the shard key." If we split shards, will Result Grouping / CollapsingQParserPlugin and number of results still work ?" With field collapsing you'll need to keep all group docs on the same shard, so you won't be abl

Re: SolrCloud Result Grouping vs CollapsingQParserPlugin

2014-01-14 Thread shamik
ntent. I'm trying to remove them by applying result grouping / CollapsingQParserPlugin. For e.g. lets say I've source ABC, MNO and XYZ. Now, ABC and MNO source contains the duplicate documents, which is identified by a field say adskdedup. I've couple of shards, the id being the url

Re: SolrCloud Result Grouping vs CollapsingQParserPlugin

2014-01-14 Thread Joel Bernstein
. > > > > > Joel Bernstein > Search Engineer at Heliosearch > > > On Tue, Jan 14, 2014 at 6:17 PM, Shamik Bandopadhyay wrote: > >> Hi, >> >> I'm planning to upgrade to Solr 4.6 to move from using Result Grouping >> to >> CollapsingQParserPl

Re: SolrCloud Result Grouping vs CollapsingQParserPlugin

2014-01-14 Thread Joel Bernstein
up on the same shard. Joel Bernstein Search Engineer at Heliosearch On Tue, Jan 14, 2014 at 6:17 PM, Shamik Bandopadhyay wrote: > Hi, > > I'm planning to upgrade to Solr 4.6 to move from using Result Grouping to > CollapsingQParserPlugin. I'm currently using SolrClo

SolrCloud Result Grouping vs CollapsingQParserPlugin

2014-01-14 Thread Shamik Bandopadhyay
Hi, I'm planning to upgrade to Solr 4.6 to move from using Result Grouping to CollapsingQParserPlugin. I'm currently using SolrCloud, couple of issues with Result Grouping are : 1. Slow performance 2. Incorrect result count from ngroup My understanding is that CollapsingQParserPlugi

Re: How to sort over all documents by score after Result Grouping / Field Collapsing

2013-09-25 Thread Erick Erickson
the search results, we would like all these results sorted by their > score. > > So If I use solr result grouping to get the top two result from each group, > then I need to un-group them. > > So my question is there any pure solr solution to handle this? I prefer it > handle by s

Re: How to sort over all documents by score after Result Grouping / Field Collapsing

2013-09-24 Thread go2jun
Thanks Erick for your response. My goal is 1. try to search from solr. In the search result, we would like show no more than two results from the same source id. 2. For the search results, we would like all these results sorted by their score. So If I use solr result grouping to get the top two

Re: How to sort over all documents by score after Result Grouping / Field Collapsing

2013-09-24 Thread Erick Erickson
ed="true"/> > indexed="true"/> > indexed="true"/> > > I know I can use solr Result Grouping / Field Collapsing to get the top 2 > result by grouping by source_id. Within each groups, documents sorted by > scroe by query like this: > http://localho

How to sort over all documents by score after Result Grouping / Field Collapsing

2013-09-23 Thread go2jun
Hi, I have solr documents like this: I know I can use solr Result Grouping / Field Collapsing to get the top 2 result by grouping by source_id. Within each groups, documents sorted by scroe by query like this: http://localhost:8983/solr/select?q=bank&group.field=source_id&gr

Re: result grouping and paging, solr 4.21

2013-07-31 Thread Erick Erickson
Not that I know of. Grouping pretty much treats all groups the same... Best Erick On Wed, Jul 31, 2013 at 4:14 AM, Gunnar wrote: > Hello, > > I'm trying to page results with grouping /field collapsing. My query is: > > ?q=myKeywords&start=0&rows=100&group=true&group.field=myGroupField&group.form

result grouping and paging, solr 4.21

2013-07-31 Thread Gunnar
Hello, I'm trying to page results with grouping /field collapsing. My query is: ?q=myKeywords&start=0&rows=100&group=true&group.field=myGroupField&group.format=simple&group.limit=1 The result will contain 70 groups, is there a way to get 100 records returned, means 70 from each group first doc

Re: Aggregate TermFrequency on Result Grouping / Field Collapsing

2013-07-06 Thread Erick Erickson
ut stating the problem. Perhaps > > there's a better approach that _is_ do-able. > > > > Best > > Erick > > > > > > On Wed, Jul 3, 2013 at 2:14 AM, Tony Mullins > >wrote: > > > > > Any suggestions please ! > > > > > >

Re: Aggregate TermFrequency on Result Grouping / Field Collapsing

2013-07-03 Thread Tony Mullins
t; On Wed, Jul 3, 2013 at 2:14 AM, Tony Mullins >wrote: > > > Any suggestions please ! > > > > > > On Tue, Jul 2, 2013 at 3:24 PM, Tony Mullins > >wrote: > > > > > Hi, > > > > > > Is it possible to perform aggregated termfreq(fie

Re: Aggregate TermFrequency on Result Grouping / Field Collapsing

2013-07-03 Thread Erick Erickson
y Mullins wrote: > Any suggestions please ! > > > On Tue, Jul 2, 2013 at 3:24 PM, Tony Mullins >wrote: > > > Hi, > > > > Is it possible to perform aggregated termfreq(field,term) on Result > > Grouping ? > > > > I am trying to get total count of

Re: Aggregate TermFrequency on Result Grouping / Field Collapsing

2013-07-02 Thread Tony Mullins
Any suggestions please ! On Tue, Jul 2, 2013 at 3:24 PM, Tony Mullins wrote: > Hi, > > Is it possible to perform aggregated termfreq(field,term) on Result > Grouping ? > > I am trying to get total count of term's appearance in a document and then > want to aggregate

Aggregate TermFrequency on Result Grouping / Field Collapsing

2013-07-02 Thread Tony Mullins
Hi, Is it possible to perform aggregated termfreq(field,term) on Result Grouping ? I am trying to get total count of term's appearance in a document and then want to aggregate that count by grouping the document on one of my field. Like this http://localhost:8080/solr/collection1/sel

Re: Result Grouping

2013-06-26 Thread Bryan Bende
ote: > I was reading this documentation on Result Grouping... > http://docs.lucidworks.com/display/solr/Result+Grouping > > which says... > > sort - sortspec - Specifies how Solr sorts the groups relative to each > other. For example, sort=popularity desc will cause the groups to be &g

Re: Result Grouping

2013-06-26 Thread Upayavira
What type of field are you grouping on? What happens when you distribute it? I.e. what specifically goes wrong? Upayavira On Tue, Jun 25, 2013, at 09:12 PM, Bryan Bende wrote: > I was reading this documentation on Result Grouping... > http://docs.lucidworks.com/display/solr/Result+Gr

Result Grouping

2013-06-25 Thread Bryan Bende
I was reading this documentation on Result Grouping... http://docs.lucidworks.com/display/solr/Result+Grouping which says... sort - sortspec - Specifies how Solr sorts the groups relative to each other. For example, sort=popularity desc will cause the groups to be sorted according to the highest

filter result by numFound in Result Grouping

2013-05-09 Thread Shalom Ben-Zvi Kazaz
Hello list In one of our search that we use Result Grouping we have a need to filter results to only groups that have more then one document in the group, or more specifically to groups that have two documents. Is it possible in some way? Thank you

SolrCloud: Result Grouping - no groups with field type with precisionStep > 0

2013-04-09 Thread Elodie Sannier
Hello, I am using the Result Grouping feature with SolrCloud, and it seems that grouping does not work with field types having precisionStep property greater than 0, in distributed mode. I updated the "SolrCloud - Getting Started" page example A (Simple two shard cluster). In my schem

Re: numFound is not correct while using Result Grouping

2013-02-26 Thread Amit Nithian
I need to write some tests which I hope to do tonight and then I think it'll get into 4.2 On Tue, Feb 26, 2013 at 6:24 AM, Nicholas Ding wrote: > Thanks Amit, that's cool! So it will also be fixed on Solr 4.2, right? > > On Mon, Feb 25, 2013 at 6:04 PM, Amit Nithian wrote: > > > Yeah I had a si

Re: numFound is not correct while using Result Grouping

2013-02-26 Thread Nicholas Ding
Thanks Amit, that's cool! So it will also be fixed on Solr 4.2, right? On Mon, Feb 25, 2013 at 6:04 PM, Amit Nithian wrote: > Yeah I had a similar problem. I filed and submitted this patch: > https://issues.apache.org/jira/browse/SOLR-4310 > > Let me know if this is what you are looking for! > A

Re: numFound is not correct while using Result Grouping

2013-02-25 Thread Amit Nithian
Yeah I had a similar problem. I filed and submitted this patch: https://issues.apache.org/jira/browse/SOLR-4310 Let me know if this is what you are looking for! Amit On Mon, Feb 25, 2013 at 1:50 PM, Teun Duynstee wrote: > Ah, I see. The docs say "Although this result format does not have as mu

Re: numFound is not correct while using Result Grouping

2013-02-25 Thread Teun Duynstee
Ah, I see. The docs say "Although this result format does not have as much information, it may be easier for existing solr clients to parse". I guess the ngroups value could be added to this format, but apparently it isn't. I do agree with you that to be usefull (as in possible to read for a client

Re: numFound is not correct while using Result Grouping

2013-02-25 Thread Nicholas Ding
Thanks Teun and Carlos, I set group.ngroups=true, but I don't have this "ngroup" number when I was using group.main = true. On Mon, Feb 25, 2013 at 12:02 PM, Carlos Maroto < cmar...@searchtechnologies.com> wrote: > Use group.ngroups, check it in the Solr wiki for FieldCollapsing > > Carlos Maroto

Re: numFound is not correct while using Result Grouping

2013-02-25 Thread Carlos Maroto
Use group.ngroups, check it in the Solr wiki for FieldCollapsing Carlos Maroto Search Architect at Search Technologies (www.searchtechnologies.com) Nicholas Ding wrote: Hello, I grouped the result, and set group.main=true. I was expecting the numFound equals to the number of groups, but act

Re: numFound is not correct while using Result Grouping

2013-02-25 Thread Teun Duynstee
You have to set group.ngroups=true (see http://wiki.apache.org/solr/FieldCollapsing). Be aware that including the number of groups is a surprisingly heavy operation, though. Teun 2013/2/25 Nicholas Ding > Hello, > > I grouped the result, and set group.main=true. I was expecting the numFound >

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-11-02 Thread ilay
/Scalability-of-Solr-Result-Grouping-Field-Collapsing-Millions-Billions-of-documents-tp4002524p4017945.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Doubts in Result Grouping in solr 3.6.1

2012-09-06 Thread Erick Erickson
, mechravi25 wrote: > Hi, > > I am currently using solr 3.6.1 version and for indexing data, i am using > the data import handler for 3.5 because of the reason posted in the > following forum link > http://lucene.472066.n3.nabble.com/Dataimport-Handler-in-solr-3-6-1-td4001149.html

Doubts in Result Grouping in solr 3.6.1

2012-09-04 Thread mechravi25
Hi, I am currently using solr 3.6.1 version and for indexing data, i am using the data import handler for 3.5 because of the reason posted in the following forum link http://lucene.472066.n3.nabble.com/Dataimport-Handler-in-solr-3-6-1-td4001149.html I am trying to achieve result grouping based

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-23 Thread Mikhail Khludnev
Tom, Feel free to find my benchmark results for two alternative joining approaches. http://blog.griddynamics.com/2012/08/block-join-query-performs.html Regards On Thu, Aug 23, 2012 at 4:40 PM, Erick Erickson wrote: > Tom: > > I thin my comments were that grouping on a field where there was > a u

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-23 Thread Erick Erickson
Tom: I thin my comments were that grouping on a field where there was a unique value _per document_ chewed up a lot of resources. Conceptually, there's a bucket for each unique group value. And grouping on a file path is just asking for trouble. But the memory used for grouping should max as a fu

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-22 Thread Lance Norskog
Yes, distributed grouping works, but grouping takes a lot of resources. If you can avoid in distributed mode, so much the better. On Wed, Aug 22, 2012 at 3:35 PM, Tom Burton-West wrote: > Thanks Tirthankar, > > So the issue in memory use for sorting. I'm not sure I understand how > sorting of gr

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-22 Thread Tom Burton-West
Thanks Tirthankar, So the issue in memory use for sorting. I'm not sure I understand how sorting of grouping fields is involved with the defaults and field collapsing, since the default sorts by relevance not grouping field. On the other hand I don't know much about how field collapsing is impl

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-22 Thread Tom Burton-West
Hi Lance and Tirthankar, We are currently using Solr 3.6. I tried a search across our current 12 shards grouping by book id (record_no in our schema) and it seems to work fine (the query with the actual urls for the shards changed is appended below.) I then searched for the record_no of the seco

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-22 Thread Tom Burton-West
Hi Tirthankar, Can you give me a quick summary of what won't work and why? I couldn't figure it out from looking at your thread. You seem to have a different issue, but maybe I'm missing something here. Tom On Tue, Aug 21, 2012 at 7:10 PM, Tirthankar Chatterjee < tchatter...@commvault.com> wr

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-22 Thread Tom Burton-West
nt work, see my thread on Solr3.6 Field collapsing > > Thanks, > > Tirthankar > > > > -Original Message- > > From: Tom Burton-West > > Date: Tue, 21 Aug 2012 18:39:25 > > To: solr-user@lucene.apache.org > > Reply-To: "solr-user@lucene.apache.o

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-22 Thread Tirthankar Chatterjee
olr-user@lucene.apache.org" > Cc: William Dueber; Phillip Farber > Subject: Scalability of Solr Result Grouping/Field Collapsing: > Millions/Billions of documents? > > Hello all, > > We are thinking about using Solr Field Collapsing on a rather large scale > and wonder if

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-21 Thread Lance Norskog
Solr3.6 Field collapsing > Thanks, > Tirthankar > > -Original Message- > From: Tom Burton-West > Date: Tue, 21 Aug 2012 18:39:25 > To: solr-user@lucene.apache.org > Reply-To: "solr-user@lucene.apache.org" > Cc: William Dueber; Phillip Farber > Subject:

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-21 Thread Tirthankar Chatterjee
ty of Solr Result Grouping/Field Collapsing: Millions/Billions of documents? Hello all, We are thinking about using Solr Field Collapsing on a rather large scale and wonder if anyone has experience with performance when doing Field Collapsing on millions of or billions of documents (details bel

Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-21 Thread Tom Burton-West
Hello all, We are thinking about using Solr Field Collapsing on a rather large scale and wonder if anyone has experience with performance when doing Field Collapsing on millions of or billions of documents (details below. ) Are there performance issues with grouping large result sets? Details: W

Re: Solr 3.5 result grouping is failing

2012-08-15 Thread Erick Erickson
Please attach the results of adding &debugQuery=on to your query in both the success and failure case, there's very little information to go on here. You might review: http://wiki.apache.org/solr/UsingMailingLists Best Erick On Wed, Aug 15, 2012 at 12:57 AM, chethan wrote: > Hi, > > I'm trying

Fwd: Solr 3.5 result grouping is failing

2012-08-14 Thread chethan
Hi, I'm trying to group (field collapse) my search results on a field called "site". The schema says that it has to be indexed: *.* But when I try to query the results with *group.field=site&group.limit=100, *I see only 1 group of results being returned. And the group value is null. This seems to

Field Collapsing / Result Grouping

2012-01-19 Thread cmathur
with sorting/ pagination by using Result Grouping? 5) Why is there a limit to the no of results in a group? Do i have to know the max number my system can have and use that to get all results in a group? group.limit seems to be odd in that respect. Either it should be that, if you dont set

Re: Difference between field collapsing and result grouping

2011-12-13 Thread Chris Hostetter
: Nope, they're the same. The original name was Field Collapsing, : but it was changed to "Grouping" later. Specificly: "Field Collapsing" is one type of usecase for the more general concept of "Result Grouping" (other types of result grouping are "gr

  1   2   >