Re: When is too many fields in "qf" is too many?

2015-05-31 Thread Tomasz Borek
inue here out of curiosity how it ends. pozdrawiam, LAFK > -Original Message- > From: Steven White [mailto:swhite4...@gmail.com] > Sent: Thursday, May 28, 2015 5:59 PM > To: solr-user@lucene.apache.org > Subject: Re: When is too many fields in "qf" is too many? > &

RE: When is too many fields in "qf" is too many?

2015-05-29 Thread Reitzel, Charles
sage- From: Steven White [mailto:swhite4...@gmail.com] Sent: Thursday, May 28, 2015 5:59 PM To: solr-user@lucene.apache.org Subject: Re: When is too many fields in "qf" is too many? Hi Folks, First, thanks for taking the time to read and reply to this subject, it is much appreciated, I have yet

Re: When is too many fields in "qf" is too many?

2015-05-28 Thread Steven White
, Reitzel, Charles < > > >> charles.reit...@tiaa-cref.org> wrote: > > >> > > >> > One request handler per view? > > >> > > > >> > I think if you are able to make the actual view in use for the > current > > >&

Re: When is too many fields in "qf" is too many?

2015-05-28 Thread Jack Krupansky
gt; > request a single value (vs. all views that the user could use over > time), > >> > it would keep the qf list down to a manageable size (e.g. specified > >> within > >> > the request handler XML). Not sure if this is feasible for you, >

RE: When is too many fields in "qf" is too many?

2015-05-28 Thread Reitzel, Charles
M To: solr-user@lucene.apache.org Subject: Re: When is too many fields in "qf" is too many? Gotta agree with Jack here. This is an insane number of fields, query performance on any significant corpus will be "fraught" etc. The very first thing I'd look at is having that many

Re: When is too many fields in "qf" is too many?

2015-05-28 Thread Erick Erickson
request handler XML). Not sure if this is feasible for you, but it >> > seems like a reasonable approach given the use case you describe. >> > >> > Just a thought ... >> > >> > -Original Message- >> > From: Steven White [mailto:swhite4...

Re: When is too many fields in "qf" is too many?

2015-05-28 Thread Jack Krupansky
st handler XML). Not sure if this is feasible for you, but it > > seems like a reasonable approach given the use case you describe. > > > > Just a thought ... > > > > -----Original Message----- > > From: Steven White [mailto:swhite4...@gmail.com] > > Se

Re: When is too many fields in "qf" is too many?

2015-05-28 Thread Steven White
t; Just a thought ... > > -Original Message- > From: Steven White [mailto:swhite4...@gmail.com] > Sent: Tuesday, May 26, 2015 4:48 PM > To: solr-user@lucene.apache.org > Subject: Re: When is too many fields in "qf" is too many? > > Thanks Doug. I might hav

RE: When is too many fields in "qf" is too many?

2015-05-27 Thread Reitzel, Charles
fields in "qf" is too many? Thanks Doug. I might have to take you on the hangout offer. Let me refine the requirement further and if I still see the need, I will let you know. Steve On Tue, May 26, 2015 at 2:01 PM, Doug Turnbull < dturnb...@opensourceconnections.com> wrote: >

Re: When is too many fields in "qf" is too many?

2015-05-26 Thread Steven White
Thanks Doug. I might have to take you on the hangout offer. Let me refine the requirement further and if I still see the need, I will let you know. Steve On Tue, May 26, 2015 at 2:01 PM, Doug Turnbull < dturnb...@opensourceconnections.com> wrote: > How you have tie is fine. Setting tie to 1 mi

Re: When is too many fields in "qf" is too many?

2015-05-26 Thread Doug Turnbull
How you have tie is fine. Setting tie to 1 might give you reasonable results. You could easily still have scores that are just always an order of magnitude or two higher, but try it out! BTW Anything you put in teh URL can also be put into a request handler. If you ever just want to have a 15 min

Re: When is too many fields in "qf" is too many?

2015-05-26 Thread Steven White
Hi Doug, I'm back to this topic. Unfortunately, due to my DB structer, and business need, I will not be able to search against a single field (i.e.: using copyField). Thus, I have to use list of fields via "qf". Given this, I see you said above to use "tie=1.0" will that, more or less, address

Re: When is too many fields in "qf" is too many?

2015-05-20 Thread Doug Turnbull
Yeah a copyField into one could be a good space/time tradeoff. It can be more manageable to use an all field for both relevancy and performance, if you can handle the duplication of data. You could set tie=1.0, which effectively sums all the matches instead of picking the best match. You'll still

Re: When is too many fields in "qf" is too many?

2015-05-20 Thread Steven White
Hi Doug, Your blog write up on relevancy is very interesting, I didn't know this. Looks like I have to go back to my drawing board and figure out an alternative solution: somehow get those group-based-fields data into a single field using copyField. Thanks Steve On Wed, May 20, 2015 at 11:17 AM

Re: When is too many fields in "qf" is too many?

2015-05-20 Thread Steven White
Thanks for calling out maxBooleanClauses. The current default of 1024 has not caused me any issues (so far) in my testing. However, you probably saw Doug Tumbull's reply, it looks like my relevance will suffer. Steve On Wed, May 20, 2015 at 11:42 AM, Shawn Heisey wrote: > On 5/20/2015 9:24 AM

Re: When is too many fields in "qf" is too many?

2015-05-20 Thread Steven White
> Also, is this 1500 fields that are always populated, or are there really a > larger number of different record types, each with a relatively small > number of fields populated in a particular document? Answer: This is a large number of different record types, each with a relatively small number

Re: When is too many fields in "qf" is too many?

2015-05-20 Thread Shawn Heisey
On 5/20/2015 9:24 AM, Steven White wrote: > I have already switched to using POST because I need to send a long list of > data in "qf". My question isn't about POST / GET, it's about Solr and > Lucene having to deal with such long list of fields. Here is the text of > my question reposted: > >>

Re: When is too many fields in "qf" is too many?

2015-05-20 Thread Steven White
Thanks Shawn. I have already switched to using POST because I need to send a long list of data in "qf". My question isn't about POST / GET, it's about Solr and Lucene having to deal with such long list of fields. Here is the text of my question reposted: > Given the above, beside the fact that

Re: When is too many fields in "qf" is too many?

2015-05-20 Thread Doug Turnbull
Steven, I'd be concerned about your relevance with that many qf fields. Dismax takes a "winner takes all" point of view to search. Field scores can vary by an order of magnitude (or even two) despite the attempts of query normalization. You can read more here http://opensourceconnections.com/blog/

Re: When is too many fields in "qf" is too many?

2015-05-20 Thread Jack Krupansky
The uf parameter is used to specify which fields a user "may" query against - the "qf" parameter specifies the set of fields that an unfielded query term "must" be queried against. The user is free to specify fielded query terms, like "field1:term1 OR field2:term2". So, which use case are you reall

Re: When is too many fields in "qf" is too many?

2015-05-20 Thread Shawn Heisey
On 5/20/2015 6:27 AM, Steven White wrote: > My solution requires that users in group-A can only search against a set of > fields-A and users in group-B can only search against a set of fields-B, > etc. There can be several groups, as many as 100 even more. To meet this > need, I build my search b